Structured Concurrency

Structured concurrency (JEP 505, still in preview as of JDK 25) treats groups of related concurrent tasks as a single unit of work — bringing lifecycle discipline, error propagation, and observability to fan-out AI patterns that virtual threads alone don't provide.

Why Assess (Not Trial)

Structured concurrency is genuinely useful for AI orchestration code but remains in preview across JDK 21–25 with API changes in each cycle. The JDK 25 version changes scope creation to static factory methods. Adopting preview APIs means potential migration work at each JDK upgrade — hence Assess rather than Trial.

If you're already on JDK 25 and comfortable with preview features, it's worth trialling in non-critical paths.

What It Solves

Virtual threads solve the performance problem of concurrent I/O. Structured concurrency solves the correctness problem:

• If one of 5 parallel LLM calls fails, cancel the other 4 immediately (no orphaned threads)
• If you abandon the outer scope (e.g., request timeout), all child tasks are cancelled
• Thread dumps show the logical task hierarchy, not a flat list of threads

// Fan out to 3 models, fail fast if any fails
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    var claude  = scope.fork(() -> callClaude(prompt));
    var gpt     = scope.fork(() -> callGPT(prompt));
    var gemini  = scope.fork(() -> callGemini(prompt));

    scope.join()           // wait for all to finish
         .throwIfFailed(); // rethrow first failure, cancels survivors

    return bestOf(claude.get(), gpt.get(), gemini.get());
}
// Leaving the block guarantees all tasks have finished or been cancelled

The guarantee — no task outlives its scope — is the structural discipline that makes concurrent AI code reliable.

The JDK 25 API (Latest Preview)

In JDK 25, scopes are created via static factory methods:

// Zero-argument form: succeed-all or fail-fast
try (var scope = StructuredTaskScope.open()) {
    var t1 = scope.fork(task1);
    var t2 = scope.fork(task2);
    scope.join();
}

// Custom policy (e.g., succeed-any — return first success)
try (var scope = StructuredTaskScope.open(Joiner.anySuccessfulResultOrFirstException())) {
    scope.fork(() -> callModelA(prompt));
    scope.fork(() -> callModelB(prompt));
    return scope.join(); // returns whichever finishes first
}

Project Loom's Three Pillars

Structured Concurrency is one of three related features — use them together:

Feature	Status	Use for
Virtual Threads	Stable (Java 21)	High-throughput I/O concurrency
Structured Concurrency	Preview (JDK 25)	Lifecycle and error management
Scoped Values	Preview (JDK 25)	Propagating context (auth, trace IDs) across forks

Scoped Values replace ThreadLocal in concurrent contexts — essential for propagating request context (user ID, trace ID) into forked LLM call tasks.

Key Characteristics

Property	Value
JEP	JEP 505 (5th preview in JDK 25)
Status	Preview — expect API changes
Use case	Fan-out AI calls with lifecycle guarantees
Companion features	Virtual Threads, Scoped Values