AI / LLM

AI / LLM Integration

AI/LLM streaming module for Atmosphere. Provides @AiEndpoint, @Prompt, StreamingSession, the AgentRuntime SPI for auto-detected AI framework adapters, and a built-in OpenAiCompatibleClient that works with Gemini, OpenAI, Ollama, a LiteLLM proxy, and any OpenAI-compatible API.

Maven Coordinates

<dependency>
    <groupId>org.atmosphere</groupId>
    <artifactId>atmosphere-ai</artifactId>
    <version>${project.version}</version>
</dependency>

Architecture

Atmosphere has two pluggable SPI layers. AsyncSupport adapts web containers — Jetty, Tomcat, Undertow. AgentRuntime adapts AI frameworks across all twelve runtimes (Built-in, Spring AI, LangChain4j, Google ADK, Embabel, Koog, Semantic Kernel, AgentScope, Spring AI Alibaba, Anthropic, Cohere, CrewAI). Same design pattern, same discovery mechanism:

Concern	Transport layer	AI layer
SPI interface	AsyncSupport	AgentRuntime
What it adapts	Web containers (Jetty, Tomcat, Undertow)	AI frameworks (all twelve AgentRuntime adapters)
Discovery	Classpath scanning	ServiceLoader
Resolution	Best available container	Highest priority() among isAvailable()
Initialization	init(ServletConfig)	configure(LlmSettings)
Core method	service(req, res)	execute(AgentExecutionContext, StreamingSession)
Fallback	BlockingIOCometSupport	Built-in AgentRuntime (OpenAI-compatible)

This is the Servlet model for AI agents: write your @Agent once, run it on any supported AgentRuntime — determined by classpath.

Quick Start — @AiEndpoint

@AiEndpoint(path = "/ai/chat",
            systemPrompt = "You are a helpful assistant",
            conversationMemory = true)
public class MyChatBot {

    @Prompt
    public void onPrompt(String message, StreamingSession session) {
        session.stream(message);  // auto-detects the best available AgentRuntime from classpath
    }
}

The @AiEndpoint annotation replaces the boilerplate of @ManagedService + @Ready + @Disconnect + @Message for AI streaming use cases. The @Prompt method runs on a virtual thread.

session.stream(message) auto-detects the best available AgentRuntime implementation via ServiceLoader — drop an adapter JAR on the classpath and it just works.

AgentRuntime SPI

The AgentRuntime SPI dispatches the entire agent loop — tool calling, memory, RAG, retries — to the AI framework on the classpath. When multiple implementations are available, the one with the highest priority() that reports isAvailable() wins.

public interface AgentRuntime {
    String name();                                    // e.g. "langchain4j", "spring-ai"
    boolean isAvailable();                            // checks classpath dependencies
    int priority();                                   // higher wins
    void configure(AiConfig.LlmSettings settings);   // called once after resolution
    Set<AiCapability> capabilities();                 // feature discovery
    void execute(AgentExecutionContext context, StreamingSession session);  // full agent loop
}

Classpath JAR	Auto-detected AgentRuntime	Priority
atmosphere-ai (default)	Built-in OpenAiCompatibleClient (Gemini, OpenAI, Ollama, LiteLLM proxy)	0
atmosphere-spring-ai	Spring AI ChatClient	100
atmosphere-langchain4j	LangChain4j StreamingChatLanguageModel	100
atmosphere-adk	Google ADK Runner	100
atmosphere-embabel	Embabel AgentPlatform	100
atmosphere-koog	JetBrains Koog AIAgent	100
atmosphere-semantic-kernel	Microsoft Semantic Kernel ChatCompletionService	100
atmosphere-agentscope	Alibaba AgentScope ReActAgent	100
atmosphere-spring-ai-alibaba	Spring AI Alibaba ReactAgent (see runtime caveat below)	100
atmosphere-anthropic	Anthropic AnthropicMessagesClient (built-in Messages API client) (requires anthropic.api.key)	100
atmosphere-cohere	Cohere CohereChatClient (built-in Chat API client) (requires cohere.api.key)	100
atmosphere-crewai	CrewAI CrewAiSidecarClient (out-of-process Python sidecar over HTTP+SSE) (requires ATMOSPHERE_CREWAI_SIDECAR_URL + live /health)	50

To switch runtimes, change a single Maven dependency — no code changes needed.

Spring AI Alibaba runtime — Spring Boot 3 only today. Spring AI Alibaba 1.1.2.2 is compiled against Spring AI 1.1.6, and spring-ai-alibaba-graph-core-1.1.2.x hardcodes Spring AI 1.1.x-only types like DeepSeekAssistantMessage, so the runtime requires Spring AI 1.1.6. Spring AI 1.1.6 itself requires Spring Boot 3 — it pins the SB3-era FQN of RestClientAutoConfiguration, which Spring Boot 4 ships at a renamed FQN. Drop atmosphere-spring-ai-alibaba into a Spring Boot 3 sample (e.g. samples/spring-boot-ai-chat -Pspring-boot3) and it round-trips end-to-end (verified via chrome-devtools against Ollama). A Spring Boot 4 path will become possible once Alibaba publishes a Spring AI 2.x-aligned spring-ai-alibaba-agent-framework. atmosphere-agentscope is unaffected and works on Spring Boot 4.

Per-Request Runtime Extensions

Each AgentRuntime runs its framework’s “happy path” by default. For requests that need framework-native composition (Spring AI advisor chain, LangChain4j AiServices, Koog graph DSL, ADK multi-agent topology), a small per-request helper attaches the framework-native object to AgentExecutionContext.metadata() and the runtime applies it for that one call — no AgentRuntime SPI growth, no mutation of shared beans. Every helper follows the CacheHint pattern: from(context) and attach(context, ...) static methods, with strict type checking that throws IllegalArgumentException on a wrong-type slot (silent drops would mask the override never firing).

Helper	Runtime	Slot it drives
SpringAiAdvisors	Spring AI	ChatClient.prompt().advisors(...) — RAG, memory, guardrails, observability (additive — multiple advisors compose into a chain)
LangChain4jAiServices	LangChain4j	Routes through caller’s AiServices-backed interface (TokenStream callbacks bridged to session) — gives access to maxSequentialToolsInvocations, custom system message providers, etc.
KoogStrategy	Koog	Swaps default chatAgentStrategy() with a custom AIAgentGraphStrategy<String, String> from the strategy {} DSL
AdkRootAgent	ADK	Replaces the runtime’s default LlmAgent with SequentialAgent / ParallelAgent / LoopAgent / any BaseAgent subclass
SemanticKernelInvocation	Semantic Kernel	Per-request InvocationContext — unlocks KernelHooks, withMaxAutoInvokeAttempts, custom PromptExecutionSettings
EmbabelPromptRunner	Embabel	UnaryOperator<PromptRunner> customizer applied AFTER the runtime’s default wiring — stack withTemperature / withModel / withGuardrails on top. Atmosphere-native dispatch path only
AgentScopeAgent	AgentScope	Per-request ReActAgent — useful when different prompts route through different agent topologies (planner vs. quick lookup) without re-installing the runtime client
SpringAiAlibabaRunnableConfig	Spring AI Alibaba	Per-request RunnableConfig — Alibaba’s natural per-invocation handle for threadId (memory thread continuation), checkPointId (resume), streamMode, metadata, store
ToolLoopPolicies	Built-in, Koog	Per-request ToolLoopPolicy(maxIterations, OnMaxIterations) — Built-in honors via OpenAI-compatible tool loop, Koog via AIAgent.maxIterations

Example — Spring AI advisor scoped to one request:

var safeGuard = SafeGuardAdvisor.builder()
        .sensitiveWords(List.of("badword"))
        .failureResponse("I cannot answer that.")
        .build();

var ctx = SpringAiAdvisors.attach(baseContext, safeGuard, new SimpleLoggerAdvisor());
runtime.execute(ctx, session);

Each helper ships with a unit-level *BridgeTest that proves the runtime honors the sidecar (e.g. SpringAiAgentRuntime.execute calls promptSpec.advisors(perRequestAdvisors) only when SpringAiAdvisors.from(context) returns non-empty). See the per-module READMEs for full DSL examples: modules/spring-ai, modules/langchain4j, modules/koog, modules/adk, and the ToolLoopPolicy section.

All eight framework-wrapping runtimes ship a per-request sidecar (SpringAiAdvisors, LangChain4jAiServices, KoogStrategy, AdkRootAgent, SemanticKernelInvocation, EmbabelPromptRunner, AgentScopeAgent, SpringAiAlibabaRunnableConfig). The three native runtimes — Anthropic and Cohere (direct HTTP clients) and CrewAI (a Python sidecar process) — wrap no third-party composition DSL, so they take per-request configuration through AgentExecutionContext (system prompt, retry policy, tool approval) rather than a dedicated sidecar. Embabel also has native streaming: when StreamingPromptRunnerBuilder.streaming().generateStream() is available the runtime emits Flux<String> chunks directly to the session, with graceful fallback to runner.generateText(...) when the streaming API is absent.

Model-Lifecycle Observability

AgentLifecycleListener exposes three model-lifecycle hooks in addition to the tool hooks (onToolCall/onToolResult):

default void onModelStart(String model, int messageCount, int toolCount) { }
default void onModelEnd(String model, TokenUsage usage, long durationMillis) { }
default void onModelError(String model, Throwable t) { }

Built-in OpenAiCompatibleClient fires these around every model dispatch (including each tool-loop round). AiEventForwardingListener is a built-in adapter that translates the hooks into AiEvent.Progress frames on the streaming session — opt in by attaching it via context.withListeners(...):

var listeners = List.of(new AiEventForwardingListener(session));
runtime.execute(context.withListeners(listeners), session);
// Browser receives wire frames like:
//   {"type":"progress","message":"model:start (gpt-4o, msgs=3, tools=2)"}
//   {"type":"progress","message":"model:end (gpt-4o, in=120, out=85, ms=842)"}

Conversation Memory

Enable multi-turn conversations with one annotation attribute:

@AiEndpoint(path = "/ai/chat",
            systemPrompt = "You are a helpful assistant",
            conversationMemory = true,
            maxHistoryMessages = 20)
public class MyChat {

    @Prompt
    public void onPrompt(String message, StreamingSession session) {
        session.stream(message);
    }
}

When conversationMemory = true, the framework:

1. Captures each user message and the streamed assistant response (via MemoryCapturingSession)
2. Stores them as conversation turns per AtmosphereResource
3. Injects the full history into every subsequent AiRequest
4. Clears the history when the resource disconnects

The default implementation is InMemoryConversationMemory (capped at maxHistoryMessages, default 20). For external storage, implement the AiConversationMemory SPI:

public interface AiConversationMemory {
    List<ChatMessage> getHistory(String conversationId);
    void addMessage(String conversationId, ChatMessage message);
    void clear(String conversationId);
    int maxMessages();
}

@AiTool — Framework-Agnostic Tool Calling

Declare tools with @AiTool and they work with every tool-capable runtime: Built-in, Spring AI, LangChain4j, Google ADK, Embabel, Koog, Semantic Kernel, Anthropic, Cohere, and CrewAI. No framework-specific annotations are needed. AgentScope and Spring AI Alibaba are still AgentRuntime adapters, but their current SDKs do not expose a native tool-dispatch loop for Atmosphere to wrap.

Defining Tools

public class AssistantTools {

    @AiTool(name = "get_weather",
            description = "Returns a weather report for a city")
    public String getWeather(
            @Param(value = "city", description = "City name to get weather for")
            String city) {
        return weatherService.lookup(city);
    }

    @AiTool(name = "convert_temperature",
            description = "Converts between Celsius and Fahrenheit")
    public String convertTemperature(
            @Param(value = "value", description = "Temperature value") double value,
            @Param(value = "from_unit", description = "'C' or 'F'") String fromUnit) {
        return "C".equalsIgnoreCase(fromUnit)
                ? String.format("%.1f°C = %.1f°F", value, value * 9.0 / 5.0 + 32)
                : String.format("%.1f°F = %.1f°C", value, (value - 32) * 5.0 / 9.0);
    }
}

Wiring Tools to an Endpoint

@AiEndpoint(path = "/ai/chat",
            systemPrompt = "You are a helpful assistant",
            conversationMemory = true,
            tools = AssistantTools.class)
public class MyChat {

    @Prompt
    public void onPrompt(String message, StreamingSession session) {
        session.stream(message);  // tools are automatically available to the LLM
    }
}

How It Works

@AiTool methods
    ↓ scan at startup
DefaultToolRegistry (global)
    ↓ selected per-endpoint via tools = {...}
AiRequest.withTools(tools)
    ↓ bridged to backend-native format
LangChain4jToolBridge / SpringAiToolBridge / AdkToolBridge
    ↓ LLM decides to call a tool
ToolExecutor.execute(args) → result fed back to LLM
    ↓
StreamingSession → WebSocket → browser

The tool bridge layer converts @AiTool to the native format at runtime:

Backend	Bridge Class	Native Format
LangChain4j	LangChain4jToolBridge	ToolSpecification
Spring AI	SpringAiToolBridge	ToolCallback
Google ADK	AdkToolBridge	BaseTool

@AiTool vs Native Annotations

	@AiTool (Atmosphere)	@Tool (LangChain4j)	FunctionCallback (Spring AI)
Portable	Any backend	LangChain4j only	Spring AI only
Parameter metadata	@Param annotation	@P annotation	JSON Schema
Registration	ToolRegistry (global)	Per-service	Per-ChatClient

To swap the AI backend, change only the Maven dependency — no tool code changes:

<!-- Use LangChain4j -->
<artifactId>atmosphere-langchain4j</artifactId>

<!-- Or Spring AI -->
<artifactId>atmosphere-spring-ai</artifactId>

<!-- Or Google ADK -->
<artifactId>atmosphere-adk</artifactId>

See the spring-boot-ai-tools sample.

AiInterceptor

Cross-cutting concerns (RAG, guardrails, logging) go through AiInterceptor, not subclassing:

@AiEndpoint(path = "/ai/chat", interceptors = {RagInterceptor.class, LoggingInterceptor.class})
public class MyChat { ... }

public class RagInterceptor implements AiInterceptor {
    @Override
    public AiRequest preProcess(AiRequest request, AtmosphereResource resource) {
        String context = vectorStore.search(request.message());
        return request.withMessage(context + "\n\n" + request.message());
    }
}

Filters, Routing, and Middleware

The AI module includes filters and middleware that sit between the @Prompt method and the LLM:

Class	What it does
PiiRedactionFilter	Buffers messages to sentence boundaries, redacts email/phone/SSN/CC
ContentSafetyFilter	Pluggable SafetyChecker SPI — block, redact, or pass
CostMeteringFilter	Per-session/broadcaster message counting with budget enforcement
RoutingLlmClient	Route by content, model, cost, or latency rules
FanOutStreamingSession	Concurrent N-model streaming: AllResponses, FirstComplete, FastestStreamingTexts
StreamingTextBudgetManager	Per-user/org budgets with graceful degradation
AiResponseCacheInspector	Cache control for AI messages in BroadcasterCache
AiResponseCacheListener	Aggregate per-session events instead of per-message noise

Cost and Latency Routing

RoutingLlmClient supports cost-based and latency-based routing rules:

var router = RoutingLlmClient.builder(defaultClient, "gemini-2.5-flash")
        .route(RoutingRule.costBased(5.0, List.of(
                new ModelOption(openaiClient, "gpt-4o", 0.01, 200, 10),
                new ModelOption(geminiClient, "gemini-flash", 0.001, 50, 5))))
        .route(RoutingRule.latencyBased(100, List.of(
                new ModelOption(ollamaClient, "llama3.2", 0.0, 30, 3),
                new ModelOption(openaiClient, "gpt-4o-mini", 0.005, 80, 7))))
        .build();

Config-driven routing (Spring Boot)

The Spring Boot starter exposes all four RoutingRule families — content, model, cost, and latency — through atmosphere.ai.routing.* properties, with no Java wiring. Off by default. When atmosphere.ai.routing.enabled=true, the starter wraps the framework-resolved LLM client in a RoutingLlmClient and installs it via AiConfig.installClient(...), so it becomes the client every AgentRuntime dispatch reads on the request critical path. When disabled, the resolved client is left untouched and the request path is byte-identical to today’s behavior.

Compose order. Rules are added to the router (and therefore evaluated first-match-wins) in the fixed order content → model → cost → latency — most-specific intent first. Within each family, rules are evaluated in config order. Requests matching no rule fall through to the resolved client and the configured default-model (or the AiConfig model when default-model is omitted). The compose order is pinned by AtmosphereRoutingAutoConfigurationTest.

Property	Type	Default	Description
atmosphere.ai.routing.enabled	boolean	false	Wrap the resolved client in a RoutingLlmClient.
atmosphere.ai.routing.default-model	string	(resolved AiConfig model)	Fallback model when no rule matches.

Content rules (atmosphere.ai.routing.content-rules[i]) match on the latest user message by case-insensitive substring; a rule with no model or no keywords is skipped with a WARN:

Property	Description
…content-rules[i].keywords	Keywords matched case-insensitively against the latest user message.
…content-rules[i].model	Model to route to when a keyword matches.
…content-rules[i].base-url / .api-key	Optional: target a different OpenAI-compatible endpoint for this rule.

Model rules (atmosphere.ai.routing.model-rules[i]) match on the incoming request.model() by literal case-insensitive equals (not regex; the request is routed unchanged — the model name is not rewritten). A blank model-pattern is skipped with a WARN:

Property	Description
…model-rules[i].model-pattern	Routed when request.model() equalsIgnoreCase this value.
…model-rules[i].base-url / .api-key	Optional: dedicated endpoint for the matched model.

Cost rules (atmosphere.ai.routing.cost-rules[i]) pick the highest-capability model whose total cost (cost-per-streaming-text × request.maxStreamingTexts()) is within max-cost. Latency rules (atmosphere.ai.routing.latency-rules[i]) pick the highest-capability model whose average-latency-ms is within max-latency-ms. Each lists candidate models[j] carrying model, cost-per-streaming-text (null → 0.0), average-latency-ms (null → 0), capability (null → 0), and optional per-option base-url / api-key. A cost/latency rule with a null budget or empty models is skipped with a WARN.

# application.yml — all four families on one router. Evaluated content → model
# → cost → latency, first match wins.
atmosphere:
  ai:
    model: gemini-2.5-flash          # resolved default client + model
    routing:
      enabled: true
      default-model: gemini-2.5-flash
      content-rules:
        - keywords: [code, function, refactor, stack trace]
          model: gpt-4o              # reuses the resolved client; only the model changes
          base-url: https://api.openai.com/v1   # optional: dedicated endpoint
          api-key: ${OPENAI_API_KEY}            # optional: key for that endpoint
      model-rules:
        - model-pattern: gpt-4o      # request.model()=="gpt-4o" → dedicated client, unchanged request
          base-url: https://api.openai.com/v1
          api-key: ${OPENAI_API_KEY}
      cost-rules:
        - max-cost: 5.0              # highest-capability model fitting the budget
          models:
            - model: gpt-4o
              cost-per-streaming-text: 0.01
              capability: 10
            - model: gpt-4o-mini
              cost-per-streaming-text: 0.001
              capability: 5
      latency-rules:
        - max-latency-ms: 100        # highest-capability model under 100ms
          models:
            - model: gemini-2.5-flash
              average-latency-ms: 50
              capability: 8

Every rule reuses the resolved client by default (same provider/credentials; only the model name changes where applicable); set base-url and/or api-key to target a different OpenAI-compatible endpoint. For routing logic beyond these property shapes (custom predicates, budget-degradation), build a RoutingLlmClient in Java and install it with AiConfig.installClient(router).

The CLI scaffolds the opt-in block for you: atmosphere new my-app --template ai-chat --routing appends a commented, ready-to-uncomment atmosphere.ai.routing.* tree to the generated application.yml. --routing is only valid for AI templates that ship an application.yml, and the emitted block is commented so the scaffold is byte-identical until you uncomment it.

Confidence Signals

Not in a release yet

Decision-level confidence (DECISION_LOGPROBS), confidence routing, decision models, intent routing and the governance feedback frames below were added after 4.0.71 and ship in the next release. They are on main (4.0.72-SNAPSHOT).

Every completed turn can carry an AiConfidence: an aggregate in [0, 1] (or unknown) and a Source that says how it was derived. Callers weight it by source:

AiConfidence.Source	Where it comes from
LOGPROBS_NATIVE	Mean native token probability over the whole response — the Built-in runtime, when the provider returns logprobs. It measures fluency, not how sure the model was of the answer
DECISION_LOGPROBS	The margin of the emitted value in the model’s distribution over a designated decision field’s allowed values (next section)
MODEL_REPORTED_FIELD	The "confidence": 0.x field the model is asked to emit — the universal fallback on every runtime
HEURISTIC	Caller-computed
PROVIDER_DISTRIBUTION	The distribution an external DecisionModel provider returns, such as the TypeSafe adapter. It appears on decision answers only, never on a streamed turn

Native logprobs are requested on the Built-in runtime only when the request carries an AiConfidenceElicitation, and only for endpoints that accept the field: atmosphere.ai.logprobs (env LLM_LOGPROBS) is auto by default, which defers to a default-deny allow-list (OpenAI, Azure OpenAI, loopback); enabled forces emission and disabled turns it off.

Decision-level confidence (DECISION_LOGPROBS)

A long, fluent answer whose one decisive token was a coin flip still scores about 0.95 on LOGPROBS_NATIVE. When the answer is a decision, score the decision instead — name the field that carries it:

enum Verdict { APPROVE, REJECT, DEFER }
record Triage(Verdict verdict, String reason) { }

pipeline.setDefaultConfidenceElicitation(
        AiConfidenceElicitation.defaults().withDecisionField("verdict"));

The Built-in runtime resolves the field against the response type’s JSON Schema: it must be a top-level property that is a string enum or a boolean (at most 16 allowed values). The client then also requests top_logprobs, locates the tokens of that value in the final round’s JSON, builds the model’s distribution over the allowed values, and emits AiConfidence.fromDecision(...) with source DECISION_LOGPROBS:

• aggregate scores the value the model emitted, DecisionDistribution.marginOf(answer): 0 unless that value is strictly the most likely, otherwise (k * p(answer) - 1) / (k - 1) for k allowed values. A coin flip between true and false scores 0, not 0.5, and a value sampled against a more likely rival also scores 0.
• decision() carries the DecisionDistribution: value → probability in schema order, plus observedMass, the share of probability carried by listed alternatives that match an allowed value. A ConfidenceRouting handler reads it through ConfidenceDecision.confidence().decision().
• tokens() holds the sampled tokens of the decision value up to the one that determined it, not the whole response.

What the provider does not show is resolved against the answer, never for it: probability outside the requested top_logprobs, or on alternatives that could still become several values, goes to the rivals first, so the reported margin is the lowest confidence in the answer consistent with what the provider returned. The distribution is not renormalised.

When the distribution cannot be built — the provider ignored top_logprobs, the field is absent from the output, the value is not an allowed one — the runtime emits no native confidence and the model-reported field applies (unknown routes to ESCALATE by default). It does not fall back to the fluency mean.

Mode scope. Decision confidence exists only where native logprobs do: the Built-in runtime’s chat-completions path, in structured-output mode (strict json_schema or the json_object fallback). A turn with a decision field is kept on that path against api.openai.com too, instead of the Responses API that a conversationId would otherwise select. Every other runtime (LangChain4j, Spring AI, ADK, Embabel, Koog, and the rest) never emits native logprobs, so on those paths withDecisionField has no effect and confidence stays on the model-reported field. The scoring is pinned against hand-authored SSE fixtures whose logprob values were chosen by hand; it has not yet been checked against a captured live-provider stream.

Confidence routing

Emitting a confidence routes nothing. ConfidenceRouting is the consumer: it turns each completed turn’s AiConfidence into a ConfidenceRoute — the answer says what, the confidence says whether to act on it.

pipeline.setDefaultConfidenceRouting(ConfidenceRouting.of(0.9, 0.5)
        .withHandler(decision -> {
            if (decision.route() == ConfidenceRoute.ESCALATE) {
                reviewQueue.submit(decision.request(), decision.confidence());
            }
        }));

Confidence	Route
>= actAt (default 0.9)	ACT
>= confirmAt (default 0.5)	CONFIRM
below confirmAt	ESCALATE
unknown — no field, out of range, nothing emitted	unknownRoute, default ESCALATE

Unknown escalates. A turn nobody measured never acts; withUnknownRoute(...) changes that explicitly.

• Both dispatch paths. @AiEndpoint (per-request ai.confidence.routing metadata) and AiPipeline (setDefaultConfidenceRouting, or the same metadata key) behave identically; the layer is part of the shared dispatch decorator chain.
• Signal. The router sees native logprobs when a runtime reports them and the parsed field otherwise. A routing with no AiConfidenceElicitation installs the default cue, because a turn nobody asked about is always unknown. For a structured decision, designate it with withDecisionField(...) so the route follows how sure the model was of that value rather than the fluency of the whole text, which can route a coin flip to ACT.
• Structured output. The cue is not appended (it would break the single-JSON-object parse). Declare the confidence field on the response record; with a routing in scope the field is read from the raw JSON.
• Delivery. Before the terminal frame the route rides the wire as the ai.confidence.route metadata frame, and the handler receives a ConfidenceDecision(route, confidence, request). The handler runs on the completing thread — hand long work off. A handler that throws is logged and does not fail the turn.
• Not routed: a turn that errored, or that a guardrail already blocked. There is no answer to act on.
• Tool calls are not gated on confidence. The confidence of a tool-call decision is not known when the tool is about to run; tool gating stays with @RequiresApproval / ToolApprovalPolicy.

ConfidenceThresholdGuardrail is the older, narrower primitive: it reads only the text field, passes when the field is missing, and blocks the stream rather than routing the turn.

Decision Models

DecisionModel (org.atmosphere.ai.decision) is a separate SPI for models that decide rather than write: typed Choice / Score / Noul (boolean) questions, one typed answer each, with the same AiConfidence so a ConfidenceRouting gates it unchanged. RuntimeDecisionModel turns any AgentRuntime into one; DecisionModelResolver discovers registered models; an external model such as the TypeSafe adapter plugs in through ServiceLoader. The LLM_CLASSIFIER injection and scope tiers and the LLM moderation detector now ask their questions through it and fail closed on an uncertain answer.

See Decision Models for the SPI, the discovery order, where the confidence comes from, and each consumer’s opt-out.

Intent Routing

IntentRouting (org.atmosphere.ai.intent) decides, before a request reaches the LLM, which handler takes it: a deterministic Java callback, the normal LLM path, or a person. It asks one Question.Choice over the declared routes through a DecisionModel and gates the answer with the same ConfidenceRouting tiers a completed turn is routed on.

It composes with model routing: intent routing picks the handler, and on an LLM route the request continues down the normal dispatch path, where the runtime (a RoutingAiSupport over a DefaultModelRouter, for example) still picks the model.

public class SupportIntents implements IntentRoutingProvider {
    @Override
    public IntentRouting intentRouting() {
        return IntentRouting.of("Which team should handle this customer message?",
                        IntentRoute.handler("track", "where an order or parcel is",
                                decision -> orders.status(decision.message())),
                        IntentRoute.llm("general", "anything else"),
                        IntentRoute.human("agent", "the customer needs a person",
                                decision -> "A person will follow up (ticket "
                                        + tickets.open(decision.request()) + ")"))
                .withThresholds(ConfidenceRouting.of(0.9, 0.5))
                .withHandler(decision -> audit.record(decision));
    }
}

@AiEndpoint(path = "/support", intentRouting = SupportIntents.class)
public class SupportChat {
    @Prompt
    public void onPrompt(String message, StreamingSession session) {
        session.stream(message);   // routed here, before the runtime is called
    }
}

On AiPipeline, call pipeline.setDefaultIntentRouting(routing). Per request, an AiInterceptor (endpoint) or the caller (pipeline) can put a routing under the ai.intent.routing metadata key; it wins over the default and is removed before the request reaches the runtime.

Classification	What runs
choice at >= actAt (default 0.9)	the chosen route
choice at >= confirmAt (default 0.5)	the requester is asked to confirm; approved runs the chosen route, denied or timed out (confirmTimeout, default 2 min, at most 1 h) runs the human route, and so does a confirmation that cannot be sent
choice below confirmAt	the human route
the model chose the human route	the human route, at any confidence, without a confirmation
no decision model, Answer.Failed, a message over 262,144 characters, a model that throws	the human route
a valid choice with no confidence at all	ConfidenceRouting.unknownRoute(), ESCALATE by default

• Routes. 2..16 routes with unique names ([A-Za-z0-9_-]{1,64}), exactly one IntentRoute.Human.
• Where the confidence comes from. Only the Built-in runtime against an endpoint that returns logprobs scores the choice from the model’s distribution (DECISION_LOGPROBS). On every other runtime RuntimeDecisionModel uses the confidence the model reports, and the ACT and CONFIRM tiers gate on that number: a model that reports 0.95 acts, and its deterministic handler runs without a confirmation. The source is in IntentDecision.reason(). For a stricter posture there, raise the tiers with withThresholds(...) or route through a DecisionModel whose confidence you trust (withDecisionModel(...)).
• Where it runs. After admission on both paths — request guardrails, governance policies, and on the endpoint the AiInterceptor pre-processing and per-request scope — so a denied request is never classified. RAG runs after routing, so a handler or human route pays no retrieval call. A request a scope policy rewrote to its redirect text is not classified; it continues to the LLM path, which renders the redirect.
• Handlers. An IntentHandler returns the reply text; the framework sends it and completes the turn, or errors the turn when the handler throws (there is no fallback to the LLM). The human route’s handler hands the request to your queue and returns the acknowledgement. Handler turns are recorded in conversation memory; cost, budget, response guardrails, structured output, confidence and the model metrics do not apply because no model is called.
• Confirmation. A CONFIRM-tier choice uses the approval machinery tools use: an approval-required frame with tool name intent:<route>, answered by /__approval/<id>/approve or /deny on the same session. On @AiEndpoint the wait is cut short to end before the endpoint’s timeout(). The OpenAI-compatible and batch surfaces have no channel to answer on, so a CONFIRM-tier choice there escalates to the human route at once.
• Cancellation. A cancelled turn is never routed: it errors with a CancellationException and runs no handler or human route.
• Wire signal. Before any reply: ai.intent.route, ai.intent.tier (ACT / CONFIRM / ESCALATE), ai.intent.choice (absent when the model gave no answer) and ai.intent.confidence (absent when unknown) as metadata frames. The OpenAI-compatible surface sends them as X-Atmosphere-Intent-Route, X-Atmosphere-Intent-Tier, X-Atmosphere-Intent-Choice and X-Atmosphere-Intent-Confidence response headers. A batch item result does not carry them.
• Bounds. One question per request, bounded by timeout (default 5 s, at most 10 min) and the decision model’s concurrency limit. Without withDecisionModel(...) the routing resolves through DecisionModelResolver and shares the fallback’s 8 slots with the LLM safety tiers; for a high-traffic endpoint, give it its own RuntimeDecisionModel.

The tiers have been driven end-to-end over WebSocket against a scripted runtime that reports a decision distribution, on both @AiEndpoint and AiPipeline; they have not yet been exercised against a live provider’s logprobs.

Governance Feedback Signal

GovernanceFeedbackInterceptor re-injects recent deny / prefer governance decisions into the next turn’s system prompt (see Governance as a learning signal). That injection is otherwise invisible to the client, so a turn that carried guidance reports it with two metadata frames just before its complete frame:

Metadata key	Value
ai.governance.feedback.injected	how many guidance lines were injected (at most maxItems, default 5)
ai.governance.feedback.lines	those lines, each capped at 512 characters

• Only what reached the model. Only lines the final system prompt still carries are reported, so an interceptor later in the chain that replaces the system prompt suppresses the signal along with the guidance. Nothing is sent when nothing was injected, nor on the error() terminal path.
• Never to other subscribers. The lines are the prompter’s own governance history, so they are sent only when the @AiEndpoint handler confirmed the reply goes to the prompter alone. On an @AiEndpoint(broadcastReply = true) room the guidance is still injected, but neither frame is sent.
• @AiEndpoint only. AiInterceptors run on the @AiEndpoint dispatch path; the resource-free AiPipeline runs none, so there the guidance is neither injected nor signalled.

The Atmosphere Console renders the frames under the turn as the Governance guidance applied panel. That is what an e2e can assert; whether the model then follows the guidance is model behaviour, not a framework guarantee.

Direct Adapter Usage

You can bypass @AiEndpoint and use adapters directly:

Spring AI:

var session = StreamingSessions.start(resource);
springAiAdapter.stream(chatClient, prompt, session);

LangChain4j:

var session = StreamingSessions.start(resource);
model.chat(ChatMessage.userMessage(prompt),
    new AtmosphereStreamingResponseHandler(session));

Google ADK:

var session = StreamingSessions.start(resource);
adkAdapter.stream(new AdkRequest(runner, userId, sessionId, prompt), session);

Embabel:

val session = StreamingSessions.start(resource)
embabelAdapter.stream(AgentRequest("assistant") { channel ->
    agentPlatform.run(prompt, channel)
}, session)

Browser — React

import { useStreaming } from 'atmosphere.js/react';

function AiChat() {
  const { fullText, isStreaming, stats, routing, send } = useStreaming({
    request: { url: '/ai/chat', transport: 'websocket' },
  });

  return (
    <div>
      <button onClick={() => send('Explain WebSockets')} disabled={isStreaming}>
        Ask
      </button>
      <p>{fullText}</p>
      {stats && <small>{stats.totalStreamingTexts} streaming texts</small>}
      {routing.model && <small>Model: {routing.model}</small>}
    </div>
  );
}

AI in Rooms — Virtual Members

var client = AiConfig.get().client();
var assistant = new LlmRoomMember("assistant", client, "gpt-5",
    "You are a helpful coding assistant");

Room room = rooms.room("dev-chat");
room.joinVirtual(assistant);
// Now when any user sends a message, the LLM responds in the same room

StreamingSession Wire Protocol

The client receives JSON messages over WebSocket/SSE:

• {"type":"streaming-text","content":"Hello"} — a single streaming text
• {"type":"progress","message":"Thinking..."} — status update
• {"type":"complete"} — stream finished
• {"type":"error","message":"..."} — stream failed

Configuration

Configure the built-in client with environment variables:

Variable	Description	Default
LLM_MODE	remote (cloud) or local (Ollama)	remote
LLM_MODEL	gemini-2.5-flash, gpt-5, o3-mini, llama3.2, …	gemini-2.5-flash
LLM_API_KEY	API key (or GEMINI_API_KEY for Gemini)	—
LLM_BASE_URL	Override endpoint (auto-detected from model name)	auto

Key Components

Class	Description
@AiEndpoint	Marks a class as an AI chat endpoint with a path, system prompt, and interceptors
@Prompt	Marks the method that handles user messages
@AiTool	Marks a method as an AI-callable tool (framework-agnostic)
@Param	Describes a tool parameter’s name, description, and required flag
AgentRuntime	SPI for AI framework backends (ServiceLoader-discovered)
AiRequest	Framework-agnostic request record (message, systemPrompt, model, userId, sessionId, agentId, conversationId, metadata)
AiEvent	Sealed interface: 15 structured event types (TextDelta, ToolStart, ToolResult, AgentStep, EntityStart, Handoff, ApprovalRequired, etc.)
AiCapability	Enum for endpoint capability requirements (TEXT_STREAMING, TOOL_CALLING, STRUCTURED_OUTPUT, etc.)
AiInterceptor	Pre/post processing hooks for RAG, guardrails, logging
AiConversationMemory	SPI for conversation history storage
MemoryStrategy	Pluggable memory selection: MessageWindowStrategy, TokenWindowStrategy, SummarizingStrategy
StructuredOutputParser	SPI for JSON Schema generation and typed output parsing (built-in: JacksonStructuredOutputParser)
StreamingSession	Delivers streaming texts, events, progress updates, and metadata to the client
StreamingSessions	Factory for creating StreamingSession instances
OpenAiCompatibleClient	Built-in HTTP client for OpenAI-compatible APIs
RoutingLlmClient	Routes prompts to different LLM backends based on rules
ToolRegistry	Global registry for @AiTool definitions
ModelRouter	SPI for intelligent model routing and failover
AiGuardrail	SPI for pre/post-LLM safety inspection
AiMetrics	SPI for AI observability (streaming texts, latency, cost)
ConversationPersistence	SPI for durable conversation storage (Redis, SQLite)
RetryPolicy	Exponential backoff with circuit-breaker semantics

Approval Gates

@RequiresApproval pauses tool execution until the client approves. The virtual thread parks cheaply on a CompletableFuture — no carrier thread consumed.

@AiTool(name = "delete_account", description = "Permanently delete a user account")
@RequiresApproval("This will permanently delete the account. Are you sure?")
public String deleteAccount(@Param("accountId") String accountId) {
    return accountService.delete(accountId);
}

Wire Protocol

When the LLM calls a @RequiresApproval tool, the client receives an approval-required event:

{"event":"approval-required","data":{
  "approvalId":"apr_a1b2c3d4e5f6",
  "toolName":"delete_account",
  "arguments":{"accountId":"user-42"},
  "message":"This will permanently delete the account. Are you sure?",
  "expiresIn":300
}}

The client responds with:

• /__approval/apr_a1b2c3d4e5f6/approve — tool executes
• /__approval/apr_a1b2c3d4e5f6/deny — tool returns cancelled

Default timeout: 5 minutes. Configurable via @RequiresApproval(timeoutSeconds = 120).

How It Works

1. AiStreamingSession.wrapApprovalGates() wraps @RequiresApproval tools with ApprovalGateExecutor
2. When the LLM calls the tool, ApprovalGateExecutor parks the virtual thread on CompletableFuture.get(timeout)
3. The session emits AiEvent.ApprovalRequired to the client
4. AiEndpointHandler fast-paths /__approval/ messages to the session’s ApprovalRegistry (before prompt dispatch)
5. ApprovalRegistry.tryResolve() completes the future, unparking the virtual thread
6. On transport reconnect, a fallback scan across all active sessions ensures the approval reaches the parked thread

ADK ToolConfirmation Bridge

When running on Google ADK, Atmosphere also calls toolContext.requestConfirmation() to give ADK native visibility into the approval pause. If ADK resolves a confirmation before Atmosphere (e.g., via its own UI), the ADK denial short-circuits without calling the executor. This creates a two-layer model: Atmosphere-level (cross-runtime) + ADK-native (runtime-specific).

All twelve runtimes with TOOL_CALLING also declare AiCapability.TOOL_APPROVAL — AgentScope bridges through AgentScopeToolBridge and Spring AI Alibaba through SpringAiAlibabaToolBridge, both routing every invocation through ToolExecutionHelper.executeWithApproval. See the per-runtime capability matrix.

Context Compaction SPI

The AiCompactionStrategy SPI controls how conversation history is compacted when it exceeds the configured limit. Unlike MemoryStrategy (which selects messages for the next request — read path), compaction permanently reduces stored history (write path).

public interface AiCompactionStrategy {
    List<ChatMessage> compact(List<ChatMessage> messages, int maxMessages);
    String name();
}

Built-in Strategies

SlidingWindowCompaction (default) — drops the oldest non-system messages until under the limit. System messages are always preserved.

SummarizingCompaction — condenses old messages into a single system-role summary, preserving the most recent messages verbatim. The recent window size is configurable (default: 6).

// Default: sliding window
var memory = new InMemoryConversationMemory(20);

// Custom: summarization with 8-message recent window
var memory = new InMemoryConversationMemory(20, new SummarizingCompaction(8));

ADK Bridge

AdkCompactionBridge.toAdkConfig() maps Atmosphere compaction settings to ADK’s EventsCompactionConfig for native compaction when using the ADK runtime.

Artifact Persistence SPI

The ArtifactStore SPI provides binary artifact persistence across agent runs. Use cases include agent-generated reports, images, code files, and content shared between coordinated agents.

public interface ArtifactStore {
    Artifact save(Artifact artifact);                              // auto-versions
    Optional<Artifact> load(String namespace, String artifactId);  // latest version
    Optional<Artifact> load(String namespace, String artifactId, int version);
    List<Artifact> list(String namespace);                         // latest of each
    boolean delete(String namespace, String artifactId);           // all versions
    void deleteAll(String namespace);
}

Artifact Record

public record Artifact(
    String id,                    // unique identifier
    String namespace,             // grouping key (session ID, agent name, user ID)
    String fileName,              // human-readable name ("report.pdf")
    String mimeType,              // MIME type ("application/pdf")
    byte[] data,                  // binary content (defensively copied)
    int version,                  // auto-incremented per save
    Map<String, String> metadata, // arbitrary key-value pairs
    Instant createdAt
) { }

Byte arrays are defensively copied on construction and on access — callers cannot mutate persisted data.

Implementations

• InMemoryArtifactStore — default, for development and testing. Data does not survive JVM restart.
• ADK bridge — AdkArtifactBridge.toAdkService() wraps an ArtifactStore as ADK’s BaseArtifactService.

Interceptor Disconnect Lifecycle

AiInterceptor includes an onDisconnect hook called before conversation memory is cleared. This enables fact extraction, summary persistence, and other cleanup that requires access to the conversation history.

public interface AiInterceptor {
    default AiRequest preProcess(AiRequest request, AtmosphereResource resource) { return request; }
    default void postProcess(AiRequest request, AtmosphereResource resource) { }
    default void onDisconnect(String userId, String conversationId, List<ChatMessage> history) { }
}

LongTermMemoryInterceptor.onDisconnect() uses this to extract facts from the full conversation on session close via OnSessionCloseStrategy.

Execution order: preProcess runs FIFO, postProcess runs LIFO, onDisconnect runs FIFO. Exceptions in one interceptor do not prevent others from being called.

AiEvent Model

The AiEvent sealed interface provides 15 structured event types emitted via session.emit(). All runtimes map their native events to this common model.

Event	Description
TextDelta	Streaming token
TextComplete	Final assembled text
ToolStart	Tool invocation begins (name + arguments)
ToolResult	Tool executed successfully (name + result)
ToolError	Tool execution failed
AgentStep	Orchestration step (ADK agent steps, Embabel planning)
StructuredField	Structured output field arrival
EntityStart / EntityComplete	Structured entity streaming
RoutingDecision	Backend routing event
Progress	Long-running operation status
Handoff	Agent handoff notification
ApprovalRequired	Human approval gate
Error	Error with recovery hint
Complete	Stream completed with usage metadata

Runtime Event Normalization

Source	Atmosphere Event
ADK event.functionCalls()	AiEvent.ToolStart
ADK event.functionResponses()	AiEvent.ToolResult
ADK event.author() (non-partial)	AiEvent.AgentStep
ADK event.usageMetadata()	ai.tokens.input/output/total metadata
Koog onToolCallStarting	AiEvent.ToolStart
Koog onToolCallCompleted	AiEvent.ToolResult
Koog onToolCallFailed	AiEvent.ToolError
Koog StreamFrame.ReasoningDelta	AiEvent.Progress
Embabel MessageOutputChannelEvent	AiEvent.TextDelta
Embabel ProgressOutputChannelEvent	AiEvent.AgentStep

Capability Matrix

Each runtime declares capabilities via AiCapability. The framework uses these flags for model routing, tool negotiation, and feature discovery. The table below mirrors the twelve-runtime snapshot pinned by each runtime’s expectedCapabilities() contract test; Y means the runtime declares the capability and the contract tests assert it.

Legend: TS=TEXT_STREAMING, TC=TOOL_CALLING, SO=STRUCTURED_OUTPUT, NSO=NATIVE_STRUCTURED_OUTPUT, SP=SYSTEM_PROMPT, AO=AGENT_ORCHESTRATION, CM=CONVERSATION_MEMORY, TA=TOOL_APPROVAL, V=VISION, A=AUDIO, MM=MULTI_MODAL, PC=PROMPT_CACHING, TU=TOKEN_USAGE, PRR=PER_REQUEST_RETRY, TCD=TOOL_CALL_DELTA, BE=BUDGET_ENFORCEMENT, CS=CONFIDENCE_SCORES, PSV=PASSIVATION.

SO is pipeline-level structured output (schema injected into the prompt + parsed by StructuredOutputCapturingSession), declared by every runtime. NSO is provider-enforced structured output — the generated JSON Schema is threaded into the provider’s own structured-output field (OpenAI response_format:json_schema, Anthropic output_config, Cohere response_format, Gemini responseSchema, etc.) so non-conforming output cannot be emitted. Governed by NativeStructuredOutputMode (AUTO default, with graceful fall-back to the prompt-injection path when a provider rejects the schema). The three runtimes without NSO carry no schema field on their wire/SDK path and stay on SO — declaring NSO for them would violate Runtime Truth.

Runtime	Priority	TS	TC	SO	NSO	SP	AO	CM	TA	V	A	MM	PC	TU	PRR	TCD	BE	CS	PSV
Built-in	0	Y	Y	Y	Y	Y		Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
Spring AI	100	Y	Y	Y	Y	Y		Y	Y	Y	Y	Y	Y	Y	Y		Y	Y	Y
LangChain4j	100	Y	Y	Y	Y	Y		Y	Y	Y	Y	Y	Y	Y	Y		Y	Y	Y
Google ADK	100	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y		Y	Y	Y
Embabel	100	Y	Y	Y		Y	Y	Y	Y	Y		Y		Y	Y		Y	Y	Y
JetBrains Koog	100	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y		Y	Y	Y
Alibaba AgentScope	100	Y	Y	Y	Y	Y		Y	Y					Y	Y		Y	Y	Y
Spring AI Alibaba	100	Y¹	Y	Y		Y		Y	Y					Y	Y		Y	Y	Y
Microsoft Semantic Kernel	100	Y	Y	Y	Y	Y		Y	Y					Y	Y		Y	Y	Y
Anthropic	100	Y	Y	Y	Y	Y		Y	Y	Y		Y		Y	Y		Y	Y	Y
Cohere	100	Y	Y	Y	Y	Y		Y	Y	Y		Y		Y	Y		Y	Y	Y
CrewAI²	50	Y	Y	Y		Y	Y		Y					Y	Y

¹ Spring AI Alibaba emits its final reply as one Atmosphere stream chunk, but the upstream ReactAgent.call() path is buffered rather than token-by-token.

² CrewAI is the only out-of-process runtime: the Java side talks HTTP + SSE to a Python sidecar (atmosphere-crewai-bridge, FastAPI + crewai 1.14). isAvailable() is config-gated on ATMOSPHERE_CREWAI_SIDECAR_URL pointing at a running sidecar whose /health responds OK — Runtime Truth gate, no classpath-only advertisement. The runtime does not own a Java-side conversation-memory store; per-task memory lives inside the sidecar’s crew rather than being declared at the Atmosphere layer.

How structured output works: AiPipeline wraps the streaming session with StructuredOutputCapturingSession and augments the system prompt with JSON-schema instructions before the runtime runs. Any runtime that honors SYSTEM_PROMPT therefore gets STRUCTURED_OUTPUT automatically via the pipeline — no per-runtime adapter code required. BuiltInAgentRuntime additionally enables native jsonMode on the OpenAI-compatible client for provider-level JSON enforcement on top of the pipeline wrap. Source: modules/ai/src/main/java/org/atmosphere/ai/pipeline/AiPipeline.java:128-135, modules/ai/src/main/java/org/atmosphere/ai/llm/BuiltInAgentRuntime.java:72-74.

Tool-dispatch bridges: every runtime that declares TOOL_CALLING routes every Atmosphere @AiTool invocation through a runtime-native bridge that calls ToolExecutionHelper.executeWithApproval, so @RequiresApproval gates fire uniformly. AgentScope ships AgentScopeToolBridge, Spring AI Alibaba ships SpringAiAlibabaToolBridge, Semantic Kernel ships SemanticKernelToolBridge, Embabel ships EmbabelToolBridge, Koog ships AtmosphereToolBridge, and the JDK runtimes (Built-in, Spring AI, LangChain4j, ADK) wire their tool callbacks through the shared helper directly.

Spring AI Alibaba token usage: ReactAgent.call() returns an AssistantMessage without usage metadata, so Atmosphere wraps the configured Spring AI ChatModel bean in a UsageCapturingChatModel decorator at auto-configuration time. Every underlying ChatModel.call(Prompt) performed by the ReAct graph during a single dispatch accumulates ChatResponseMetadata.getUsage() into a per-thread collector; the runtime emits one typed TokenUsage record via session.usage(...) after the agent returns. Token-based AiBudget breaches therefore trip uniformly alongside wall-clock breaches. Custom ReactAgent beans that bypass the auto-config also bypass the wrapper — see AtmosphereSpringAiAlibabaAutoConfiguration for the wrapping point.

Cross-Runtime Contract Tests (TCK)

The AbstractAgentRuntimeContractTest base class in atmosphere-ai-test enforces a minimum contract across all runtime adapters.

public abstract class AbstractAgentRuntimeContractTest {
    protected abstract AgentRuntime createRuntime();
    protected abstract AgentExecutionContext createTextContext();
    protected abstract AgentExecutionContext createToolCallContext();
    protected abstract AgentExecutionContext createErrorContext();

    // Enforced contracts:
    // - runtimeDeclaresMinimumCapabilities (TEXT_STREAMING)
    // - runtimeHasNonBlankName
    // - runtimeIsAvailable
    // - textStreamingCompletesSession (10s timeout)
    // - toolCallExecutesIfSupported
    // - errorContextTriggersSessionError
}

Add atmosphere-ai-test as a test dependency and extend the base class:

<dependency>
    <groupId>org.atmosphere</groupId>
    <artifactId>atmosphere-ai-test</artifactId>
    <scope>test</scope>
</dependency>

The RecordingSession test double captures all events, text chunks, metadata, and errors for assertion. The contract suite is implemented for all twelve runtime adapters.

Samples

• Spring Boot AI Chat — built-in client with Gemini/OpenAI/Ollama
• Spring Boot AI Tools — framework-agnostic @AiTool pipeline
• Spring Boot AI Classroom — rooms-based multi-room AI with an Expo client
• Spring Boot RAG Chat — Spring AI VectorStore-backed RAG agent
• Quarkus AI Chat — five @AiEndpoint demos on Quarkus + LangChain4j bridge
• Spring Boot Dentist Agent — @Agent with tools, memory, and approval gates

See Also

• Spring AI Adapter
• LangChain4j Adapter
• Google ADK Adapter
• Embabel Adapter
• Decision Models — DecisionModel SPI, the LLM safety tiers, intent routing
• TypeSafe Adapter — an external DecisionModel
• MCP Server — AI-MCP bridge for tool-driven streaming
• Rooms & Presence — AI virtual members in rooms
• atmosphere.js — useStreaming React/Vue/Svelte hooks
• Module README