attemptApiRequest: streaming LLM calls and retries

源码版本v4.0.10

Responsibilities

attemptApiRequest is the generator method in Task that handles "talking to the LLM provider". It is an async * function that yields stream chunks one by one for the outer recursivelyMakeClineRequests to consume. Before yielding, it does a large amount of preparation: wait for MCP servers to connect, read rule files, assemble the system prompt, and run context truncation. When the first chunk fails, it decides whether to auto-retry, ask the user, or give up.

It sits in the middle layer of the agent loop. recursivelyMakeClineRequests calls attemptApiRequest once per recursion, hands the stream off to the outer while loop to consume chunks, parse them, and call presentAssistantMessage. So this method does not consume the stream itself — it is only responsible for "preparing the stream and handing it off safely".

Its return type is ApiStream, essentially an async iterator. The outer layer first calls iterator.next() to probe the first chunk (first chunk probe:2389). If the first chunk succeeds, it yield* iterators the rest; if it fails, the error is classified and retry logic kicks in. This "probe with the first chunk" design separates "pre-stream failure" from "mid-stream failure", because the former has clean state and can be retried with no side effects, while the latter may have already executed some tools and cannot be retried trivially.

Design motivation

• MCP wait has a cap: It uses pWaitFor to wait for mcpHub.isConnecting to go false; on a 10-second timeout it logs an error and continues (mcp wait:2177). MCP is not allowed to stall the whole request.
• SystemPromptContext as one bucket: cwd, IDE, provider info, rule files, clineignore, skills, editor tabs, parallel tool calling — all of it goes into one context object (promptContext:2300). getSystemPrompt consumes it in one shot, so prompt-assembly logic does not get scattered.
• Context truncation before the call: contextManager.getNewContextMessagesAndMetadata trims the conversation history to fit the model's context window (context truncate:2354). If the trim updates the deletion range, it immediately writes back to clineMessages for persistence.
• First-chunk probe strategy: An isWaitingForFirstChunk flag plus a dedicated try/catch around iterator.next() — when the first chunk fails, state is still clean and a retry is safe (first chunk try:2388).
• Error classification drives retries: auth, spend limit, quota, entitlement, ClinePass cap, insufficient credits — errors where retrying is pointless — skip the auto-retry path (shouldRetry gate:2495); other errors auto-retry up to 3 times with 2s/4s/8s exponential backoff (backoff:2509).

Key files

• attemptApiRequest:2175 — the method body, from MCP wait all the way to yield*.
• pWaitFor mcpHub:2177 — waits for MCP servers to connect, 10-second timeout.
• SystemPromptContext:2300 — collects all prompt inputs into one object, ready for getSystemPrompt.
• getSystemPrompt:2350 — produces the final systemPrompt and tools array.
• getNewContextMessagesAndMetadata:2354 — the context manager trims conversation history; returns the truncated messages and metadata.
• deleted range update:2366 — when the trim updates the deletion range, it immediately persists to clineMessages.
• createMessage:2378 — uses the api handler to create the stream, passing systemPrompt, truncated history, and tools.
• first chunk probe:2388 — a dedicated try/catch pulls the first chunk; on failure, error classification kicks in.
• context window check:2393 — checkContextWindowExceededError decides whether the context is too long.
• handleContextWindowExceededError:2409 — auto-truncates and retries once on context-too-long.
• shouldRetry gate:2495 — excludes error types where retry is pointless.
• exponential backoff:2509 — 2s/4s/8s three auto-retries with increasing delay.
• yield* recurse:2620 — on retry success, calls itself recursively and forwards chunks from the new stream to the outer layer.

Data flow

The main path from preparation to yield is "MCP wait → assemble prompt → truncate history → create stream → probe → hand off". The history-truncation step is critical — it decides what the model actually sees:

// apps/vscode/src/core/task/index.ts
const contextManagementMetadata =
    await this.contextManager.getNewContextMessagesAndMetadata(
        this.messageStateHandler.getApiConversationHistory(),
        this.messageStateHandler.getClineMessages(),
        this.api,
        this.taskState.conversationHistoryDeletedRange,
        previousApiReqIndex,
        await ensureTaskDirectoryExists(this.taskId),
        this.stateManager.getGlobalSettingsKey("useAutoCondense") &&
            isNextGenModelFamily(this.api.getModel().id),
    );

if (contextManagementMetadata.updatedConversationHistoryDeletedRange) {
    this.taskState.conversationHistoryDeletedRange =
        contextManagementMetadata.conversationHistoryDeletedRange;
    await this.messageStateHandler.saveClineMessagesAndUpdateHistory();
    // saves task history item which we use to keep track of conversation history deleted range
}

ContextManager takes the full conversation history + the last recorded deletion range + the token usage of the previous request, and computes the new truncation range. If the range changed, clineMessages is flushed to disk immediately — because the next request depends on this new range. useAutoCondense is the auto-compression switch for new (next-gen) models; when enabled, it replaces early messages with summaries generated by the model itself (autocondense flag:2362).

Only after truncation finishes does it create the stream and probe:

// apps/vscode/src/core/task/index.ts
const stream = this.api.createMessage(
    systemPrompt,
    truncatedConversationHistory,
    tools,
);

const iterator = stream[Symbol.asyncIterator]();

try {
    // awaiting first chunk to see if it will throw an error
    this.taskState.isWaitingForFirstChunk = true;
    const firstChunk = await iterator.next();
    yield firstChunk.value;
    this.taskState.isWaitingForFirstChunk = false;
} catch (error) {
    // ... error classification, auto-retry, or ask the user
    yield* this.attemptApiRequest(previousApiReqIndex);
    return;
}

The isWaitingForFirstChunk flag tells the outer layer "I'm waiting for the first chunk; don't treat me as mid-stream". The catch branch for first-chunk failure dispatches by error type: if the context window is exceeded and we have not auto-retried yet, call handleContextWindowExceededError to auto-truncate and retry; for other errors, shouldRetry decides between auto backoff retry or ask("api_req_failed") to hand the decision to the user.

Boundaries and failure modes

• MCP timeout is not fatal: If MCP cannot connect within 10 seconds, it only logs an error and continues (mcp timeout catch:2179). MCP tools may then be missing from the system prompt, but the request does not fail because of it.
• Context-too-long auto-retries only once: The didAutomaticallyRetryFailedApiRequest flag guarantees handleContextWindowExceededError runs at most once (auto retry flag:2407). The second time, it goes through ask("api_req_failed") and hands the decision to the user.
• Conversation already short but still too long: If the truncated message count is ≤ 3, it tells the user "context window exceeded, retry to truncate" but no longer auto-truncates (conversation bricked:2423). The conversation is essentially bricked at this point and needs user intervention.
• api_req_started status update: Each retry finds the last api_req_started message and updates its streamingFailedMessage and retryStatus (update api_req_started:2433). The UI uses this field to show "retrying" / "retries exhausted".
• error_retry display dedup: On auto-retry, it first say("error_retry", ...)s the full message, then removes streamingFailedMessage from api_req_started to prevent the same error from showing in both ErrorRow and error_retry (dedupe error display:2548).
• Retry counter reset manually: When the user clicks yes to retry, autoRetryAttempts is zeroed (reset counter:2586), leaving three more auto-retry slots for subsequent failures.
• Failures after the first chunk are not handled here: Mid-stream failures after the first chunk succeeds are caught by recursivelyMakeClineRequests's outer try/catch (stream-mid failure:3935). This method only handles "before the stream starts" failures.

Summary

attemptApiRequest is three things in one: prepare, probe, and retry. The preparation phase runs rules, skills, MCP, and context truncation; the probe phase uses the first chunk to decide whether to retry; the retry phase uses three layers — error classification, exponential backoff, and a user fallback — to handle failures. Once the stream starts, it is yield*ed out and no longer this method's concern. To see how the stream is consumed and sliced into blocks once it leaves, go to /agent-loop/present-assistant-message; to see the outer recursive driver, go to /agent-loop/task-class.

See official docs: Cline docs · README