BrowserToolHandler: Puppeteer browser automation

源码版本v4.0.10

Responsibilities

BrowserToolHandler is Cline's entry point for operating the browser, corresponding to the ClineDefaultTool.BROWSER tool name. The model picks a specific action via the action parameter: launch (open a URL), click (click by coordinates), type (type text into the focused element), scroll_up/scroll_down (scroll the page), close (close the browser) (class declaration:12-13). After every action it returns a screenshot plus the console logs from that window, so the model can understand page state visually.

Itself it is only a "facade": on receiving a tool_use it does parameter validation, approval, and hooks, then delegates the action to config.services.browserSession, a BrowserSession instance. The actual browser process, Puppeteer API, screenshot encoding, and remote browser connection all live in BrowserSession (class BrowserSession:38). BrowserSession uses puppeteer-core to connect to a local Chrome or a remote browser, obtaining Browser/Page objects via connect/launch.

Within Cline, this tool sits in the "capability tool" category alongside file ops and web fetch — all are one-shot execution units launched by ToolExecutor. What makes it special is that it is stateful: within a Task there is only one browser session; after launch, all subsequent click/type/scroll calls reuse the same Page until close or an error. That is also why the handler calls closeBrowser() on every error branch — a browser process must not be left lingering.

Design motivation

• Single session reuse: after launch the page is reused until close, avoiding reopening on every click and letting the model preserve login state and scroll position across multiple actions (launchBrowser:155).
• Screenshot + console log dual channel: after each action, doAction waits for the console to go quiet for 500ms before taking a screenshot, returning both visual and log data to the model to compensate for the fact that pure vision cannot capture JS errors (doAction:378).
• Separate approval for launch: opening a browser is a high-risk action, so launch goes through a dedicated ask type browser_action_launch; other actions (click/type/scroll) are auto-approved by default once launch has been approved (launch flow:81).
• Remote browser fallback: when the remote browser connection fails it degrades to local mode rather than crashing the whole tool (remote fallback:181-198).
• Wait for navigation after click: a click may trigger a page navigation, so doAction listens for request events on click to detect network activity and calls waitForNavigation accordingly (click:528-561).
• Must close browser before switching tools: the returned result appends a "REMEMBER ... you must close the browser before using non-browser_action tools" reminder so the model does not switch to editing files while the browser is open (result reminder:190-194).

Key files

• class declaration:12 — export class BrowserToolHandler implements IFullyManagedTool, name = ClineDefaultTool.BROWSER.
• handlePartialBlock:19 — streaming branch: launch goes through the browser_action_launch ask, other actions just say the current coordinate/text.
• execute:62 — main flow: validate action → launch branch or click/type/scroll/close branch → return screenshot + logs.
• launch flow:81-127 — full launch flow: validate url, approval, PreToolUse hook, applyLatestBrowserSettings to refresh the session, launchBrowser + navigateToUrl.
• click/type param check:130-145 — when click is missing coordinate or type is missing text, each reports an error and calls closeBrowser.
• action dispatch:163-179 — switch dispatching to browserSession.click/type/scrollDown/scrollUp/closeBrowser.
• result format:184-203 — launch/click/type/scroll return screenshot + logs, close returns a plain-text message.
• error close:205-208 — any exception triggers closeBrowser, no lingering browser process.
• launchBrowser:155 — remote first, falls back to local on failure, then browser.newPage() opens a new tab.
• doAction:378 — attaches console/pageerror listeners, runs the action, waits 500ms of silence, takes a screenshot, cleans up listeners.
• navigateToUrl:481 — page.goto uses domcontentloaded + networkidle2, then waitTillHTMLStable polls HTML size.
• click:528 — splits coordinates, page.mouse.click, listens for request to decide whether to wait for navigation.
• screenshot:441 — webp first, falls back to png on failure, throws and emits telemetry if both fail.

Data flow

The launch branch is BrowserToolHandler's most complete flow: validate url → approval → hook → refresh browserSession → open browser → navigate → return screenshot and logs. Key code:

// apps/vscode/src/core/task/tools/handlers/BrowserToolHandler.ts
if (action === "launch") {
    if (!url) {
        config.taskState.consecutiveMistakeCount++
        const errorResult = await config.callbacks.sayAndCreateMissingParamError(this.name, "url")
        await config.services.browserSession.closeBrowser()
        return errorResult
    }
    config.taskState.consecutiveMistakeCount = 0
    // ... approval + PreToolUse hook ...
    await config.callbacks.say("browser_action_result", "")
    config.services.browserSession = await config.callbacks.applyLatestBrowserSettings()
    await config.services.browserSession.launchBrowser()
    browserActionResult = await config.services.browserSession.navigateToUrl(url)
}

Note the line config.services.browserSession = await config.callbacks.applyLatestBrowserSettings(): it does not just refresh settings, it also replaces the browserSession reference held by ToolExecutor, so subsequent browser tool calls pick up the same new session (applyLatestBrowserSettings:125).

navigateToUrl internally goes through doAction, and every non-launch action also goes through doAction. doAction packs the screenshot + console logs into a BrowserActionResult:

// apps/vscode/src/services/browser/BrowserSession.ts
const screenshotType = this.useWebp ? "webp" : "png"
let screenshotBase64 = await this.page.screenshot({ ...options, type: screenshotType })
let screenshot = `data:image/${screenshotType};base64,${screenshotBase64}`
if (!screenshotBase64) {
    screenshotBase64 = await this.page.screenshot({ ...options, type: "png" })
    screenshot = `data:image/png;base64,${screenshotBase64}`
}
return {
    screenshot,
    logs: logs.join("\n"),
    currentUrl: this.page.url(),
    currentMousePosition: this.currentMousePosition,
}

The handler wraps this result as formatResponse.toolResult and appends the strong reminder "close the browser before switching to other tools".

Boundaries and failure modes

• Invalid action: when the block completes and the action is not in the browserActions list, consecutiveMistakeCount++ + sayAndCreateMissingParamError + closeBrowser (invalid action:69-75).
• launch missing url: launch must carry a url; if missing, same mistake++ and closeBrowser (missing url:82-87).
• click missing coordinate / type missing text: missing the corresponding parameter reports an error and calls closeBrowser so the session does not linger (param checks:131-145).
• Remote browser connection failure: when launchRemoteBrowser throws, the catch falls back to launchLocalBrowser and telemetry records remote_browser_launch_error (remote fallback:181-198).
• Screenshot failure: webp failure retries with png, and if both fail throws "Failed to take screenshot." and emits screenshot_error telemetry (screenshot retry:448-466).
• Page not started: doAction checks that this.page exists and throws "Browser is not launched" otherwise; this usually means a non-browser_action tool was called (which in fact has no permission) or a call after a prior close (page check:379-383).
• Any exception closes the browser: the handler's try/catch fallback is browserSession.closeBrowser(), guaranteeing no Chrome process is left holding resources (error close:205-208).

Summary

BrowserToolHandler is a thin shell; the real work is done by BrowserSession. The model picks launch/click/type/scroll/close via the action parameter, the handler does parameter validation and approval, delegates the action to BrowserSession's Puppeteer calls, then packs the screenshot and console logs back to the model. launch requires separate approval; other actions reuse the already-open session; any error forces a browser close.

To dig deeper, continue with:

• File and command tools: /cap-tools/file-ops
• Web fetch/search: /cap-tools/web
• Tool scheduling chain: /agent-loop/task-class

See official docs: Cline 文档 · README