Skip to content

AI Tools Reference

nick3 edited this page May 28, 2026 · 1 revision

AI Tools Reference

The complete catalog of tools the model can call. ~52 tools across 9 categories. Each table lists name, key arguments, what it returns, and when to use it.

All tools dispatch through toolRegistry.dispatch(name, args, ctx) — see Tool-Registry for the internals.


Step protocol (2)

Personas mandate a declare_step → action → verify_step cycle for non-trivial work, so the model has to articulate plan + verification rather than acting blindly. Source: src/main/ai-tools/step-protocol.ts.

Tool Key args Returns Use when
declare_step number, title, action, success_criteria {ok, message} Before a substantive action. State what you'll do and how you'll know it worked.
verify_step number, passed (bool), observation {ok, message} After the action. Confirm against the success criteria.

Pane management (6)

Workspace-level inventory and per-pane UI control. Source: src/main/ai-tools/pane.ts.

Tool Key args Returns Use when
list_panes AIPaneInfo[] (id, label, type, isConnected, position) First thing to do when entering a new workspace — get the lay of the land
capture_screenshot pane_id? (or whole window) Data URL Visual debugging; for browser visual verification prefer [[Vision-Verification
focus_pane pane_id confirmation Bring a pane into focus before typing into it
maximize_pane pane_id confirmation When you need more room to read output
create_workspace name, grid new workspace ID Spin up a fresh layout for a new task
restart_terminal pane_id confirmation When a shell died or you want a clean slate

Terminal control (4)

Read from and write to a terminal's PTY. Source: src/main/ai-tools/terminal.ts.

Tool Key args Returns Use when
write_to_terminal pane_id, text, press_enter?, wait_timeout_ms?, terminal_type? command output (tail) Sending a command. Default polls for completion; set terminal_type="claude_code" and longer timeout for Claude Code instances.
read_terminal_output pane_id, cursor?, max_bytes? PagedTextResult envelope Inspecting scrollback. Returns {content, hasMore, nextCursor} — paginate with cursor to read more.
poll_terminal_status pane_id {is_idle, last_activity_ms} Quick "is this terminal still doing something?" check.
wait_for_output pane_id, pattern (regex), timeout_ms? matched text or timeout Waiting for a specific prompt or marker (e.g., "Compilation succeeded") before next step.

Agent orchestration (8)

Coordinate per-pane agents in a multi-pane goal. Source: src/main/ai-tools/orchestration.ts.

Tool Key args Returns Use when
get_fleet_status array of agent states Snapshot of every pane's role/status/task
set_agent_role pane_id, role confirmation Assigning Builder/Monitor/Tester/Deployer/General to a pane
assign_task pane_id, task task id Queue a task on a specific pane
complete_task pane_id, task_id, result confirmation Mark task done, unblocks waiting agents
fail_task pane_id, task_id, reason confirmation Mark task failed
wait_for_agent waiting_pane_id, target_pane_id confirmation Block one agent until another completes
share_context from_pane_id, to_pane_id, context confirmation Push a context snippet from one agent to another
create_goal description, assigned_panes[] goal id High-level multi-pane goal (different from a single-pane GoalRunner goal — see Agent-Orchestration)

Browser navigation (7)

Source: src/main/ai-tools/browser/navigation.ts. All take a pane_id of a browser pane.

Tool Key args Returns Use when
browser_navigate pane_id, url {success, finalUrl, title} Going to a new page
browser_back / browser_forward / browser_reload pane_id confirmation Standard browser controls
browser_get_content pane_id, cursor?, max_bytes? PagedTextResult of page text Reading the page (cheaper than get_axtree for plain text)
browser_screenshot pane_id file path + dims Viewport-only screenshot, saved to disk
browser_execute_js pane_id, code serialized result Running arbitrary JS in the page (sync or async IIFE; result must be JSON-safe)

Browser interaction T1 (9)

The "common" interactions. Source: src/main/ai-tools/browser/interaction-t1.ts.

Tool Key args Returns Use when
browser_click pane_id, selector {success, urlBefore, urlAfter, navigated} Clicking by CSS selector
browser_type pane_id, selector, text confirmation Typing into an input. Subject to password-field approval gate.
browser_wait_for_selector pane_id, selector, timeout_ms? confirmation Wait for element to appear
browser_wait_for_navigation pane_id, timeout_ms? new URL After clicking a link, wait for the next page to load
browser_wait_for_text pane_id, text, timeout_ms? confirmation Wait for specific text to appear
browser_keypress pane_id, key (e.g., Enter, Escape) confirmation Synthetic keyboard event
browser_scroll pane_id, selector?, direction, amount? confirmation Scroll the page or an element
browser_select_option pane_id, selector, value confirmation Pick an <option> in a <select>
browser_check pane_id, selector, checked confirmation Toggle a checkbox/radio

Browser interaction T2 (9)

Higher-power inspection + interaction. Source: src/main/ai-tools/browser/interaction-t2.ts.

Tool Key args Returns Use when
browser_query pane_id, selector element info (text, attrs) Inspect a single element
browser_query_all pane_id, selector array of element infos Inspect all matches
browser_get_axtree pane_id accessibility tree (paged) Best for understanding page structure — gives roles, labels, text. Preferred over raw DOM.
browser_set_files pane_id, selector, paths[] confirmation File upload. Subject to approval gate.
browser_click_at pane_id, x, y confirmation Click at coordinates (CDP-based, trusted events — for strict-input sites where browser_click fails)
browser_hover pane_id, selector confirmation Hover over an element
browser_drag pane_id, source_selector, target_selector confirmation Drag-and-drop
browser_screenshot_full_page pane_id file path + dims Full scrolled-page screenshot
browser_screenshot_annotated pane_id, selectors[]? file path + dims Screenshot with numbered red boxes drawn over given selectors — for "click box N" decision making

Browser advanced (9)

Workflows, observability, and pane-conversion. Source: src/main/ai-tools/browser/advanced.ts.

Tool Key args Returns Use when
browser_smart_click pane_id, selector?, aria_label?, role?, text? {success, matchedBy, navigated} Click with fallback strategies (selector → aria-label → role+text → visible text). Use when browser_click is too brittle.
browser_run_recipe pane_id, recipe_name? or steps_json per-step results Replay a saved sequence — see Browser-Recipe-System
browser_get_action_log pane_id?, limit? last N browser tool calls (ring buffer of 500) Debugging "what happened?" after a failure
browser_get_cookies pane_id, url? cookie array Inspect cookies
browser_set_cookie pane_id, cookie confirmation Set a cookie programmatically
browser_save_pdf pane_id, path {success, path} Save current page as PDF
browser_save_html pane_id, path {success, path} Save raw HTML
convert_pane_to_browser pane_id confirmation Replace a terminal pane with a browser. Kills the PTY.
convert_pane_to_terminal pane_id confirmation Inverse — kills the webview.

Browser vision (2)

Vision-grounded verification. Source: src/main/ai-tools/browser/vision.ts. Requires the active provider to have a visionModel (or be dual-purpose like Claude/GPT-4o).

Tool Key args Returns Use when
browser_verify_visual_state pane_id, expected_state (free text) {verdict: 'yes'|'no'|'unclear', explanation, screenshot_path} After an action, when DOM check is ambiguous (modal might be over your target, error toast might have appeared, etc.)
browser_describe_screen pane_id, prompt? {description, screenshot_path} "Where am I?" — free-form vision description for when the page doesn't match expectations

See Vision-Verification for the full story.


Paginated tool results

Several tools return a PagedTextResult envelope:

{
  success: true,
  content: string,        // chunk for this page
  hasMore: boolean,
  nextCursor?: string,    // pass back on next call to get the next chunk
  totalBytes: number,
  truncated?: boolean
}

Affected tools: read_terminal_output, browser_get_content, browser_get_axtree. To fetch the next page, call the same tool with cursor: <nextCursor> from the previous response.

This prevents large terminal scrollback or rich pages from blowing out the conversation token budget. The model is taught about pagination in the default system prompt.


Transient tools (goal-runner-only)

These are registered globally but only meaningful inside a GoalRunner invocation. Outside one they return a no-op result so the model gets clear feedback.

Tool Key args Use when
claim_complete rationale You believe the goal is done. Runner verifies the success criterion; if it fails, the loop resumes with the failure reason.
abort_with_report reason, what_was_learned Genuine giveup. Goal is marked aborted; what_was_learned ends up in the final report.

See Goal-Runner-Overview.


Tool result truncation

Results > 3000 characters get truncated with a … [truncated] suffix, except when the tool returns a PagedTextResult envelope (those are designed for paginated reads). When you see a truncation, the original is not lost — the model can ask for a paginated read of the same source, or re-issue the call with a narrower scope.


See also

Clone this wiki locally