CLIs

CLI SDK

See which commands Claude Code, Cursor and other coding agents run with your CLI, next to people and CI.

Coding agents use command-line tools all day. The CLI SDK records each command your CLI runs: its name, flag names, exit code and duration, the CLI version and OS, and who ran it. That shows you which commands agents rely on, where they fail more often than people do, the commands they guess at that don't exist, and how often they fall back to --help before getting it right.

Add a CLI

Open Project settings → Setup → CLIs and add one (or tick CLI when you create the project). Each CLI has its own key (al_cli_…). It's public: it ships inside your CLI and can only send that CLI's events, like the browser SDK's site key.

Record the command

One call in your CLI's entry point, for the framework it parses arguments with. The setup page has the same code with your key filled in, and a prompt for each framework that has your coding agent add it.

npm i @agentstats/cli
import { agentstats } from '@agentstats/cli'

const stats = agentstats({ cli: 'acme', version: pkg.version, apiKey: 'al_cli_…' })

stats.commander(program) // commander 8+, before program.parse()
  • yargs: commands defined inside a builder function only exist while that command runs, so the command list has the top-level ones. Nested commands are still recorded when they run.
  • citty: lazy subcommands still load lazily, but then the command list isn't sent on its own (it would only hold the ones loaded so far); pass it to stats.commands([...]) if you want it.

Each integration records the command that ran (nested ones as db migrate, aliases under the command they stand for), help, commands that don't exist, parse errors such as an unknown flag or a missing argument, and errors your commands raise, and sends the command list. Your CLI's output and exit codes don't change. From parse errors only the flag involved is kept, never a value that was typed: invalid value for --env, not the value.

If you never call command(), the run is recorded under the CLI's name, which suits single-command tools.

Who ran it

Coding agents set environment variables in every command they run, and the SDK reads them:

ClientDetected from
claude-codeCLAUDECODE, CLAUDE_CODE_ENTRYPOINT
cursorCURSOR_AGENT
codexCODEX_SANDBOX, CODEX_THREAD_ID
gemini-cliGEMINI_CLI
ampAMP_CURRENT_THREAD_ID (checked before Claude Code, which it also sets)
copilot-cli, opencode, augment, antigravityCOPILOT_CLI, OPENCODE_CLIENT/OPENCODE, AUGMENT_AGENT, ANTIGRAVITY_AGENT
anyAI_AGENT, the proposed standard (e.g. AI_AGENT=devin@1), which wins over the rest
ciCI, GITHUB_ACTIONS, GITLAB_CI and other CI providers
humana terminal on stdin or stderr
unknownnone of the above: a script, or an agent that sets nothing

An agent running inside CI is recorded as the agent, with the CI provider kept alongside. These are hints, not proof: any process can set them, so they're for analytics, never for authorization.

Commands share a session when an agent exposes a thread id (Codex, Amp), or otherwise when the same client runs them in the same directory with less than 30 minutes between them.

In the dashboard

Each command is a tool call named after the command, so the overview, tools, errors, sessions, funnels, alerts, Ask AI and the export API all work for CLIs, and the client breakdown shows agents next to people and CI. A non-zero exit code or an unknown command is an error; help is its own entry (deploy --help), so journeys show agents checking help before running a command. Pick the CLI in the overview's source menu to see it alone, or open its Commands page in the sidebar.

The command list

The CLI sends its own command list (names, descriptions, arguments, options) at runtime, from installed copies, whenever it changes. Commands show up in the dashboard as soon as one copy with them runs, so commands nobody uses appear with 0 calls. There's nothing to add to your release pipeline.

Because the CLI key ships with your CLI, a list sent with it can only add commands: a removed command stays listed, with no new calls. If you want removals and changes recorded and alerted on, you can optionally run the CLI once in release CI with the secret publish key from the CLI's card under Project settings → Setup (AGENTSTATS_CLI_PUBLISH_KEY=… acme --version); after that, its lists replace the runtime ones.

Leaked keys

The CLI key is public by design, like Sentry's DSN or PostHog's project key: it can only send events, never read them. What a copied key can do is bounded:

  • Rate limits per IP, per IP and key, and per key. A rate-limited CLI keeps its events and sends them on a later run.
  • A per-machine budget of 5,000 events an hour per CLI. No person, agent or CI job gets near it.
  • Spike protection: each CLI gets an hourly ceiling of 6× its normal volume over the last week, never below 1% of your plan's monthly events. Above it, events are dropped and you get one alert an hour.
  • Dropped events never count against your quota.
  • The command list can only grow with the CLI key; only the optional publish key can remove commands.

If a key is abused anyway, rotate it in project settings and ship a release; copies with the old key stop reporting.

Speed and privacy

Nothing waits on the network. At exit the SDK appends one line to a spool file in ~/.cache/agentstats and starts a detached background process that sends it, so the command returns immediately. If AgentStats can't be reached, events wait in the spool (up to 512 KB) for the next run. In Node, compiled single-file binaries skip the background send; Go and Rust CLIs start their own binary again to send, and Python CLIs start the same interpreter.

Argument values are never sent. Flags are reduced to their names: --env=prod becomes --env, -pSECRET becomes -p, and anything after -- is ignored. Pass captureFlags: false to send none, or a function to choose: captureFlags: (command, flags) => flags.filter(f => f !== '--token').

Error messages are sent for failed commands, cut at 300 characters, because they explain the failure. They can also carry data, such as a connection string or the row that failed. captureErrors: false sends none (the command still counts as failed, with its exit code). A function decides per command: it gets { command, message, name, code, exitCode, error } and returns the text to keep, or null for none.

const stats = agentstats({
  cli: 'acme',
  captureErrors: ({ command, message, code }) => command.startsWith('db ') ? code ?? null : message,
})

Go takes ErrorText and FlagNames (with errors.As on ErrorInfo.Err), Rust capture_errors_with and capture_flags_with, and Python a function for capture_errors= and capture_flags= (its code comes from code, errno or pgcode). The framework integrations pass the thrown error itself, so code (such as ECONNREFUSED) is there too. If the function throws, nothing is kept. To stop keeping messages without shipping a release, turn off Keep error messages on the CLI's card under Project settings → Setup.

Telemetry notice and opt-out

On the first run in a terminal the SDK prints a one-time notice to stderr. Pass your own text as notice, or notice: false if your CLI already tells users. Any of these turns it off:

  • ACME_TELEMETRY_DISABLED=1 (your CLI's name, uppercased)
  • DO_NOT_TRACK=1
  • stats.setEnabled(false), saved for future runs: wire it to an acme telemetry disable command

Options

The Node names are below; Go (cli.Options), Rust (cli::Options) and Python (Tracker(...)) have the same options in their own style, such as APIKey, api_key(…) and api_key=.

OptionDefault
clirequiredYour CLI's name
versionShown per command
apiKeyAGENTSTATS_CLI_KEYWithout a key the SDK does nothing
noticebuilt-in textString, or false
captureFlagstrueFlag names only. false, or a function returning the names to keep
captureErrorstrueMessage of the error passed to error(), first 300 characters. false, or a function returning the text to keep
sessionIdleMs30 minutes
dir~/.cache/agentstatsSpool, session and notice files
debugAGENTSTATS_DEBUG=1Logs to stderr