Observability

Runtime gives you session-level visibility into every interaction, down to the tool call and chain of thought. Track who is doing what, how much it costs, and where time is being spent. The dashboard exposes three observability views: Activity, Metrics, and Traces.

​

Activity

Activity is the human-readable view of what happened in your org. It answers questions like “what has my team built today?” and “which agents are running right now?”. Each activity event includes:

• Who - user id, email, organization
• What - prompt text, file changes, commands run, tool calls
• When - timestamp in UTC
• How much - tokens used, dollars spent, time elapsed
• Where - which session, which template, which agent

​

Metrics

Metrics aggregate activity into time-series data you can chart and alert on. Useful for tracking trends, capacity planning, and cost analysis. Available metrics include:

• Sessions created per day / week / month
• Prompts sent per user, per agent type, per template
• Token consumption by model, by user, by org
• Deploy frequency and success rate
• Agent run completion rate
• Cost in dollars, broken down by team member or project

See Team Deploy Metrics and Team Summary for the API surface.

​

Traces

Traces give you the deepest level of detail: every step an agent took inside a session, in order, with timing and metadata. A single trace captures:

• The original prompt
• The agent’s plan (if plan mode was used)
• Every tool call (read file, write file, run command, search, etc.) with arguments and results
• Every LLM round trip with token counts and latency
• File changes with diffs
• Errors and recovery attempts
• Final output (commit, PR, deploy)

Use traces for:

• Debugging - figure out exactly where an agent went wrong
• Replaying - reproduce a session step-by-step
• Audit - prove what an agent did and why
• Optimization - find slow tool calls or expensive prompt patterns

See Team Traces for the API surface.

​

Cost tracking

Every prompt, tool call, and deploy is priced and attributed:

• Per session - total cost for one session, broken down by component
• Per user - daily and monthly spend per team member
• Per org - aggregate spend with month-to-date and forecast
• Per template - which templates drive the most cost

Cost data flows into activity events and aggregates into metrics. Guardrails can cap spend at any level.

​

Real-time observability

For live views into running sessions, use the Collaboration WebSocket. It streams presence and prompt-status events as they happen, which is what powers the “X is prompting…” indicators in the dashboard. For watching an individual prompt run, the Prompt WebSocket emits structured events for every tool call, file edit, and command in real time.

​

Exporting

Activity, metrics, and traces are all exportable. Use the Team Events endpoint to pull a stream of structured events into your own data warehouse, SIEM, or BI tool.

​

Why observability matters

Coding agents are powerful but they make a lot of decisions per minute. Without observability, you cannot:

• Verify that agents are doing the right thing
• Diagnose when something goes wrong
• Improve prompts, templates, and skills based on what works
• Control cost as agent usage scales across the team
• Audit what happened for security or compliance

Runtime treats observability as a first-class part of the platform, not something you bolt on later.

​

Related guides