The Problem
When you run dozens of AI agent sessions per day across multiple frameworks (Claude Code, Codex, Gemini CLI), you lose visibility. How much are you spending? Which sessions are stalled? Which agents are idle? What’s the error rate? How do you onboard a new service?
Traditional APM tools (Datadog, New Relic) don’t understand AI agent sessions. They track HTTP requests and container metrics, not token consumption, session state, or agent decision quality.
Hammurabi is purpose-built observability for AI agents.
Modules
Agents Monitor
The core dashboard. Shows all active and recent agent sessions with:
- Session state (running, idle, completed, failed)
- Token consumption and cost per session
- Duration and activity timeline
- Live progress indicators for running sessions
Telemetry Hub
Receives and stores telemetry from all connected agents. Built on OpenTelemetry with custom attributes for AI-specific metrics:
token.input_count,token.output_countmodel.id,model.providersession.id,conversation.idcost.usd(computed from token counts and model pricing)
The normalizer handles format differences between agent frameworks — Claude Agent SDK sends session.id as a resource attribute, while Codex SDK sends conversation.id as a log attribute.
Services Manager
Registry of all connected services with health checks. Each service declares:
- What agent framework it uses
- Expected telemetry format
- Health check endpoint
- Alert thresholds (cost per session, session duration, error rate)
Factory
Template-based provisioning for new agent services. Instead of manually configuring telemetry for each new agent, Factory provides starter configs that wire up the correct OTEL exporters and normalized attribute mappings.
Settings
Global configuration: pricing tables per model, retention policies, alert channels, user management via Auth0.
Architecture
- Frontend — React with shadcn/ui components
- Backend — Express API server
- Database — Prisma ORM with PostgreSQL
- Auth — Auth0 for user management and RBAC
- Telemetry ingestion — OTEL collector with custom receiver
Key Design Decisions
OTEL-native, not custom protocol. Agents export standard OpenTelemetry traces and logs. Hammurabi’s receiver understands the AI-specific attributes but doesn’t require a custom SDK. Any OTEL-compatible exporter works.
Cost is a first-class metric. Every session has a running cost counter computed from token usage and model pricing tables. Cost alerts are as important as error alerts.
Agent framework abstraction. The normalizer layer translates between framework-specific telemetry formats so the dashboard shows a unified view regardless of whether the session ran on Claude, Codex, or Gemini.
Lessons Learned
Session ID mapping is framework-specific. We assumed all frameworks would send session.id consistently. They don’t. Claude sends it as an OTEL resource attribute. Codex sends conversation.id as a log attribute. The normalizer must handle both, with fallbacks.
Telemetry volume scales fast. A single agent session can generate hundreds of trace spans. Retention policies and aggregation are essential from day one, not afterthoughts.
Onboarding friction kills adoption. If it takes 30 minutes to configure a new agent service to report telemetry, people won’t do it. Factory’s template approach reduced onboarding to under 5 minutes.
Status
Active. Hammurabi monitors all agent sessions across the Gehirn ecosystem, tracking cost, performance, and health across Claude Code, Codex, and Gemini CLI agents.