Sidecar-Workspace / Sidecar /knowledge /PRIOR_ART_LOGS.md
smlflg's picture
Initial public upload from Projekte/Sidecar-Workspace (part 4)
e6b2bd6 verified
|
Raw History Blame Contribute Delete
16.8 kB
# Prior Art: Tool-Call Log Viewers & Trace Viewers
**Research date:** 2026-03-04
**Researcher:** logs-researcher (panel-research team)
**Purpose:** Redesigning GTK3 Logs Tab for Claude Code sessions β€” from zero-value to proper trace/activity viewer
---
## 1. Tools Surveyed
### Langfuse (Open Source LLM Engineering Platform)
- Hierarchical trace model: **Trace β†’ Observations (nested spans)**
- Observation types: generations, toolcalls, RAG retrieval steps
- Sessions layer groups traces by user/conversation
- Filter by: environment, tags, user, session, metadata key-value pairs
- Batches traces locally and sends async to avoid blocking
- Launched a beta timeline view (Gantt-style) after user demand
- Source: https://langfuse.com/docs/observability/data-model
### LangSmith (LangChain's Observability Platform)
- Traces display as **tree of sub-steps**: agent thought process + each tool call
- Filter types: metadata, input, output, tool_call names via Output Key search
- AI Query feature: natural language β†’ filter DSL (LLM-powered)
- Grouping: by metadata attributes in Monitoring tab for A/B stats
- Same filter UI available in trace view AND the main runs table
- Source: https://docs.langchain.com/langsmith/filter-traces-in-application
### Helicone (Open Source LLM Observability)
- Sessions: high-level grouping with average latency, total cost, time filters
- Unified filter component β€” persistent across pages
- Cost per request shown inline (derived from provider pricing)
- Session-level metrics: cost, latency, request count per session
- Multi-step workflow visualization for agent traces
- Source: https://docs.helicone.ai/features/sessions
### Braintrust (AI Observability)
- **Visual timeline replay**: when each op started, duration, return value
- **Nested spans**: parent-child in multi-step agent runs (10+ tool calls)
- Token granularity: prompt/cached/completion/reasoning tokens separately
- Error categorization: LLM errors, tool errors, timeouts
- Connects failures to root cause spans directly
- Source: https://www.braintrust.dev/articles/best-llm-tracing-tools-2026
### AgentPrism (Open Source React Component Library)
- 4 view modes: **Tree, Timeline (Gantt), Details Panel, Sequence Diagram**
- Tree view: red highlighting for problematic patterns, collapse repetitive sequences
- Timeline: Gantt-style with color-coded status (green/red/yellow) + real-time cost
- Details Panel: compressed input/output + metrics in single view
- Sequence Diagram: step-by-step replay for decision chains, play/pause
- Progressive disclosure to keep deep structures navigable
- Source: https://evilmartians.com/chronicles/debug-ai-fast-agent-prism-open-source-library-visualize-agent-traces
### Grafana Tempo Trace Viewer (Standard OTel/Distributed Tracing UI)
- **3-panel layout**: Header (name + trace ID + search) | Minimap | Span Timeline
- Minimap: draggable zoom for dense traces
- Span rows: service name + operation name + visual duration bar (waterfall)
- Click span β†’ detail side panel with: attributes, resource attrs, events, links
- Filters: service name, span name, duration (with unit selector), tags
- "Show matches only" toggle + "Show critical path only"
- Source: https://grafana.com/docs/grafana/latest/visualizations/explore/trace-integration/
### Datadog APM / General OTel UI Pattern
- Span list as master, detail as side panel (master-detail layout)
- Minimap for zooming into dense timelines
- Critical path highlighting in darker color (CRISP algorithm)
- Source: https://docs.datadoghq.com/tracing/trace_explorer/trace_view/
### Claude Code Session JSONL (Our Data Source)
Key fields available per tool call:
- `timestamp`: ISO-8601
- `sessionId`, `uuid`, `parentUuid`
- `message.content[].type`: text | tool_use | tool_result
- `message.content[].name`: tool name (Bash, Read, Edit, Glob, Grep...)
- `message.content[].input`: full tool arguments
- `message.usage`: input_tokens, output_tokens, cache_creation_input_tokens, cache_read_input_tokens
- `message.model`: claude model used
- `toolUseResult`: full tool output
- Source: https://liambx.com/blog/claude-code-log-analysis-with-duckdb
---
## 2. UI Patterns Found
### A. Hierarchical Timeline / Waterfall (Most Common for Traces)
**What it is:** Horizontal bar chart where each call gets a row, bars show duration, indentation shows parent-child nesting.
**Pros:**
- Immediately shows parallelism vs. sequential calls
- Duration at a glance β€” long bars catch the eye
- Nesting reveals call structure without extra clicks
- Standard pattern β€” users who know Chrome DevTools/Jaeger already know it
**Cons:**
- Requires timing data (we have timestamps, can derive duration if needed)
- Becomes dense with 50+ spans β€” minimap needed
- Poor for text-heavy output (truncation required)
**When to use:** Primary view when timing and structure matter most.
### B. Tree List / Expandable List
**What it is:** Indented list with expand/collapse toggles, no time axis.
**Pros:**
- Works without timing data
- Better for deep hierarchies than waterfall
- Easier to read tool inputs/outputs inline
- GTK3 TreeView widget directly supports this pattern
**Cons:**
- No timing information visible without extra column
- Harder to spot slow calls without duration bars
**When to use:** Best for our GTK3 context β€” maps naturally to GtkTreeView.
### C. Flat Table / Log List (Reverse Chronological)
**What it is:** Scrolling list of events, newest on top (or bottom), tabular columns.
**Pros:**
- Simplest to implement
- Easy to scan text
- Works well with search/filter
- Activity Log pattern standard (reverse-chron with entity filter)
**Cons:**
- Loses parent-child structure
- Can't see call hierarchy
**When to use:** Secondary view or for the raw sidecar log format.
### D. Session Group View
**What it is:** Sessions as cards or collapsible groups, with aggregate metrics per session.
**Pros:**
- Quick session comparison (cost, call count, errors)
- Click to drill into session detail
- Good for multi-session overview
**Cons:**
- Extra navigation layer before seeing individual calls
**When to use:** Top-level navigation before drilling into a session's calls.
### E. Sequence Diagram / Replay
**What it is:** Step-by-step replay of call sequence with play/pause, like a debugger.
**Pros:**
- Great for understanding complex decision chains
- Can show agent "reasoning" per step
**Cons:**
- Complex to implement, high effort
- Overkill for a GTK3 desktop panel
**When to use:** Skip for now β€” too complex, diminishing returns vs. list/tree.
---
## 3. Filtering Best Practices
### Filters that actually matter (ranked by utility):
1. **By tool name** β€” "Show only Bash calls" or "hide Read calls" is the most frequent need
2. **By session** β€” single most important grouping; usually implicit (viewing one session at a time)
3. **By time range** β€” "last 30 min", "today", custom range
4. **By error status** β€” "show only failed calls" (non-zero exit, exception messages)
5. **By text/search** β€” free-text search across tool inputs/outputs
6. **By cost threshold** β€” "calls over $0.01" (less useful for individual calls, better for sessions)
### Implementation patterns:
- **Filter bar at top** with chips/tags that are toggleable β€” most common pattern
- **Persistent filters** across tab switches (Helicone's design insight)
- **Quick toggles** for common filters (errors only, tool type buttons)
- **"Show matches only" toggle** vs. "show all with matches highlighted" β€” both useful
- **Filter by text in input/output** β€” powerful for debugging specific tool behavior
- **Sidebar filter panel** (LangSmith/Datadog style) works for complex multi-dimensional filtering but adds UI complexity β€” probably overkill for our tab
### What NOT to do:
- Don't require users to type raw query DSLs
- Don't hide filters behind a hamburger menu (discoverability drops to zero)
- Don't reset filters on every navigation
---
## 4. Grouping Strategies
### By Session (Primary)
- Most natural grouping for Claude Code β€” each session is a distinct task
- Show aggregate per session: total cost, call count, error count, duration
- Session ID from `sessionId` field in JSONL
- Sidecar state gives us: `files_touched`, `total_calls`, `concerns`, `score`
### By Tool Type
- Group Bash calls, Read calls, Edit calls, etc.
- Useful for "how many files did I read?" type analysis
- Second-level grouping within a session
### By Request Chain (Parent-Child)
- `parentUuid` links messages; `tool_use_id` links tool calls to results
- Can reconstruct the call tree within a session
- Useful for seeing "which assistant turn triggered which tools"
### By Error Status
- Surface sessions/calls with errors separately
- Failed Bash commands (non-zero exit) are the most actionable errors
- Sidecar `concerns` field already categorizes some of this
### By Time Period
- Today / Yesterday / This week groupings for session list
- Within a session, time is usually sequential so this matters less
---
## 5. Key Metrics
### Per Tool Call
| Metric | Source | Value |
|--------|--------|-------|
| Tool name | `message.content[].name` | High β€” what happened |
| Duration | derived from timestamps | High β€” how long |
| Input summary | `message.content[].input` truncated | High β€” what args |
| Success/fail | exit code, error in output | High β€” did it work |
| Output size | len(toolUseResult) | Medium β€” how much data |
| Session ID | `sessionId` | Context only |
### Per Assistant Turn (LLM Call)
| Metric | Source | Value |
|--------|--------|-------|
| Input tokens | `message.usage.input_tokens` | High |
| Output tokens | `message.usage.output_tokens` | High |
| Cache hit tokens | `cache_read_input_tokens` | Medium β€” cost savings |
| Model | `message.model` | Medium |
| Cost (derived) | tokens Γ— model pricing | High |
| Tool calls made | count of tool_use blocks | High |
### Per Session (Aggregates)
| Metric | Source | Value |
|--------|--------|-------|
| Total cost | sum of per-turn costs | High |
| Total calls | sidecar `total_calls` or count | High |
| Files touched | sidecar `files_touched` | High |
| Error count | count of failed calls | High |
| Duration | last_ts - first_ts | Medium |
| Score | sidecar score (0-100) | Medium |
| Concerns | sidecar `concerns` list | Medium |
### Aggregate Dashboard (All Sessions)
- Cost per day (bar chart)
- Most used tools (horizontal bar)
- Error rate over time
- Average session cost
---
## 6. View Modes: When Is Each Appropriate
### Timeline / Waterfall
- **Best for:** Debugging performance, understanding parallel calls, timing analysis
- **Our case:** Useful for seeing "did Claude do things in sequence or parallel?"
- **GTK3:** Custom cairo drawing required β€” significant effort
- **Verdict:** Nice-to-have, defer to v2
### Tree List (Expandable)
- **Best for:** Understanding call hierarchy, reading input/output per call
- **Our case:** Perfect fit β€” GtkTreeView is built for this
- **GTK3:** GtkTreeView + GtkTreeStore, easy to implement
- **Verdict:** PRIMARY view mode β€” implement first
### Flat List / Log Scroll
- **Best for:** Raw log tailing, quick scan, search results
- **Our case:** Current sidecar log format is already flat β€” map directly
- **GTK3:** GtkListBox or GtkTextView (for raw log)
- **Verdict:** SECONDARY view β€” implement as "raw log" sub-tab or toggle
### Session Cards / Summary View
- **Best for:** Choosing which session to inspect
- **Our case:** Session selector in sidebar or top dropdown
- **GTK3:** GtkListBox with custom cells
- **Verdict:** NAVIGATION component, not a view mode itself
---
## 7. Recommendations for Our Logs Tab
### Current State
The current tab "offers zero value" β€” presumably just raw log text or nothing useful.
### Recommended Architecture: Two-Panel Layout
```
+--[Filter Bar: Tool | Session | Time | Errors Only | Search]--+
| |
| LEFT: Session + Call Tree RIGHT: Detail Panel |
| +----------------------------------+ +---------------------+ |
| | [Session: abc123] $0.12 42calls | | Tool: Bash | |
| | [+] Turn 1 (3 tools) | | Input: git status | |
| | Bash [OK] 0.3s | | Output: On branch.. | |
| | Read [OK] 0.1s | | | |
| | Edit [OK] 0.2s | | Duration: 0.3s | |
| | [+] Turn 2 (1 tool) | | Exit code: 0 | |
| | Bash [ERR] 1.2s | | Turn: 2 | |
| +----------------------------------+ +---------------------+ |
+---------------------------------------------------------------+
| [Status bar: Session abc123 | 42 calls | $0.12 | 3 errors] |
+---------------------------------------------------------------+
```
### Concrete Implementation Steps
1. **Data source priority:**
- Start with sidecar log (`/tmp/claude-sidecar.log`) β€” simplest, already structured
- Add usage JSONL for richer data
- Add session JSONL for full tool I/O
2. **Session list:** GtkListBox at top (or left sidebar) showing sessions with
aggregate cost + call count + error badge
3. **Call tree:** GtkTreeView with 4 columns:
- Icon (tool type color-coded)
- Tool name + truncated input
- Duration (if derivable from timestamps)
- Status (green checkmark / red X)
4. **Detail panel:** Right-side GtkStack with:
- Full tool input (GtkTextView, monospace)
- Full tool output (GtkTextView, monospace, scrollable)
- Metrics row: duration, tokens, cost (if available)
5. **Filter bar:** Top bar with:
- Tool type buttons (toggle: All | Bash | Read/Write | Search | Other)
- "Errors only" toggle
- Search field (filters tree in real-time)
- Session selector (GtkComboBox or tabs)
6. **Status bar:** Bottom bar showing session totals
(derived from sidecar state JSON)
7. **Live updates:** Watch sidecar log file with GLib.io_add_watch
for file changes β€” append to tree in real-time
### What Makes It Good (Lessons from Research)
- **Color code tool types:** Bash=orange, Read=blue, Edit=green, Search=purple, API=yellow
- **Error calls always visible in red** even when not filtering errors
- **Expandable groups** (by turn/assistant message) so 42 calls don't overwhelm
- **Detail panel on click** β€” click any row to see full input/output
- **Never reset filters on refresh** β€” persist across live updates
- **Show cost prominently** β€” users care most about "$0.12 this session"
- **Progressive disclosure** β€” collapsed by default, expand to see sub-calls
### What to Skip (Not Worth the GTK3 Effort)
- Gantt/waterfall timeline: requires custom cairo rendering, high effort, moderate value
- Sequence diagram / replay: overkill for a desktop sidecar panel
- AI-powered query (LangSmith's NL filtering): fun but not needed at this stage
- Cross-session comparison charts: belongs in a separate analytics view
---
## Sources
- [Langfuse Data Model](https://langfuse.com/docs/observability/data-model)
- [Langfuse LLM Observability Overview](https://langfuse.com/docs/observability/overview)
- [LangSmith Filter Traces](https://docs.langchain.com/langsmith/filter-traces-in-application)
- [LangSmith Visibility with Tracing](https://victorleungtw.wordpress.com/2025/04/26/langsmith-visibility-while-building-with-tracing/)
- [Helicone Sessions](https://docs.helicone.ai/features/sessions)
- [Helicone Smarter Sessions Design](https://www.helicone.ai/changelog/20250506-smarter-sessions-insights)
- [Braintrust: Best LLM Tracing Tools 2026](https://www.braintrust.dev/articles/best-llm-tracing-tools-2026)
- [Braintrust: Best AI Observability Tools 2026](https://www.braintrust.dev/articles/best-ai-observability-tools-2026)
- [AgentPrism: Debug AI Fast (Evil Martians)](https://evilmartians.com/chronicles/debug-ai-fast-agent-prism-open-source-library-visualize-agent-traces)
- [Grafana Trace Integration](https://grafana.com/docs/grafana/latest/visualizations/explore/trace-integration/)
- [Grafana Span Filters](https://grafana.com/docs/grafana/latest/datasources/tempo/span-filters/)
- [Datadog Trace View](https://docs.datadoghq.com/tracing/trace_explorer/trace_view/)
- [Maxim: Agent Tracing for Debugging Multi-Agent AI Systems](https://www.getmaxim.ai/articles/agent-tracing-for-debugging-multi-agent-ai-systems/)
- [Claude Code Log Analysis with DuckDB](https://liambx.com/blog/claude-code-log-analysis-with-duckdb)
- [Inside Claude Code: The Session File Format](https://databunny.medium.com/inside-claude-code-the-session-file-format-and-how-to-inspect-it-b9998e66d56b)
- [Activity Log Pattern (alguidelines.dev)](https://alguidelines.dev/docs/navpatterns/patterns/activity-log/)
- [SigNoz: Jaeger vs Zipkin](https://signoz.io/blog/jaeger-vs-zipkin/)