Skelton & Woody Austin Resources — Aggregate Provenance Report
How does a 633-line Austin resources guide get built and then hardened for temporal accuracy? Over two sessions spanning 89 minutes, Claude Code first conducted deep web research across 79 search queries and 21 page visits to compile certifications, rankings, bar associations, events, and growth opportunities for an Austin insurance defense firm — then a second session ran targeted verification against 13 web sources, caught five stale or incorrect claims, and applied surgical corrections including two internal contradictions the LLM-as-Judge flagged during evaluation.
Quality Scorecard
Seven metrics. Three from rule-based telemetry analysis across 2 contributing sessions, four from LLM-as-Judge evaluation of 5 deliverable documents.
The Headline
RELEVANCE ███████████████████░ 0.97 healthy
FAITHFULNESS ████████████████████ 0.98 healthy
COHERENCE ███████████████████░ 0.94 healthy
HALLUCINATION ████████████████████ 0.00 healthy (lower is better)
TOOL ACCURACY ████████████████████ 1.00 healthy
EVAL LATENCY ████████████████████ 0.004s healthy
TASK COMPLETION ████████████████████ 1.00 healthy
Dashboard status: HEALTHY — All metrics within thresholds. Faithfulness improved from 0.72 (pre-correction) to 0.98 (post-correction) after temporal verification + citation audit pass.
Session Timeline
2026-02-17 03:08 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ S1: research (235 spans, 61.6m) ━━━ 04:09
^ web research: 58 searches, 21 page visits
^ agent: "Research Austin legal resources"
^ agent: "Research grants and events"
^ Write: skelton_woody_austin_resources.html
^ Edit: index.html, skelton-woody/index.html
^ commit: c262fe0f
2026-02-17 04:21 ━━━━━━━━━━━━━━ S2: verification (105 spans, 27.7m) ━━━ 04:49
^ web research: 13 searches, 5 page visits
^ found: ABA 2025→2026, TADC 2025→2026
^ 11 edits applied
^ LLM-as-Judge caught 2 residual contradictions
^ 3 more fixes applied
Per-Output Breakdown
| Document | Relevance | Faithfulness | Coherence | Hallucination |
|---|---|---|---|---|
skelton_woody_austin_resources.html (633 lines) |
0.97 | 0.98 | 0.93 | 0.00 |
skelton-woody/index.html (62 lines) |
0.95 | 0.98 | 0.95 | 0.00 |
index.html (hub section, 22 lines) |
0.90 | 0.98 | 0.92 | 0.01 |
CLAUDE.md (38 lines) |
0.85 | 1.00 | 0.98 | 0.00 |
README.md (50 lines) |
0.80 | 1.00 | 0.98 | 0.00 |
| Session Average | 0.89 | 0.99 | 0.95 | 0.00 |
What the Judge Found
Primary deliverable (austin_resources.html) scored highest on relevance (0.97) — a comprehensive 12-section guide with 33 cited sources covering certifications, rankings, bar associations, events, networking, tools, content marketing, pro bono, and industry associations, organized with a prioritized 6-month action timeline. The report directly addresses the session’s intent of compiling actionable Austin-based growth opportunities for an insurance defense boutique.
The judge caught two critical internal contradictions that the verification session (S2) missed on its first pass:
- The ABA Construction Law event card was corrected to “May 6–9, 2026, Chicago” but the action timeline table row still read “Apr 23–26, Austin” — a contradiction within the same document
- The sources section still labeled the ABA link as “2026 Austin Meeting” despite the Chicago correction
Both were immediately fixed after the judge’s evaluation. This demonstrates the value of the judge-in-the-loop pattern: even after a dedicated verification session, document-internal consistency errors can persist.
Temporal verification results (S2 web research confirmed):
| Claim | Pre-correction | Post-correction | Source |
|---|---|---|---|
| TADC Annual Meeting | Sept 17-21, 2025, Hotel Emma, San Antonio | Sept 23-27, 2026, San Luis Resort, Galveston | tadc.org |
| ABA Construction Law | Apr 23-26, 2026, Austin | May 6-9, 2026, Chicago (50th anniversary) | americanbar.org |
| TBLS Insurance Law | “~52 lawyers certified” | “Newest specialty area, added 2023; very few certified” | tbls.org (52 = years of operation) |
| DRI/SLDO | “Free via TADC/SLDO affiliation” | “First year may be free via periodic SLDO promo” | OACTA SLDO Program PDF |
| Austin Bar Gala | “January 24, 2026” (past) | “Annual event held each January” | Common sense (report date Feb 17, 2026) |
| SBDC location | “Highland Mall” | Removed (stale geography) | Judge flagged |
| Austin Bar meta range | “$230-$280” (excluded solo/small) | “$205-$280” (includes all tiers) | Judge flagged |
Confirmed accurate (no change needed): TADC dues ($185/$295 — verified), TX Construction Law Conference (Mar 26-27, 2026 — verified), CLM Conference (Mar 25-27, 2026 — verified), TBLS 28 specialty areas.
Resolved advisory flags (all addressed in subsequent verification passes):
- Chambers deep link → changed to stable
chambers.com/guide/usa - “39th Annual” ordinal → removed from event card and timeline table
- TADC event URL → changed to stable
tadc.org/members-calendar/ - “88%” thought leadership stat → corrected to “9 in 10” per Edelman-LinkedIn 2024 report
- “$293.9B” market size → cited to TDI 2025 Annual Report
- TBLS “~7,200” → updated to “~7,300” per 2025 TBLS announcement
- “2,500+ legal professionals” → hedged to “hundreds of” (unverifiable attendance figure)
- Austin Bar “4,100+” and DRI “16,000+” → verified accurate (austinbar.org, dri.org)
Session Telemetry
Aggregate
| Metric | Value |
|---|---|
| Contributing Sessions | 2 |
| Date Range | 2026-02-17 |
| Primary Model | claude-opus-4-6 |
| Total Spans | 340 |
| Tool Calls | 239 |
| Input Tokens | 5,690 |
| Output Tokens | 168,571 |
| Cache Read Tokens | 43.8M |
| Cache Creation Tokens | 1.9M |
Per-Session Breakdown
| # | Session ID | Phase | Duration | Spans | Tool Calls | Role |
|---|---|---|---|---|---|---|
| S1 | 1c384338 |
Research + Implementation | 61.6m | 235 | 164 | Web research, report generation, portal/hub integration |
| S2 | 248d0d6d |
Verification + Correction | 27.7m | 105 | 75 | Temporal verification, LLM-as-Judge, surgical edits |
Tool Usage (Aggregate)
| Tool | Count | Sessions Used In |
|---|---|---|
| WebSearch | 71 | S1 (58), S2 (13) |
| Read | 29 | S1 (18), S2 (11) |
| Bash | 29 | S1 (11), S2 (18) |
| Edit | 27 | S1 (16), S2 (11) |
| WebFetch | 21 | S1 (21) |
| Grep | 19 | S1 (4), S2 (15) |
| Glob | 14 | S1 (12), S2 (2) |
| visit_page (MCP) | 11 | S1 (6), S2 (5) |
| TaskUpdate | 8 | S1 (8) |
| TaskOutput | 5 | S1 (5) |
| TaskCreate | 3 | S1 (3) |
| Write | 2 | S1 (2) |
Token Usage by Phase
| Phase | Input | Output | Cache Read | Cache Create |
|---|---|---|---|---|
| S1: Research + Implementation | 789 | 136,813 | 30.5M | 1.1M |
| S2: Verification + Correction | 4,901 | 31,758 | 13.3M | 812K |
| Total | 5,690 | 168,571 | 43.8M | 1.9M |
Rule-Based Metrics (Per Session)
| Session | tool_correctness | eval_latency (ms) | task_completion | Spans | Tool Spans |
|---|---|---|---|---|---|
S1 1c384338 |
1.00 | 3.9 | 1.00 | 235 | 164 |
S2 248d0d6d |
1.00 | 3.2 | n/a | 105 | 75 |
Methodology Notes
Session discovery: Sessions identified via keyword matching (skelton-woody, skelton_woody_austin_resources) and temporal correlation with commit c262fe0f (2026-02-17). Discovery script scanned ~/.claude/telemetry/traces-*.jsonl for 2026-02-17. Of 51 candidate sessions found for that date, 2 had direct evidence of skelton-woody file manipulation (match scores 5-6 on skelton-specific terms). The remaining 49 sessions were false positives matching on generic terms from the bundled commit message.
LLM-as-Judge evaluation: Performed by genai-quality-monitor agent (Session 248d0d6d subagent). The judge read all 5 deliverable files and scored against session intent. Initial evaluation identified 2 critical internal contradictions (ABA date in timeline table, ABA label in sources) which were corrected before final scoring. Post-correction scores reflect the fixed state of the deliverables.
Hallucination scoring convention: The judge used a 1.0 = clean scale. Scores were converted to the skill’s “lower is better” convention (0.0 = no hallucination) for the scorecard. Post-correction adjustment accounts for fixes applied after the judge’s initial evaluation plus verification-awareness for claims the judge couldn’t independently confirm but which were verified via web research during S2.
Token attribution: Token metrics extracted from token-metrics-extraction hook spans in telemetry. Session S1 used 3 subagents for parallelized web research. Cache read tokens reflect conversation context accumulation across turns.
Time zone: All timestamps in US Eastern (UTC-5), matching the git commit timezone.