← back
Skill issue: Lessons from skilling up coding agents to use Langfuse - Marc Klingen, Clickhouse
Takeaway
Product skills should guide agents toward current documentation and application-specific workflows, then improve through trace feedback.
Summary
- Skills provide reusable procedural guidance while preserving an agent’s ability to combine tasks that would otherwise require separate routed workflows.
- Production traces reveal new use cases that need skills and existing skills that have become outdated or inefficient.
- Langfuse’s large documentation surface and evolving SDK make pretrained knowledge insufficient for correct integration.
- Without guidance, an agent may implement obsolete instrumentation before fetching current documentation; the skill aims to supply expert setup guidance and help choose suitable observability and evaluation patterns.
langfuseagent-skillsobservability
Original description
Without a skill, Claude Code adds Langfuse using stale pre-training context, ships broken instrumentation, then catches the failure and fetches current docs to fix it. The resulting trace captures two LLM calls with no visibility into what the agent actually did. Marc Klingen covers the six learnings from building a skill to close that gap: surfacing a natural language search endpoint so agents stop crawling 478 documentation pages, why pointing to references beats duplicating content, and what happened when they ran an auto-research loop on the skill itself. Three of six suggested improvements shipped, but their target function nearly backfired by optimizing out the documentation-fetching steps that make the skill reliable over time. Speaker info: https://x.com/marcklingen / marcklingen Timestamps: 00:00 - Introduction to Marc Klingen and Langfuse 01:22 - Conceptual mental model for agent skills 04:04 - The problem: scaling documentation and onboarding for coding agents 09:03 - Six key learnings from building a Langfuse skill 09:12 - Learning 1: Looking at traces for debugging 10:47 - Learning 2: Helping agents navigate information (sitemaps/formats) 11:40 - Learning 3: Surfacing a search endpoint for docs 12:36 - Learning 4: Implementing basic evaluation (eval) setups 13:53 - Learning 5: Referencing rather than duplicating content 14:24 - Learning 6: Using auto-research loops with target functions 17:24 - Discussion on challenges: distribution, versioning, and target functions 19:21 - Roadmap and future outlook for agent automation 20:09 - Q&A session