The Usefulness of Agent Skills
I really like skills for agents. The way the community has rallied around creating and sharing skills has been great to see. Most of them though, are very much written for devs building software with Claude Code, Codex etc.
We haven't yet realized the potential (or maybe even the most robust shape) of agents that run autonomously over knowledge work. More concretely - agents that are triggered by system events (e.g. email arrived, invoice posted to ERP), and investigate, recommend or act to resolve a case.
That's a different kind of agent than the one skills were designed around. Most folks I talk to still think of one type of agent - the chatbot that calls tools. That definition took off thanks to ChatGPT, and a lot of the abstractions that followed (tools, skills, MCP, memory) were shaped by it. The other type of agent - scoped, system-triggered, sitting inside a high-stakes business process - is now expected to inherit the same harness components.
The bellwether use case of software development has given some convergence on what a great agent harness looks like for long-running, deep reasoning and coding tasks. But is that same harness really needed for an agent that is scoped to a domain-specific knowledge task? In many cases, I think the reflex of jumping to a harness designed for software building, is in reality:
rushing to use what is most fashionable in the moment.
Business processes in many cases are much more constrained. They need less deep reasoning, less agent turns and more careful curation of the right context in the agent's message array to output a well-informed recommendation to a human for approval. In some cases, this may not need an "agent" by its traditional definition (LLM calling tools in a loop) and a simple workflow with LLM calls suffices. I see more of these scenarios than cases warranting an agent with a sophisticated harness.
This brings me back to skills...
Why are skills so useful and popular?
- Simple to create
- Easy to understand
- Quick to share, install and use
- You can attach a LOT of skills to your agent without overloading the context window
One of the fundamental ideas of skills is that you can have many of them for doing many different things. An agent might have access to dozens of skills and only the YAML frontmatter is loaded in the session, so you avoid the "dumb zone" of the context window (when compared to loading up tons of tools and MCPs - although things like Tool Search have improved here...).
This is GREAT for an agent that needs to do many things - like quickly switch from plumbing through a back end architecture, to evaluating UX design...to building a Powerpoint deck that communicates how something was built! The wide-ranging, unbounded input scope to those agents calls for modularized sets of procedural memory or knowledge.
This approach doesn't always translate to agents that run against business context in core operations. An invoice exception that needs a policy lookup, a PII redaction step, maybe an escalation rule - still doesn't need to discover its job at runtime. The workflow already knows what this agent is for. Attaching a high quantity of skills, and letting the model route between them, is solving a problem this agent doesn't really have.
I'll admit - the architectural simplicity of skills is nice - the decoupling of instructions that can live outside a prompt. That's the part I think actually transfers. You can version the file, an SME can PR a change, you can reuse the same "redact PII" instructions across a few scoped agents. That's real leverage, and I wouldn't throw it out. But I think it's less often that you need a high quantity of skills attached to an agent that augments a business process - these agents often sit inside workflows with a smaller, bounded set of inputs.
Progressive disclosure is also debatable in terms of its benefits to tightly-scoped agents. The idea of progressively disclosing data makes sense in conjunction with the notion that you are attaching lots of skills to the agent and want to avoid overwhelming the context window. Additionally, it's helpful with long-running agents that need to iterate across many turns and pull in skills data dynamically.
With single-task agents, we often design them to perform a shorter-lived, focused task in the background and are optimizing for high accuracy and speed. These agents benefit from having strong instructions in the system prompt. The risk of the agent failing to load the skill may be too high to tolerate relying on the LLM to decide when to read it.
That's the bit I think gets under-discussed. In a Claude Code session, if the model doesn't pull in a skill, you can just tell it to. The user is still there. In a background agent - invoice posted, recommendation comes back - if the model skips the skill you thought you'd attached, you don't get an error. You get a fluent answer produced without the procedure. That's a silent miss, and for this class of work it might be the whole product.
So if I were designing a "skill" for this kind of agent, I'd want something a bit more boring. Pinned, not selected at runtime. The workflow already knows which procedure applies, so just load it - don't make the model go looking for its job. Versioned, and ideally run against a golden dataset before you promote a change, because a silent edit to the file is otherwise a silent change in behavior. The authoring convention of skills still helps here. It's the runtime discovery I'd strip.
This is a version of a gap I keep coming back to. The chat and coding kind of agent shipped first and grabbed a lot of the roadmap. Evals, observability, regression gates, the compliance layer a workflow agent actually needs - that stuff is still thin, because the unconstrained agent was the one everyone was building for. Skills are a good example of the mismatch. A really nice abstraction, pointed at a problem it wasn't designed for.
I love Daniel Warfield's writing - this conclusion from his IAEE article on skills:
"Skills are starting to blur the lines between constrained and unconstrained agents, but are still thoroughly in the unconstrained territory. I think it’s important for developers to appreciate the difference between the loose constraints of a skill and the more rigid constraint of a graphical agent. Personally, I see Skills as a convenience that makes unconstrained agents less wrong, but doesn’t mean I’ll now trust them to operate consistently or scale in a maintainable manner as an application becomes more complex"
That's roughly where I land too. Skills are useful. They're useful for the agents they were designed for. For the ones sitting inside a business process, I'd keep the files - and drop the idea that the model should go looking for them.