How I Create/Choose Skills

The difference between a good skill and a bad one is massive.

The biggest help I had with skills comes from this paper: “SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks”.

This paper tested 7,308 agent runs across 84 tasks. Curated skills boosted pass rated by 16.2% on average. The wrong skills made things much worse, proving this is important to get right.

Here’s how I go about creating and choosing them.

A skill is a structured set of instructions that lives in your project or global folder. It tells Claude Code how to approach a type of task. Not what to do once, but how to do something every time.

If you think of like onboarding a new engineer that helps.

Now the most important thing that’s important is to understanding where Claude Code/AI tools strengths are without skills.

Claude Code has in build features it good at like searching for files and copying file conventions, similar to a new engineer you wouldn’t need to explain pattern matching as they are likely to copy the repo.

So if Claude Code can already pattern match and search your codebase, what’s left?

The stuff it can't see. Architecture decisions. Why something is built a certain way. Your team's workflow. Domain knowledge that isn't in the repo.

The SkillsBench data showed this clearly. Software engineering tasks only improved 4.5% with skills. Healthcare jumped 51.9%. The less the model already knows, the more a skill matters.

I just want to note as well, a 4.5% increase for a skill in software engineering is not great, as you should not overwhelm your codebase with skills as this has a negative impact.

Before writing any skill, ask: “Can Claude Code figure this out from my codebase?” If yes, skip it. If no, that’s your skill.

The signal is always the same. If you catch yourself prompting a process, that isn’t specifically related to pure coding implementation, then a skill is a good idea.

For example, we have a legacy application that has an extremely different style of validation around tests, linting, and pushing code.

In that repo whenever I would go to push after AI had implemented something, it would always get it wrong. This is a process that isn’t easy for Claude Code, this is a great example for having a skill.

If you have setup a green field project or personal project and you are already running into a lot of issues, ask yourself can I simplify this process for AI.

The skills I love testing and adopting the most are more around process. Not ones that help slight bug fixes or react/next standards.

Instead ones that help scoping a ticket out, documenting decisions, a quick code reviewer/refactor skill.

This is where I have found the more time spent the better the outcome.

The paper was clear here. 2 to 3 focused modules gave an 18.6% boost. 4 or more only gave 5.9%.

Keep it under 50 lines. Lead with steps, not background. Include one working example. State your constraints explicitly. That’s it.

When models in the study wrote their own skills before solving tasks, performance dropped 1.3%.

They wasted time writing bad docs then tried to follow them. Create skills separately, review them, then use them fresh.

The full paper is at arxiv.org/abs/2602.12670.