Effect is a TypeScript library with strong opinions. Services use Context.Tag and Layer. Errors are Schema.TaggedError. Functions wrap in Effect.fn. Sequencing happens through Effect.gen. Get any of it wrong and the types don't compile.
AI coding agents struggle with it. They hallucinate imports from @effect-ts/core (a package that hasn't existed since 2023). They use async/await instead of generators. They build services without layers. The training data is stale or conflicting, and the API surface is large enough that guessing usually fails.
There are several ways to fix this. effect-first is a curated plain-text reference .. 15 endpoints, ~12k tokens total .. with rules, examples, and anti-patterns designed specifically for agents. effect.solutions (Kit Langton) is a high-level prescriptive guide for writing idiomatic Effect .. comprehensive and well-structured, but explicitly not designed for agents. opensrc (Vercel Labs) takes a different approach entirely: clone the actual library source to disk and let the agent read it.
Which helps most? We ran all three against a bare baseline on gpt-5.2.
The Three Approaches
EFFECT-FIRST (~2,800 TOKENS)
A Cloudflare Worker serving plain text. 9 rules, copy-paste examples, anti-patterns. Designed for token efficiency .. agents fetch only what they need. The content encodes the conventions an experienced Effect developer knows: Effect.fn for all named functions, globally qualified Context.Tag IDs, layers composed at the entry point, Schema.Class for records, branded types for domain primitives.
EFFECT.SOLUTIONS (~10,000 TOKENS)
A prescriptive guide by Kit Langton covering basics, services & layers, data modeling, error handling, and testing. Comprehensive, well-structured, with code examples. Explicitly not designed for AI agents .. it's a human-readable field manual. We used it to test whether human-oriented docs work as well as agent-oriented ones.
OPENSRC (~7,700 TOKENS OF EXTRACTED SOURCE)
A Vercel Labs CLI that clones library source to disk. For our benchmark, we extracted the export signatures from Effect.ts, Context.ts, Layer.ts, and Schema.ts .. the same files an agent with filesystem access would grep. Raw type signatures, no usage guidance.
Three design philosophies. effect-first says: tell the agent how to use the library, tersely. effect.solutions says: teach the developer the patterns, thoroughly. opensrc says: show the agent what the library is. Recipe, cookbook, raw ingredients.
The Experiment
Four conditions. Five tasks of increasing complexity. Deterministic regex judges.
Baseline: task prompt only. No additional context.
effect-first: /rules + /examples (~2,800 tokens) prepended to the task.
effect.solutions: basics + services + data modeling + errors + testing (~10k tokens).
opensrc: extracted source signatures (~7,700 tokens) prepended to the task.
The judges check structural compliance: does the code use Effect.fn? Schema.TaggedError? Is the Context.Tag ID globally qualified? Does it avoid async/await, try/catch, throw new Error, Promise constructors, and .then() chains?
Methodology Note
Regex judges verify idiom compliance, not semantic correctness. Code that passes every rule might still not compile. We're measuring whether the agent uses the right APIs .. not whether the program works. The opensrc condition uses realistic extracted snippets: the same export signatures an agent would find by grepping the cloned source.
Model: gpt-5.2 (Dec 2025). Temperature: 0.2. Single run per condition. The benchmark harness is open-source.
Results
Five tasks, four conditions, scored by percentage of judge rules passed.
Effect.fn, Effect.gen, NodeRuntime.runMain + 5 absence checks
TaggedError, catchTag + 5 absence checks
Context.Tag, Layer.succeed, global tag ID, testLayer, it.effect + 5 absence checks
Schema.Class, brand, encode, decodeUnknown + 5 absence checks
All rules: TaggedError, Class, brand, catchTag, Context.Tag, Layer, Config, resilience, it.effect, testLayer + 5 absence checks
gpt-5.2 aces the first four tasks at baseline. It already knows basic Effect patterns well enough to score 100% on hello, errors, services, and schema without help. This is a major shift from gpt-4o-mini, which scored 56–58% on services and schema.
In this run, the opensrc condition scored 25 points below baseline on services and schema. The added context contained raw type signatures from Effect.ts, but this experiment does not isolate which part of that context caused the lower score.
Both effect-first and effect.solutions caused a minor regression on tagged errors (−10%). The model already knew this pattern perfectly, so the extra reference material introduced noise instead of useful context.
On the full-stack task, baseline scored 76%, effect-first 95%, effect.solutions 90%, and opensrc 76%. The two curated guides scored higher in this single run. More runs and stronger judges are needed before treating the difference as a general effect.
Why Raw Source Isn't Enough
The opensrc condition scored lower on two tasks and matched baseline on the full-stack task. The task outputs suggest several possible explanations, but the experiment was not designed to separate them.
TYPE SIGNATURES ≠ CONVENTIONS
The source shows export const Tag: <const Id extends string>(id: Id) => .... It does not say "tag IDs must be globally qualified as @scope/Name." It does not say "always create a testLayer alongside your live layer." The rules that matter most are conventions .. and conventions aren't in the type system.
Signal-To-Noise Ratio
The opensrc context was ~7,700 tokens .. nearly 3x the effect-first context. But most of that is generic type machinery: overloads, conditional types, internal generics. The signal (how to use the APIs) is buried in noise (how they're implemented). More tokens, less useful information per token.
No Negative Examples
effect-first includes anti-patterns: "Never use async/await for effectful code." "Never throw .. use Schema.TaggedError." Raw source can't tell you what not to do. And telling a model what not to do is cheap and highly effective .. every absence check passed in every effect-first run.
This matches what we know about in-context learning. Min et al. (2022) found that demonstrations in prompts primarily teach the model about the format and label space of a task, not the underlying reasoning. effect-first is doing exactly that: correcting the vocabulary and conventions. opensrc provides implementation details the model can't easily translate into usage patterns.
The Effect.solutions Question
effect.solutions is the most interesting comparison because it's the closest in spirit to effect-first. Both are curated, prescriptive guides. Both cover the same territory: services, layers, schemas, errors, testing. The key differences are audience and token budget.
Designed for Humans
effect.solutions includes explanatory prose, multiple code examples per concept, inline comments, and progressive disclosure. It's a teaching document. At ~10,000 tokens, it's 3.5x the size of effect-first's treatment context.
Designed for Agents
effect-first strips to essentials: terse rules, copy-paste patterns, anti-patterns as corrections. No explanation of why .. just what. At ~2,800 tokens, it leaves more room in the context window for the actual task.
On the full-stack task, effect-first scored 95% and effect.solutions 90%, while using about 29% as many context tokens. That is an encouraging result from one run, not enough evidence to rank the guides generally.
effect.solutions wasn't built for this. Kit Langton has been explicit about that. It's a field manual for developers, yet its structure also gave the agent useful context on the full-stack task. Well-structured human documentation is therefore the baseline that agent-specific formatting has to beat.
Open Questions
One model, one run per condition, regex judges. Directional, not definitive.
Opensrc With Agent Selection
We extracted source signatures manually. A real agent with opensrc access would search more strategically .. reading examples, tests, README. The opensrc condition here is pessimistic. Would an agent that can interactively browse the source do better?
Compilation Testing
Regex judges verify idiom compliance, not correctness. Adding tsc --noEmit would answer the harder question: does the code actually compile?
Token Budget
effect-first used ~2,800 tokens and won. opensrc used ~7,700 tokens and barely helped. Is the effect-first signal strong enough at ~800 tokens (/rules alone)? What's the minimum effective dose?
Combined Approach
What if you used both? effect-first for conventions, opensrc for implementation details when the agent gets stuck. The approaches aren't mutually exclusive.
Where This Leaves Me
The results surprised me. I expected a clean hierarchy: agent-designed reference > human docs > raw source > nothing. Instead:
- gpt-5.2 is strong at baseline. It aces 4 of 5 tasks cold. The era of models that can't write Effect is ending.
- More context can hurt. opensrc degraded performance on 2 tasks. Even the curated guides caused a minor regression on tagged errors. Injecting reference material has a cost.
- Curated guides dominate on complex tasks. effect-first +19%, effect.solutions +14%. Both far ahead of raw source (+0%) and baseline (76%).
- Agent-optimized beats human-optimized, slightly. effect-first: 95% in 2,800 tokens. effect.solutions: 90% in 10,000 tokens. Better score, fewer tokens.
- Raw source is actively harmful. opensrc made a model that was getting 100% drop to 75%. Type signatures without conventions confuse more than they help.
In this benchmark, both curated references scored above baseline on the full-stack task. effect-first used fewer context tokens; effect.solutions provided more explanation. The raw-source condition did not improve that task and scored lower on two others. I would repeat the runs before using those differences to choose documentation for another library.
The benchmark is open source. The reference is live at effect-first.coey.dev. Run it yourself.
Sources
effect-first reference server. effect-first.coey.dev
effect-first benchmark suite and harness. github.com/acoyfellow/effect-first
effect.solutions .. Kit Langton. effect.solutions
opensrc .. Vercel Labs. github.com/vercel-labs/opensrc
Effect TypeScript library. effect.website
Min et al. "Rethinking the Role of Demonstrations." EMNLP, 2022. arxiv.org/abs/2301.00234
Related
- effect-first.coey.dev .. The live reference server. Fetch any endpoint to see what agents receive.
- Context Cues .. Research on context window position, compaction, and what survives memory loss.
- Prompts .. The full prompt library, including the Effect integration prompt.