Keeping your analysis correct: mycelium and scilintr

Coding agents have completely changed the time it takes to analyze data. The time between having a question and having the answer used to be measured in days or weeks. Now it’s minutes. I can hand Claude Code a messy dataset and a somewhat-baked idea and watch it write the loading code, the QC, the model, and the plot. We have gone from analytic scarcity to analytic abundance, and the potential of that to change how we do science is enormous.

It also moved the bottleneck. When producing an analysis was the expensive step, that’s where attention was spent. Now that producing analysis is cheap, the expensive step is deciding whether to believe it. And from what I’ve seen, you should be *very* cautious in trusting these vibe-coded analyses.

https://github.com/arjunrajlaboratory/mycelium

https://github.com/arjunrajlaboratory/scilintr

Claude is an extraordinary software engineer, and that is precisely the problem. Its instincts are tuned on an enormous corpus of production software, where graceful degradation is a design virtue: catch the exception, return a sensible default, keep the service running no matter what comes down the pipe. Those instincts are totally appropriate for a web server—and completely wrong for data analysis.

For instance, hand an agent a statistical test that errors on some inputs and it will wrap the call in a `try`, return `NA`, and move on. Great software engineering, terrible analysis, because a failed test has now silently become a dropped sample and your p-value is computed over a different set of samples than you think. Give it a missing input file and it falls back to the most recent one it can find, gracefully publishing last week’s numbers. It fills a gap with a mean, eats up warnings, hardcodes thresholds that happened to work once. None of these are bugs in the software sense. For data analysis, these instincts are precisely wrong. You want to fail hard, flag everything, and stop completely.

The maddening part is that you can’t reliably instruct your way out of it. You tell the agent, in the strongest possible terms, never to swallow an error, and it agrees and then just does it anyway. Even if you fix it, it will just reintroduce the same pattern a few edits later, sometimes reverting the very fix you just watched it make, completely silently. It’s the “don’t think about an elephant” problem: a prohibition in a CLAUDE.md file is no match for behavior baked this deep into what good code looks like in the training data. The agent can’t help itself. Mycelium and scilintr are our answers to this challenge.

Mycelium is an open-source, MIT-licensed Claude Code plugin that wraps agentic analysis in the scaffolding it needs to be trustworthy. You install it and point it at a project:

claude plugin marketplace add arjunrajlaboratory/mycelium

claude plugin install mycelium@mycelium

# then, in any project, just tell the agent: “set up mycelium”

Here’s how it helps:

It steers the work toward rigor. /mycelium:analyze runs the analysis against a pack of defensive-analysis conventions: validation checks, sensitivity sweeps, null-hypothesis testing, and a long list of the silent failure modes above. The conventions ship as swappable packs (`robust-analysis` by default, with domain packs like `bioinformatics` and `image-analysis` one sentence away), so the agent is working against codified discipline instead of its own untempered instincts.

It interrogates the result before you believe it. /mycelium:review is the heavy artillery: a panel of agents, each with a different lens—statistical and causal inference, data pipeline and leakage, bioinformatics, LLM-specific coding antipatterns, documentation fidelity, code quality—that tear the analysis apart and hand back a prioritized report. It catches the things nothing else does. It is where you go when you are about to trust a headline number. The patterns it finds have come from some blood, sweat, and tears from mistakes caught in the past.

It writes the result up with its provenance intact. /mycelium:report turns a finished analysis into a structured document in which every number traces back to the code that produced it, rather than to a value someone typed into a draft and forgot to update. Also, the report is tuned to be easy to read while also having all relevant details.

And when you’re stuck, it helps you get unstuck. /mycelium:ideas brainstorms new directions through a set of disciplinary personas, a key strength of LLM ideation. It has helped me solve problems I couldn’t figure out on my own

Underneath all of that, mycelium also gives a project a structure and a memory: a consistent file layout with a manifest in every folder, and a .living/ layer that records decisions, learnings, and findings as you work and reloads them when you return, so the next session doesn’t start cold.

The deep review is extremely effective, but it has one real drawback: it’s heavy. It spins up many agents, burns a lot of tokens, takes a while, and on a large analysis it often has to run several times before it catches everything. Worse, by construction it runs at the *end*. You build the whole analysis, run the review, and discover that an important but erroneous choice was made back in step two. Now you have to run it all over again. Darn.

What I actually wanted was something to keep the agent on track *while it writes* — a guardrail cheap enough to run on every edit, that fires the moment one of these patterns appears, instead of a post-mortem once the analysis is already built.

Software solved a version of this problem a long time ago, and the tool is the code linter. A linter reads your code without running it, matches it against a catalog of suspicious patterns, and flags each one on the spot—fast, deterministic, no LLM calls, cheap enough to run on every minor iteration. It doesn’t prove your code is correct; it just keeps the known, mechanical mistakes from ever happening in the first place.

Ordinary linters check for *software* problems, and they have nothing to say about whether a line hides a *scientific* problem, because by every software standard those lines are fine. So I wrote scilintr, whose rule catalog is specifically about analysis failure modes. It lints R and Python analysis code against the patterns that are bad. Like, “try catch” for error handling, or wrong defaults or hard-coded sample IDs.

The design choice I’m most pleased with is who the linter is *for*: the primary reader is the agent, not me. A human-facing linter has to tune hard for precision, because people stop reading a tool that cries wolf. An agent reading a finding and deciding “fix or justify” costs almost nothing, so scilintr goes the other way: high recall over precision. Flag anything that *might* be a meaningful choice, and let the agent or its reviewer decide. When a flag really is fine, you don’t delete the rule or scatter suppressions. You write a waiver:

# ANALYSIS_OK[broad-exception]: wilcox.test fails only on all-tie input here;

# NA is the documented sentinel and is dropped with a logged count downstream.

result <- tryCatch(wilcox.test(x, y), error = function(e) NA)

The point is that it makes each such choice deliberate and explicit to the LLM.

Because it’s programmatic, scilintr is reliable, comprehensive, fast, and costs no tokens, which means it can run in-line: the agent writes code, runs scilintr, fixes what it flags, runs again, and only moves on once the file is clean or every exception is explicitly waived.

These two tools are meant to be used together. Scilintr is designed to catch the cheap, mechanical mistakes before they’re ever run. The review runs when you’re ready to believe a result and catches the structural and scientific problems a pattern-matcher can’t see.

What I have found in my own usage is that scilintr reduces the amount of stuff the deep review catches pretty dramatically, presumably allowing it to spend its attention more efficiently and comprehensively on the parts it is good at. Mycelium:analyze handles scilintr for you, you don’t even have to think about it.

There’s one more place the trick pays off: the write-up. A number can be computed correctly in the analysis and still go stale by the time it reaches the abstract, which is a very common problem. This numerical “drift” is a huge problem, and can affect conclusions dramatically. So there’s a sibling, `scitexlintr`, that lints the LaTeX source of a report against a manifest of registered values and figures and blocks the stale, undocumented number before it ships. Same principle as scilintr, but applied to reporting. I have found this tool also catches a surprising number of serious issues.

I’ve spent most of these words on guardrails, because that’s the part I reach for every day. But the reason mycelium is a framework and not just a handy set of tools is the other half of it: every project it touches accumulates a memory. The decisions, the learnings, the gotchas that harden into conventions get written down as you work and loaded back when you return — so a project, and eventually a whole lab, compounds what it knows instead of letting it evaporate session after session. That’s the reason for the name itself: mycelial networks move nutrients between trees to wherever they’re needed, and the eventual goal is projects that share what they’ve learned with their siblings rather than rediscovering it the hard way.

Of course, all of these things are still in development, and for sure much of it will change once myths and fables become reality :). Feedback very welcome!