The experiment is over: we tried to outsmart the algorithm; here’s what happened

Over the past year, I have been treating Substack as a kind of black box: introducing one significant perturbation (i.e., a hybrid paid tier, without changing my editorial rhythm or closing new posts) and observing how our socio-technical system responded. This was an experiment involving an algorithm, a public space and a remarkably supportive community adapting together: the result was a sustained shift to a higher growth regime, strong enough to survive a formal counterfactual analysis, and a milestone that — at least to me — once looked almost unreasonable.

Dear readers,

At the end of July 2025 I switched #ComplexityThoughts from fully free to a hybrid model, with the promise that this would be an experiment, measured in public, and that I would report back. Following up, I received a great response by the community, and I have provided a mid-term report in December 2025.

On July 31, 2025, we had 2,515 subscribers and I set a somehow unreasonable public goal of 4,000 within one year, asking for almost 1,500 new subscribers: roughly +60%.
It sounds slightly insane even to me, but the ambition of that number was proportional to my educated guess that the platform could reward newsletters differently once they introduced a paid tier, even if the core content remained open.
I did not know if I was right, but at least the hypothesis was testable.

I was not really prepared for your response: hundreds of messages, where about 80% of the feedback supported the experiment. Many of you encouraged the attempt, defended open knowledge, suggested alternatives, or simply said: try it, measure it and tell us what happens.

You reminded me that #ComplexityThoughts was much more than a newsletter: it was already a community of curious, generous and engaged minds. That trust made the experiment possible, and here we go.

Insane or not, the 4,000-subscriber milestone was reached before the deadline (officially, on July 11, 2026). Since we were already growing before July 2025, the right questions to ask is:

How much of the subsequent growth exceeded what would probably have happened anyway?

I therefore built a counterfactual using only pre-hybrid data: since followers and subscribers move almost together, I modelled them jointly, validating before the transition, then freezing and projecting into the post-hybrid period.
Across the complete post-hybrid window, the estimated excess was:

• 2.93 additional followers per day, with a 95% interval of 1.56–3.91;

• 2.21 additional subscribers per day, with a 95% interval of 1.01–3.11.

In cumulative terms over nearly one year, this corresponds to approximately:

• 1,004 additional followers, with a 95% interval of 536–1,343;

• 757 additional subscribers, with a 95% interval of 347–1,068.

It is worth remarking that these numbers are the gains estimated beyond what the previous trajectory would probably have produced anyway, not just total gains.

The weekly growth became visibly larger after the transition, while the relative percentage growth appears less dramatic because the audience itself is now much larger.

The full history remains noisy and episodic, but the typical post-hybrid growth regime stays above our prespecified baseline.

I would rather say: I was not wrong.

The increase is sustained, practically meaningful and remains visible after discounting the growth already expected. Even the lower uncertainty bounds exceed one additional user per day for both followers and subscribers.

This does not prove that the platform’s reward mechanism is the only cause, of course. A counterfactual is still a model, not an alternate universe: we are modelers and we have to keep this in mind, since viral events, external shares, recommendations and platform changes also matter.

But the result is consistent with the original hypothesis: introducing a paid tier changed the conditions under which this publication could grow, without forcing me to close new knowledge behind a paywall or to change posting habits.

The forecast published in December was deliberately simple: reality exceeded both projections during several bursts, and the 4,000-subscriber milestone was reached before “our deadline”.

There is, however, one more conclusion from this experiment, and this one is much more personal.

Over the past year, I have learned that growth here is possible. I now have evidence of that. But I have also learned something about what sustained growth seems to require. In practice, the requests are rather clear: publish shorter pieces, publish very often, accompany them with a constant presence in Notes1 and participate actively in the social layer of the platform. Network effects matter. Sociality matters. And this is not necessarily a bad thing: when done well, intellectual exchange, recommendation and community can genuinely help good ideas travel.

But there is another side to this. Trying to counter the amount of smoke that is inevitably sold on a platform like this requires a continuous effort: more presence, more repetition, more participation in the game of visibility. And I have gradually realized that this effort probably goes beyond the mission of this newsletter.

I cannot really play that game, or, more precisely, I do not want #ComplexityThoughts to become optimized for playing it.

I care about sharing science and I care about taking my time to think about writing, especially when I believe there is something worth writing on complexity without forcing every idea into the unsustainable rhythm required by an attention economy. I am interested in network effects as a network scientist and I am grateful for the extraordinary network emerged around this newsletter. But I cannot — and do not want to — become a full-time slave competing for visibility inside the platform.

Maybe this experiment has also allowed me to clarify a boundary: we can grow and we now know more about how to do it, but growth is not the objective function I want to optimize at any cost.

After all, I am here to share science and thoughts about complexity.

As promised, we are not going not change the tested model: that was my agreement with you, one year ago. Accordingly, we have run the experiment, measured it publicly and reassessed it at the milestone, which was reached 3 weeks ahead. A magic prediction? Not really: I’d better say a plausible scenario with plausible hypotheses being tested.

Therefore, the new posts from #ComplexityThoughts will remain freely available when published, while paid subscriptions will continue to provide access to the whole archive and the long-term sustainability of this space: exactly as agreed with you.

However, the most important result, today, is beyond numbers: it is that this community trusted an uncertain experiment, challenged it and helped it succeed without asking #ComplexityThoughts to lose its original spirit.

Therefore, this project is no longer mine alone: it is ours. And this puts some (positive) pressure.

With deep gratitude,
Manlio

I have converted daily cumulative counts into weekly net additions, and then I used a joint dynamic-factor model to capture the shared evolution of followers and subscribers. Its structure was selected using an entirely pre-hybrid validation period and then refitted only through July 30, 2025. From that boundary, 5,000 joint no-hybrid trajectories were simulated.

We divided the data into three strictly separated periods:

• T1, from November 17, 2022 to January 29, 2025, for model training;

• T2, the 26 weeks from January 30 to July 30, 2025, for pre-hybrid out-of-sample validation;

• T3, from the hybrid transition on July 31, 2025 to July 8, 2026, for counterfactual evaluation. Only complete seven-day blocks were retained.

The difference between observed and simulated growth provides the estimated excess and its uncertainty.

The model was tested on six months of unseen pre-hybrid observations before being used for the counterfactual analysis.

Observed post-hybrid growth is compared with the expected joint trajectory had the previous dynamics continued.

The cumulative gap translates the daily excess into the estimated number of additional readers gained over the experiment.

Short windows remain uncertain; over the full period, the estimated gain becomes both clearer and operationally meaningful.