What does the government think an algorithm is?

A few years ago, the British government decided that—due to the pandemic—it would be impractical to administer the A-level exams given to school leavers. They instead used an algorithm that determined each pupil’s predicted examination result based on a variety of factors including the student’s prior exam history, the historical performance of the school, and an estimated score from the teacher. The number of top grades given out to those attending independent schools (versus state-funded comprehensive schools) was significantly higher that year than it had been in previous years. Many students were incredibly pissed off about this, as some had lost their places on university courses.

Said students organised a demonstration outside the Department for Education in London. Their rallying cry:

Fuck the algorithm!

If an algorithm had meant that my plans to get into university or become a doctor or whatever had been completely undone, I’d be shouting fuck the algorithm too, because that’s a very bad algorithm.

The problem is the folk definition of an algorithm is rather more complicated than the strict computer science definition. In the strict sense, an algorithm is literally just a sequence of operations that are defined by reasonably precise rules. Long division is an algorithm. Calculating the dates of various religious holidays like Easter or Rosh Hashanah are algorithms that existed long before computers.

When the kids are shouting “fuck the algorithm”, or trade union activists discuss the perils of “algorithmic management”, or commentators worry about how young people’s brains are being poisoned through a feedback loop of bullshit (or “misinformation” as we now call it) on platforms like TikTok, they’re talking about algorithms differently than the way mathematicians and computer scientists do. Computer scientists and mathematicians talk about algorithms; politicians, journalists and wider society talks about The Algorithm. The spooky one that demonetises your YouTube videos, or incorrectly estimates your A-level grade, or channels toddlers towards a five-hour video called “FROZEN ELSA HUGE SNOT ARM BROKEN SURPRISE KINDER EGG SURPRISE PEPPA PIG FINGER GAME NURSERY RHYME 4K HD” that was created by a mysterious content farm for reasons nobody really understands.

And that difference would normally be fine. Words mean lots of things. We can probably come up with some kind of rough-and-ready definition of The Algorithm in the ordinary language sense, as opposed to the algorithms that are the daily concern of people who are cursed to have opinions about things like Tailwind, Kubernetes and open source supply chain security.

My rough stab at it? The algorithms people are concerned with are generally of a reasonably high level of complexity, are tasked with determining decisions that have sociopolitical importance, and often do so in a way that is opaque to the user.

Now, your socio-political ideals may vary (obviously), but there’s a pretty reasonable argument that governments do have a role in regulating algorithms, when those algorithms might end up causing injustice or unfairness. If you’re not convinced of this, I’ll pursue the standard approach of philosophers—namely, using an over-the-top example.

Let’s start with the assumption that you live in a country that forbids discrimination in the workplace on the basis of the usual categories (ethnicity, gender, sexual orientation, disability etc.) Then, say, your employer decided to handle your complaints of alleged discriminatory behaviour by a colleague in the form of a Dungeons & Dragons-style ability check. Was it sexism or just workplace banter? Let’s roll a D20 and find out. That kind of process would clearly be unfair to both the complainant and the accused. Compare it to the actual obligation that employers have in most places, namely, make a reasonable attempt to do a proper investigation of the allegation and then, if it is substantiated, take the appropriate remedial action as circumstances dictate.

(Incidentally, if the above example makes you think that “rolling a die” does not amount to a fair process for resolving, say, an employment dispute, well, do keep that in mind when government ministers who spend considerable amounts of time using ChatGPT start talking about how magical and amazing artificial intelligence will be in, say, fixing the ailing criminal justice system or playing Musical Chairs with whatever’s left of the NHS workforce.)

You can think of plenty more examples where either the adoption of an algorithm in place of careful considered thought, or an inappropriate algorithm, would be something society would reasonably find intolerable, and which governments might have good reason to seek to regulate. Some examples:

• is “dynamic pricing” of tickets (for concerts, sports matches, and airline flights) a form of unlawful price discrimination?
• could a company like Google use their significant market control in search to favour particular partners in an anticompetitive way?
• when do shady “dark patterns” breach consumer protection law?
• how far should online gambling providers be able to manipulate gamblers with narrowly-targeted, incredibly specific offers?
• could pricing algorithms become a form of collusion in a market?

(Many of these are from the excellent work that was being done by the Competition & Markets Authority. We shall have to wait and see how well they follow through on enforcing these.)

In some domains, then, there is a compelling reason to seek to regulate The Algorithm. You know, the one the kids were rightly incensed about when it magically generated their A-level grades.

But how does one regulate not just specific algorithms, but The Algorithm? How do we differentiate The Algorithm from mere algorithms? The problem is we only really define The Algorithm putatively. That is, we talk about things like the TikTok algorithm (when we mean content recommendation algorithms), or “that bloody dynamic pricing thing that made Oasis tickets cost £900”, or whatnot. Picking out what distinguishes algorithms from The Algorithm is really hard.

We can see an illustration of this in an excitingly named regulation passed pursuant to the UK’s Online Safety Act, namely The Online Safety Act 2023 (Category 1, Category 2A and Category 2B Threshold Conditions) Regulations 2025.

Some background is needed to understand this. The Online Safety Act 2023 regulates some “user-to-user services”—that is, websites, apps or other online stuff where users can talk to one another. It divides those services up into three categories. Search engines go into Category 2A. Everything else goes into Category 1 or Category 2B. If you are a Category 1 service, you have to do a bunch more bureaucratic paperwork than you would otherwise.

The intuition behind this is that Category 1 services are larger and thus have heavier burdens placed on them.

What, then, makes something a Category 1 service? The Online Safety Act delegates the making of this decision to the relevant Secretary of State. Behold the Threshold Conditions Regulations, which tells us:

The Category 1 threshold conditions are met by a regulated user-to-user service where, in respect of the user-to-user part of that service, it—

(a) (i) has an average number of monthly active United Kingdom users that exceeds 34 million, and (ii) uses a content recommender system, or

(b) (i) has an average number of monthly active United Kingdom users that exceeds 7 million, (ii) uses a content recommender system, and (iii) provides a functionality for users to forward or share regulated user-generated content on the service with other users of that service.

Or, to translate this algorithm out of legalese and into Python (in case you are more familiar)…

def is_category_one(
    content_recommender_system_used: bool,
    allows_forwarding_or_sharing: bool,
    average_monthly_uk_users: int,
) -> bool:
    if not content_recommender_system_used:
        return False
    if allows_forwarding_or_sharing:
        if average_monthly_uk_users > 7_000_000:
            return True
    else:
        if average_monthly_uk_users > 34_000_000:
            return True
    return False

The first question you might ask: “what’s with those magical numbers?” 34 million is roughly 50% of the current UK population, and 7 million is roughly 10%.

The important thing here is if you don’t use a “content recommender system”, you aren’t a Category 1 service. So what’s a content recommender system?

a “content recommender system” means a system, used by the provider of a regulated user-to-user service in respect of the user-to-user part of that service, that uses algorithms which by means of machine learning or other techniques determines, or otherwise affects, the way in which regulated user-generated content of a user, whether alone or with other content, may be encountered by other users of the service.

It’s an algorithm. Arguably, the drafters of the Regulations are trying to grasp for not just any algorithm, but The Algorithm.

Why is the use of content recommendation algorithms so significant? The Explanatory Notes are silent on this. The Impact Assessment does not even mention the word algorithm.

The Online Harms White Paper does set out the problem with algorithms at a number of points.

4. There is also a real danger that hostile actors use online disinformation to undermine our democratic values and principles. Social media platforms use algorithms which can lead to ‘echo chambers’ or ‘filter bubbles’, where a user is presented with only one type of content instead of seeing a range of voices and opinions. This can promote disinformation by ensuring that users do not see rebuttals or other sources that may disagree and can also mean that users perceive a story to be far more widely believed than it really is.

23. […] The regulator will also have powers to require additional information, including about the impact of algorithms in selecting content for users and to ensure that companies proactively report on both emerging and known harms.

3.22. As part of a movement towards greater transparency, companies should also work in conjunction with the regulator to build a shared understanding of the mechanics of their associated platforms or services. Where necessary, to establish that companies are adequately fulfilling the duty of care, the regulator will have the power to request explanations about the way algorithms operate. The regulator may, for example, require companies to demonstrate how algorithms select content for children, and to provide the means for testing the operation of these algorithms.

Okay, but what algorithms? The White Paper—forgivably, perhaps—only talks about algorithms in the popular and putative sense and does not really help me determine what kind of algorithms I need to be worried about.

Let me introduce you to a content recommendation algorithm.

SELECT *
FROM posts
WHERE draft = false
ORDER BY published_at DESC
LIMIT 10;

We’ll call this one most recent posts. It “uses algorithms which by means of machine learning or other techniques determines, or otherwise affects, the way in which regulated user-generated content of a user, whether alone or with other content, may be encountered by other users of the service”. If I changed the ORDER BY published_at DESC to ORDER BY RANDOM(), the content which you would encounter is different.

“Sure, but ORDER BY published_at DESC is clearly not what the government meant! They mean, as they said, filter bubbles and nasty TikTok stuff”, says the handily placed interlocutor.

Absolutely. That’s clearly the harm the entire legislative scheme purports to fix. But writing a legal definition that cleaves “the kind of clever machine learning stuff Facebook and TikTok do” from the ten most recent posts is really hard.

This may seem needlessly pedantic (I’ll take it as a compliment), but if I am operating a web forum that happens to have over 7 million UK users and I have a “share” button, I need to know whether I provide a content recommendation system in order to know whether I need to comply with the obligations imposed on me as an operator of a Category 1 service?

(Sure, Ofcom might just roll up and tell me I’m operating a Category 1 service. But the whole point of that rather unglamorous notion, the rule of law, is they do so based on some rules that are—to quote the late Lord Bingham, “accessible and so far as possible intelligible, clear and predictable”. Ofcom might make a mistake in their assessment, and I might want to take them to court to challenge this error.)

The technical weirdness is the main point of the recent case Wikimedia Foundation and BLN v Secretary of State for Science, Innovation and Technology [2025] EWHC 2086 (Admin). The applicants in the case operate a rather well-known website called Wikipedia, and were not successful in their claim (in part due to issues of standing).

I go back to what I said earlier. The hard bit of this is we don’t have a clear way of distinguishing ordinary algorithms from The Algorithm (the kind we might be worried about). The rough categories I suggested that distinguish the two are essentially complexity, opacity and importance. The A-level students shouting “Fuck The Algorithm” outside the Department of Education did so because the A-level grade estimation algorithm had a role that was really important in their life (it decided whether they could go to the university of their choice), and was operating in a complex and opaque way which they had no real control over.

It is perfectly reasonable to want to prevent the harms of The Algorithm. Whether we can do so in a practical and effective way is a really tough question. We can all provide a putative definition of The Algorithm by pointing to stuff we don’t like on popular online platforms. But writing laws is quite a lot tougher. (Who knew?) And finding a technical basis for differentiating between the TikTok recommendation algorithm and “get the last ten posts in reverse chronological order” turns out to be very difficult, as the Threshold Conditions Regulations demonstrate.