AI Is Starting to Decide Who Gets Considered... And the Internet May Never be the Same

I was reading the Montréal Declaration on Responsible AI this week, which is a sentence that makes me sound more emotionally stable than I was while doing it.

I had more browser tabs open than any ethical framework could reasonably justify, and the growing suspicion that every interesting AI question eventually becomes an ethics question wearing a product roadmap.

One line kept bothering me.

The declaration says AI systems should contribute to a just and equitable society. It also says they should be intelligible, justifiable, accessible, and compatible with social and cultural diversity.

All good. Hard to oppose without twirling a small villain mustache.

But I kept thinking about a quieter problem.

What happens when AI does not make the final decision, but decides who gets considered in the first place?

We are moving from a web where people search through options to one where AI systems increasingly synthesize and recommend them. [OpenAI now describes ChatGPT shopping as a place where people can compare options and figure out what to buy. Its commerce protocol is designed to ingest structured merchant data, understand inventory, and surface relevant products in context.

That sounds like a shopping feature.

It is also a preview of a much larger shift.

The agent does not need to reject you. It only needs to build a reasonable shortlist without you on it.

For most of the commercial internet, visibility worked roughly like this:

→ Search

→ list of links

→ human visits a few

→ human compares

→ human chooses

That process was never perfectly fair. Search rankings concentrated attention. Advertising bought access. Large brands accumulated links, reviews, data, and familiarity that smaller organizations could not manufacture on command.

But the options were at least visible as options.

The emerging version looks more like this:

→ Human states a need

→ AI interprets the need

→ AI retrieves evidence

→ AI assembles candidates

→ AI compares the candidates

→ human sees a compressed answer

I have written before that your next buyer might meet you as a shortlist before they meet you as a website. The more I sit with that shift, the less it feels like an SEO problem.

It is an access problem.

A small manufacturer may make the best object for a particular buyer but have incomplete product data. A local nonprofit may be unusually effective but explain its outcomes inside a PDF last updated by someone who has since moved to Vermont. A specialist consultancy may have deep expertise but little third-party coverage. A new company may solve the problem cleanly while an older competitor owns ten years of backlinks, reviews, comparison pages, and machine-readable proof.

The machine is not necessarily hostile to any of them.

It may simply have more usable evidence for someone else.

That distinction matters because legibility is not merit.

Legibility is the ability to be found, understood, compared, and justified by a system. Merit is whether you are actually good, relevant, trustworthy, useful, or right for the person asking.

They overlap.

They are not the same thing.

This is not an argument that AI invented inequality.

The old web was already easier for organizations with money, links, technical teams, strong distribution, and enough content to occupy every possible question. Search optimization has always favored people who knew how the system worked… or could at least afford to hire someone who did.

The difference is compression.

Ten blue links still allowed a person to wander, ignore the first result, notice an unfamiliar name, open six tabs, distrust all of them, and eventually text a friend.

A synthesized answer can do more of that work before the human arrives.

That is useful. I do not want to return to manually comparing 37 nearly identical product pages while each one announces that it is “revolutionary.” I would happily delegate more research to a system that understands my budget, constraints, taste, and unreasonable emotional relationship with objects made of old metal.

But convenience changes the location of the gate.

If the system retrieves only a narrow set of candidates, the user cannot evaluate the options that never entered the answer. The shortlist becomes an invisible border around the decision.

This is why machine-readable proof matters. Clear product records, current pricing, structured details, explicit fit, credible evidence, third-party sources, and dated claims give an AI system something it can use.

That is good marketing advice.

It is not yet an equitable information system.

There is an easy version of this article where I declare that AI recommendations automatically favor the biggest brands, cite one alarming chart, and spend the rest of the essay looking concerned near a graph.

The research is more complicated.

One recent audit of generative search in the Web3 domain found that larger follower bases and more concentrated creator networks were associated with higher-ranked citation exposure. The authors warn that this could entrench prominent voices, but they are also explicit about the limits: one domain, an external prominence metric, API-versus-interface differences, and exposure estimates rather than observed user behavior.

A broader study of generative search found something that cuts against the clean doom story… across several systems, generative engines often cited less-popular domains than organic search. The exact pattern changed by engine and dataset.

And a 2026 critical survey of 45 GEO studies concluded that the field still has inconsistent terminology, metrics, and evidence standards. Visibility depends on multiple partially observable stages: whether search activates, what gets crawled, what gets retrieved, where a source appears in context, whether it is cited, whether its information shapes the answer, and what the user eventually does.

In other words… we do not have enough evidence to say AI-mediated discovery always entrenches incumbents.

We have enough evidence to know the question is real.

That is the more honest boundary.

The point is not to announce that every small organization is being erased by a robot concierge. The point is that systems are beginning to mediate consideration at scale, and we still do not have mature ways to inspect who repeatedly disappears, why they disappear, or whether absence reflects poor fit, missing evidence, popularity, technical inaccessibility, stale information, or a flaw in the retrieval system.

A reasonable answer can still produce an unreasonable pattern.

This is the idea I cannot shake:

In an AI-mediated market, legibility becomes access to consideration.

Not guaranteed selection. Not guaranteed revenue. Not proof that the organization deserves to win.

Consideration.

The chance to enter the set of options that a person, buyer, donor, applicant, patient, student, or partner may evaluate.

That is why I think we need a concept I am calling AI Legibility Equity:

Legitimate organizations should have a fair chance to be accurately understood and considered by AI systems without needing the resources to manufacture an overwhelming volume of machine-readable authority.

There are two responsibilities hiding inside that sentence.

The first belongs to organizations.

If the facts about you are vague, stale, contradictory, inaccessible, or trapped inside a beautiful page that only makes sense when viewed as a whole, the system has less to work with. You cannot demand accurate representation while treating product truth like a scavenger hunt.

The second belongs to the systems doing the selecting.

They should not quietly treat popularity as quality, documentation volume as legitimacy, or missing machine-readable evidence as proof that an organization is irrelevant. They should expose sources, preserve uncertainty, distinguish absence from poor fit, and make consequential recommendation patterns possible to audit.

The Montréal Declaration gives us useful language for this: intelligibility, justifiability, equity, diversity, inclusion, prudence, and human responsibility. UNESCO’s AI ethics recommendation similarly treats inclusion, justice, diversity, access to information, and the digital divide as part of the ethical surface of AI.

Those principles should not begin only after an AI system makes a dramatic decision.

They should apply to the quieter decisions that shape the menu.

If you build, buy, audit, or market through an AI system that recommends people, products, organizations, services, sources, or opportunities, ask these seven questions.

Could the system consider only organizations with structured feeds, certain review volumes, strong domain authority, approved integrations, or coverage from a known group of sources?

The rule may be technically reasonable. Make it visible anyway.

Those are different states.

A company may be wrong for the user. It may also be relevant but poorly documented, inaccessible to the retrieval layer, newly launched, geographically underrepresented, or described in language the system did not connect to the request.

Do not flatten every absence into irrelevance.

Not with “AI-generated recommendations may be inaccurate.”

What evidence mattered? Which constraints were used? Which source established the feature, price, qualification, location, or outcome? What information was missing?

A shortlist without a reason is a ranking wearing conversational clothes.

Popularity can be evidence of familiarity, adoption, or social trust.

It can also be the result of age, budget, distribution, existing privilege, or being very good at publishing comparison pages about yourself.

Ask whether the system can recognize credible smaller sources, original evidence, domain expertise, and strong fit without treating raw volume as merit.

Can an organization update stale information, dispute a false claim, expose a current source, or understand why its record is incomplete?

If machine-readable reality becomes consequential, correction cannot depend on winning a support-ticket séance.

One reasonable answer proves very little.

Audit patterns across repeated queries, locations, languages, buyer stages, constraints, and user profiles. Look at who is retrieved, cited, named, compared, recommended, and omitted. Preserve the denominator. Separate what the system could not find from what it evaluated and rejected.

Fairness lives in the distribution, not the demo.

Can the user inspect alternatives, broaden the search, change the criteria, see uncertainty, or choose outside the generated set?

The goal is not to make the machine choose correctly on humanity’s behalf.

The goal is to help people make better decisions without hiding the shape of the choice.

There is a temptation to treat all of this as another optimization game.

Publish more. Add schema. Create comparison pages. Feed the machine. Become impossible to ignore.

Some of that is useful. Organizations should make their truth easier to find. They should maintain current product records, publish evidence, clarify fit, and give both humans and machines accurate material to work with.

But if the answer to unequal machine visibility is simply “everyone should become better at GEO,” we will reproduce the old internet with fewer visible links and more confident summaries.

The organizations with the most money will hire the most sophisticated teams to manufacture legibility. The people already fluent in the system will compound their advantage. Everyone else will be told the algorithm found the best options.

I want a better bargain.

Organizations should earn consideration with clear, current, credible evidence.

AI systems should make the conditions of consideration inspectable.

And humans should retain enough visibility into the process to notice when a convenient shortlist has quietly become the whole world.

That is not anti-AI.

It is taking AI seriously enough to ask what happens before the answer reaches us.

Reply and tell me: where do you already see AI narrowing the set of options before a person gets to choose?

-zc

P.S. If you want more practical essays on AI, human judgment, startup marketing, and the strange systems forming underneath modern work, subscribe to Don’t Feed The Algorithm.

- The Montréal Declaration on Responsible AI — ten principles for responsible AI, including intelligibility, equity, diversity, inclusion, prudence, and human responsibility.

- UNESCO’s Recommendation on the Ethics of Artificial Intelligence — a broader normative framework connecting AI to human rights, inclusion, access to information, and the digital divide.

- When Attention Becomes Exposure in Generative Search — a useful domain-specific audit of citation exposure and prominence, with unusually clear limitations.

- Characterizing Web Search in the Age of Generative AI — important counterweight to the simple incumbent-dominance story; generative systems sometimes surfaced less-popular sources than organic search.

- Optimizing Visibility in Generative Engines: A Critical Survey — the best current reminder that visibility is multistage, stochastic, and not reducible to one score or universal trick.

- Your Next Buyer Might Be a Shortlist — my earlier essay on the shift from search results to AI-mediated shortlists.

- The Agent-Readable Content Checklist — the practical companion for making evidence easier to retrieve without turning your writing into bot bait.