Mapping the ecosystem behind artificial intelligence

Since 2022 I write about complex biological and socio-technical systems. Click here to easily find all the posts about your topic of interest. Listen to our free podcast on Spotify and Apple.

AI is increasingly embedded in economic activity, education, scientific research and everyday information processing, faster than most institutions — and people — can adapt to it. Generative AI reached mass adoption within only a few years, faster than either the personal computer or the Internet, while organizations, schools, regulators and infrastructure providers are still adjusting to what these systems can do, as I have recently described in the post below:

I find it useful to think about this as a co-evolutionary process, where new AI capabilities modify how people work, search, program, communicate or produce knowledge. Those changes affect demand for particular capabilities, attract investment, create new computational infrastructure and generate incentives for another round of model development. Technology changes its social environment, while that environment influences which technologies are developed and deployed, closing the feedback loop.

However, behind the small number of AI products familiar to most users lies another rapidly changing system: the population of models from which many of those products are built.

Public repositories contain an enormous number of such models, and it is even possible to reconstruct their interdependencies, such as fine-tuning, adapters, quantization, merging and successive training checkpoints. Therefore, a model uploaded today can occupy one position in a much longer history of modifications originating from an earlier model.

Horwitz and colleagues have tried to make that history visible in a recent paper. In We Should Chart an Atlas of All the World’s Models, they organize a substantial part of the public model population as a network they call the Model Atlas, and the accompanying interactive atlas allows one to explore some of these families directly.

The reconstruction is conceptually simple:

• Each node represents a model at a particular point in its development;

• A directed link records a transformation from one model to another;

Merged models can descend from several parents, which makes the resulting structure a directed network rather than a conventional genealogical tree.
Information about architecture, task, performance, popularity or license can be attached to the nodes, while kinks can contain information about how the transformation was performed.

Several properties exist only once models are considered together. For instance, we can measure how large a descendant family becomes, how deeply a lineage branches, which transformations repeatedly occur around particular foundation models, and whether different areas of machine learning modify their models in similar ways.

A conventional repository makes most of this difficult to see, since the information is distributed among model pages and, often, it was never documented in the first place.

Horwitz and colleagues therefore also discuss model forensics: reconstructing where a model came from and which transformations contributed to its current state. An interesting byproduct of this approach is that the same structure could improve model discovery, because ancestry and functional properties offer information that simple text search cannot recover1.

The atlas begins from Hugging Face metadata, including declarations about which base model was used to produce another model, and allowing related models to be grouped into families.

It is worth stressing that much of the underlying history is missing, because model documentation might be highly incomplete, including basic information about parent models, performance and other attributes. The observable network therefore contains gaps that do not necessarily correspond to real breaks in the genealogy.

Those missing relationships can be inferred from the models themselves, though. Similarity between weights provides one clue, while upload times provide another; other features such as quantization, sequential checkpoints, hyperparameter searches and model merging also leave characteristic structural patterns that help distinguish plausible relationships2.

However, I will leave the reconstruction algorithm aside here.

Once thousands of models are represented together, distinct development practices become visible.

The language-model region is particularly deep and heterogeneous: especially quantization and model merging appear frequently, producing complex families around widely reused models. Computer-vision models show different patterns, with generative image models that have progressively adopted parameter-efficient adapters such as LoRA, while conventional full fine-tuning remains prevalent in other parts of vision. Audio models show another distribution of adaptation strategies.

However, these differences require some caution, because the atlas describes a public repository with incomplete documentation, and the communities being compared differ in model architectures, computational costs, tasks and software ecosystems. A network pattern does not — and likely cannot — identify its cause, but it can turn differences in engineering practice into something that can be measured.

Let us consider model merging. The atlas shows that merging has been adopted much more extensively around language models than around vision models. From individual model releases we can observe particular examples of merging: from the population we can identify where the practice concentrates, which families generate merged descendants and whether its use changes over time or not.

There is also an important distinction between genealogical proximity and functional similarity. Models descended from the same ancestor may become behaviorally different after training, while unrelated models may eventually acquire similar capabilities. The current atlas emphasizes ancestry and transformations, while performance, behavior and other measurements can be layered onto that structure. That separation could become especially useful as the model population grows.

Most machine-learning evaluation concerns individual systems: their accuracy, capabilities, robustness, efficiency or failure modes. Those measurements tell us little about how a population of models develops through repeated reuse.

The network adds variables defined by the relations among models: some ancestors generate large families, some lineages accumulate many rounds of modification, particular transformations concentrate in specific communities. The topology records part of that collective history of model development.

This approach is similar in spirit to the study of citation networks. A scientific paper can be studied through its contents, while a citation network reveals how ideas and communities are organized across thousands of papers. The network itself does not replace analysis of individual papers, but it can answer questions that cannot be formulated from one paper alone.

Here, individual AI models remain the objects whose capabilities we ultimately use and evaluate, while their network of ancestry records how the technological population producing those capabilities is organized.

Note that the present atlas is necessarily partial, as explained above, and even the authors explicitly present charting the complete Model Atlas as an open research problem rather than a finished map.

Still, the experiment provides a useful level of description for a technology that is changing very quickly: as AI co-evolves with economic activity, institutions and human behavior, the models supporting that process are also accumulating their own history of descent and modification.

Mapping that history gives us a way to study how the machinery behind contemporary AI develops as a population, rather than encountering its products only one model at a time.

→ Please, remind that if you find value in #ComplexityThoughts, you might consider helping it grow by subscribing, or by sharing it with friends, colleagues or on social media. See also this post to learn more about this space.