“The Middle Point Between Researcher and Engineer” – Alan Aboudib on AI, Bittensor, IOTA, and the Future

Sat in a restaurant, a coffee by his side, the white mountains of the French alps resting behind him through the window, Alan Aboudib chronicled his life to me. His time earning a PhD in AI, how he conceptualises ideas, his involvement in Bittensor, and work on SN9 (both its current IOTA era, and its former Pretraining time).

As the AI Research Lead at Macrocosmos, his expertise and insights facilitate our machine learning success. He’s been instrumental in our recently published work where we created a new transformer-based architecture to facilitate frontier AI model training in low-bandwidth, high-latency environments: ResBM. It took less than five minutes for me to truly grasp the level of intellect sitting across the table. Alan was also imperative to our Orion-100B distributed pretraining run, a major milestone for IOTA.

No two academics are alike - the way you orient yourself in the educational world determines much about your character, and your trajectory. While many PhDs are purely into the theoretical, Alan was precise in how that uniform never really fit. “I enjoy thinking about new ideas and new frontiers in AI, but I like coding, I like building and getting hands-on.”

When detailing his position in the field, he described himself as “the perfect middle-point between researcher and engineer.” That hybrid is the great separator between those who possess and expand our knowledge of AI, and those who both engage in this and apply it in practicality.

Despite feeling “most intellectually at home in research”, that desire to give new ideas form was not as strongly found in a modern day academic route as it was in the early days of the deep learning revolution where companies still needed to do their own research which was mostly on real-world problems. Working on AI at Macrocosmos gave him something institutes often struggle to: the opportunity to write world-class research papers whilst testing their hypotheses in a live environment.

Alan envisaged a role in AI, just not distributed AI. When he first entered the field, the deep learning revolution, sparked by the landmark AlexNet paper, had just begun, though it hadn’t yet reached the heights we see today. “I did my PhD and postdoc at Collège de France, very well known in France,” he told me, “and my research stay at Brown University, an Ivy League institution highly engaged in AI. During my research stay I was with Thomas Serre, inventor of HMAX, just a few years after it was state of the art image recognition.”

Rather than in the pre-deep learning era, Alan was operating right on the cusp of the shift. While HMAX was a hierarchical model of object recognition inspired by biology, attempting to mimic the brain’s visual pathway as an early precursor to layered visual representations, the modern revolution was already knocking. At Brown, his work pivoted to the bleeding edge: figuring out how to integrate memory and attention into AlexNet. It was a remarkably prescient project; today, both memory and attention serve as the fundamental building blocks of modern LLM-based systems.

“It was very early days. At that time we were exploring what deep learning could be.” His academic journey further brought him to meet Jean Ponce, one of the rapporteurs for his PhD - a senior reviewer who read his academic work. Ponce is a key figure in computer vision (the process of teaching machines how to interpret images, who works at WILLOW a joint Inria-ENS-CNRS research team founded in 2007 by Ponce himself).

His PhD opened the way to his postdoc in Collège de France, where he worked under the supervision of Gérard Berry. Berry is a key figure in French technology, a member of the French Academy of Sciences and a holder of the Légion d’Honneur, the highest order of merit in France. Alan’s arrival came just a year after Yann LeCun had been teaching there. LeCun is about as close as you can get to a household name in the AI field, co-recipient of the 2018 ACM A.M. Turing Award, and founder of Meta’s FAIR (AI research) lab.

The gravity of that environment was evident immediately. “Before I officially joined for my postdoc, my supervisor Gérard Berry organized a private event,” Alan recalled. It wasn’t just any gathering; it was a closed-door conference with AI pioneers like LeCun, Yoshua Bengio (co-winner of the Turing Award with LeCun), and researchers from CIFAR [a Canadian research organization that anchors and shapes Canada’s national AI ecosystem]. “It was a private event, but I attended.”

Looking out the window, thinking back on it, he recalled a specific anecdote from that day. “I asked the person next to me if they were a PhD student. They replied ‘I’m one of the inventors of VAEs’”. Variational Autoencoders are a type of neural network that can learn a compressed representation of data and generate new, similar data - a milestone in generative AI.

“These people are rockstars now. But this was the beginning. They were gaining some recognition, but it was early days. I was surrounded by these types of leaders. I was excited. I was curious. I was just getting out of my PhD and I was ready to absorb what they had to say. I’d sit with them and just observe how they debated deep research ideas. It was enlightening seeing what types of questions they asked each other - what interested them when reading a paper. It’s an observant kind of learning. A different type of education.”

Learning about AI was not enough. It’s a beautiful field, and one moving at such a lightning pace, but to learn is not the same as to partake. Alan, surrounded by such instrumental figures, possessed a hunger to participate - specifically at the intersection of AI and decentralisation. But Bittensor was far from his first exposure into this niche. Back in 2019, he began organising the Paris meetup for OpenMined, a research organization founded by Andrew Trask, an Oxford PhD who was also working at DeepMind at the time.

OpenMined’s mission to democratise privacy-preserving, decentralised AI resonated deeply with Alan, leading him to head up their natural language and cognition team. It was there he created SyferText, a decentralised natural language library so effective it was eventually absorbed into the widely used PySyft library.

That deep-rooted expertise made his eventual transition to Bittensor a natural progression. Having already spent years architecting decentralised AI, he recognized the potential of what the team was building. “I joined Macrocosmos because I thought the work was cool. Using AI in a decentralised space interested me,” he recalled. What was particularly striking was hearing how Alan had come to navigate the Bittensor ecosystem specifically, and the intense interplay between its users.

“It’s an aggressive space. You can have multiple teams working on the same topics going against each other. But it’s also collaborative, you want people to be part of a collective”. Alan has an especially intimate viewpoint on this. His time at Macrocosmos has focused primarily on Bittensor’s subnet 9. Its current iteration, IOTA, is a collaborative distributed training architecture, where miners across the globe work together to build the same model. However, a long time ago, subnet 9 operated under a different set of goals.

It used to be a subnet where individuals would submit their own pretraining models and compete against each other for the highest rewards. It was a winner-takes-all setup, meaning only the top scorers would earn. Back then, there was fighting, both between miners and between other subnets who had the same goals. Bittensor’s built on this push-and-pull dynamic.

What became present was those challenges fed Alan’s curiosity. Those constraints set new conditions. However, there’s a difference between a challenge born from stress, and one born from connectivity. “When I’m passionate I produce my best work. Being at Macrocosmos, I have the opportunity to be an engineer and to be a researcher, and that’s not a luxury afforded to all. I can focus on the areas that interest me, and push them forward.”

That ideology: that harmony brings innovation, is what drives IOTA in its current day. It’s felt both by the architects behind it, and the miners operating on it. Everybody works in unison to build the same models, all reaping the benefits. It’s a collective striving for a distributed future.

Even when SN9 was a winner-takes-all arena, Alan noted that even though the environment was more adversarial, it was from a state of peace that the team were able to stay ahead of the curve. “Back in the old SN9, we didn’t train models like we do now. But we knew what the correct recipe was for the best models at the time. It’s not dark magic, there’s a formula to it. We were playing the scientifically correct games to succeed. For instance, we evaluated miners based on their data-mixing, which improved their benchmarks”. This is where data within a set is labeled specifically, and where each model receives a specific percentage of topic-data within its pretraining, a method that’s proven to create robust models. The peace of knowing you can spend time thinking deeply on a problem gives you opportunities to experiment with different avenues.

I was granted a door into how he thinks about problems. He’s a visualiser. He builds these rich systems in his mind’s eye, used to conceptualise his ideas. “When you’re teaching as a PhD student you’re taught to use analogies to explain concepts to people. But I don’t make them for others. I create analogies primarily when I need to understand something. I’m not satisfied with theorems or mathematical solutions. It’s too nebulous. I need intuition. I need to create a mental image and manipulate it” he says as he slowly rotates his hand.

That’s the mindset he brings to IOTA. Before detailing the specific bottleneck the team is currently working on, he clarified exactly what this subnet does: training large language models on a supercomputer built over the internet. However, utilizing the internet introduces a unique constraint; one that must be solved to enable any serious, large-scale LLM training. “Let me break it down for you. State of the art AI models all use transformer architecture. ChatGPT, Gemini, Llama, Qwen, they all implement it. Each neural network has layers, and in each layer they have transformer blocks. Every time a layer sends an output, it becomes an input to the next layer. This input, we call a tensor, which is essentially a vector, or a collection of numbers that goes as an input”.

“Now, these collections of numbers, or tensors, are huge. They generally cannot fit onto a single GPU or machine,” he continued. “That means we need to cut the layers up into a few parts, to put some on one GPU, some on another, and so forth. To be clear, this cutting is done even in traditional supercomputers. It’s just that centralized supercomputers have incredibly high-bandwidth connections, called NVLinks and InfiniBand, to handle the transfer.” But IOTA operates without the luxury of a centralized data center. “With IOTA, we’re training a model across the globe, so we have to use the internet. And these tensors are simply too slow to transmit over standard internet speeds.”

“If we were working in a centralised setting we wouldn’t have that problem. The systems would be low latency and high bandwidth, built of data centres. These centres are hundreds or even thousands of times faster than the internet”. He paused to give me a moment to assimilate.

“What we do with IOTA is we cut the models up, and distribute them across our miners. We’re treating the internet like one giant compute cluster. The issue is that the internet’s speed means we need to find ways to optimise that transmission process. The machines used to train belong to people around the world, with different specifications. You want to transmit this tensor from one machine to another at a reasonable speed. But it’s not easy. In order for this to work over the internet, we have to redesign the LLM architecture from the ground up in a decentralised-native approach. We worked on our own architectures that we call bottleneck networks.”

“In truth, it’s a problem we’re in the process of solving. Very few are trying. We knew this was an issue before we launched IOTA. But we launched it as a kind of trust within ourselves. We believed we could do it.” He takes a sip of his drink. “Let’s get back to transformers. These are neural networks designed for centralized systems. If we use them for our decentralised ones we will inevitably run into some problems - at the very least it won’t be efficient. But what if transformers could be designed as decentralised native? So it was my strategy to design them from the ground up. Of course, I didn’t want to reinvent the wheel, I took a transformer architecture and rethought how the architecture operated with the internet in mind.”

“I asked myself, how would the original inventors build the transformer if they didn’t have a centralized composition? How would they change the architecture? That was the intuition strategy”.

In case it wasn’t obvious, what Alan’s working on is far from trivial. He’s taking an architecture built precisely for networks with high connectivity where all machines are located geographically near, and rebuilding it for a network that spans the entire world which is conceptually the same yet fundamentally different. That’s profoundly complex. His work was recently published on arXiv, where he and the IOTA team created a new transformer-based architecture (ResBM). This is what makes IOTA operate.

Figure: Bottleneck layer placement across a pipeline-parallel communication boundary.

Attached closely to the bottleneck problem is the role of identity mapping. “This is a cornerstone of today’s deep-network design,” Alan said. In plain terms: modern architectures keep a shortcut route where the input is added back to the output (a “skip connection” or identity connection). That makes optimization easier and helps information and gradients flow through very deep networks.

In the context of distributed training, the key nuance is that compression is a separate mechanism - and if you insert a compression module (for example, an autoencoder bottleneck) in the wrong place, you can accidentally break that clean identity route. Alan’s point was that some compression schemes may be mathematically at odds with the residual/identity pathway that Transformers inherit from ResNets.

“However, you cannot make any type of compression,” he added. “You need to be careful about its properties - especially that you preserve the identity path.” In other words: the goal isn’t “zip it and push it,” but to design the distributed bottleneck so you get major bandwidth savings without damaging the architecture’s core residual structure.

The night fell. The cafe got louder. It was time for us to wrap up. The conversation drifted away from the hard science behind IOTA, and towards the broader landscape. I asked him what brings him joy in the AI space. What excites him right now.

“My answer is not technical. It’s about technical possibilities, but it’s not technical. We really are starting to live a life of science fiction. And we will live it more in the coming years. There’s some hype, and there’s some exaggeration, that’s for sure. But even when you strip that away, it doesn’t take from this reality. It’s coming. The future is actually coming this time.”

As we got ready to leave, he turned to me. “Now, that future isn’t amazing in all aspects. AI also negatively affects people’s lives. AI is replacing people. It’s good for businesses, but bad for people, and therefore it’s something we should all care about. We need to ask, who’s going to protect the people? Society is set up for productivity. You enter an agreement. You are productive, and in return you get some of the outcomes or benefits of the company’s productivity. What happens when the machines are more productive than us? Who gets the benefits? There may be a future where people cannot find work, and so they’re stressed because AI replaced them. It’s already happening. There may also be a future where people don’t need to work as much, because AI has replaced them. But for that, we need to consider something like productivity sharing. AI is built by ingesting human knowledge, so all humans should be benefactors. But nobody seems to seriously talk about it. Some entrepreneurs are talking about it, but the governments rarely do, which is weird.”

“In some countries, like France, we’ve started to reduce the social system that protects people. At the exact same time as these technologies attack their livelihood. All governments across the world have a duty to talk about this. AI is only exciting if we all get a slice of the pie. Otherwise people are getting harmed by it, and the standards of living will decay.”

We paid the bill. We walked out. It might sound like a somber note to end on, but I don’t see it like that. I didn’t suspect Alan did either. It was a realist’s view from the inside. To hold the elation of the future in one hand, and its sorrows in the other is a rare thing. Commonly, when people get so engrossed in their field or niche, they struggle to accept its underbelly. The joy overshadows the pragmatism. That’s not something Alan can relate to.

His final points framed his entire conversation. AI is meant for human flourishing. That’s what distributed technologies are trying to do: give people the autonomy to build their own collective systems that serve them. The field has an egalitarian streak, one that’s been slowly forgotten in the frenzy of continuous breakthroughs. But when developers remember this as their north star, that’s when we really benefit. Alan asked who’s going to protect the people? I don’t think he noticed that, as of right now, he’s one of them.