What Happened to Cloud Labs?

Imagine if there was a way to run a scientific experiment the same way you would write a line of code. You hit enter on your keyboard and a sequence of robotic activities process biological samples, such as DNA or stem cells, to generate experimental data without any user intervention. Imagine that such a system could service hundreds of user requests in parallel to run a huge variety of wet-lab experimental protocols on a broad range of laboratory instruments. If such an idea were successfully implemented, it could change the entire biotech industry by decoupling the ability to pursue novel hypotheses from the need to invest in expensive equipment and floorspace, hire technicians, and manage the day-to-day logistics of lab operations. It would solve the pervasive frustration that highly educated professionals working at the forefront of biology research sometimes have for spending a huge amount of time on repetitive manual labor.

If you are really optimistic about the increasing power of large language models, a platform like the one described probably sounds like a dream come true. A highly intelligent AI could interact with this platform to design and perform its own experiments, autonomously collecting data in pursuit of self-driven research. Leveraging the inhumanly abundant knowledge and reasoning abilities of an LLM in combination with a highly capable experimental platform would result in a powerful engine of scientific progress. In fact, the biggest advocates of the world-changing potential of AI almost always cite an interface between AI and automated scientific experiments as a path towards accelerating technological progress (often without expanding much on the details). So who is working on this?

Well, a few companies have been trying for over 10 years. The results have not lived up to expectations. Emerald Cloud Labs and Strateos are the main companies operating in the space known as “cloud labs,” labs which provide the capability for remote execution of a broad range of experimental protocols. Users interact with these remote execution platforms by writing code that translates to a set of real world instructions. Some of these instructions are sent to robots and automated lab equipment, and others are sent to human operators to fill in the gaps. The platforms are intended to run a highly diverse set of experiments including mammalian cell engineering, enzyme screening, and protein purification. In theory, the possibilities should be endless, but the excitement over what seemed possible resulted in many overlooking fundamental obstacles that would almost guarantee that such an expansive vision could not be fulfilled.

To put it simply, wet-lab biology entails a ridiculously and unintuitively broad set of techniques and protocols. Even a single high-impact paper will likely involve many unique experimental workflows that each require specialized and expensive lab equipment. The workflows typically require the utmost degree of care and precision in order to ensure the accuracy of results. The proposal that a single facility could encompass such a wide range of capabilities such that scientific users could expect to execute entire pipelines “in the cloud” is not realistic with current technology. I’ll break down the specific reasons below.

Capital expense

Consider automating an experimental pipeline (such as the one described in this Cell Immunity paper) that investigates signal transduction in cancer-fighting immune cells. This is a very popular class of experiments that will probably be relevant to the biotech industry for many years to come. The equipment needed to run this experiment includes a confocal microscope, a flow cytometer, a mass spectrometer, an incubator, a freezer, a centrifuge, and a spectrophotometric plate reader, as well as other less expensive instruments like thermal cyclers. We will also need a liquid-handling robot for dispensing reagents and samples, and robotic arms for moving plates and tubes between equipment. All of the above equipment would likely entail a price tag of about $5 million, and this is just for one experimental pipeline.

Caveat: Instead of having a set of equipment dedicated to one type of experiment (how automation is typically used in biotech), cloud labs share equipment across many different types of experiments, similar to how a server can handle requests from many different clients. This leads to cases where an experiment requires bringing a sample from one instrument to another on the opposite end of the lab, since the facility was not planned with that (or any) particular experiment in mind. Which brings us to the next problem.

Reagent and sample handling

Almost all experiments run in a cloud lab will involve samples being moved between shared equipment, and there is currently no way of moving arbitrary containers of samples between any two distant instruments other than by using humans. This means that every experiment will rely on a human technician interpreting a set of computer-generated instructions to interact with one of perhaps hundreds of automated instruments to perform a step in an experimental protocol for which they have essentially no context. A mistake could result in the failure of an entire experimental protocol, potentially in a way that isn’t even detectable by the remote end-user. A cloud lab technician won’t have any context for interpreting the physical condition of an experiment, recovering from errors, or knowing if a step in a protocol has failed.

Interpretability

When you run an experiment in a cloud lab, you have very limited access to information about how exactly your experiment was performed. It’s very common for biology experiments to fail the first (or hundredth) time, and these failures are frequently due to subtle factors like mixing speed or volume inaccuracies during pipetting. Often, failures can only be diagnosed through direct visual observation in combination with deep contextual knowledge of the experiment. This means that for the vast majority of experiments which do not succeed the first time, it can be extremely difficult for a remote cloud lab user to diagnose why. If an experiment fails because a reagent is contaminated or expired, or an instrument is out of calibration, a user usually won’t have the capacity to detect this or troubleshoot errors, and a lab technician might not even be aware of the failure at all.

Protocol Design

Creating an automated version of a manual wet-lab protocol is a pretty tricky business for many of the reasons above. It is easy to underestimate how much tacit finesse is required to perform a biology protocol successfully. Wet-lab scientists can use their hands, eyes, and years of experience to perfectly calibrate how a pipette aspirates a sample. This fine-tuned calibration is very important because biological samples are often sensitive, have unique physical properties, and are only available in very low volumes. Automating a wet-lab protocol is not just about writing code, it means painstakingly formalizing all of the implicit micro-actions and sensory feedback that an experienced biologist would “just know” to perform. In a cloud lab, the protocol design is done fully remotely by a user who has no ability to observe all of the different actions performed in the cloud lab. The process of optimizing an automated protocol through trial-and-error, which is how biotech automation is done in every other case, is completely omitted.

Planning and scheduling

The task of orchestrating the flow of experiments through a cloud lab in a time and resource efficient manner is a unique software challenge. There are big differences between this kind of scheduling problem and typical examples such as shop-floor scheduling. One is that the timing between certain steps is very sensitive for many biology protocols. This can be disastrous for situations where a sample has to be delayed from moving onto the next step in a protocol due to random malfunctions or heavy sample traffic. In such a complex system running many experiments in parallel, it is inevitable that some protocols will suffer from extended and unpredictable wait times. Another problem is that in cases where the schedule has to be adjusted, such as due to an equipment malfunction or operator error, it might be difficult to recalculate a new schedule for all of the affected protocols due to the timing dependencies between steps. The unpredictability of a system that has to service many different types of experiments in parallel on shared equipment can make it very challenging to depend on for sensitive experiments.

Final thoughts

Cloud labs are very complicated propositions, and the reasons for why they are borderline impossible to create with current technology are more numerous than what I explained here. I think a lot of these problems can be individually addressed, but before they are, I believe it’s very unlikely for cloud labs to be a worthwhile business model. Here are some lessons I think can be applied to future attempts at really ambitious biotech automation platforms.

• Bet on specific pipelines. Do not over-generalize.

• There is no way to decouple scientific judgement from the execution of experiments.

• There is always more going on in an experiment than can be captured by a finite set of automated instrument measurements.

I am overall really optimistic about the future of automation in biology. It seems like there are a lot of people who are really interested in these kinds of platforms, and who are willing to make huge bets on building scalable automation infrastructure. I am fully on their side in that goal, and hopefully these bets will have a higher chance of succeeding if we can learn lessons from an earlier round.