In the first chapter we make the case that AI agents could transform children’s healthcare globally and locally. We’ve seen the dramatic advances in consumer AI applications. AI applications are able to recognize common images with greater accuracy than humans. So how was this achieved?
In 2010, researchers at Stanford University launched the ImageNet competition. They assembled a database of 14 million images with 856 types of birds, 993 types of trees, and 157 musical instruments. The competition was to see how well a computer could be trained to recognize the images. It only took five years for computers to become more accurate than people.¹²³
As you can see in the figure above, Microsoft was already more accurate than a human in 2015. So, how was an AI application able to outperform humans? Neural networks. While neural networks have been discussed in research papers for a long time since the 1960s and 1970s, it was only recently, with advances in cloud computing, that these results could be achieved.
As the diagram above (courtesy of Jeff Dean, head of Google Brain) shows, the basic principle is simply that the more computing power and data provided, the higher the degree of accuracy neural networks are able to achieve. For those of you in major metro areas, this is the reason why you’ve likely seen cars equipped with lots of data-collecting cameras driving around town. What they’re doing is collecting as much real-world data as possible to more accurately train autonomous driving AI applications.
What is a neural network?
Neural networks can take different shapes and structures, but in general are modeled on our brain biology and the basic way that a neuron functions. A simple way to graphically visualize the basic building block of a neural network, a neuron, is shown below.
Here inputs (x), are multiplied by weights (w) and summed. That number is then passed through an activation function that results in an output, which can be fed to another neuron. Image data can be represented as a matrix of pixel values, which can be fed as input x) to a neural network for image classification.
A neural network is organized into layers of neurons. Deep learning occurs when more and more layers (hidden layers) are added beyond the first layer. Models can have hundreds of these hidden layers.
Executing a neural network (aka an AI application) is merely carrying out this computation on any text (e.g. a query to Claude or ChatGPT), image (e.g. Focal Cortical Dysplasia diagnostic) or voice (e.g. Siri).
Of course, the magic is how do we train an accurate neural network. Training a neural network is done by first taking a data set and dividing it into a training set and a test set. The training set is used to determine the various weights (w) and number of layers to achieve the objective. Once you think you have the level of accuracy you want, you then run the test data set to see if you have indeed trained the neural network.
In the case of ImageNet, information from individual pixels causes neurons in the first layer to pass signals to the second layer, which then passes its analysis to the third. Each layer deals with increasingly abstract concepts such as edges, shadows, and shapes until the output layer attempts to categorize the entire image. Each team in the competition made many choices, including the structure of the number of layers and the weighting function, to achieve the increasing degrees of accuracy shown in the competition.
Advances have been based on a centralized AI architecture, which assumes there is a large centrally managed database of images. In the case of ImageNet, the database is 150GB, containing 4 million images. As a central repository of a vast number of annotated images, which have already proven their worth in enhancing neural networks’ ability to accurately recognize and identify objects, it’s no wonder there’s interest in applying lessons learned from ImageNet to other AI-driven image recognition opportunities. In healthcare, giving access to a large number of ultrasounds, MRI, CT or X-ray images could similarly serve to improve the accuracy with which we can recognize and diagnose even rare conditions. Efforts are thus underway to build centralized architectures in the form of imaging data commons, image data repositories, or data lakes.
Makes sense, right? Well, yes and no. Yes, in the desire to gather up as much data as possible. But the sourcing of health care data unfortunately isn’t easy. In fact, data used to train adult AI applications comes from only a few states in the United States and a very limited set of countries in the world. Three states (California, New York, and Massachusetts) provide 70% of the training data, and you can guess the data is not from rural California.⁴
Globally, two countries (US and China) provide 50+% of the data used to train AI applications; only eight of the remaining 193 countries supply the rest.⁵
The result is that many AI applications lack accuracy in a wide variety of populations, or as a recent paper⁶ concluded: “Nearly all of the studies published in the study period that evaluated the performance of AI algorithms for diagnostic analysis of medical images were designed as proof-of-concept technical feasibility studies and did not have the design features that are recommended for robust validation of the real-world clinical performance of AI algorithm.”
In summary, centralized aggregation of larger data sets, coupled with scalable compute, has enabled neural network software to achieve increasing degrees of accuracy. But aggregation of large datasets in medicine has proven itself problematic at best.
Stay tuned to learn why centralized AI infrastructure will never work for healthcare and life sciences.
³ http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-convnet-on-imagenet/
⁴ https://pubmed.ncbi.nlm.nih.gov/32960230/; Kaushal A, Altman R, Langlotz C. JAMA. 2020;324: 1212–1213.
⁵ Celi LA, Cellini J, Charpignon M-L, Dee EC, Dernoncourt F, Eber R, et al. Sources of bias in artificial intelligence that perpetuate healthcare disparities—A global review. PLOS Digital Health. 2022. https://doi.org/10.1371/journal.pdig.0000022
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.