Why have we seen rapid advances in consumer AI but not AI in medicine?

In the first post we make the case that pediatric AI applications could transform children’s healthcare globally and locally. We’ve seen the dramatic advances in consumer AI applications. AI applications are able to recognize common images with greater accuracy than humans. So how was this achieved?

In 2010, researchers at Stanford University launched the ImageNet competition. They assembled a database of 14 million images with 856 types of birds, 993 types of trees, and 157 musical instruments. The competition was to see how well a computer could be trained to recognize the images. It only took five years for computers to become more accurate than people. [1] [2] [3]

As you can see in the figure above the by 2015 the Microsoft AI application was more accurate than a human. So how was an AI application able to outperform humans? Neural networks. While neural networks have been discussed in research papers since the 1960s and 1970s, it was only recently, with advances in cloud computing, that these results could be achieved.

As the diagram above (courtesy of Jeff Dean, head of Google Brain) shows, the basic principle is simply that the more computing power and data provided, the higher the degree of accuracy neural networks are able to achieve. For those of you in major metro areas, this is the reason why you’ve likely seen cars equipped with lots of data-collecting cameras driving around town. What they’re doing is collecting as much real-world data as possible to more accurately train autonomous driving AI applications.

What is a neural network?

Neural networks can take different shapes and structures, but in general are modeled on our brain biology and the basic way the neurons function. A graphical visualization of the basic building block of a neural network, a software neuron, is shown below.

The software neuron can have many inputs (x), each are multiplied by weights (w) and summed. That number is then passed through an activation function that results in an output, which can be fed to another neuron. As an example image data can be represented as a matrix of pixel values, which can be fed as input x) to a neural network for image classification.

A software neural network is organized into layers of many neurons. Deep learning occurs when more and more layers (hidden layers) are added beyond the first layer. Models can have hundreds or thousand of these hidden layers.

Executing a neural network (aka an AI application) is merely carrying out this computation on any text (e.g. a query to ChatGPT), image (e.g. Focal Cortical Dysplasia diagnostic) or voice (e.g. Siri).

Of course the magic is in how we train an accurate neural network. Training a neural network is performed by first taking a data set and dividing it into a training set and a test set. The training set is used to determine the various weights (w) and number of layers to achieve the objective. Once you think you have the level of accuracy you want, you then run the test data set to see if you have indeed trained the neural network.

In the case of ImageNet, information from individual pixels cause neurons in the first layer to pass signals to the second layer, which then passes its analysis to the third. Each layer deals with increasingly abstract concepts such as edges, shadows, and shapes until the output layer attempts to categorize the entire image. Each of the teams in the competition made many choices, including the structure, the number of layers and the weighting functions, to achieve the increasing degrees of accuracy shown in the competition.

Advances have been based on a centralized architecture, which assumes there is a large centrally managed database of images. In the case of ImageNet, that database is 150GB of 4 million images. As a central repository of a vast number of images already proven its worth in enhancing neural network’s ability to accurately recognize and identify, it’s no wonder there’s interest in applying lessons learned in ImageNet to other AI-driven image recognition opportunities. In healthcare, greater access to a larger number of ultrasound, MRI, CT, or Xray images could similarly serve to improve the accuracy with which we can recognize and diagnose even rare conditions like Focal Cortical Dysplasia (FCD). Efforts are thus underway to build centralized architectures in the form of imaging data commons, image data repositories, or data lakes.

Makes sense, right? Well, yes and no. Yes in the desire to gather up as much data as possible. But the sourcing of diverse health care data unfortunately isn’t easy. In fact, data used to train adult AI applications comes from only a few states in the United States and a very limited set of countries in the world. Three states (California, New York, and Massachusetts) provide 70% of the training data, and you can guess the data is not from rural California.[4]

Globally, two countries (US and China) provide 50+% of the data used to train AI applications; only eight of the remaining 193 countries supply the rest.[5]

The result is that many AI applications lack accuracy in a wide variety of populations, or as a recent paper [6] concluded: “Nearly all of the studies published in the study period that evaluated the performance of AI algorithms for diagnostic analysis of medical images were designed as proof-of-concept technical feasibility studies and did not have the design features that are recommended for robust validation of the real-world clinical performance of AI algorithm.”

In summary, while centralized aggregation of larger data sets coupled with scalable compute has enabled neural network software to achieve increasing degrees of accuracy, aggregation of large data sets in medicine has proven itself problematic at best. Why? Read on to understand why centralized architectures are not the answer to deploying and training pediatric AI applications.

Many thanks for extensive editing by Laura Jana, Pediatrician, Social Entrepreneur & Connector of Dots; Leanne West, Chief Engineer of Pediatric Technology at Georgia Tech. Special thanks to Alberto Tozzi, Head of Predictive and Preventive Medicine Research Unit at Ospedale Pediatrico Bambino Gesù for the translation to Italian.