Applying AI to microscopy shortens image processing to seconds

Scientists at the European Molecular Biology Laboratory (EMBL) have combined artificial intelligence (AI) algorithms with two cutting-edge microscopy techniques to shorten the time for image processing from days to seconds, while ensuring that the resulting images are crisp and accurate.

To observe the swift neuronal signals in a fish brain, scientists can use a technique called light-field microscopy, which makes it possible to image fast biological processes in three-dimensions (3D). However, the images are often lacking in quality and it takes hours or days for massive amounts of data to be converted into 3D volumes and movies – highlighting the need for the new AI methodology.

“Ultimately, we were able to take ‘the best of both worlds’ in this approach,” said Nils Wagner, one of the paper’s two lead authors. “AI enabled us to combine different microscopy techniques, so that we could image as fast as light-field microscopy allows and get close to the image resolution of light-sheet microscopy.”

Light-field microscopy captures large 3D images that allow researchers to track and measure remarkably fine movements, such as a fish larva’s beating heart, at very high speeds. However, this technique produces massive amounts of data, which can take days to process and the final images usually lack resolution.

Light-sheet microscopy homes in on a single two-dimensional (2D) plane of a given sample at one time, so researchers can image samples at higher resolution. Compared with light-field microscopy, light-sheet microscopy produces images that are quicker to process, but the data are not as comprehensive, since they only capture information from a single 2D plane at a time.

To take advantage of the benefits of each technique, the researchers developed an approach that uses light-field microscopy to image large 3D samples and light-sheet microscopy to train the AI algorithms, which then create an accurate 3D picture of the sample.

“If you build algorithms that produce an image, you need to check that these algorithms are constructing the right image,” explained Anna Kreshuk, group leader whose team brought machine learning expertise to the project.

In the new study, the researchers used light-sheet microscopy to make sure the AI algorithms were working. 

The researchers say this approach could potentially be modified to work with different types of microscopes, eventually allowing biologists to look at dozens of different specimens and see much more, much faster. For example, it could help to find genes that are involved in heart development or could measure the activity of thousands of neurons at the same time.

Next, the researchers plan to explore whether the method can be applied to larger species, including mammals.

The findings are published in Nature Methods.