Advancing electron microscopy using deep learning

Electron microscopy, a sub-field of microanalysis, is critical to many fields of research. The widespread use of electron microscopy for imaging molecules and materials has had an enormous impact on our understanding of countless systems and has accelerated impacts in drug discovery and materials design, for electronic, energy, environment and health applications. With this success a bottleneck has emerged, as the rate at which we can collect data has significantly exceeded the rate at which we can analyze it. Fortunately, this has coincided with the rise of advanced computational methods, including data science and machine learning. Deep learning (DL), a sub-field of machine learning capable of learning from large quantities of data such as images, is ideally suited to overcome some of the challenges of electron microscopy at scale. There are a variety of different DL approaches relevant to the field, with unique advantages and disadvantages. In this review, we describe some well-established methods, with some recent examples, and introduce some new methods currently emerging in computer science. Our summary of DL is designed to guide electron microscopists to choose the right DL algorithm for their research and prepare for their digital future.