The definition of AI medical imaging

The application of artificial intelligence in medical imaging mainly realizes the analysis and judgment of medical images by machines through deep learning, which is an auxiliary tool to assist doctors in diagnosis and treatment, help doctors obtain image information faster, conduct qualitative and quantitative analysis, improve the efficiency of doctors’ viewing/reading of images, and help find hidden lesions.

By means of image classification, target detection, image segmentation and image retrieval, artificial intelligence completes the functions of lesion identification and labeling, three-dimensional reconstruction, automatic delineation of target area and adaptive radiotherapy, etc., and is applied in the stage of disease screening, diagnosis and treatment. The development of artificial intelligence technology has accelerated the speed of medical image diagnosis, improved the accuracy of image diagnosis, and brought changes to the way of “film reading” of imaging doctors, which is mainly manifested in:

(1) The way of reading films is changed. The application of artificial intelligence directly realizes the machine automatically screening, judging, selecting lesions, etc., and the doctor only needs to be responsible for the final judgment.

(2) The reading speed changes. Artificial intelligence automatically and quickly screening, and tick the lesions, the doctor is only responsible for the review of key parts, saving the doctor a lot of tedious preliminary screening process. Time is greatly reduced and efficiency is improved.

(3) Accuracy change. Artificial intelligence has the characteristics of stability and comprehensive two-sided, is not affected by the length of working time, and can achieve the complete observation of the film without omission, quickly and statically complete the preliminary screening and judgment, and finally the key parts are reviewed by professional doctors. Therefore, the accuracy of film reading is doubly guaranteed.

Main application scenarios of AI medical imaging

At present, there are two main application directions of artificial intelligence in the field of medical imaging, namely image case classification, target or lesion detection segmentation.

(1) Image case classification. Case classification is mainly to analyze a typical set of multiple pictures, so as to obtain the corresponding case classification results. In this kind of problem, there is usually a small amount of case image data for the corresponding task, which also leads to the transfer learning algorithm in computer vision is usually used to deal with this kind of problem. Most transfer learning algorithms use network models pre-trained by natural images, which are usually used as feature extractors and fine-tune medical image data in pre-trained models.

These two methods are very effective and have been widely used. However, in some classification problems, the model is difficult to converge, the accuracy is not high, and even the accuracy cannot surpass the classical manual analysis algorithm. The fundamental reason is that the amount of data is not enough and overfitting occurs in the process of transfer learning algorithm. However, with the iterative update of different deep learning network algorithms, especially the emergence of InceptionV3 network architecture in the United States, the problem of skin cancer classification detection has achieved results beyond human experts. The disadvantages of difficult convergence and low accuracy of the model have been gradually solved.

The new generation of artificial intelligence technology has been applied to the case classification of medical images in the early germination stage. As early as 2013, Japanese scholars published the application of DBNs network and SAEs network to brain MRI image classification. After the widespread application of convolutional neural networks (CNNs) in computer vision, the standard configuration of image classification problems has become convolutional neural networks and their variants.

Of the 47 articles on medical image cases from 2015 to 2017, 36 used convolutional neural network models, five used AEs models, and six used RBMs models, which were used in a wide range of medical image applications, from brain MRI to lung CT scans. In summary, convolutional neural networks are a standard model algorithm in medical images, especially the skills of pre-trained model transfer learning algorithms have demonstrated their powerful capabilities.

(2) Target or lesion detection segmentation

Different from the above image and case classification, the target or lesion classification pays more attention to the classification of a certain part of the image or small tissues, lesions and other local differences, such as the detection and classification of common lung nodules. For many tasks, regional and global conceptual information plays a very important role in these classification results. Many scholars have adopted a new multi-information fusion architecture for network topology (such as residual network structure) and the combination of information of different scales to make targeted model input and calculation adjustment for medical images.

Target delineation: radiation therapy, surgery and chemotherapy are the three main means of tumor treatment. Using medical image guidance, radiotherapy patients do not need surgery, short hospital stay, quick recovery. Before radiotherapy, each patient needs to take dozens or even hundreds of medical images (CT, MRI, etc.). The radiotherapy doctor needs half an hour to several hours to sketch the radiotherapy target area of each patient by experience, which takes time and energy, resulting in limited treatment of patients and unsatisfactory accuracy in the sketch.

Affected by the doctor’s experience, emotion, patience and other factors, different doctors will produce different sketching effects of the same patient’s medical image target area. Target mapping and treatment plan design have a certain technical content and require a doctor’s experience, but they involve a lot of repetitive work, these labor-intensive tasks are the specialty of AI, and using AI to do these things will save oncologists a lot of time.

The application of artificial intelligence technology in the field of radiotherapy is a major research and development direction for many AI + medical companies. At present, Lianxin Medical, medical Nuo technology, global medical, Purun Medical, Hui soft technology and other companies are developing related products.

The core technology of medical image diagnosis system construction

The core technology of medical image diagnosis system construction includes model design, model construction, algorithm selection and service establishment.

1. Model design

The selection of clinical problems, namely the design of AI models, is crucial.

First, the problems solved by the model must be of general concern to clinicians and imaging physicians, and the improvement of its solving efficiency or accuracy can generally benefit patients.

Second, the model design needs to refer to the latest clinical guidelines in the relevant field and contribute to the diagnosis and treatment of diseases on the existing medical procedures.

Third, it is necessary to use sufficient data and data labeling for learning, such as focusing on the identification of common tumors rather than the diagnosis of rare tumors. Therefore, the key to model design is to select the problem that is most conducive to the decision of doctors and the benefit of patients, and the problem selected must also have a large number of easy to obtain and annotate learning data.

2. Model construction

The establishment of the model includes the structured construction of learning data, the establishment of the model using the learning algorithm, and the verification of the model. High-quality structured data is the foundation of learning tasks.

First, data collection. Various models of image data acquisition equipment, different parameters, and different quality control will affect the final application of AI. Therefore, the requirements of AI model on data parameters and quality should be planned first during image data acquisition. For example, thin-layer resolution CT is used for lung nodule detection instead of thick layer data. On the basis of the application potential of AI, it is possible to cover different manufacturers, parameters, image quality and disease types.

Second, data annotation. The learning labels of the data should be directly oriented to the target problems that need to be learned, such as pulmonary nodule detection task labeling nodule coordinate outline, benign and malignant identification task labeling nodule pathological types. Try to use the “gold standard” label in the labeling tasks, such as pathology, genotype, survival time, etc. The quantitative knowledge of the radiologist was used, such as lesion location, scope, benign and malignant score, etc. The quality control of data sets is very important, and improving the accuracy of data sets can effectively improve the accuracy and robustness of the model. Therefore, the key to the construction of high-quality structured data lies in the quality and wide representation of image data acquisition, as well as the accuracy of data annotation.

3. Algorithm selection

Different from the machine vision algorithms and machine learning algorithms used in traditional computer-aided diagnosis, the new generation of AI algorithms can apply larger sample data to break through the bottleneck of accuracy, so that the model can be truly efficient in clinical use. The choice of different modeling methods should be planned according to the data volume and complexity of the learning data, including:

First, for a large number of learning data, it is recommended to use deep learning including various neural networks as a learning model;

Second, for the medium learning data, we can try to use deep learning modeling. If the effect is not good, we can consider using neural network to extract features and use machine learning method to build a model.

Third, for a small amount of learning data, it is recommended to use the imaging omics method to first conduct high-throughput examination, extract image features within the scope of the lesion, and use the machine learning method to establish a model. Fourth, although there is only a medium amount of learning data, there are a large number of similar modal data facing other problems. We can try to use transfer learning to apply the experience of large sample data to small sample data learning.

No matter which model building algorithm is used, it is necessary to verify the accuracy, robustness and generalization of the model. Cross-validation method can be used to verify the stability of the model in the training data set. In addition, independent data sets are needed to verify the robustness and generalization of the model, and finally evidence in clinical use will provide an evaluation of the model’s performance in the real world.

Therefore, the key point of AI algorithm selection and model building lies in data – and problem-oriented algorithm selection and model verification. The CheXNet deep convolutional neural network model proposed by Stanford University considers the interpretability of the model based on the use of chest X-ray to judge the disease of pneumonia patients.

This model uses Dense Net deep neural network model to analyze image features. Not only does the accuracy exceed the average level of human doctors in the case that chest X-rays are used as diagnostic basis, but also the weight sum of various image features on each pixel of the model is calculated to measure the importance of each location of the image in classification decision making. Explain the decision-making process and help human doctors understand the patient’s condition.

Professor Xing Bo of Carnegie Mellon University recently proposed a multi-task collaborative framework to accurately locate and summarize abnormal regions by introducing a collaborative attention mechanism. Not only the image content is described by labels, but also the medical image analysis report in text form is generated by hierarchical long short-term memory (LSTM) model, and the analysis results are described and interpreted by text description.

4. Service establishment

According to the application characteristics, clinical needs and doctors’ working habits, a reasonable service model is established.

First, the current rapid development of cloud imaging technology, its combination with AI technology can better provide medical institutions, especially grassroots hospitals with image transmission, storage, auxiliary diagnosis package solutions, which is conducive to improving the operating efficiency and diagnostic accuracy of medical institutions.

Second, in terms of combining with the existing workflow, the RIS system can be combined to provide AI structured reports, while the AI comprehensive analysis report can be combined with the PACS system to submit to the PACS system in DICOM format, and the lesions can be marked and suggested when the doctor browsed the images.

In general, although the pros and cons of a specific AI medical imaging technology depend on many links, the main issues that should be concerned at the current stage are reflected in the application object setting, service mode and accuracy of AI technology products. Good sensitivity and diagnostic accuracy are the foundation of the service.

In order to achieve this goal, in addition to excellent image segmentation, recognition algorithms and AI classification algorithms, more attention should be paid to building high-quality structured data sets including databases and knowledge bases. In addition, it is also necessary to pay attention to the purpose setting of AI technology with clinical diagnostic application value and in line with clinical norms, and in line with the application habits of clinicians.