TY - GEN
T1 - Unsupervised learning of endoscopy video frames’ correspondences from global and local transformation
AU - Armin, Mohammad Ali
AU - Barnes, Nick
AU - Khan, Salman
AU - Liu, Miaomiao
AU - Grimpen, Florian
AU - Salvado, Olivier
N1 - Publisher Copyright:
© Springer Nature Switzerland AG 2018.
PY - 2018
Y1 - 2018
N2 - Inferring the correspondences between consecutive video frames with high accuracy is essential for many medical image processing and computer vision tasks (e.g. image mosaicking, 3D scene reconstruction). Image correspondences can be computed by feature extraction and matching algorithms, which are computationally expensive and are challenged by low texture frames. Convolutional neural networks (CNN) can estimate dense image correspondences with high accuracy, but lack of labeled data especially in medical imaging does not allow end-to-end supervised training. In this paper, we present an unsupervised learning method to estimate dense image correspondences (DIC) between endoscopy frames by developing a new CNN model, called the EndoRegNet. Our proposed network has three distinguishing aspects: a local DIC estimator, a polynomial image transformer which regularizes local correspondences and a visibility mask which refines image correspondences. The EndoRegNet was trained on a mix of simulated and real endoscopy video frames, while its performance was evaluated on real endoscopy frames. We compared the results of EndoRegNet with traditional feature-based image registration. Our results show that EndoRegNet can provide faster and more accurate image correspondences estimation. It can also effectively deal with deformations and occlusions which are common in endoscopy video frames without requiring any labeled data.
AB - Inferring the correspondences between consecutive video frames with high accuracy is essential for many medical image processing and computer vision tasks (e.g. image mosaicking, 3D scene reconstruction). Image correspondences can be computed by feature extraction and matching algorithms, which are computationally expensive and are challenged by low texture frames. Convolutional neural networks (CNN) can estimate dense image correspondences with high accuracy, but lack of labeled data especially in medical imaging does not allow end-to-end supervised training. In this paper, we present an unsupervised learning method to estimate dense image correspondences (DIC) between endoscopy frames by developing a new CNN model, called the EndoRegNet. Our proposed network has three distinguishing aspects: a local DIC estimator, a polynomial image transformer which regularizes local correspondences and a visibility mask which refines image correspondences. The EndoRegNet was trained on a mix of simulated and real endoscopy video frames, while its performance was evaluated on real endoscopy frames. We compared the results of EndoRegNet with traditional feature-based image registration. Our results show that EndoRegNet can provide faster and more accurate image correspondences estimation. It can also effectively deal with deformations and occlusions which are common in endoscopy video frames without requiring any labeled data.
KW - Convolutional neural network
KW - Image correspondences
KW - Registration
KW - Unsupervised learning
UR - http://www.scopus.com/inward/record.url?scp=85054841196&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-01201-4_13
DO - 10.1007/978-3-030-01201-4_13
M3 - Conference contribution
SN - 9783030012007
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 108
EP - 117
BT - OR 2.0 Context-Aware Operating Theaters, Computer Assisted Robotic Endoscopy, Clinical Image-Based Procedures, and Skin Image Analysis - 1st International Workshop, OR 2.0 2018 5th International Workshop, CARE 2018, 7th International Workshop, CLIP 2018, 3rd International Workshop, ISIC 2018 Held in Conjunction with MICCAI 2018
A2 - Malpani, Anand
A2 - Zenati, Marco A.
A2 - Oyarzun Laura, Cristina
A2 - Celebi, M. Emre
A2 - Sarikaya, Duygu
A2 - Codella, Noel C.
A2 - Halpern, Allan
A2 - Erdt, Marius
A2 - Maier-Hein, Lena
A2 - Xiongbiao, Luo
A2 - Wesarg, Stefan
A2 - Stoyanov, Danail
A2 - Taylor, Zeike
A2 - Drechsler, Klaus
A2 - Dana, Kristin
A2 - Martel, Anne
A2 - Shekhar, Raj
A2 - De Ribaupierre, Sandrine
A2 - Reichl, Tobias
A2 - McLeod, Jonathan
A2 - González Ballester, Miguel Angel
A2 - Collins, Toby
A2 - Linguraru, Marius George
PB - Springer Verlag
T2 - 1st International Workshop on OR 2.0 Context-Aware Operating Theaters, OR 2.0 2018, 5th International Workshop on Computer Assisted Robotic Endoscopy, CARE 2018, 7th International Workshop on Clinical Image-Based Procedures, CLIP 2018, and 1st International Workshop on Skin Image Analysis, ISIC 2018, held in conjunction with the 21st International Conference on Medical Imaging and Computer-Assisted Intervention, MICCAI 2018
Y2 - 16 September 2018 through 20 September 2018
ER -