Joint Super-Resolution and Tissue Patch Classification for Wholeslide Histological Images

Мұқаба

Дәйексөз келтіру

Толық мәтін

Ашық рұқсат Ашық рұқсат
Рұқсат жабық Рұқсат берілді
Рұқсат жабық Тек жазылушылар үшін

Аннотация

Segmentation of wholeslide histological images through the classification of tissue types of small fragments is an extremely relevant task in digital pathology, necessary for the development of methods for automatic analysis of wholeslide histological images. The extremely large resolution of such images also makes the task of increasing image resolution relevant, which allows storing images at a reduced resolution and increasing it if necessary. Annotating whole slide images by histologists is complex and time-consuming, so it is important to make the most efficient use of the available data, both labeled and unlabeled. In this paper we propose a novel neural network method to simultaneously solve the problems of super-resolution of histological images from 20× optical magnification to 40x and classifying image fragments into tissue types at 20× magnification. The use of a single encoder as well as the proposed neural network training scheme allows to achieve better results on both tasks compared to existing approaches. The PATH-DT-MSU WSS2v2 dataset presented for the first time in this paper was used for training and testing the method. On the test sample, an accuracy value of 0.971 and a balanced accuracy value of 0.916 were achieved in the classification task on 5 tissue types, for the super-resolution task, values of PSNR = 32.26 and SSIM = 0.89 were achieved. The source code of the proposed method is available at: https://github.com/Kukty/WSI_SR_CL.

Толық мәтін

Рұқсат жабық

Авторлар туралы

Zh. Sun

Lomonosov Moscow State University

Хат алмасуға жауапты Автор.
Email: sbitb0009@gse.cs.msu.ru
Ресей, Moscow

A. Khvostikov

Lomonosov Moscow State University

Email: khvostikov@cs.msu.ru
Ресей, Moscow

A. Krylov

Lomonosov Moscow State University

Email: kryl@cs.msu.ru
Ресей, Moscow

A. Sethi

Indian Institute of Technology Bombay

Email: asethi@ee.iitb.ac.in
Үндістан, Mumbai

I. Mikhailov

Lomonosov Moscow State University

Email: imihailov@mc.msu.ru
Ресей, Moscow

P. Malkov

Lomonosov Moscow State University

Email: pmalkov@mc.msu.ru
Ресей, Moscow

Әдебиет тізімі

  1. Hu W. et al. A state-of-the-art survey of artificial neural networks for whole-slide image analysis: from popular convolutional neural networks to potential visual transformers // Computers in Biology and Medicine. 2023. V. 161. P. 107034.
  2. Rodriguez J.P.M. et al. Artificial intelligence as a tool for diagnosis in digital pathology whole slide images: A systematic review // Journal of Pathology Informatics. 2022. P. 100138.
  3. Wulczyn E. et al. Deep learning-based survival prediction for multiple cancer types using histopathology images // PloS one. 2020. V. 15. № 6. P. e0233678.
  4. Khvostikov A., Krylov A., Mikhailov I., Malkov P. Visualization and Analysis of Whole Slide Histological Images // Lecture Notes in Computer Science. 2023. V. 13644. P. 403–413.
  5. Juhong A. et al. Super-resolution and segmentation deep learning for breast cancer histopathology image analysis // Biomedical Optics Express. 2023. V. 14. № 1. P. 18–36.
  6. Afshari M., Yasir S., Keeney G.L., Jimenez R.E., Garcia J.J., Tizhoosh H.R. Single patch super-resolution of histopathology whole slide images: a comparative study // Journal of Medical Imaging. 2023. V. 10. № 1. P. 017501–017501.
  7. Chen Z., Wang J., Jia C., Ye X. Pathological image super-resolution using mix-attention generative adversarial network // International Journal of Machine Learning and Cybernetics. 2023. P. 1–11.
  8. Xie L., Li C., Wang Z., Zhang X., Chen B., Shen Q., Wu Z. SHISRCNet: Super-resolution And Classification Network For Low-resolution Breast Cancer Histopathology Image // Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. 2023. P. 23–32.
  9. Salgueiro L., Marcello J., Vilaplana V. SEG-ESRGAN: A Multi-Task Network for Super-Resolution and Semantic Segmentation of Remote Sensing Images // Remote Sensing. 2022. V. 14. № 22. P. 5862.
  10. Wang L., Li D., Zhu Y., Tian L., Shan Y. Dual super-resolution learning for semantic segmentation // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020. P. 3774–3783.
  11. Khvostikov A., Krylov A., Mikhailov I., Malkov P., Danilova N. Tissue type recognition in whole slide histological images // CEUR Workshop Proceedings. 2021. V. 3027. P. 50.
  12. Larkin K.G. Reflections on shannon information: In search of a natural information-entropy for images //arXiv preprint arXiv:1609.01117, 2016.
  13. Ledig C. et al. Photo-realistic single image super-resolution using a generative adversarial network // Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. P. 4681–4690.
  14. Schonfeld E., Schiele B., Khoreva A. A U-Net based discriminator for generative adversarial networks // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020. P. 8207–8216.
  15. Wang X., Xie L., Dong C., Shan Y. Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data // Proceedings of the IEEE/CVF international conference on computer vision. 2021. P. 1905–1914.
  16. Zhang Y., Tian Y., Kong Y., Zhong B., Fu Y. Residual dense network for image super-resolution // Proceedings of the IEEE conference on computer vision and pattern recognition. 2018. P. 2472–2481.
  17. Shi W. et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network // Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. P. 1874–1883.
  18. He K., Zhang X., Ren S., Sun J. Deep residual learning for image recognition // Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. P. 770–778.
  19. Schreiber S., Geldenhuys J., De Villiers H. Texture synthesis using convolutional neural networks with long-range consistency and spectral constraints // 2016 Pattern Recognition Association of South Africa and Robotics and Mechatronics International Conference (PRASA-RobMech). IEEE. 2016. P. 1–6.
  20. Krichen M. Generative adversarial networks // 14th International Conference on Computing Communication and Networking Technologies (ICCCNT). IEEE. 2023. P. 1–7.
  21. Müller R., Kornblith S., Hinton G.E. When does label smoothing help? // Advances in neural information processing systems. 2019. P. 4696–4705.
  22. Yoo J., Ahn N., Sohn K.A. Rethinking data augmentation for image super-resolution: A comprehensive analysis and a new strategy // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. P. 8375–8384.
  23. Huang G. et al. Densely connected convolutional networks // Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. P. 4700–4708.
  24. Tan M., Le Q. Efficientnetv2: Smaller models and faster training // International conference on machine learning. PMLR. 2021. P. 10096–10106.
  25. Howard A.G. et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications // arXiv preprint arXiv:1704.04861, 2017.
  26. Chen Z., Guo X., Woo P.Y., Yuan Y. Super-resolution enhanced medical image diagnosis with sample affinity interaction // IEEE Transactions on Medical Imaging. 2021. V. 40. № 5. P. 1377–1389.
  27. Wang X. et al. ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks // Computer Vision–ECCV 2018 Workshops: Munich, Germany, September 8–14, 2018. Proceedings. Part V 15. Springer International Publishing. 2019. P. 63–79.

Қосымша файлдар

Қосымша файлдар
Әрекет
1. JATS XML
Жүктеу
2. Fig. 1. Example of a full-slide histological image from the PATH-DT-MSU WSS2v2 set with polygonal markup performed on 5 tissue classes, including the background

Жүктеу (916KB)
Метадеректер
3. Fig. 2. Architecture of the auxiliary neural network for resolution enhancement

Жүктеу (337KB)
Метадеректер
4. Fig. 3. Basic neural network architecture for resolution and classification enhancement

Жүктеу (239KB)
Метадеректер

© Russian Academy of Sciences, 2024