Cross-Modulated Few-Shot Image Generation for Colorectal Tissue Classification

Amandeep Kumar; Ankan Kumar Bhunia; Sanath Narayan; Hisham Cholakkal; Rao Muhammad Anwer; Jorma Laaksonen; Fahad Shahbaz Khan

doi:10.1007/978-3-031-43898-1_13

Cross-Modulated Few-Shot Image Generation for Colorectal Tissue Classification

Amandeep Kumar^*, Ankan Kumar Bhunia, Sanath Narayan, Hisham Cholakkal, Rao Muhammad Anwer, Jorma Laaksonen, Fahad Shahbaz Khan

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

1 Citation (Scopus)

Abstract

In this work, we propose a few-shot colorectal tissue image generation method for addressing the scarcity of histopathological training data for rare cancer tissues. Our few-shot generation method, named XM-GAN, takes one base and a pair of reference tissue images as input and generates high-quality yet diverse images. Within our XM-GAN, a novel controllable fusion block densely aggregates local regions of reference images based on their similarity to those in the base image, resulting in locally consistent features. To the best of our knowledge, we are the first to investigate few-shot generation in colorectal tissue images. We evaluate our few-shot colorectral tissue image generation by performing extensive qualitative, quantitative and subject specialist (pathologist) based evaluations. Specifically, in specialist-based evaluation, pathologists could differentiate between our XM-GAN generated tissue images and real images only $$55\%$$ time. Moreover, we utilize these generated images as data augmentation to address the few-shot tissue image classification task, achieving a gain of 4.4% in terms of mean accuracy over the vanilla few-shot classifier. Code: https://github.com/VIROBO-15/XM-GAN.

Original language	English
Title of host publication	Medical Image Computing and Computer Assisted Intervention – MICCAI 2023
Subtitle of host publication	Proceedings of 26th International Conference
Editors	Hayit Greenspan, Hayit Greenspan, Anant Madabhushi, Parvin Mousavi, Septimiu Salcudean, James Duncan, Tanveer Syeda-Mahmood, Russell Taylor
Publisher	Springer
Pages	128-137
Number of pages	10
ISBN (Print)	978-3-031-43897-4
DOIs	https://doi.org/10.1007/978-3-031-43898-1_13
Publication status	Published - 2023
MoE publication type	A4 Conference publication
Event	International Conference on Medical Image Computing and Computer Assisted Intervention - Vancouver, Canada Duration: 8 Oct 2023 → 12 Oct 2023 Conference number: 26

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14222 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	International Conference on Medical Image Computing and Computer Assisted Intervention
Abbreviated title	MICCAI
Country/Territory	Canada
City	Vancouver
Period	08/10/2023 → 12/10/2023

Keywords

Cross Modulation
Few-shot Image generation

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1007/978-3-031-43898-1_13

https://arxiv.org/abs/2304.01992

Cite this

Kumar, A., Bhunia, A. K., Narayan, S., Cholakkal, H., Anwer, R. M., Laaksonen, J., & Khan, F. S. (2023). Cross-Modulated Few-Shot Image Generation for Colorectal Tissue Classification. In H. Greenspan, H. Greenspan, A. Madabhushi, P. Mousavi, S. Salcudean, J. Duncan, T. Syeda-Mahmood, & R. Taylor (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2023: Proceedings of 26th International Conference (pp. 128-137). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14222 LNCS). Springer. https://doi.org/10.1007/978-3-031-43898-1_13

Kumar, Amandeep ; Bhunia, Ankan Kumar ; Narayan, Sanath et al. / Cross-Modulated Few-Shot Image Generation for Colorectal Tissue Classification. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023: Proceedings of 26th International Conference. editor / Hayit Greenspan ; Hayit Greenspan ; Anant Madabhushi ; Parvin Mousavi ; Septimiu Salcudean ; James Duncan ; Tanveer Syeda-Mahmood ; Russell Taylor. Springer, 2023. pp. 128-137 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{a236d55407bd4ace82754abe52777220,

title = "Cross-Modulated Few-Shot Image Generation for Colorectal Tissue Classification",

abstract = "In this work, we propose a few-shot colorectal tissue image generation method for addressing the scarcity of histopathological training data for rare cancer tissues. Our few-shot generation method, named XM-GAN, takes one base and a pair of reference tissue images as input and generates high-quality yet diverse images. Within our XM-GAN, a novel controllable fusion block densely aggregates local regions of reference images based on their similarity to those in the base image, resulting in locally consistent features. To the best of our knowledge, we are the first to investigate few-shot generation in colorectal tissue images. We evaluate our few-shot colorectral tissue image generation by performing extensive qualitative, quantitative and subject specialist (pathologist) based evaluations. Specifically, in specialist-based evaluation, pathologists could differentiate between our XM-GAN generated tissue images and real images only $$55\%$$ time. Moreover, we utilize these generated images as data augmentation to address the few-shot tissue image classification task, achieving a gain of 4.4% in terms of mean accuracy over the vanilla few-shot classifier. Code: https://github.com/VIROBO-15/XM-GAN.",

keywords = "Cross Modulation, Few-shot Image generation",

author = "Amandeep Kumar and Bhunia, {Ankan Kumar} and Sanath Narayan and Hisham Cholakkal and Anwer, {Rao Muhammad} and Jorma Laaksonen and Khan, {Fahad Shahbaz}",

note = "Funding Information: We extend our heartfelt appreciation to the pathologists who made significant contributions to our project. We are immensely grateful to Dr. Hima Abdurahiman from Government Medical College-Kozhikode, India; Dr. Sajna PV from MVR Cancer Center and Research Institute, Kozhikode, India; Dr. Binit Kumar Khandelia from North Devon District Hospital, UK; Dr. Nishath PV from Aster Mother Hospital Kozhikode, India; Dr. Mithila Mohan from Dr. Girija{\textquoteright}s Diagnostic Laboratory and Scans, Trivandrum, India; Dr. Kavitha from Aster MIMS, Kozhikode, India; and several other unnamed pathologists who provided their expert advice, valuable suggestions, and insightful feedback throughout various stages of our research work. This work was partially supported by the MBZUAI-WIS research program via project grant WIS P008. Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.; International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI ; Conference date: 08-10-2023 Through 12-10-2023",

year = "2023",

doi = "10.1007/978-3-031-43898-1_13",

language = "English",

isbn = "978-3-031-43897-4",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "128--137",

editor = "Hayit Greenspan and Hayit Greenspan and Anant Madabhushi and Parvin Mousavi and Septimiu Salcudean and James Duncan and Tanveer Syeda-Mahmood and Russell Taylor",

booktitle = "Medical Image Computing and Computer Assisted Intervention – MICCAI 2023",

address = "Germany",

}

Kumar, A, Bhunia, AK, Narayan, S, Cholakkal, H, Anwer, RM , Laaksonen, J & Khan, FS 2023, Cross-Modulated Few-Shot Image Generation for Colorectal Tissue Classification. in H Greenspan, H Greenspan, A Madabhushi, P Mousavi, S Salcudean, J Duncan, T Syeda-Mahmood & R Taylor (eds), Medical Image Computing and Computer Assisted Intervention – MICCAI 2023: Proceedings of 26th International Conference. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14222 LNCS, Springer, pp. 128-137, International Conference on Medical Image Computing and Computer Assisted Intervention, Vancouver, British Columbia, Canada, 08/10/2023. https://doi.org/10.1007/978-3-031-43898-1_13

Cross-Modulated Few-Shot Image Generation for Colorectal Tissue Classification. / Kumar, Amandeep; Bhunia, Ankan Kumar; Narayan, Sanath et al.
Medical Image Computing and Computer Assisted Intervention – MICCAI 2023: Proceedings of 26th International Conference. ed. / Hayit Greenspan; Hayit Greenspan; Anant Madabhushi; Parvin Mousavi; Septimiu Salcudean; James Duncan; Tanveer Syeda-Mahmood; Russell Taylor. Springer, 2023. p. 128-137 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14222 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

TY - GEN

T1 - Cross-Modulated Few-Shot Image Generation for Colorectal Tissue Classification

AU - Kumar, Amandeep

AU - Bhunia, Ankan Kumar

AU - Narayan, Sanath

AU - Cholakkal, Hisham

AU - Anwer, Rao Muhammad

AU - Laaksonen, Jorma

AU - Khan, Fahad Shahbaz

N1 - Conference code: 26

PY - 2023

Y1 - 2023

N2 - In this work, we propose a few-shot colorectal tissue image generation method for addressing the scarcity of histopathological training data for rare cancer tissues. Our few-shot generation method, named XM-GAN, takes one base and a pair of reference tissue images as input and generates high-quality yet diverse images. Within our XM-GAN, a novel controllable fusion block densely aggregates local regions of reference images based on their similarity to those in the base image, resulting in locally consistent features. To the best of our knowledge, we are the first to investigate few-shot generation in colorectal tissue images. We evaluate our few-shot colorectral tissue image generation by performing extensive qualitative, quantitative and subject specialist (pathologist) based evaluations. Specifically, in specialist-based evaluation, pathologists could differentiate between our XM-GAN generated tissue images and real images only $$55\%$$ time. Moreover, we utilize these generated images as data augmentation to address the few-shot tissue image classification task, achieving a gain of 4.4% in terms of mean accuracy over the vanilla few-shot classifier. Code: https://github.com/VIROBO-15/XM-GAN.

AB - In this work, we propose a few-shot colorectal tissue image generation method for addressing the scarcity of histopathological training data for rare cancer tissues. Our few-shot generation method, named XM-GAN, takes one base and a pair of reference tissue images as input and generates high-quality yet diverse images. Within our XM-GAN, a novel controllable fusion block densely aggregates local regions of reference images based on their similarity to those in the base image, resulting in locally consistent features. To the best of our knowledge, we are the first to investigate few-shot generation in colorectal tissue images. We evaluate our few-shot colorectral tissue image generation by performing extensive qualitative, quantitative and subject specialist (pathologist) based evaluations. Specifically, in specialist-based evaluation, pathologists could differentiate between our XM-GAN generated tissue images and real images only $$55\%$$ time. Moreover, we utilize these generated images as data augmentation to address the few-shot tissue image classification task, achieving a gain of 4.4% in terms of mean accuracy over the vanilla few-shot classifier. Code: https://github.com/VIROBO-15/XM-GAN.

KW - Cross Modulation

KW - Few-shot Image generation

UR - http://www.scopus.com/inward/record.url?scp=85174720463&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-43898-1_13

DO - 10.1007/978-3-031-43898-1_13

M3 - Conference article in proceedings

AN - SCOPUS:85174720463

SN - 978-3-031-43897-4

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 128

EP - 137

BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2023

A2 - Greenspan, Hayit

A2 - Madabhushi, Anant

A2 - Mousavi, Parvin

A2 - Salcudean, Septimiu

A2 - Duncan, James

A2 - Syeda-Mahmood, Tanveer

A2 - Taylor, Russell

PB - Springer

T2 - International Conference on Medical Image Computing and Computer Assisted Intervention

Y2 - 8 October 2023 through 12 October 2023

ER -

Kumar A, Bhunia AK, Narayan S, Cholakkal H, Anwer RM , Laaksonen J et al. Cross-Modulated Few-Shot Image Generation for Colorectal Tissue Classification. In Greenspan H, Greenspan H, Madabhushi A, Mousavi P, Salcudean S, Duncan J, Syeda-Mahmood T, Taylor R, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2023: Proceedings of 26th International Conference. Springer. 2023. p. 128-137. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-43898-1_13