4th COV19D Competition

part of “Domain adaptation, Explainability and Fairness in AI for Medical Image Analysis (DEF-AI-MIA)” CVPR 2024 Workshop

The “Domain adaptation, Explainability and Fairness in AI for Medical Image Analysis (DEF-AI-MIA)” Workshop and 4th COV19D Competition will be held in conjunction with IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR) 2024 at the Seattle Convention Center in June 17-21, 2024 .

For any requests or enquiries regarding the Competition, please contact: d.kollias@qmul.ac.uk

Organizers

General Chairs

Program Chairs

  • Dimitrios Kollias (Queen Mary University London)                   d.kollias@qmul.ac.uk 
  • Xujiong Ye (University of Lincoln)                                                xye@lincoln.ac.uk 
  • Francesco Rundo (STMicroelectronics ADG—Central R&D)    francesco.rundo@st.com 

Data Chairs

  • Anastasios Arsenos (National Technical University of Athens )
  • Paraskevi Theofilou (National Technical University of Athens )
  • Manos Seferis ( National Technical University of Athens )
  • James Wingate ( University of Lincoln )

Description

4th COV19D Competition

Scope

The 4th COV19D Competition is the 4th in the series of COV19D Competitions following the first 3 Competitions held in the framework of  ICCV 2021, ECCV 2022 and ICASSP 2023 Conferences respectively. It includes two Challenges: i) Covid-19 Detection Challenge and ii) Covid-19 Domain Adaptation Challenge.

Both Challenges are  based on the COV19-CT-DB database, described in the provided References, including  chest CT scan series. Each chest CT scan series consists of a sequence of 2-D CT slices, the number of which is between 50 and 700.

I. Covid-19 Detection Challenge

Many CT scans have been aggregated, each one of which has been manually annotated in terms of Covid-19 and non-Covid-19 categories. The resulting dataset is split into training, validation and test partitions. The training and validation sets along with their annotations will be provided to the Competition participating teams to develop AI/ML/DL models for Covid-19 and non-Covid-19 prediction. Performance of the different approaches will be evaluated on the test set in terms of the ‘macro’ F1 score.

II. Covid-19 Domain Adaptation Challenge

CT scans have been aggregated from various hospitals and medical centres. Each CT scan has been manually annotated with respect to Covid-19 and non-Covid-19 categories. The resulting dataset is split into training, validation and test partitions. Participants will be provided with a training set that consists of: i) the annotated data of the 1st Challenge which are aggregated from some hospitals and medical centres (case A); ii) a small number of annotated data and a larger number of non-annotated data (case B), all of which are aggregated from other hospitals and medical centres and their distribution is different from that of case A. Participants will be provided with a validation set that consists of a small number of annotated data of case B. Competition participating teams will need to develop AI/ML/DL models for Covid-19 prediction. Performance of the different approaches will be evaluated on a test set (that contains data of case B) in terms of the ‘macro’ F1 score.

Call for Participation

The participating teams are invited to participate in at least one of these Challenges. In the end, they will submit their Covid-19 predictions for the respective test datasets. The winners of the Competitions will be selected based on the achieved performance, in terms of the ‘macro’ F1 score, on the test sets.

There will be one winning team for each Competition. The top-3 performing teams are expected to contribute a paper describing their approach, methodology and results to our DEF-AI-MIA CVPR 2024 Workshop. All other teams will be able to submit a paper describing their solutions and final results to the Workshop. The accepted papers will be part of the CVPR 2024 proceedings. 

The research groups will be able to use any publicly available datasets as long as they report it in their submitted final paper. We will provide -at a later stage- a white paper describing the Competitions, the datasets used, as well as baseline models and their results for comparison purposes.

General Information

In order to participate in the Competitions, teams will need to register. At the end of the Competitions, each team will have to send us:
i) their predictions on the test set,
ii) a link to a Github repository where their solution/source code will be stored, and
iii) a link to an ArXiv paper with 2-8 pages describing their proposed methodology, data used and results.
After that, the winner of each Competition will be announced and a leaderboard will be published. The winner and the two runner-ups in each Competition will be asked to also share their trained models so as to check out the validity of the approach.

Participation Information

In order to participate, teams will have to register; the lead researcher should send an email from their official address (no personal emails will be accepted) to d.kollias@qmul.ac.uk with:

i) subject “4th COV19D Competition: Team Registration”;
ii) this EULA filled in, signed and attached;
iii) the lead researcher’s official academic/industrial website; please note that the lead researcher cannot be a student (UG/PG/Ph.D.);
iv) the emails of each team member
v) the team’s name
vi) the point of contact name and email address (which member of the team will be the main point of contact for future communications, data access etc.)

There is a maximum number of 8 participants in each team.

As a reply, you will receive access to the dataset’s images and annotations.

Final Paper Submission Information

The paper format should adhere to the paper submission guidelines for main CVPR 2024 proceedings style. Please have a look at the Submission Guidelines Section here.

We welcome full long paper submissions (between 4 and 8 pages, excluding references or supplementary materials). All submissions must be anonymous and conform to the CVPR 2024 standards for double-blind review.

All papers should be submitted using this CMT website.

All accepted manuscripts will be part of CVPR 2024 conference proceedings.

Important Dates

The timeline for the competitions will be as follows:

  • January, 8, 2024:     Opening of the Competition; start of registration, data available
  • March 19, 2024:      Final submission of results deadline (Results, Code and ArXiv paper
                                      sent to d.kollias@qmul.ac.uk)
  • March 21, 2024:      Winners announcement
  • March 29, 2024:      Paper submission deadline (AoE); Start of review period
  • April 10, 2024:           Review decisions sent to authors; Notification of acceptance
  • April 14, 2024:         Camera ready version deadline

Competition Results

21 Teams participated in the 4th COV19D Competition (COVID19 Detection Challenge and COVID19 Domain Adaptation Challenge).

12 Teams submitted their results to the COVID19 Detection Challenge; 10 Teams submitted their results to the COVID19 Domain Adaptation Challenge.

6 and 4 Teams scored higher than the baseline and made valid submissions, respectively; their results are shown in the leaderboards below.

————————————————————————————————————————

The winner of the COVID19 Detection Challenge is the team MDAP consisting of: Robert Turnbull and Simon Mutch (University of Melbourne, Australia).

The runner-up is team Deep-Adaptation consisting of: Bougourzi Fares, Feryal WINDAL MOULAÏ and Abdelmalik Taleb-Ahmed (Polytechnique Hauts-de-France), Halim Benhabiles (Centre d’ Enseignement, de Recherche et d ’Innovation Systèmes Numériques, Lille), Fadi Dornaika (University of the Basque Country UPV/EHU).

The team that ranked third is ACVLab consisting of: Chih-Chung Hsu, Chia-Ming Lee, Yang Fan Chiang, Yi-Shiuan Chou, Chih-Yu Jiang, Shen-Chieh Tai, Chi-Han Tsai (National Cheng Kung University, Taiwan).

The leaderboard can be found  here

It is worth mentioning that in this Challenge, the top four performing teams had up to 0.65 % difference in their performance.

Congratulations to all teams!

————————————————————————————————————————

The winner of the COVID19 Domain Adaptation Challenge is the team FDVTS consisting of: Runtian Yuan, Qingqiu Li, Jilan Xu, Rui Feng, and Yuejie Zhang (Fudan University, China), Junlin Hou, Hao Chen (Hong Kong University of Science and Technology).

The runner-up is team MDAP consisting of: Robert Turnbull and Simon Mutch (University of Melbourne, Australia).

The team that ranked third is Deep-Adaptation consisting of: Bougourzi Fares, Feryal WINDAL MOULAÏ and Abdelmalik Taleb-Ahmed (Polytechnique Hauts-de-France), Halim Benhabiles (Centre d’ Enseignement, de Recherche et d ’Innovation Systèmes Numériques, Lille), Fadi Dornaika (University of the Basque Country UPV/EHU).

The leaderboard can be found  here

It is worth mentioning that in this Challenge, the top two performing teams had only 0.34 % difference in their performance.

Congratulations to all teams!

————————————————————————————————————————

References

If you want to reference the 4th AI-MIA COV19D Competition, or if you want to use the Competition datasets, you must cite all following papers and the white paper that will be distributed at a later stage:

  • D. Kollias, et. al.: “Domain adaptation, Explainability & Fairness in AI for Medical Image Analysis: Diagnosis of COVID-19 based on 3-D Chest CT-scans“, 2024

@article{kollias2024domain, title={Domain adaptation, Explainability \& Fairness in AI for Medical Image Analysis: Diagnosis of COVID-19 based on 3-D Chest CT-scans}, author={Kollias, Dimitrios and Arsenos, Anastasios and Kollias, Stefanos}, journal={arXiv preprint arXiv:2403.02192}, year={2024}}

  • D. Kollias, et. al.: “AI-Enabled Analysis of 3-D CT Scans for Diagnosis of COVID-19 & its Severity“, 2023

@inproceedings{kollias2023ai, title={AI-Enabled Analysis of 3-D CT Scans for Diagnosis of COVID-19 \& its Severity}, author={Kollias, Dimitrios and Arsenos, Anastasios and Kollias, Stefanos}, booktitle={2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW)}, pages={1–5}, year={2023}, organization={IEEE}}

  • A. Arsenos, et. al.: “Data-Driven Covid-19 Detection Through Medical Imaging“, 2023

@inproceedings{arsenos2023data, title={Data-Driven Covid-19 Detection Through Medical Imaging}, author={Arsenos, Anastasios and Davidhi, Andjoli and Kollias, Dimitrios and Prassopoulos, Panos and Kollias, Stefanos}, booktitle={2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW)}, pages={1–5}, year={2023}, organization={IEEE}}

  • D. Kollias, et. al.: “A deep neural architecture for harmonizing 3-D input data analysis and decision making in medical imaging“, 2023

@article{kollias2023deep,title={A deep neural architecture for harmonizing 3-D input data analysis and decision making in medical imaging}, author={Kollias, Dimitrios and Arsenos, Anastasios and Kollias, Stefanos}, journal={Neurocomputing}, volume={542}, pages={126244}, year={2023}, publisher={Elsevier}}

  • D. Kollias, et. al.: “AI-MIA: COVID-19 Detection & Severity Analysis through Medical Imaging“, 2022

@inproceedings{kollias2022ai, title={Ai-mia: Covid-19 detection and severity analysis through medical imaging}, author={Kollias, Dimitrios and Arsenos, Anastasios and Kollias, Stefanos}, booktitle={European Conference on Computer Vision}, pages={677–690}, year={2022}, organization={Springer}}

  •     A. Arsenos, et. al.: “A Large Imaging Database and Novel Deep Neural Architecture for Covid-19 Diagnosis“, 2022

@inproceedings{arsenos2022large, title={A Large Imaging Database and Novel Deep Neural Architecture for Covid-19 Diagnosis}, author={Arsenos, Anastasios and Kollias, Dimitrios and Kollias, Stefanos}, booktitle={2022 IEEE 14th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP)}, pages={1–5}, year={2022}, organization={IEEE} }

  •      D. Kollias, et. al.: “MIA-COV19D: COVID-19 Detection through 3-D Chest CT Image Analysis“, 2021

@inproceedings{kollias2021mia, title={Mia-cov19d: Covid-19 detection through 3-d chest ct image analysis}, author={Kollias, Dimitrios and Arsenos, Anastasios and Soukissian, Levon and Kollias, Stefanos}, booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision}, pages={537–544}, year={2021} }

  •      D. Kollias, et. al.: “Deep transparent prediction through latent representation analysis”, 2020

@article{kollias2020deep, title={Deep transparent prediction through latent representation analysis}, author={Kollias, Dimitrios and Bouas, N and Vlaxos, Y and Brillakis, V and Seferis, M and Kollia, Ilianna and Sukissian, Levon and Wingate, James and Kollias, S}, journal={arXiv preprint arXiv:2009.07044}, year={2020}}

  •  D. Kollias, et. al.: “Transparent Adaptation in Deep Medical Image Diagnosis”, 2020

@inproceedings{kollias2020transparent, title={Transparent Adaptation in Deep Medical Image Diagnosis.}, author={Kollias, Dimitris and Vlaxos, Y and Seferis, M and Kollia, Ilianna and Sukissian, Levon and Wingate, James and Kollias, Stefanos D}, booktitle={TAILOR}, pages={251–267}, year={2020}}