FAQs and Common Error Message Causes¶
------- Paper¶
The manuscript giving an overview and the major outcomes of the COVID-19 Lung CT Lesion Segmentation Challenge – 2020, including the data, winning approaches, and statistical analysis of results, is now available here:¶
Rapid Artificial Intelligence Solutions in a Pandemic - The COVID-19-20 Lung CT Lesion Segmentation Challenge¶
Holger Roth, Ziyue Xu, Carlos Tor Diez, Ramon Sanchez Jacob, Jonathan Zember, Jose Molto, Wenqi Li, Sheng Xu, Baris Turkbey, Evrim Turkbey, Dong Yang, Ahmed Harouni, Nicola Rieke, Shishuai Hu, Fabian Isensee, Claire Tang, Qinji Yu, Jan Sölter, Tong Zheng, Vitali Liauchuk, Ziqi Zhou, Jan Moltz, Bruno Oliveira, Yong Xia, Klaus Maier-Hein, Qikai Li, Andreas Husch, Luyang Zhang, Vassili Kovalev, Li Kang, Alessa Hering, João Vilaça, Mona Flores, Daguang Xu, Bradford Wood, Marius Linguraru¶
------- Mini-symposium¶
The COVID-19-20 mini-symposium will be held on January 11th, 2021 at 9am EST and it will be an opportunity to hear from the organizers and top performers in the challenge, offer clinical insights into the imaging and clinical management of COVID-19, and unveil the winners. Find more information and a recording of the symposiums under the Mini-symposium tab.¶
------- Test Phase¶
Test phase is open now, note that the leaderboard will not be shown until Dec. 18. Multiple submissions are allowed, but only the last one will be used for final ranking.¶
The test data has been shared by email with all challenge participants who submitted results on the validation phase.¶
The mandatory additional information required to participate in the test evaluation must be submitted through the form available on the Test Data page by December, 11th at 11:59pm GMT.¶
Submission of results on the test dataset will be accepted until December, 11th at 11:59pm GMT.¶
Preliminary results on the test set will be available online on December 18th, 2020. Note that these results will not be ranked nor final. Challenge winners and final ranking will be presented on January 11th, 2021.¶
Notes:¶
- The category "semi-automatic" refers to any manual interference in the testing that makes the testing process not fully-automated.
- The rules are one entry for each team. It's up to the team to select their representative approach, i.e. category.
------- Validation Phase¶
A sample submission could be found in the "Download" page. If you submitted results on the validation data and they do not appear on the leader board, please resubmit them; the system has been updated.¶
Results on the Leaderboard are ranked by Dice only, not by the statistical method that will be used to determine the challenge winners, which combines several metrics with their relative statistical significance.¶
Submission of results on the validation dataset will be accepted until December, 8th at 11:59pm GMT.¶
------- Submission¶
Participants are expected to submit a single .zip file containing all the binary predications for each of the validation data. Please make sure all filenames follows the same correspondence as: Input image: volume-covid19-A-0004_ct.nii.gz -> Ouput prediction: 004.nii.gz i.e. remove the pre-text "volume-covid19-A-0" and post-text "_ct", keeping only the 3-digit case ID number. Example: submission.zip file, containing 50 files "004.nii.gz ... 180_0.nii.gz ... 697.nii.gz"