THE MICCAI 2014 MACHINE LEARNING CHALLENGE (MLC)
Predicting Binary and Continuous Phenotypes from Structural Brain MRI Data
NOTE: The challenge is over. The results will be presented at the MLC Workshop at MICCAI 2014. For program details go here.
Welcome to the home page of the MICCAI 2014 Machine Learning Challenge (MLC). This challenge is organized in conjunction with the 17th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), which will take place at the Massachusetts Institute of Technology, Cambridge, MA, USA. The challenge workshop will happen on September 18, 2014, on the last day of MICCAI 2014.
In this challenge, we aim to attract submissions from machine-learning practitioners who are interested in trying out their favorite (novel or not) supervised learning algorithm(s) on brain imaging (neuroimaging) data and for clinically relevant prediction problems. We will distribute image-based features derived from brain MRI scans and subject-specific (clinical) labels in an easy-to-read-fashion. Our focus is not on the clinical context but rather on the learning algorithms that can best leverage information contained in brain MRI. In a complementary fashion, our sister challenge CADDementia (http://caddementia.grand-challenge.org/) focuses on the specific problem of computer-aided diagnosis of Alzheimer’s dementia from brain MRI scans. MLC's aim is not to identify the machine learning algorithm with the best test performance. Instead, our goal is to quantify and study the variation and uncertainty in test performance and potentially uncover global patterns. In this sense, we do not consider this "Challenge" as a competition, rather we view it as a collective experiment.
There are several critical aspects of our machine learning challenge, which we would like to highlight:
- Hassle-free data: To keep the focus on machine learning algorithms, we will make available easy-to-read image-derived features (as simple text files). [For data description go here. For data download go here.]
- Binary and continuous labels: We will have two problems. In the first one, the target variable (the label that is to be predicted from the image data) is continuous. And, in the second problem, the target variable is binary.
- Secret labels: Once again, to keep the focus on machine learning, we will not release any information on what the labels are until the challenge workshop (September 18, 2014).
- Independent test data and validation: For each problem (binary or continuous), there will be two datasets: training and testing. For the training data, we will provide the ground-truth values for the target variables (i.e., labels), while for the testing data we will not distribute the labels. The participants (I.e., you) will be required to do two things: estimate prediction accuracy via cross-validation on the training data and produce predictions on the test data. These predictions will then be submitted to the organizers (via CodaLab), which will be assessed based on ground truth data. The agreement between cross-validation estimated prediction accuracy and performance on the independent test data will also be quantified.
- Brain MRIs will also be available: For those participants who are interested in using their own image processing pipelines, we will make the original images available for download, as well. This way, the participants are free to derive their own features to feed into their machine-learning algorithm.
- Any method allowed: We do not require that the utilized machine-learning algorithm is novel. Our goal is to attract as many submissions as possible and assess a wide range of methods. You can use this challenge to gain further experience by “playing with” novel data, without having to worry about the specifics of neuroimaging or the clinical context. Alternatively, you can use these data to evaluate your novel algorithm. Each team can submit multiple entries.
- Results will be published: We plan to write up a journal paper that will summarize our findings from this challenge. All participating teams will be represented with a co-author on this paper.
For further details, including challenge instructions, we recommend you to navigate these pages via the available links or the menu on the top of each page. Downloading/reading the data (provided as simple text files), and submitting your results (open April 15, 2014) should be very straightforward.
We look forward to a large number of submissions and a successful MICCAI Challenge.