Train a first image model on chest X-rays, split the data properly, and look at what the model is looking at
Step 1 · The question
Can a model tell pneumonia from normal on a chest X-ray, and what is it looking at?
Pneumonia, an infection of the lung, often shows on a chest X-ray as a white area where air spaces have filled with fluid. Teaching a computer to recognize that pattern is the classic first project in imaging AI, and it teaches the lessons that matter for every later project: how an image becomes numbers, how to keep the test fair, and how to check what the model has really learned.
This lab uses PneumoniaMNIST, part of MedMNIST, a free collection of small, standardized medical image datasets made for teaching and benchmarking. It contains 5,856 chest X-rays of children from a hospital in Guangzhou, China (first published by Kermany and colleagues in 2018), each shrunk to 28 by 28 pixels and labeled pneumonia or normal. The dataset comes split into 4,708 training, 524 validation and 624 test images. Unlike Labs 1 to 3, everyone uses the same images here, so step 5 asks two fixed questions about the dataset plus your own model's accuracy, which varies from run to run.
By the end of this lab you can:
sens_spec().Step 2 · The idea
A digital image is a grid of pixels, and each pixel is just a number. In CT, that number is in Hounsfield units (HU), a scale fixed by physics: water is 0, air is about −1000, fat about −100, soft tissue and blood roughly 20 to 80, and bone several hundred and up. A CT image holds thousands of different values, but a screen shows only a few hundred shades of gray, and our eyes tell apart far fewer. So the viewer picks a window: a width (how wide a range of values is spread from black to white) and a level (the value in the middle of that range).
A drawn, simplified chest CT slice (not a real patient). Try the three presets, then move the sliders yourself. Point at the image to read the number stored in any pixel. Can you find the small nodule in the right lung?
Nothing about the stored numbers changed when you moved the sliders; only the mapping to gray did. The model never sees "a lung" either, only the grid of numbers. X-rays are not stored in Hounsfield units, but the idea is the same, and the PneumoniaMNIST images are already grayscale values from 0 to 255 that the notebook rescales to between 0 and 1 before training.
The model in this lab is a convolutional neural network (CNN): a model that slides small filters across the image to pick up edges, then shapes, then larger patterns, and learns which filters help tell pneumonia from normal. It learns from the training set, is tuned on the validation set, and is judged once on the test set, which it has never seen. If the same patient has images in both training and test, the model can score well by recognizing the patient rather than the disease. That error is called data leakage, and splitting by patient prevents it.
A good test score still does not show what the model relied on. A saliency map colors the pixels that most changed the model's answer. A well-known 2018 study by Zech and colleagues found that pneumonia models had partly learned which hospital an X-ray came from, using clues such as text markers on the film, and performed worse at hospitals they had not seen. Testing on images from somewhere else, called external validation, is how such shortcuts come to light.
From a DICOM file to numbers. PneumoniaMNIST comes already converted to small arrays, but hospital images arrive as DICOM files (Lesson 2.3). To see how a DICOM file is read in Python and turned into a NumPy array (Python's grid of numbers), work through the Kaggle notebook Basic understanding of DICOM: convert to NumPy. It's optional, and a good bridge to the open datasets in the Getting involved section.
Step 3 · Warm-up
Two short reading checks. They are not graded and always show the explanation. They also work as a skip test: they cover what the notebook's first parts teach.
What does this code print?
import numpy as np
img = np.array([[0, 50],
[200, 255]])
print(img.shape, img.max())
img is a tiny 2-by-2 image: two rows of two pixels. shape gives the number of rows and columns, (2, 2), and max() gives the brightest pixel, 255 (white in an 8-bit image, where 0 is black). A PneumoniaMNIST image has shape (28, 28). NumPy, imported here as np, is the library Python uses for grids of numbers.
Drag the lines into the order that builds a model, trains it and measures it on the test set. (The notebook's exact names may differ.)
Build the model, train it on the training images only (fit; an epoch is one pass through all of them), then ask it about test images it has never seen. Its outputs are probabilities, so a threshold turns them into yes or no, exactly as in Lab 3. Accuracy is the share of test images where the prediction matches the label. Training on the test images, even by accident, would make this number meaningless.
Step 4 · Notebook
The notebook Lab_4_Imaging_AI.ipynb downloads PneumoniaMNIST and works through the four ideas above. Part 1 opens the images as grids of numbers and shows them under different windows. Part 2 makes a patient-level split and includes AI-written splitting code with a leak for you to fix. Part 3 trains a small CNN (switch Colab to a GPU runtime, a faster kind of processor, under Runtime > Change runtime type) and measures it with the sens_spec() function from your toolkit. Part 4 draws saliency maps and tests the model on a second set of images standing in for another hospital.
Type it into the notebook's first cell, as in the earlier labs:
GITHUB_USERNAME = "your-username"
Colab needs a Google account. It opens the course's copy of the notebook; follow "Save your work to GitHub" below to keep your own.
Tick each part off when you finish it. Your ticks show as progress on the Research Lab map.
Every task in the notebook is tagged with one mode. In Write tasks you write a few lines yourself from an empty cell, with a hint and a hidden solution. In Fill in tasks most of the code is there and you complete the blanks. Reuse tasks import a function you wrote in an earlier lab from your own toolkit.py, the way real projects grow. In Direct the AI tasks you ask an AI assistant (Colab's built-in one is suggested, but any is fine) to write the code, paste it in, run the test cell, and note in the AI-use log what you asked and what you had to change. Fix tasks give you code "written by an AI" with one realistic mistake to find. Read tasks ask you to put lines in order or predict what code will print before you run it.
Early labs lean on writing code yourself, because that is how the basics stick. Later labs lean on reusing your code, directing an AI and checking its work, because that is how research code is written now, and checking is the part a person must still do.
Colab runs the notebook, but it does not keep your work for you. You keep it in your own copy of the course repository on GitHub (a repository, or repo, is a project folder on GitHub with the full history of every saved change). The first time, you make that copy, called a fork; after that, you save into it whenever you finish a part.
github.com/your-username/CA2i-Research-Launchpad.your-username/CA2i-Research-Launchpad, branch main, and keep the file path lesson-notebooks/Lab_4_Imaging_AI.ipynb. Write a short note about what changed (for example "Part 2 done") and click OK.Step 5 · Report back
The notebook's last cell prints the numbers below, worked out from your own dataset. Enter them here. If one doesn't match, you get a hint first, then an explanation, and after a third try the answer, so you can find where your analysis went differently. Getting help is fine; the lab is then marked "finished with help".
Loading the answer checker...