Search
Quick links: Schedule | Forum | BRUTE | Lectures | Labs
The main task is to fine-tune a pretrained CNN for a new classification task (transfer learning).
Skills: data loader from an image folder, data preprocessing, loading pretrained models, remote GPU servers, training part of the model. Insights: convolutional filters, error case analysis
In this lab we start from a model already pretrained on the ImageNet classification dataset (1000 categories and 1.2 million images) and try to adjust it for solving a small-scale but otherwise challenging classification problem.
It is a good time now to start working with GPU servers. Check How To page. The recommended setup is as follows:
Beware: VScode tends to keep the connection active even after you turn off your computer. As the GPU memory is expensive, login to the server regularly and check if your processes still occupy some GPUs. You may call pkill -f ipykernel to kill these processes.
pkill -f ipykernel
SOTA pretrained architectures are available in PyTorch. We will use the following models:
import torchvision.models model1 = torchvision.models.vgg11(weights=torchvision.models.VGG11_Weights.DEFAULT) model2 = torchvision.models.squeezenet1_0(weights=torchvision.models.SqueezeNet1_0_Weights.DEFAULT)
print(model)
The data will be placed in /local/temporary/butterflies/ on both servers (for a faster access and to avoid multiple copies). You can also download the dataset (e.g. to use on your computer):
/local/temporary/butterflies/
The dataset contain color images 224×224 pixels of 10 categories. The scientific (Latin) names of the butterfly categories are:
01: Danaus plexippus 02: Heliconius charitonius 03: Heliconius erato 04: Junonia coenia 05: Lycaena phlaeas 06: Nymphalis antiopa 07: Papilio cresphontes 08: Pieris rapae 09: Vanessa atalanta 10: Vanessa cardui
template.zip
This lab is substantially renewed this year, please let us know of any problems you encounter with the template or the task.
The first task will be just to load the pretrained network, apply it to test image and visualize the convolution filters and activations in the first layer. For this task, squeezenet is more suitable as it has 7×7 convolution filters in the first layer. There are a couple of technicalities, prepared in the template.
Sequential
model.features[0:2]
To address the classification task, we first need to load in the data: create dataset, split into training and validation, create loaders. Fortunately, there are convenient tools for all the steps. The respective technicalities are prepared in the template.
datasets.ImageFolder
from torchvision import datasets, transforms train_data = datasets.ImageFolder('/local/temporary/butterflies/train', transform.ToTensor()) train_loader = torch.utils.data.DataLoader(train_data, batch_size=1, shuffle=True, num_workers=0)
mean=[0.485, 0.456, 0.406] std=[0.229, 0.224, 0.225]
train_loader
val_loader
sampler
DataLoader
We will first try to learn only the last layer of the network on the new data. I.e. we will use the network as a feature extractor and learn a linear classifier on top of it, as if it was a logistic regression model on some features. This task is somewhat simpler with VGG architecture (SqueezeNet uses fully convolutional architecture and global pooling in the end).
model.to(dev)
for param in model.parameters(): param.requires_grad = False
model.train(False)
requires_grad = True
optimizer
nll_loss
to(dev)
torch.save
Report the final test classification accuracy of the best model (selected on the validation set). The test set is specified as a separate folder:
test_data = datasets.ImageFolder('/local/temporary/butterflies/test', transform) test_loader = torch.utils.data.DataLoader(test_data, batch_size=8, shuffle=False, num_workers=0)
Because we have very limited training / testing data available, it is a good idea to use also data augmentation. Let us select some transforms, which can be expected to result in realistic images of the same class. A possible set is
See Torchvision transform examples. Note that transforms inherit torch.nn.Module and therefore can be used the same way as layers, or as functions applied to data Tensors (however, not batched). They can be also built-in the Dataset by setting the transform argument. They can process PIL.Image or a Tensor. For efficiently reasons it is better to use them as functions on Tensors.
torch.nn.Module