Table of Contents

HW 3 - Segmentation

In this assignment, your task will be to train a neural network with multi-loss objective, namely: hierarchical classification and semantic segmentation. The dataset consists of images of pets, where each image corresponds to a species (cat or dog) and a breed (25 dog breeds and 12 cat breeds). For each image there is also a semantic segmentation map with three classes: foreground, background & boundary. The task is to train a model that can determine the species p(species|image), the breed p(breed|image) and segmentation mask p(mask|image). hw03.zip

UPDATE: make sure in your image and mask transforms you use transforms.Resize(128) and not transforms.Resize((128,128)) as was originally in the homework template!!

Scoring

Task 1 - Species classification (1 point)

Task 2 - Breed classification (3 points)

Task 3 - Semantic segmentation (6 points)

Submission and Evaluation

Submit a .zip file containing all your training & inference code. There needs to be a model.py file, containing a Net class which has a method predict.

There also needs to be a weights.pth file, which will be loaded in BRUTE with:

model.load_state_dict(torch.load(weights_path, map_location=torch.device('cpu')))

You can save the model with:

torch.save(model.state_dict(), "weights.pth")

The method takes a single 3 x 128 x 128 image as input. (processed with the same transform as in the template: Resize, CenterCrop, ToTensor, Normalize)

After computing the predictions, it outputs them in the following format:

Tournament

Accompanying the assignment, there will be a tournament in which the models will be ranked based on their performance. There will be a separate ranking for each task: species accuracy, top-3 breed accuracy and mean IoU. The final ranking will be determined by the sum of the ranks in all three tasks. The scoring will be based on ranking as follows:

Rules

Good luck, if you get stuck feel free to consult the web or various chatbots, just make sure to acquire true understanding in the process and not just copy stuff, in the case of any questions or concerns please contact siproman@fel.cvut.cz.