Assignment: Nested Cross-Validation

📅 Deadline: 22.5.2026 21:59

🏦 Points: 4

In this assignment, you will implement a nested cross-validation procedure and use it to train an RBF-kernel SVM classifier. You will also estimate its generalization performance by constructing a confidence interval. The full assignment details are available in the Assignment PDF.

You are provided with a template containing the following files:

• train_svm.py: This file includes the functions main, select_best_c that you are required to implement.
• main.py: This file evaluates performance of the SVM classifier and validatity of the confidence interval.
• utils.py: Contains helper functions for loading and saving data, and visualizations. You do not need to modify this file.
• X_train.npy: Contains development inputs.
• y_train.npy: Contains development labels.
• test-cases: A folder containing public test cases to help you verify your implementation before submitting to BRUTE.

Your objective is to implement the function main, which performs the nested CV, including C tuning, SVM training and CI computation, and select_best_c.py which selects the regularization parameter C via the cross-validation.

After completing your implementation, you can test your solution using the following commands before submitting it to BRUTE:

python main.py test-cases/public/instances/public_test.json

Expected output:

Loading model and scaler...
Model, scaler, and results loaded successfully.
X shape: (3804, 10)
Scaling hidden features...
Making predictions...
Error on Hidden Test Set: 0.1338
CV Error: 0.1269, Computed CI: [0.1133, 0.1404]
Reference Error: 0.1338, Reference CI: [0.1133, 0.1404]
Test OK

—-

• Submit the following files as a .zip via BRUTE:

cv_results.json
feature_scaler.joblib
main.py
rbf_svm_model.joblib
utils.py

• All files must be stored in the root of the .zip sent for submission.
• Make sure your implementation passes the test cases provided above. Good luck! 😊