Lectures

Regularly updated list of all lecture recordings is here.

date	week	topic	slides + videos + materials
23.09.2024	1	Lec 1: Machine learning 101: engineering view on model, loss, learning, learning issues, regression, classification	00_outline.pdf 01_intro_learning_regression_classification.pdf 01_lecture_recording
30.9.2024	2	Lec 2: Under the hood of a linear classifier: two-class and multi-class linear classifier on RGB images	02_classification.pdf 02_lecture_recording
7.10.2024	3	Lec 3: Where the hell do the loss and overfitting come from? MAP and ML estimate, KL divergence and losses.	03_kl_mle_loss_overfitting.pdf 03_lecture_recording central limit theorem Bayes theorem
14.10.2024	4	Lec 4: Under the hood of auto-differentiation: Computational graph of fully-connected NN, Vector-Jacobian-Product (VJP) vs chainrule and multiplication of Jacobians !!! Lecture canceled please look at the recording from the last year !!!	04_fcnn_autodiff_vjp.pdf 04_lecture_recording what is convolution
21.10.2024	5	Lec 5: The story of the cat's brain surgery: cortex + convolutional layer and its Vector-Jacobian-Product (VJP), fun with backpropagation	05_convnets.pdf
28.10.2024	6	Independence Day of Czechoslovakia
4.11.2024	7	Midterm test	urob_2023_midterm_test_1_.pdf vir_2022_midterm_test.pdf vir_2022_midterm_solution.pdf vir_2022_training_questions_midterm_test.pdf (“pure SGD” = “SGD”, i.e. momentum = 0)
11.11.2024	8	Lec 6: Why is learning prone to fail? - Structural issues: issues emerging from convolution and activation layers, initialization, normalization (BN, LN, GN, BN, BIN), regularization (weight-decay, drop-out)	06_layers.pdf 06_lecture_recording
18.11.2024	9	Lec 7: Why is learning prone to fail? - Optimization issues: optimization vs learning, KL divergence, SGD, momentum, convergence rate, Adagrad, RMSProp, AdamOptimizer, diminishing/exploding gradient, oscillation, double descent	07_training.pdf 07_lecture_recording
25.11.2024	10	Lec 8: Advanced layers that are known to work well I: Classical blocks (Skip-connection, Spatial/Channel attention, Inception module), Segmentation, Transformers	08a_architectures_class_segm.pdf 08b_transformers.pdf 08_lecture_recording
02.12.2024	11	Lec 9: Advanced layers that are known to work well II: Detection (Yolo, fast-RCNN), Regression (OpenPose), Spatial Transformer Nets, Generative networks	09a_architectures_simple_reg_det_stn.pdf 09b_generative_models.pdf 09_lecture_recording
09.12.2024	12	Lec 10: Reinforcement learning: Approximated Q-learning, DQN, DDPG, Derivation of the policy gradient (REINFORCE), A2C, TRPO, PPO, Reward shaping, Inverse RL, Applications,	10_reinforcement_learning.pdf l0_lecture_recording
16.12.2024	13	Lec 11: Implicit layers: Backpropagation through unconstrained and constrained optimization problems, ODE solvers, roots, fixed points + existing end-to-end differentiable modules cvxpy, gradSLAM, gradMPC, gradODE, pytorch3d	11a_recurrent_nets.pdf 11b_implicit_layers.pdf
6.1.2025	14	Exam test	exam_vir_2023.pdf exam_vir_2022.pdf vir_2022_training_questions_exam_test.pdf exam_2021.pdf exam_2022.pdf

Lectures 2023

date	week	topic	slides + videos + materials
25.09.2023	1	Lec 1: Machine learning 101: model, loss, learning, issues, regression, classification	vir_outline.pdf 01_intro_learning_regression_classification.pdf 01_lecture_recording
02.10.2023	2	Lec 2: Under the hood of a linear classifier: two-class and multi-class linear classifier on RGB images	02_classification.pdf 02_lecture_recording
9.10.2023	3	Lec 3: Under the hood of auto-differentiation: Computational graph of fully-connected NN, Vector-Jacobian-Product (VJP) vs chainrule and multiplication of Jacobians.	03_fcnn_autodiff_vjp.pdf 03_lecture_recording
16.10.2023	4	Lec 4: The story of the cat's brain surgery: cortex + convolutional layer and its Vector-Jacobian-Product (VJP)	04_convnets.pdf 04_lecture_recording
23.10.2023	5	Lec 5: Where the hell does the loss come from? MAP and ML estimate, KL divergence and losses.	05_kl_mle_loss_overfitting.pdf 05_lecture_recording
30.10.2023	6	Midterm test	vir_2022_midterm_test.pdf vir_2022_midterm_solution.pdf vir_2022_training_questions_midterm_test.pdf (“pure SGD” = “SGD”, i.e. momentum = 0)
06.11.2023	7	Lec 6: Why is learning prone to fail? - Structural issues: layers + issues, batch-norm, drop-out	layers.pdf 06_lecture_recording
13.11.2023	8	Lec 7: Why is learning prone to fail? - Optimization issues: optimization vs learning, KL divergence, SGD, momentum, convergence rate, Adagrad, RMSProp, AdamOptimizer, diminishing/exploding gradient, oscillation, double descent	training.pdf 07_lecture_recording
20.11.2023	9	Dean's day
27.11.2023	10	Lec 8: What can('t) we do with a deep net?: Classification (ResNet, Squeeze and Excitation Nets), Segmentation (DeepLab), Detection (Yolo, fast-RCNN), Regression (OpenPose), Spatial Transformer Nets,	08a_architectures_class_segm_simplified.pdf 08b_architectures_simple_reg_det_stn.pdf 08_lecture_recording
04.12.2023	11	Lec 9: Reinforcement learning: Approximated Q-learning, DQN, DDPG, Derivation of the policy gradient (REINFORCE), A2C, TRPO, PPO, Reward shaping, Inverse RL, Applications,	09b_reinforcement_learning.pdf 09_lecture_recording
11.12.2023	12	Lec 10: What I wanted to speak about from the very beginning, but it did not fit in: Implicit layers (Backpropagation through unconstrained and constrained optimization problems, ODE solvers, roots, fixed points) + existing end-to-end differentiable modules cvxpy, gradSLAM, gradMPC, gradODE, pytorch3d, Memory and attention (recurrent nets, Image transformers with attention module)	10a_memory_attention.pdf generative_models.pdf l0_lecture_recording
18.12.2023	13	Exam test	exam_vir_2022.pdf vir_2022_training_questions_exam_test.pdf exam_2021.pdf exam_2022.pdf
8.1.2024	14	Lec 11: Deep learning for satellite imagery: Guest lecture of Michal Reinstein (chief scientist in Spaceknow)

The playlist of all lecture recordings is here

Lectures 2022

date	week	topic	slides + videos
19.09.2022	1	Lec 1: Machine learning 101: model, loss, learning, issues, regression, classification	01_intro_learning_regression_classification.pdf lec1_recording
26.09.2022	2	Lec 2: Under the hood of a linear classifier: two-class and multi-class linear classifier on RGB images	02_classification.pdf lec2_recording
03.10.2022	3	Lec 3: Where the hell does the loss come from? MAP and ML estimate, KL divergence and losses.	03_mle.pdf lec3_recording
10.10.2022	4	Lec 4: The story of the cat's brain surgery: fully-connected NN + fast backpropagation via Vector-Jacobian-Product (VJP), cortex + convolutional layer	04b_convnets.pdf lec4_recording
17.10.2022	5	Lec 5: Under the hood of auto-differentiation: Vector-Jacobian-Product (VJP) vs chainrule and multiplication of Jacobians, convolutional layer and its VJP	04a_fully_connected_neural_nets.pdf lec5_recording
24.10.2022	6	Midterm test	vir_2022_training_questions_midterm_test.pdf (“pure SGD” = “SGD”, i.e. momentum = 0)
31.10.2022	7	Lec 6: Why is learning prone to fail? - Structural issues: layers + issues, batch-norm, drop-out	layers.pdf lec6_recording
07.11.2022	8	Lec 7: Why is learning prone to fail? - Optim. issues: optimization vs learning, KL divergence, SGD, momentum, convergence rate, Adagrad, RMSProp, AdamOptimizer, diminishing/exploding gradient, oscillation, double descent	training.pdf lec7_recording
14.11.2022	9	Lec 8: What can('t) we do with a deep net?: Classification (ResNet, Squeeze and Excitation Nets), Segmentation (DeepLab), Detection (Yolo, fast-RCNN), Regression (OpenPose), Spatial Transformer Nets, Memory and attention (recurrent nets, Image transformers with attention module)	08a_architectures_class_segm_simplified.pdf 08b_architectures_simple_reg_det_stn.pdf lec8_recording
21.11.2022	10	Lec 9: Reinforcement learning: Approximated Q-learning, DQN, DDPG, Derivation of the policy gradient (REINFORCE), A2C, Inverse RL, Applications, Lec 9 3/4:Learning to optimize: Backpropagation through unconstrained and constrained optimization problems, application (end-to-end differentiable modules cvxpy, gradSLAM, gradMPC, gradODE, pytorch3d) Lec 9 9/10:Self-supervision: (pseudo-labeling, privileged information, monodepth)	09_reinforcement_learning.pdf lec9_recording
28.11.2022	11	Lec 10: Generative Adversial Networks: Guest lecture by David Coufal, ÚI AV ČR	lec10_recording vir2022_a.pdf
05.12.2022	12	Lec 11: Normalizing Flows: Guest lecture by David Coufal, ÚI AV ČR	lec11_recording vir2022_b.pdf
12.12.2022	13	Exam test	vir_2022_training_questions_exam_test.pdf
09.01.2023	15	Lec 12: Deep learning for satellite imagery: Guest lecture of Michal Reinstein (chief scientist in Spaceknow)

The playlist of all lecture recordings is here.

Lectures 2021

date	week	topic	slides + videos
20.09.2021	1	Lec 1: MLE regression: derivation of L2 loss, prior	lec_01_MLE_regression
27.09.2021	2	Lec 2: MLE classifier: derivation of cross-entropy loss and logistic loss and linear classifier	lec_02_MLE_classifiction(pdf) lec_02_recording codes
04.10.2021	3	Lec 3: Fully-connected network: computational graph + backpropagation	lec_03_NN(pdf) lec_03_recording
11.10.2021	4	Lec 4: ConvNet: convolutional layer + backpropagation	lec_04_ConvNets(pdf) lec_04_recording
18.10.2021	5	Lec 5: Training: SGD, momentum, convergence rate, Adagrad, RMSProp, AdamOptimizer, diminishing/exploding gradient, oscilation	lec_05_training(pdf) lec_05_recording
25.10.2021	6	Lec 6: Layers: Convolution, Activation functions, Batch-Instance Norm, MaxPooling, Losses + backpropagation Test T1	lec_06_ConvNets training_examples.zip
01.11.2021	7	Lec 7: Architectures I: classification (ResNet, Squeeze and `excitation Nets)	lec_07_classification_segmentation(pdf) lec_07_recording
08.11.2021	8	Lec 8: Architectures II: segmentation (DeepLab), pose regression (OpenPose),	lec_08_classification_segmentation(pdf) lec_08_recording
15.11.2021	9	Lec 9: Architectures III: detection (Yolo), depth regression, spatial transformer nets	lec_09_regression_detection_stn(pdf) SVTI streams and recordings
22.11.2021	10	Lec 10: Reinforcement Learning I: Approximated Q-learning, DQN, DDPG, Policy gradient (REINFORCE),	lec_10_DQN_DDPG_REINFORCE(pdf) lec_10_recording
29.11.2021	11	Lec 11: Generative Adversarial Networks: guest lecture by David Coufal, ÚI AV ČR)	lect_11_GANs(pdf) lec_11_recording
06.12.2021	12	Lec 12: Reinforcement Learning II: Derivation of the policy gradient, A2C, Inverse Reinforcement Learning, Applications, end-to-end differentiable modules (cvxpy, gradSLAM, pytorch3d)	lec_12_A2C_IRL(pdf) lec_12_recording
13.12.2021	13	Lec 13: Memory and attention: Recurrent nets, Image transformers with attention module, ~~Depth completition (monodepth), Graph convolution,~~	lec_13_memory_attention(pdf) lec_13_recording
03.01.2022	15	Lec 14: Exam Test ET

All SVTI streams

Lectures 2020

date	week	topic	slides+videos+codes
21.09.2020	1	Lec 1: Regression/Classification as MLE: derivation of L2-loss, cross-entropy loss, logistic loss Lec 2: Neural networks: Fully-connected layer + computational graph + backpropagation	lec_1_video (incorrect audio sync!) vir_outline.pdf mle_01_regression.pdf mle_regression.py lec2_internal_viewer lec_2_video mle_02_classification.pdf mle_linear_classifier.py
28.9.2020	2	State holidays
05.10.2020	3	Lec 3: ConvNet: convolutional layer + backpropagation	lec3_internal_viewer lec3.video neural_nets.pdf, convnets.pdf convolution_codes.zip
12.10.2020	4	Lec 4: Training: SGD, momentum, convergence rate, Adagrad, RMSProp, AdamOptimizer, diminishing/exploding gradient, oscilation	lec4_internal_viewer lec4_video training.pdf (corrected slides)
19.10.2020	5	Lec 5: Layers: Convolution, Activation functions, Batch-Instance Norm, MaxPooling, Losses + backpropagation	lec5_internal_viewer, layers.pdf (corrected)
26.10.2020	6	Lec 6: Architectures I: classification (ResNet), segmentation (DeepLab) Test T1	lec6_internal_viewer architectures_i.pdf
02.11.2020	7	Lec 7: Learning from unlabelled data: Self-supervision: Contrastive learning, rotation, jigsaw, Colorization Weak-supervision: Multiple-instance learning, physical constraints; guest lecture by Patrik Vacek	lec7_internal_viewer unlabelled_data.pdf
09.11.2020	8	Lec 8: Architectures II: pose regression, detection (Yolo), depth regression, spatial transformer nets, LIFTs	lec8_internal_viewer architectures_ii.pdf
16.11.2020	9	Lec 9: Structured inputs: Recurrent neural networks, Convolution in 1,2 and 3D and other structures: guest lecture by Teymur Azayev	lec9_internal_viewer lecture_9_structure.pdf
23.11.2020	10	Lec 10: Reinforcement Learning I: DQN, GAE+TD(lambda), DDPG	lec10_internal_viewer reinforcement_learning.pdf
30.11.2020	11	Lec 11: Reinforcement Learning II: Policy gradients (REINFORCE), Inverse Reinforcement Learning, Applications, What did not fit to lectures: domain transfer, MAML, monodepth, depth from symmetries, Pytorch3D, CvxPyLayer (it did not fit even here )	lec11_internal_viewer reinforcement_learning.pdf
07.12.2020	12	Lec 12: Generative Adversarial Networks: guest lecture by David Coufal, ÚI AV ČR)	lec12_internal_viewer gans_coufal_vir_2019
14.12.2020	13	Exam Test ET	see BRUTE for results
04.01.2021	14	Selected teams present their semestral works (the rest presents during labs)

Table of Contents

Lectures

Lectures 2023

Lectures 2022

Lectures 2021

Lectures 2020