Lectures

date	week	topic	slides + videos
19.09.2022	1	Lec 1: Machine learning 101: model, loss, learning, issues, regression, classification	01_intro_learning_regression_classification.pdf lec1_recording
26.09.2022	2	Lec 2: Under the hood of a linear classifier: two-class and multi-class linear classifier on RGB images	02_classification.pdf lec2_recording
03.10.2022	3	Lec 3: Where the hell does the loss come from? MAP and ML estimate, KL divergence and losses.	03_mle.pdf lec3_recording
10.10.2022	4	Lec 4: The story of the cat's brain surgery: fully-connected NN + fast backpropagation via Vector-Jacobian-Product (VJP), cortex + convolutional layer	04b_convnets.pdf lec4_recording
17.10.2022	5	Lec 5: Under the hood of auto-differentiation: Vector-Jacobian-Product (VJP) vs chainrule and multiplication of Jacobians, convolutional layer and its VJP	04a_fully_connected_neural_nets.pdf lec5_recording
24.10.2022	6	Midterm test	vir_2022_training_questions_midterm_test.pdf (“pure SGD” = “SGD”, i.e. momentum = 0)
31.10.2022	7	Lec 6: Why is learning prone to fail? - Structural issues: layers + issues, batch-norm, drop-out	layers.pdf lec6_recording
07.11.2022	8	Lec 7: Why is learning prone to fail? - Optim. issues: optimization vs learning, KL divergence, SGD, momentum, convergence rate, Adagrad, RMSProp, AdamOptimizer, diminishing/exploding gradient, oscillation, double descent	training.pdf lec7_recording
14.11.2022	9	Lec 8: What can('t) we do with a deep net?: Classification (ResNet, Squeeze and Excitation Nets), Segmentation (DeepLab), Detection (Yolo, fast-RCNN), Regression (OpenPose), Spatial Transformer Nets, Memory and attention (recurrent nets, Image transformers with attention module)	08a_architectures_class_segm_simplified.pdf 08b_architectures_simple_reg_det_stn.pdf lec8_recording
21.11.2022	10	Lec 9: Reinforcement learning: Approximated Q-learning, DQN, DDPG, Derivation of the policy gradient (REINFORCE), A2C, Inverse RL, Applications, Lec 9 3/4:Learning to optimize: Backpropagation through unconstrained and constrained optimization problems, application (end-to-end differentiable modules cvxpy, gradSLAM, gradMPC, gradODE, pytorch3d) Lec 9 9/10:Self-supervision: (pseudo-labeling, privileged information, monodepth)	09_reinforcement_learning.pdf lec9_recording
28.11.2022	11	Lec 10: Generative Adversial Networks: Guest lecture by David Coufal, ÚI AV ČR	lec10_recording vir2022_a.pdf
05.12.2022	12	Lec 11: Normalizing Flows: Guest lecture by David Coufal, ÚI AV ČR	lec11_recording vir2022_b.pdf
12.12.2022	13	Exam test	vir_2022_training_questions_exam_test.pdf
09.01.2023	15	Lec 12: Deep learning for satellite imagery: Guest lecture of Michal Reinstein (chief scientist in Spaceknow)

The playlist of all lecture recordings is here.

Lectures 2021

date	week	topic	slides + videos
20.09.2021	1	Lec 1: MLE regression: derivation of L2 loss, prior	lec_01_MLE_regression
27.09.2021	2	Lec 2: MLE classifier: derivation of cross-entropy loss and logistic loss and linear classifier	lec_02_MLE_classifiction(pdf) lec_02_recording codes
04.10.2021	3	Lec 3: Fully-connected network: computational graph + backpropagation	lec_03_NN(pdf) lec_03_recording
11.10.2021	4	Lec 4: ConvNet: convolutional layer + backpropagation	lec_04_ConvNets(pdf) lec_04_recording
18.10.2021	5	Lec 5: Training: SGD, momentum, convergence rate, Adagrad, RMSProp, AdamOptimizer, diminishing/exploding gradient, oscilation	lec_05_training(pdf) lec_05_recording
25.10.2021	6	Lec 6: Layers: Convolution, Activation functions, Batch-Instance Norm, MaxPooling, Losses + backpropagation Test T1	lec_06_ConvNets training_examples.zip
01.11.2021	7	Lec 7: Architectures I: classification (ResNet, Squeeze and `excitation Nets)	lec_07_classification_segmentation(pdf) lec_07_recording
08.11.2021	8	Lec 8: Architectures II: segmentation (DeepLab), pose regression (OpenPose),	lec_08_classification_segmentation(pdf) lec_08_recording
15.11.2021	9	Lec 9: Architectures III: detection (Yolo), depth regression, spatial transformer nets	lec_09_regression_detection_stn(pdf) SVTI streams and recordings
22.11.2021	10	Lec 10: Reinforcement Learning I: Approximated Q-learning, DQN, DDPG, Policy gradient (REINFORCE),	lec_10_DQN_DDPG_REINFORCE(pdf) lec_10_recording
29.11.2021	11	Lec 11: Generative Adversarial Networks: guest lecture by David Coufal, ÚI AV ČR)	lect_11_GANs(pdf) lec_11_recording
06.12.2021	12	Lec 12: Reinforcement Learning II: Derivation of the policy gradient, A2C, Inverse Reinforcement Learning, Applications, end-to-end differentiable modules (cvxpy, gradSLAM, pytorch3d)	lec_12_A2C_IRL(pdf) lec_12_recording
13.12.2021	13	Lec 13: Memory and attention: Recurrent nets, Image transformers with attention module, ~~Depth completition (monodepth), Graph convolution,~~	lec_13_memory_attention(pdf) lec_13_recording
03.01.2022	15	Lec 14: Exam Test ET

All SVTI streams

Lectures 2020

date	week	topic	slides+videos+codes
21.09.2020	1	Lec 1: Regression/Classification as MLE: derivation of L2-loss, cross-entropy loss, logistic loss Lec 2: Neural networks: Fully-connected layer + computational graph + backpropagation	lec_1_video (incorrect audio sync!) vir_outline.pdf mle_01_regression.pdf mle_regression.py lec2_internal_viewer lec_2_video mle_02_classification.pdf mle_linear_classifier.py
28.9.2020	2	State holidays
05.10.2020	3	Lec 3: ConvNet: convolutional layer + backpropagation	lec3_internal_viewer lec3.video neural_nets.pdf, convnets.pdf convolution_codes.zip
12.10.2020	4	Lec 4: Training: SGD, momentum, convergence rate, Adagrad, RMSProp, AdamOptimizer, diminishing/exploding gradient, oscilation	lec4_internal_viewer lec4_video training.pdf (corrected slides)
19.10.2020	5	Lec 5: Layers: Convolution, Activation functions, Batch-Instance Norm, MaxPooling, Losses + backpropagation	lec5_internal_viewer, layers.pdf (corrected)
26.10.2020	6	Lec 6: Architectures I: classification (ResNet), segmentation (DeepLab) Test T1	lec6_internal_viewer architectures_i.pdf
02.11.2020	7	Lec 7: Learning from unlabelled data: Self-supervision: Contrastive learning, rotation, jigsaw, Colorization Weak-supervision: Multiple-instance learning, physical constraints; guest lecture by Patrik Vacek	lec7_internal_viewer unlabelled_data.pdf
09.11.2020	8	Lec 8: Architectures II: pose regression, detection (Yolo), depth regression, spatial transformer nets, LIFTs	lec8_internal_viewer architectures_ii.pdf
16.11.2020	9	Lec 9: Structured inputs: Recurrent neural networks, Convolution in 1,2 and 3D and other structures: guest lecture by Teymur Azayev	lec9_internal_viewer lecture_9_structure.pdf
23.11.2020	10	Lec 10: Reinforcement Learning I: DQN, GAE+TD(lambda), DDPG	lec10_internal_viewer reinforcement_learning.pdf
30.11.2020	11	Lec 11: Reinforcement Learning II: Policy gradients (REINFORCE), Inverse Reinforcement Learning, Applications, What did not fit to lectures: domain transfer, MAML, monodepth, depth from symmetries, Pytorch3D, CvxPyLayer (it did not fit even here )	lec11_internal_viewer reinforcement_learning.pdf
07.12.2020	12	Lec 12: Generative Adversarial Networks: guest lecture by David Coufal, ÚI AV ČR)	lec12_internal_viewer gans_coufal_vir_2019
14.12.2020	13	Exam Test ET	see BRUTE for results
04.01.2021	14	Selected teams present their semestral works (the rest presents during labs)

Table of Contents

Lectures

Lectures 2021

Lectures 2020