Quick links: [[https://intranet.fel.cvut.cz/cz/education/rozvrhy-ng.B231/public/html/predmety/43/58/p4358506.html|Schedule]] | [[https://cw.felk.cvut.cz/forum/forum-1849.html|Forum]] | [[https://cw.felk.cvut.cz/brute/|BRUTE task submission system]] | [[.labs/start|Labs]] ====== BE5B33RPZ -- Pattern Recognition and Machine Learning ====== ===== Summary ===== This course introduces statistical decision theory and surveys canonical and advanced classifiers such as perceptrons, AdaBoost, support vector machines, and neural nets. ===== Basic info ===== ** Winter semester 2023/2024 ** ** Teaching: ** [[http://cmp.felk.cvut.cz/~matas | Jiří Matas]] (JM) , [[http://cmp.felk.cvut.cz/~drbohlav| Ondřej Drbohlav]] (OD) ** Where and when:** KN:E-301 at [[ http://cyber.felk.cvut.cz/contact/#maps | Building G, Karlovo namesti]], Monday 16:15-17:45 ** Lectures are streamed live ** on YouTube at the time of lecture: [[https://www.youtube.com/playlist?list=PLQL6z4JeTTQmU701oNZ_sYpakFTkXcgyF | online stream of the lecture room KN:E-301]] ** Recorded lectures ** [[https://www.youtube.com/playlist?list=PLQL6z4JeTTQm2m2_6SR6mZW3l7v8VgKhA | playlist ]] [[https://www.youtube.com/playlist?list=PLQL6z4JeTTQliSd8UclIb07j6TcAJ-7GR | Last year (2022) lecture playlist]] /* For online feedback, connect via zoom link: https://feectu.zoom.us/j/6135640703 */ ===== Lecture plan 2023/2024 ===== ^ Week ^ Date ^ Lect. ^ Slides ^ Topic ^ Wiki ^ Additional material ^ | 1 | 25.9. |JM| {{.lectures/pr_01_intro_and_bayes_2021.pdf|pdf}} | Introduction. Basic notions. The Bayesian recognition problem |[[https://en.wikipedia.org/wiki/Machine_learning|Machine_learning]] [[https://en.wikipedia.org/wiki/Naive_Bayes_classifier|Naive_Bayes_classifier]]| {{:courses:b4b33rpz:pr_01_extra.pdf|some simple problems}} | | 2 | 2.10. |OD| {{.lectures/pr_02_non_bayes_2023.pdf|pdf}} | Non-Bayesian tasks |[[https://en.wikipedia.org/wiki/Minimax|Minimax]] | | | 3 | 9.10. |JM| {{.lectures/pr_03_parameter_estimation_2022.pdf|pdf}} | Parameter estimation of probabilistic models. Maximum likelihood method |[[http://en.wikipedia.org/wiki/Maximum_likelihood|Maximum_likelihood]] | | | 4 | 16.10. |JM| {{.lectures/pr_04_non_parametric_knn_2022.pdf|pdf}} | Nearest neighbour method. Non-parametric density estimation. | [[http://en.wikipedia.org/wiki/K-nearest_neighbor_algorithm|K-nearest_neighbor_algorithm]] | | | 5 | 23.10. |JM| {{.lectures/pr_05_logistic_regression_2022.pdf |pdf}} | Logistic regression |[[https://en.wikipedia.org/wiki/Logistic_regression|Logistic_regression]] | | | 6 | 30.10. |JM| {{.lectures/pr_06_perceptron_2022.pdf|pdf}} | Classifier training. Linear classifier. Perceptron. |[[https://en.wikipedia.org/wiki/Linear_classifier|Linear_classifier]] [[https://en.wikipedia.org/wiki/Perceptron|Perceptron]] | | | 7 | 6.11. |JM| {{ :courses:b4b33rpz:pr_07_svm_2018.pdf |pdf}} | SVM classifier |[[https://en.wikipedia.org/wiki/Support_vector_machine|Support_vector_machine]]| [[https://jgreitemann.github.io/svm-demo|demo]] | | | 8 | 13.11. |JM| {{.lectures/pr_08_adaboost_2017.pdf|pdf}} | Adaboost learning |[[https://en.wikipedia.org/wiki/AdaBoost|Adaboost]] | | | 9 | 20.11. |JM| //no teaching// | Dean's day | | 10 | 27.11. |JM|{{:courses:b4b33rpz:neural_networks_2020.pdf|pdf}} | Neural networks. Backpropagation |[[https://en.wikipedia.org/wiki/Artificial_neural_network|Artificial_neural_network]] | | 11 | 4.12. |JM| {{:courses:b4b33rpz:pr_10_k_means_2015_12_04.pdf|pdf}} | Cluster analysis, k-means method |[[https://en.wikipedia.org/wiki/K-means_clustering|K-means_clustering]] [[https://en.wikipedia.org/wiki/K-means%2B%2B|K-means++]] | | | 12 | 11.12. |JM| {{ :courses:be5b33rpz:lectures:em_2020.pdf |pdf}} | EM (Expectation Maximization) algorithm. |[[https://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm|Expectation_maximization_algorithm]] |{{:courses:b4b33rpz:2010.12.10-em-hoffmann.pdf|Hoffmann}},{{:courses:b4b33rpz:2010.12.10-em-bishop.pdf|Bishop}}, {{.lectures/flach-2013.12.02-em_algorithm.pdf|Flach}}|| | 13 | 18.12. |JM| {{.lectures/pr_12_pca_2017_01_02.pdf|pdf}} | Feature selection and extraction. PCA, LDA. |[[https://en.wikipedia.org/wiki/Principal_component_analysis|Principal_component_analysis]] [[https://en.wikipedia.org/wiki/Linear_discriminant_analysis|Linear_discriminant_analysis]] | Optimalizace (CZ): [[https://cw.fel.cvut.cz/wiki/_media/courses/b0b33opt/05aplikace.pdf|PCA slides]], [[https://cw.fel.cvut.cz/wiki/_media/courses/b0b33opt/opt.pdf| script 7.2]] || | 14 | 8.1. | JM | {{.lectures/pr_13_dec_trees_2017_01_09.pdf|pdf}} | Decision trees. |[[https://en.wikipedia.org/wiki/Decision_tree|Decision_tree]] [[https://en.wikipedia.org/wiki/Decision_tree_learning|Decision_tree_learning]] |[[http://ocw.mit.edu/courses/sloan-school-of-management/15-097-prediction-machine-learning-and-statistics-spring-2012/lecture-notes/MIT15_097S12_lec08.pdf|Rudin@MIT]]|| /* Old PCA links: [[http://www.csd.uwo.ca/~olga/Courses/CS434a_541a/Lecture8.pdf|Veksler]], {{.pca-2016.01.15-franc.pdf|Franc}}, {{.lda_2014_06_08.pdf |ver1}} ^ datum ^ č.t. ^ S/L ^ náplň ^ | 25.09.2023 | 1 | L | | | 02.10.2023 | 2 | S | | | 09.10.2023 | 3 | L | | | 16.10.2023 | 4 | S | | | 23.10.2023 | 5 | L | | | 30.10.2023 | 6 | S | | | 06.11.2023 | 7 | L | | | 13.11.2023 | 8 | S | | | 20.11.2023 | 9 | L | | | 27.11.2023 | 10 | S | | | 04.12.2023 | 11 | L | | | 11.12.2023 | 12 | S | | | 18.12.2023 | 13 | L | | | 08.01.2024 | 15 | S | | ===== Recommended literature ===== * Duda R.O., Hart, P.E.,Stork, D.G.: Pattern Classification, John Willey and Sons, 2nd edition, New York, 2001 * Schlesinger M.I., Hlaváč V.: Ten Lectures on Statistical and Structural Pattern Recognition, Springer, 2002 * Bishop, C.: Pattern Recognition and Machine Learning, Springer, 2011 * Goodfellow, I., Bengio, Y. and Courville, A.: Deep Learning, MIT Press, 2016. [[http://www.deeplearningbook.org/|www]] /* ** Further resources ** * [[http://cmp.felk.cvut.cz/%7Ehlavac/Public/TeachingLectures/ROD-UvodRozpozn.pdf|Introduction to recognition]], V. Hlaváč (Czech only) * [[http://cmp.felk.cvut.cz/%7Ehlavac/Public/TeachingLectures/PravdepStatistikaOpakov.ppt|Probability and statistics overview]] (Czech only) * [[http://cmp.felk.cvut.cz/%7Ehlavac/Public/TeachingLectures/ReceiverOperCharact.pdf|ROC curve]] * [[http://cmp.felk.cvut.cz/%7Ehlavac/Public/TeachingLectures/LP-Theory.ppt|Linear programming]] * [[http://cmp.felk.cvut.cz/%7Ehlavac/Public/TeachingLectures/DualTaskLinearProgramming.pdf|Dual tasks in linear programming]] * [[http://cmp.felk.cvut.cz/%7Ehlavac/Public/TeachingLectures/P33ROD-2StatModels.pdf|Two statistical models]] * [[http://cmp.felk.cvut.cz/%7Ehlavac/Public/TeachingLectures/UceniBezUcitele.pdf|Unsupervised learning]] * [[http://cmp.felk.cvut.cz/%7Ehlavac/Public/TeachingLectures/Bishop-ECCV-04-tutorial-B.ppt|Unsupervised learning]] (lecture by Ch. Bishop) * [[http://cmp.felk.cvut.cz/%7Ehlavac/Public/TeachingLectures/UmeleNN.ppt|Artificial neural networks]] * [[http://cmp.felk.cvut.cz/%7Ehlavac/Public/TeachingLectures/MarkovianPR.pdf|Markov sequences]] * [[http://cmp.felk.cvut.cz/%7Ehlavac/Public/TeachingLectures/HMMalaRabiner.pdf|Markov sequences]] (tutorial by Rabiner) */ ===== Assessment (zápočet) ===== Conditions for assessment are in the [[./labs/|lab section]]. ===== Exam ===== * Only students who receive all credits from the lab work and are granted the assessment ("zápočet") can be examined. * The labs contribute 50% to your final evaluation, the written part of the exam contributes 40% and the oral part 10%. * For passing the written exam, at least 40% of maximum number of points is needed (16 out of 40 points). * Your grade chances after the written exam test and before the oral exam are illustrated in the image below. Beware: The scheme is **approximate only**, but illustrates how the oral exam influences the final grade. * The questions used in the test are available [[http://cmp.felk.cvut.cz/cmp/courses/recognition/Exam-questions|here]] (if one can solve these questions, one will likely do well in the exam). * The oral starts approximately 2 hours after the end of the test, as soon as the tests are graded. Alternatively, the oral might take place the following day, if the number of tests to correct is high. * The oral part is compulsory! Note that if during the oral examination it becomes clear that the student cannot explain how they obtained a result in the written exam, the examiner may re-assess the grading of the written exam. * Example oral exam questions are available [[http://cmp.felk.cvut.cz/cmp/courses/recognition/Exam-questions/exam-questions-eng.pdf|here]]. {{ :courses:be5b33rpz:lectures:rpz_oral_chances.png?nolink&400 |}} /*===== Exam dates ===== * written part: January 22, 2020 in lecture room K1, 12:00 * oral part: January 23 (the location will be specified) */