===== Foundations of generative learning =====

==== Overview ====
This teaching block provides fundamentals of generative learning and covers the Maximum Likelihood Estimator (MLE) and its properties, the Expectation Maximisation Algorithm as well as basics of Bayesian learning. Finally, it introduces the classes of Hidden Markov models and Markov Random Fields and shows how to apply the discussed generative learning approaches to them.

  * **Teacher:** [[http://cmp.felk.cvut.cz/~flachbor/ | Dr. Boris Flach]]

  * **Prerequisites:**
    * probability theory and statistics
    * linear algebra and optimisation
    * pattern recognition and decision theory

==== Lectures ====
^Topic ^Pdf ^Recording ^Notes ^
| **1. Generative learning, Maximum Likelihood estimator** | {{ :courses:be4m33ssu:ml-em.pdf | }} | [[http://ptak.felk.cvut.cz/recordings/flachbor/BE4M33SSU/BE4M33SSU-2021_1116-Flach-8.m4v| mp4 ]] | |
| **2. EM algorithm, Bayesian learning** | {{ :courses:be4m33ssu:em_bayesian-ws2021.pdf | }} | no recording | |
| **3. Hidden Markov Models** | {{ :courses:be4m33ssu:hmms.pdf | }} | [[http://ptak.felk.cvut.cz/recordings/flachbor/BE4M33SSU/BE4M33SSU-2021_1130-Flach-10.m4v| mp4 ]]| |
| **4. Markov Random Fields** | {{ :courses:be4m33ssu:mrfs.pdf | }} | [[http://ptak.felk.cvut.cz/recordings/flachbor/BE4M33SSU/BE4M33SSU-2021_1207-Flach-11.m4v| mp4 ]]| |

==== Theoretical assignments ====
^Topic ^Pdf ^
| **Seminar: lecture 1** | {{ :courses:be4m33ssu:sem-glearn-mle-ws21.pdf| }} |
| **Seminar: lecture 2** | {{ :courses:be4m33ssu:sem-em-bayesian-ws21.pdf| }} |
| **Seminar: lecture 3** | {{ :courses:be4m33ssu:sem-hmm1-ws21.pdf| }} |

==== Homework ====
EM algorithm {{ :courses:be4m33ssu:shape_em_binary.pdf | task}} {{ :courses:be4m33ssu:em_data.tgz | data}}

==== Textbooks and References ====
    * [1] C.M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006 [[https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf|[PDF]]]
     * [2] M. Sugiyama, Introduction to Statistical Machine Learning, Elsevier, 2015 [[https://www.elsevier.com/books/introduction-to-statistical-machine-learning/sugiyama/978-0-12-802121-7|[Elsevier]]]