BE0M33BDT – Big Data Technologies

Because the English version of this course has to be held only for few students, the form will be different:

• self-study of literature and presentations
• two whole-day (or four half-day) workshops with training in technologies:
  • 5. 11. 9.00–16.00
  • 4. 12. 9.00–16.00
• homework
• test
• oral exam

• registration in Metacentrum (group CVUT:FEL:B0M33BDT or CVUT:FEL:A4M33BDT)
• Linux basic skills (file and directory management)
• SQL basic skills (creation of table, simple SELECT, GROUP BY, JOIN)
• Python basic skills (list, tuple, dict, string manipulation and functions, basic regexp)
• general skills in programming/scripting, using console and shell

• Syllabus in questions
• Big Data & Hadoop basics (presentation in PDF)
• Storage & Hive (presentation in PDF)
• MapReduce (presentation in PDF)
• Spark RDD (presentation in PDF, examples in Python)
• Spark SQL (presentation in PDF, examples in Python)

_

• Linux & HDFS (tasklist in PDF)
• Hive (tasklist in PDF)
• Spark RDD (tasklist in PDF)
• Spark SQL (tasklist in PDF)

_

PDF

• for assessment: at least 25 points (50 possible) got for tests and homeworks; the more points, the better position for the exam
• for exam: a short interview on theoretical topics, the final mark is “sum” of assessment points and exam performance

• Metacentrum Hadoop reference page
• HDFS DFS commands
• Learn python in Y minutes
• Official python documentation
• Regular expressions at Ryan's tutorials
• Hive language manual
• Apache Spark manual
• PySpark SQL manual
• CSV files import/export

Course coordinator: Jan Hučín

Hadoop: The Definitive Guide, 4th Edition, by Tom White