SURV753: Machine Learning II (ML II)

Data Analysis

Social scientists and survey researchers are confronted with an increasing number of new data sources such as apps and sensors that often result in (para)data structures that are difficult to handle with traditional modeling methods. At the same time, advances in the field of machine learning (ML) have created an array of flexible methods and tools that can be used to tackle a variety of modeling problems. Against this background, this course discusses advanced ML concepts such as cross validation, class imbalance, Boosting and Stacking as well as key approaches for facilitating model tuning and performing feature selection.  In this course we also introduce additional machine learning methods including Support Vector Machines, Extra-Trees and LASSO among others. The course aims to illustrate these concepts, methods and approaches from a social science perspective. Furthermore, the course covers techniques for extracting patterns from unstructured data as well as interpreting and presenting results from machine learning algorithms. Code examples will be provided using the statistical programming language R.

The course is structured such that each session focuses on specific prediction tasks and presents tools that can be used to tackle modeling problems in this setting. Topics include, e.g., accounting for informative data structures in the context of model training and tuning, dealing with class imbalance in categorical outcomes, building effective prediction models by applying cutting edge ML methods, and performing feature selection in high-dimensional data settings. The presented methods will be motivated from a social and survey science perspective and critically discussed with respect to their advantages and limitations.

Code examples will be provided using the statistical programming language R.

Course objectives: 

By the end of the course, students will…

  • will have a profound understanding of advanced (ensemble) prediction methods
  • have built up a comprehensive ML toolkit to tackle various learning problems
  • know how to (critically) evaluate and interpret results from ''black-box'' models

Grading will be based on

  • 4 homework assignments (10% each)
  • 8 online quizzes (5% each)
  • Participation in discussion during the weekly online meetings (20% of grade)

Dates of when assignment will be due are indicated on Canvas. Late assignments will not be accepted without prior arrangement with the instructors.


Topics covered in SURV751: Introduction to Machine Learning and Big Data (ML I), i.e.:

  • Conceptual basics of machine learning (training vs. test data, model evaluation basics)
  • Decision trees with CART
  • Random forests

Familiarity with the statistical programming language R is strongly recommended.

Participants are encouraged to work through one or more R tutorials prior to the first-class meeting. Some resources can be found here:

Course syllabus: 

Course Dates


Summer Term (June – August)


Fall Semester (September – December)