SURV667: Introduction to Record Linkage with Big Data Applications

Area: 
Data Curation/Storage, Data Generating Process
Credit(s)/ECTS: 
2/4
Core/Elective: 
Elective

The demand for using different data sets in a “combined way” to analyse research questions is increasing. This is where record linkage comes into play as the common technique to integrate seperate data sets.

The course will provide an introduction to record linkage: it will address methods to combine data on given entities (people, households, firms etc.) that are stored in different data sources. By showing the strengths of these methods and by providing numerous practical examples ranging from linked survey and administrative data to Big Data applications, the course will demonstrate the various benefits of record linkage. Participants will also learn about potential challenges record linkage projects may face.

The schedule of the course will follow a prototypical record linkage process:

  • the need for common identifiers (e.g., names, addresses, birth dates) and the importance of assuring high data quality even during the planning phase of each project,
  • preparation of these identifiers before the actual linkage,
  • increasing the efficiency of the matching step (different blocking techniques),
  • alternative ways of conducting the comparison step, namely rule-based, distance-based and probabilistic record linkage,
  • as data protection requirements are an important issue in many applications, methods of privacy preserving record linkage are discussed,
  • evaluation and visualization of different quality aspects of the linkage result.

Numerous practical examples will give participants an opportunity to create and discuss their own ideas for promising record linkage projects. By the end of the course participants will be able to assess the feasibility of, plan and manage record linkage projects as well as to perform each step along the linkage process using the R software.

Course objectives: 

By the end of the course, students will…

  • be familiar with a host of record linkage applications from different countries or jurisdictions that link a variety of data sources and use different types of linkage
  • know how to improve the quality of linkage identifiers by applying pre-processing routines
  • be familiar with different methods of increasing the efficiency of record linkage
  • be able to understand, select and apply appropriate record linkage methods (e.g., deterministic and probabilistic linkage)
  • be able to evaluate the success of data linkage
  • be able to perform each step in the record linkage process using the R software
Grading: 

Grading will be based on:

  • 3 online quizzes (worth 10% each, 30% in total)
  • Participation in the weekly online meetings (20% of grade): engagement in discussions during the meetings and submission of questions in the forum on the course website (deadline: Tuesday, 1:00 PM EST/7:00 PM CET, i.e. 24 hours before each online meeting)
  • 3 homework assignments (worth 50% in total)

Dates of when assignment will be due are indicated in the syllabus. Extensions will be granted sparingly and only with prior arrangement with the instructors.

Prerequisites: 

Students should have knowledge of basic statistical concepts. They need to have an intermediate knowledge of R. Familiarity with regular expressions, the R packages ggplot2 and tidyverse is useful but not required.

Course syllabus: 

Course Dates

2020

Summer Term (June – August)

2021

Summer Term (June – August)