Prerequisites

• DS-GA-1001: Intro to Data Science or its equivalent
• DS-GA-1002: Statistical and Mathematical Methods or its equivalent
• Solid mathematical background, equivalent to a 1-semester undergraduate course in each of the following: linear algebra, multivariate calculus (primarily differential calculus), probability theory, and statistics. (The coverage in the 2015 version of DS-GA 1002, linked above, is sufficient.)
• Python programming required for most homework assignments.
• Recommended: Computer science background up to a "data structures and algorithms" course
• Recommended: At least one advanced, proof-based mathematics course
• Some prerequisites may be waived with permission of the instructor
• You can also self-assess your preparation by filling out the Prerequisite Questionnaire

Grading

Homework (40%) + Midterm Exam (30%) + Final Exam (30%)

Many homework assignments will have problems designated as “optional”. At the end of the semester, strong performance on these problems may lift the final course grade by up to half a letter grade (e.g. B+ to A- or A- to A), especially for borderline grades. You should view the optional problems primarily as a way to engage with more material, if you have the time. Along with the performance on optional problems, we will also consider significant contributions to Piazza and in-class discussions for boosting a borderline grade.

Important Dates

• Midterm Exam (100 min) Tuesday, March 12th, 5:20–7pm.
• Final Exam (100 min) Thursday, May 16th, 6-7:50pm (confirmed).
• See Assignments section for homework-related deadlines.

Textbooks

The Elements of Statistical Learning (Hastie, Friedman, and Tibshirani)

This will be our main textbook for L1 and L2 regularization, trees, bagging, random forests, and boosting. It's written by three statisticians who invented many of the techniques discussed. There's an easier version of this book that covers many of the same topics, described below. (Available for free as a PDF.)

An Introduction to Statistical Learning (James, Witten, Hastie, and Tibshirani)

This book is written by two of the same authors as The Elements of Statistical Learning. It's much less intense mathematically, and it's good for a lighter introduction to the topics. (Available for free as a PDF.)

Understanding Machine Learning: From Theory to Algorithms (Shalev-Shwartz and Ben-David)

Last year this was our primary reference for kernel methods and multiclass classification, and we may use it even more this year. Covers a lot of theory that we don't go into, but it would be a good supplemental resource for a more theoretical course, such as Mohri's Foundations of Machine Learning course. (Available for free as a PDF.)

Pattern Recognition and Machine Learning (Christopher Bishop)

Our primary reference for probabilistic methods, including bayesian regression, latent variable models, and the EM algorithm. It's highly recommended, but unfortunately not free online.

Bayesian Reasoning and Machine Learning (David Barber)

A very nice resource for our topics in probabilistic modeling, and a possible substitute for the Bishop book. Would serve as a good supplemental reference for a more advanced course in probabilistic modeling, such as DS-GA 1005: Inference and Representation (Available for free as a PDF.)

Other tutorials and references

• Carlos Fernandez-Granda's lecture notes provide a comprehensive review of the prerequisite material in linear algebra, probability, statistics, and optimization.
• Brian Dalessandro's iPython notebooks from DS-GA-1001: Intro to Data Science
• The Matrix Cookbook has lots of facts and identities about matrices and certain probability distributions.
• Stanford CS229: "Review of Probability Theory"
• Stanford CS229: "Linear Algebra Review and Reference"
• Math for Machine Learning by Hal Daumé III

Software

• NumPy is "the fundamental package for scientific computing with Python." Our homework assignments will use NumPy arrays extensively.
• scikit-learn is a comprehensive machine learning toolkit for Python. We won't use this for most of the homework assignments, since we'll be coding things from scratch. However, you may want to run the scikit-learn version of the algorithms to check that your own outputs are correct. Also, studying the source code can be a good learning experience.

Section Leaders

Sreyas Mohan (Head TA)

sm7582@nyu.edu

Sreyas is a second year PhD student in the Data Science Program at CDS working with Prof. Carlos Fernandez-Granda and Prof. Eero Simoncelli.



Xintian Han

xh1007@nyu.edu

Xintian is a second year PhD student in the Data Science Program at CDS working with Prof. Rajesh Ranganath.