What is this course about?

This graduate-level course will focus on an advanced study of frameworks, algorithms and methods in NLP -- including state-of-the-art techniques for problems such as language modeling, text classification, machine translation, and question answering. The course will contain multiple programming assignments, paper readings, a mid-term and a final project. Students are expected to have taken at least one introductory course in NLP/machine learning prior to this class, and be comfortable with programming in Python.

The course includes introductory lectures to related topics. These lectures are joint with COS 484, see lecture schedule here.

In addition, we will have precepts every Friday that cover more advanced material. Students are expected to attend these precepts, read the pre-assigned material and engage in Perusall, as well as answer pre-lecture questions on the topic of discussion.

Prerequisites:

Students are expected to have taken at least one introductory course in NLP/machine learning (484/429 or similar) prior to this class, and be comfortable with programming in Python.

Information

For most logistics and useful resources, if not specified below, please follow the guidelines in the COS484 page.

Time/location:

All the lectures/precepts/office hours are held on Zoom and all the Zoom links can be found on Canvas.

• Lectures: Monday, Wednesday 1:30-2:50pm ET, shared with COS 484
• Precepts: Friday 11-11:50am ET

Grading

• Assignments (30%): Assignments will be partially shared with 484, with extra questions potentially for 584. Please see the 484 page for logistics. The breakdown for assignments are: Assignment 0 (2%), Assignment 1-4 (7% each).
• Midterm exam (25%): The course has a take-home midterm that will test your knowledge and problem-solving skills on all material up to and including lecture on March 8th.
• Final project (35%): The final project requires you to write a research paper, similar as a conference submission. The paper should begin with an abstract and introduction, clearly describe the proposed idea or exploration, present technical details, give results, compare to baselines, provide analysis and discussion of the results, and cite any sources you used. You can work as a team of 1 or 2. You are required to turn in a project proposal (due on Mar 22) and complete a paper written in the style of a conference (e.g., ACL) submission (due on May 5). There will be also project presentations at the end of the semester.
• Precept partipation (10%), including pre-lecture quizzes, Perusall participation and in-class discussion.

Precept Schedule

Week	Date	Topics	Readings
1	Fri (2/5)	Language Models	J&M, 3.5 - 3.6
2	Fri (2/12)	Text classification	Wang and Manning (2012)
3	Fri (2/19)	Word embeddings	Levy et al., (2015)
4	Fri (2/26)	Feedforward Neural Networks	Iyyer et al., (2015)
5	Fri (3/5)	Conditional Random Fields	Sha and Pereira (2003)
6	Fri (3/12)	No meeting (midterm)
7	Fri (3/19)	Recurrent neural networks and neural language models	Grave et al., (2017)
8	Fri (3/26)	Dependency parsing	Kiperwasser and Goldberg (2016)
9	Fri (4/2)	Machine translation	Sennrich et al., (2016)
10	Fri (4/9)	Transformers	Rush (2018)
11	Fri (4/16)	Pre-training	Yang et al., (2019)
12	Fri (4/23)	Language Grounding	Bisk et al., (2020)