1 of 55

CSCI-SHU 376: Natural Language Processing

Hua Shen

Course Agenda: 2026 Fall-NLP-[CSCI-SHU-376]-Class Schedule

2026-08-31

Fall 2026

Lecture 1: Introduction

2 of 55

Welcome to NLP course👏 !

Hello from Your Instructor!

Hua Shen

Assistant Professor of Computer Science

huashen@nyu.edu | huashen218

Research: Human-AI Alignment

Website: https://hua-shen.org/

3 of 55

Outline

  1. What is Natural Language Processing (NLP)?
  2. Course Overview and Logistics
  3. Discussion: Know more about You 🧡 !

4 of 55

Outline

  1. What is Natural Language Processing (NLP)?
  2. Course Overview and Logistics
  3. Discussion: Know more about You 🧡 !

5 of 55

What is Natural Language Processing

6 of 55

What is NLP?

  • Natural Language Processing:
    • build program to automatically analyse, understand and generate human language in text
    • Important branch of Artificial Intelligence
  • NLP is an interdisciplinary field
    • Healthcare, Law, Finance, etc

​

​

7 of 55

What is NLP?

8 of 55

What is NLP?

9 of 55

What is NLP?

Gemini achieves gold-medal standard at the International Mathematical Olympiad

10 of 55

What is NLP?

Generative Agents: Interactive Simulacra of Human Behavior

11 of 55

  • 1950s–1980s: Rule-based NLP
    • Hand-crafted rules, grammars, lexicons
    • Tasks: parsing, MT, QA (ELIZA)
    • Brittle, non-scalable
  • 1990s–2000s: Statistical NLP
    • Data-driven, probabilistic models
    • N-grams, HMMs, PCFGs, MaxEnt
    • Tasks: POS, NER, SMT
    • Enabled by large corpora (Penn Treebank)
  • 2000s–2013: Feature-Based ML NLP
    • SVMs, CRFs, heavy feature engineering
    • Sequence labeling, parsing, IE
    • Limited transfer, costly features

NLP Landscape & History

12 of 55

  • 2013–2017: Neural NLP
    • Word embeddings (Word2Vec, GloVe)�RNNs, LSTMs, attention
    • Seq2Seq models
  • 2018–2020: Transformers & Pretraining
    • Self-attention, pretrain–finetune
    • BERT, GPT, T5
    • Benchmark leaps (GLUE)
  • 2021–Now: Foundation Models / LLMs
    • Scale, in-context learning, RLHF
    • Zero-/few-shot generalization
    • New challenges: bias, hallucination, cost, safety

NLP Landscape & History

13 of 55

  • Task Landscape
    • Syntax → Semantics → Discourse
    • Tagging, parsing, NER, QA, MT, summarization, dialogue
  • Resources
    • Data (Wikipedia, Common Crawl)
    • Compute (GPUs/TPUs)
    • Tooling (NLTK, spaCy, HuggingFace)
  • Open Directions
    • Multimodal NLP
    • Low-resource languages
    • Explainability & Controllability
    • Human–AI collaboration
    • Safety and Alignment

NLP Tasks & Directions

ACL 2026 Reference: https://2026.aclweb.org/calls/main_conference_papers/

14 of 55

Know More about State-Of-The-Art (SOTA) NLP

Three top *CL conferences

  • ACL
  • EMNLP
  • NAACL

ACL 2026 Reference: https://2026.aclweb.org/calls/main_conference_papers/

15 of 55

Know More about State-Of-The-Art (SOTA) NLP

Three top Machine Learning conferences

  • ICLR
  • NeurIPS
  • ICML

ICML 2026 Reference: https://openreview.net/group?id=ICML.cc/2026

16 of 55

Accepted Top-Conference Papers from Prior NYU Shanghai Course

Congratulations!

  • Baosheng Jin, Yushen Liang, Hua Shen. STAGE: Diagnosing Semantic Transfer at Grounded Execution in Embodied Agents. EMNLP 2026 Findings. (From CSCI-SHU 376: Natural Language Processing, Spring 2026)
  • Jiajun Chen, Hua Shen. Value Alignment Tax : Measuring Value Trade-offs in LLM Alignment. ACL 2026 Findings. (From CSCI-SHU 205: Topics in Computer Science – Human-AI Alignment)

17 of 55

NLP History 1: Statistical and Feature-Based NLP

https://medium.com/@antoine.louis/a-brief-history-of-natural-language-processing-part-1-ffbcb937ebce

Rule-based NLP

18 of 55

Rule Based NLP

  • Rule based system, require careful programming
  • Limited Domains

19 of 55

Statistical NLP

20 of 55

Statistical NLP

  • Use machine learning approaches for NLP
  • Statistical Machine Translation

21 of 55

https://medium.com/@antoine.louis/a-brief-history-of-natural-language-processing-part-1-ffbcb937ebce

NLP History 2: NLP with Deep Learning

22 of 55

NLP with Deep Leaning

  • Significant Progress in NLP

​

  • Lots of Compute resources, large corpora

​

  • Little Feature Engineering

​

23 of 55

NLP with Deep Leaning – Neural Machine Translation

24 of 55

NLP with LLMs

  • Pre-trained on large corpus
  • Fine-tuning / prompting on tasks
  • One model for many tasks

​

​

25 of 55

State-of-the-art LLMs

26 of 55

State-of-the-art LLMs

Model

Maker

Note for class

Claude Opus 5 / Claude Fable 5

Anthropic

Top of agentic + text leaderboards

GPT-5.6 (Luna/Terra/Sol tiers)

OpenAI

Leads math benchmarks (LiveBench ~96)

Gemini 3.1 Pro

Google

~98% on ARC-AGI-1, ties human panel; native search grounding

Grok 4.6

xAI

Strong agentic performance, real-time X data

Kimi K3

Moonshot

Best open-weight model — 2.8T params (MoE), 1M-token context

DeepSeek V4

DeepSeek

Best price-performance; ~50× cheaper than frontier closed models

GLM 5.3

Z.ai

MIT-licensed, near-frontier

August 2026 frontier (source, release timeline):

  1. The open-weight gap has nearly closed — Kimi K3 scores within a few points of the best closed models, and half this table is from Chinese labs (Moonshot, DeepSeek, Z.ai, plus Qwen). Very relevant for your students.
  2. Price collapse: frontier-quality inference costs fell ~10× in two years — why agentic systems (Lecture topic!) suddenly became economical.

27 of 55

Language Model Hallucinates

28 of 55

Hallucination is an open question

  • BingChat: Retrieval Augmentation

29 of 55

NLP Subfields

  • Machine Translation
  • Question Answering
  • Information Extraction
  • Text Summarization
  • Syntactic Parsing
  • Semantic Parsing
  • ….

​

30 of 55

Why NLP is (Still) Hard?

  • Ambiguous
  • Listener has to infer --- Pragmatics
  • ….

31 of 55

Lexical Ambiguity

32 of 55

Syntactic Ambiguity

33 of 55

Semantic Ambiguity

34 of 55

Pragmatics

  • Language conveys information about a user’s preference

​

​

35 of 55

Outline

  1. What is Natural Language Processing (NLP)?
  2. Course Overview and Logistics
  3. Discussion: Know more about You 🧡 !

36 of 55

Logistics

  • Lectures: Mon/Wed 11:15am – 12:30pm, S311
  • Office Hour: Friday 2:30-4:30pm (send me an email beforehand is preferred)
  • Course Site: Brightspace
    • Contains all information about the course (dates, contents, readings etc)
    • Announcements will be posted on Brightspace as well
    • Slides will be available before each class

37 of 55

Course Topics

38 of 55

Course Goal

  • Understand basics of NLP (standard frameworks, algorithms etc)
  • Know recent advances (e.g., LLM) and NLP applications
  • Get hand-on experience, through assignments and final projects

39 of 55

Course Structure

Grading breakdown

      • Attendance and Quizzes: 15%
      • Homework Assignments: 15%
      • Mid-term Exam: 25%
      • [Project] Proposal: 5%
      • [Project] Mid-term Presentation: 10%
      • [Project] Final Report and Presentation: 30%

40 of 55

Course Structure

Quizzes (15%)

    • There will be 7 quizzes, each quiz will focus on the previous topic we discussed in class (e.g., the first quiz will be about N-gram LMs).
    • Each quiz will include one single/multi-option questions, free-answer questions (for 10-15 mins), about the key concepts in the corresponding topic.
    • Quiz will be closed book.
    • Your lowest quiz score will be removed.

41 of 55

Course Structure

Assignments (15%)

1: LM, text classification, word embedding (5%)

2: sequence modelling (5%)

3: Transformer (5%)

    • Each assignment consists of programming (in Python) and written components, and usually has 2 weeks
    • You will have 48 free late hours for all assignments

​

42 of 55

Course Structure

Midterm Exam (25%)

    • True / False, Multiple-choice, free-answers
    • You are allowed to bring a double sided A4 sheet
    • Grades will be curved

​

​

43 of 55

Course Structure

Final Project (45%)

  • Research Topic Candidates:
    • Topic 1 — Alignment & Safety of Agentic AI Systems
    • Topic 2 — Dynamic Alignment in Human-AI Co-Evolution
    • Topic 3 — Agentic AI for Scientific Research
    • Topic 4 — Agentic AI for Healthcare
    • Topic 5 — Multi-Modal Agents & World Models
  • Target Conferences:
    • ACL 2027
      • https://2027.aclweb.org/
    • ICML 2027

44 of 55

Course Structure

Final Project (45%)

Work on a specific problem with provided datasets and baselines. E.g., Agents, aiming for a research paper.

    • A team project with (usually) <= 3 people per team
    • Meet Hua at second week with your project ideas.
    • [Project] Proposal (5%): your ideas and a related paper (5%)
    • [Project] Mid-term Presentation (10%): relevant papers, and your progress / challenges (10%)
    • [Project] Final Presentation (10%): your final project
    • [Project] Final Report (20%): a conference style report

​

45 of 55

Final Project Team Registration

NLP-Fall-2026-Team Registration: https://docs.google.com/spreadsheets/d/1nsTyIjofy0XXook3JpRys8R-8Sx_AYAZ0KbwpvFj60A/edit?usp=sharing

​

46 of 55

2026: Human + AI in Classs

Human-AI Collaboration is HIGHLY Encouraged!: Leverage AI as you need in this course

47 of 55

  • Code editor + LLM

​

​

​

​

2026: AI + Coding

48 of 55

  • Operating system written by Cursor Only!!
  • Do we still need software engineer ???

​

​

​

​

2026: AI + Coding

49 of 55

2026: AI + Research

Van Noorden, R., & Perkel, J. M. (2023). AI and science: what 1,600 researchers think. Nature, 621(7980), 672-675.

50 of 55

Course Books

Textbooks

  • The most popular textbook in NLP
  • The book is frequently Updated

​

Link: https://web.stanford.edu/~jurafsky/slp3/

51 of 55

Prerequisite

  • We assume you have learned (or at least know a bit about) the following:
    • Python
    • Calculus, Probabilities and statistics
    • Supervised Learning and. Unsupervised Learning
    • Basics of Neural Networks

​

52 of 55

Computing Resource

— Generative AI Tools and Services in NYU Shanghai

Service

How to Access

Collect data?

Commercial

  • OpenAI.com (account needed)
  • Microsoft Bing (with built-in ChatGPT-5 functionality)
  • Public AI Tools (ChatGPT, Claude, Bard) - Direct Platform access; strictly prohibited for any NYU-related information

Yes (personal use only)

Institutional Licence

@NYU IT

  • Google Gemini & NotebookLLM - Available through NYU IT with NetID

No (NYU wide license)

Private By Request

  • Private Generative AI Pilot (OpenAI ChatGPT) - Submit project proposal via NYU IT; 3-5 days approval;

No

53 of 55

Outline

  1. What is Natural Language Processing (NLP)?
  2. Course Overview and Logistics
  3. Discussion: Know more about You 🧡 !

54 of 55

Love to know more about you!

What’s your experience + expectation on this course

55 of 55

​

​

  • Your name, major, year
  • Why do you choose this course / What do you know about NLP?
  • Do you have specific topics that you find interesting?

​

Students Introduction