1 of 54

ECE 5755: Modern Computer Systems and Architecture

Lecture 1: Introduction

Ada Lovelace

​

Credited for being the first computer programmer on the theoretical machine called the “Analytical Engine”

2 of 54

Initial questions:

  • How many students have:
    • Never taken a digital logic or computer architecture course
    • Taking an introductory digital logic and computer organization course
    • Taken an upper undergraduate or graduate computer architecture course
  • How many students are:
    • ECE majors
    • CS majors
    • IS majors
    • Other?

Photos, illustrations, graphics here.

3 of 54

Today’s Agenda

  1. Why take this course?
  2. Course logistics

Photos, illustrations, graphics here.

4 of 54

Photos, illustrations, graphics here.

5 of 54

The first digital computer! (1945)

Electronic, programmable, general purpose

  • Designed primarily by John Mauchly and J. Pesper Eckert

​

  • Programmed primarily by six female mathematicians:
    • Jean Jennings, Marlyn Wescoff, Ruth Lichterman, Betty Snyder, Frances Bilas, and Kay McNulty

Photos, illustrations, graphics here.

6 of 54

The first digital computer! (1945)

Electronic, programmable, general purpose

ENIAC consisted 18,000 vacuum tubes, 7,200 crystal diodes, 1,500 relays, 70,000 resistors, 10,000 capacitors, and approximately 5,000,000 hand-soldered joints.

  • It weighed 30 Tons!
  • Took up a room 1800 sq. feet in size
  • And consumed 150 kW of electricity

How many FLOPS?

Photos, illustrations, graphics here.

7 of 54

The first digital computer! (1945)

Electronic, programmable, general purpose

ENIAC consisted 18,000 vacuum tubes, 7,200 crystal diodes, 1,500 relays, 70,000 resistors, 10,000 capacitors, and approximately 5,000,000 hand-soldered joints.

  • It weighed 30 Tons!
  • Took up a room 1800 sq. feet in size
  • And consumed 150 kW of electricity

500!

Photos, illustrations, graphics here.

8 of 54

Modern computers today

iPhone 14

100mm2

5W

~TFLOPs

NVIDIA A100

12x5x2 inches

250+Watts

~100 TFLOPs

Photos, illustrations, graphics here.

9 of 54

Modern computers today

iPhone 14

100mm2

5W

~TFLOPs

NVIDIA A100

12x5x2 inches

250+Watts

~100 TFLOPs

Photos, illustrations, graphics here.

10 of 54

Original prediction by Gordon Moore

Photos, illustrations, graphics here.

11 of 54

Original prediction by Gordon Moore

Photos, illustrations, graphics here.

12 of 54

The decades long impact of Moore’s Law

2x/1.5yrs

(52% per year)

Compute performance →

Image source: Hennessy and Patterson

Photos, illustrations, graphics here.

13 of 54

The decades long impact of Moore’s Law

2x/1.5yrs

(52% per year)

Slowing Moore’s Law

Compute performance →

Moore’s law has died at the age of 51 after an extended illness…

for now, lawbreaking is going to be the new normal - Ars Technica

Image source: Hennessy and Patterson

Photos, illustrations, graphics here.

14 of 54

The impact of Moore’s Law slowing down

Power eff. trends

CPU perf trends

Disk $/byte trends

DRAM $/byte

trends

Technology

Plateaus

​

Photos, illustrations, graphics here.

15 of 54

But computing demand continues to grow!

ELMo (93.6M)

BERT(340M)

GPT-2 (1.5B)

Megatron-LM (8.3B)

GPT-3 (175B)

LaMBDA (137B)

Gopher (280B)

PaLM (540B)

Megatron-Turing NLG (530B)

T5 (11B)

~10x/yr trajectory

Scaling of "dense" large language models

Photos, illustrations, graphics here.

16 of 54

Emerging applications introduce new requirements

Photos, illustrations, graphics here.

17 of 54

Computing faces new, exacerbating challenges

Photos, illustrations, graphics here.

18 of 54

Computing is drawing massive amounts of power!

  • Virginia electricity prices rose by 267% over 5 years due to data centers

​

  • Utilities restarting coal plants to meet data center demand

​

  • Cost burden falls on residential ratepayers, not operators

Photos, illustrations, graphics here.

19 of 54

Water: Medium data center uses similar water to small town

  • Google’s Uruguay data center: 2M galloons/day (~55000 households)

​

​

  • Meta faced similar backlash to Arizona data center

​

  • Water is #1 reason communities push back against data centers

20 of 54

Computing has a tremendous global impact too

20

1.2-2.2 Billion tons of CO2

​

  • On par with the aviation industry’s footprint
  • Around 4% of worldwide emissions

Mobile

Communication

Data center

Computing’s emissions are rising given its growing demand!

21 of 54

Technology companies pledging to be carbon neutral

Apple produces 25 million metric tons of CO2 every year

Photos, illustrations, graphics here.

22 of 54

Technology companies pledging to be carbon neutral

Apple produces 25 million metric tons of CO2 every year

Photos, illustrations, graphics here.

23 of 54

Slowing

Supply

(Moore’s law)

Growing Demand

(New applications)

Problem

Exacerbating Challenges (Security/Privacy, Sustainability)

Photos, illustrations, graphics here.

24 of 54

Slowing

Supply

(Moore’s law)

Growing Demand

(New applications)

Problem

Exacerbating Challenges (Security/Privacy, Sustainability)

Opportunities

Photos, illustrations, graphics here.

25 of 54

“A new golden age of computer architecture”

  • Dire need for innovation to meet demands and new challenges
    • End of Dennard Scaling and Moore’s Law demands architecture & systems innovation to improve perf./cost/energy
  • Security needs architectural innovation to prevent attacks
  • Domain Specific Architectures
    • AI is a prime example!
  • Open architectures and open implementations for everyone to contribute!

Photos, illustrations, graphics here.

26 of 54

Changing dynamics with ICT supply chain

Photos, illustrations, graphics here.

27 of 54

NVIDIA GPUs (GH200 and GB200)

Photos, illustrations, graphics here.

28 of 54

Consumer Devices: Intel Lunar Lake

Photos, illustrations, graphics here.

29 of 54

Consumer Devices: Apple SoC

Photos, illustrations, graphics here.

30 of 54

Processing in Memory units

Photos, illustrations, graphics here.

31 of 54

Themes throughout the class

Parallelism

Trade-offs

Co-design

  • Performance
  • Power efficiency
  • Energy
  • Cost
  • Task level
  • Core level
  • Instruction level
  • Data level

Photos, illustrations, graphics here.

32 of 54

5 minute break!

Photos, illustrations, graphics here.

33 of 54

Today’s Agenda

  1. Why take this course?
  2. Course logistics

Photos, illustrations, graphics here.

34 of 54

Topics covered

  • Processor Microarchitecture
    • Single cycle processor
    • Pipelined processors
  • Advanced Processor Microarchitecture
    • Branch prediction
    • Out of order execution
  • ML Overview
  • Memory Systems
    • Overview and motivation
    • Caching
    • Virtual memory
    • DRAM architecture

​

Photos, illustrations, graphics here.

35 of 54

Topics covered

  • Multi-core Architecture
    • Overview
    • Caching and cache coherence
    • Locks and barriers
    • Cache consistency
  • Advanced Topics
    • GPU architecture
    • Specialized Accelerators
    • Systems & Data centers
  • Student Presentations

Photos, illustrations, graphics here.

36 of 54

Course staff

Professor

Udit Gupta

Head TA

Yahya Emara

Photos, illustrations, graphics here.

37 of 54

Instructor: Udit Gupta

Graduated from Cornell in 2016 (ECE Major and CS Minor)!

​

PhD from Harvard in 2022

Enabling High Performance, Efficient, and Sustainable Deep Learning At Scale

​

Visiting Research Scientist at Meta (2019-2022)

​

I love to play chess, try new restaurants and food (vegetarian), hike, and find new TV shows!

Photos, illustrations, graphics here.

38 of 54

TA: Yahya Emara

  • 1st year PhD at Cornell Tech CS, working on efficient systems for Language Models
  • Graduated from ETH Zurich in 2025 (Zurich, Switzerland)
  • I like reading classics and swimming!

Photos, illustrations, graphics here.

39 of 54

Prerequisites

  • There is no official textbook for this course
  • However, if you want to follow a textbook you can read:
    • Computer Architecture: A Quantitative Approach (The Morgan Kaufmann Series in Computer Architecture and Design) 6th Edition�

​

Photos, illustrations, graphics here.

40 of 54

Evaluation

  • In class quizzes (30%)
  • Lab Assignments (50%)
  • Final paper & presentation (15%)
  • Class participation (5%)

Photos, illustrations, graphics here.

41 of 54

Lab Assignments (50%)

Over the semester you will build an ML Kernel library in C.

As you build this library you will optimize its performance gathering detailed performance counters along the way.

Photos, illustrations, graphics here.

42 of 54

Emerging LLM on CPU for deployment

Photos, illustrations, graphics here.

43 of 54

Lab Assignments (50%)

0. Getting to know your tools!, 5%

1. Implement the core ML Kernels (e.g., MatMul, Conv, ReLu), 10%

2. Optimize the ML operations using tiling, blocking, mixed precision 10%

3. Optimize the ML operations using multi-threading, 10%

4. Optimize the ML operations implemented using sparsity, 10%

5. Integrate the ML operations together for end-to-end NN inference benchmarks, 5%

​

Each assignment will come with theoretical questions to test your knowledge of course material

Photos, illustrations, graphics here.

44 of 54

Intel Top Down for Profiling

Photos, illustrations, graphics here.

45 of 54

Term Paper & Presentation (15%)

  • A critical assessment of how principles of computer architecture touched upon in this course are being investigated in modern day research (< 10 years) or having impact on real products
    • The role of speculation in Spectre and Meltdown attacks
    • New branch predictor methods (e.g., neuro branch prediction)
    • Hardware specialization for ML, AR/VR
    • Adding security protection mechanisms in processors
    • Emerging memory technologies
    • And many more!

​

Photos, illustrations, graphics here.

46 of 54

Term Paper & Presentation (15%)

  • Will be done as a pair (or a group of 3 if we have an odd number)!
  • Paper will be ~5 pages including figures but excluding references
  • You should reach ~8 papers or detailed industry reports or technical reports in your particular topic and summarize/compare-and-contrast them
  • Will be due as your final assessment

​

​

Photos, illustrations, graphics here.

47 of 54

Term Paper & Presentation (15%)

  • Based on your term paper you will prepare a group presentation to be done in class during the final weeks of the course
  • We will have signups for groups and times shortly!
    • You can also use Ed to find partners with similar interests!
  • The group presentation will be roughly 25 minutes long (~3 groups per class)
    • Make it fun and engaging!

​

Photos, illustrations, graphics here.

48 of 54

In-class Quizzes (30%)

  • At the end of each topic there will be a short in-class quiz.
  • Roughly 8 in total
  • We will drop your lowest score
  • You will know which classes have quizzes so be prepared!
    • If you are out of town, you must let us know in advance

​

Photos, illustrations, graphics here.

49 of 54

In class participation (5%)

  • Students are expected to contribute at least once every 2 lectures with either a question or idea!

Photos, illustrations, graphics here.

50 of 54

Course Tech

  • Webpage: https://s4ai-cornelltech.github.io/ece5755-fall2026/
  • Canvas Link: https://canvas.cornell.edu/courses/88439
  • Piazza: https://edstem.org/us/courses/103217/discussion
  • Zoom: Everyone should be attending class in person
    • If you have a medical emergency or extenuating circumstances you may email me and request a Zoom link.
  • You will get access to our compute server for doing performance analysis for the lab assignments
  • Submitting assignments: Canvas (zip up your code and submit it there!)

Photos, illustrations, graphics here.

51 of 54

Office Hours

  • Professor:
    • TBA
    • Bloomberg 268
  • TA: TBA

Photos, illustrations, graphics here.

52 of 54

Course Philosophy

  1. Fundamentals:
    • Most of the material and topics we will be covering are the fundamental concepts to modern computer systems and architecture. Using this foundation you will be able to adapt to various research and industry environments!
  2. Research and state-of-the-art:
    • You will be reading papers and technical reports to grasp new architectural concepts learned in the class. These will extend your ability to apply concepts learned in class to the real world, whether graduate school or industry.
  3. Practical:
    • You will implement the backend of ML frameworks that are used by researchers and engineers every day! This experience will give you practice hardware and software engineering experiences.
  4. Fun:
    • Assignments, presentations and papers should be fun! We will have “competitions” (extra-credit) on the final lab that you build to integrate the end-to-end NN framework.

Photos, illustrations, graphics here.

53 of 54

Honor Code

  • Plagiarism tools will be used on all submitted code and text.
  • If you use any third-party code/text, please cite it.
  • It is OK to discuss ideas with your colleagues but it is not OK to copy code.
  • See Cornell’s Code of Academic Integrity

Photos, illustrations, graphics here.

54 of 54

Summary

  • Excited for a fun semester!
  • Why take ECE 5755?
  • Course logistics

Photos, illustrations, graphics here.