How do we prepare students for
a world we cannot imagine?

Dylan Wiliam (@dylanwiliam)

Overview: Science and Design

  • We need to improve student achievement
  • This requires improving teacher quality
  • Improving the quality of entrants takes too long
  • So we have to make the teachers we have better
  • We can change teachers in a range of ways
  • Some will benefit students, and some will not
  • Those that do involve changes in teacher practice

  • Changing practice requires new kinds of teacher learning
  • And new models of professional development



Raising achievement matters

  • For individuals:
    • Increased lifetime salary
    • Improved health
    • Longer life
  • For society:
    • Lower criminal justice costs
    • Lower healthcare costs
    • Increased economic growth
      • Net present value to Canada of a 25-point increase on PISA: $4 trillion (seven times the National Debt)
      • Net present value to Canada of getting all students to 400 on PISA: $3 trillion (Hanushek & Woessman, 2010)


What is the purpose of education?

  • Four main philosophies of education
    • Personal empowerment
    • Cultural transmission
    • Preparation for citizenship
    • Preparation for work
  • All are important
  • Any curriculum is a (sometimes uneasy) compromise between these four forces

The coming war for jobs (Clifton, 2011)

  • Right now
    • 7 billion people on earth
    • 5 billion adults
    • 3 billion people who want to work
    • 90% of these want to work full time
  • As a consequence
    • 2.7 billion full-time formal jobs are wanted
    • with only 1.2 billion full-time formal jobs available
  • A shortfall of 1.5 billion jobs
  • So, for every Canadian worker, there are 80 people who would like their job…

International comparisons

  • Programme for International Student Assessment (PISA)

2009 2012

    • United States 496 492
    • United Kingdom 500 502
    • Australia 519 512
    • Canada 527 518
    • Finland 544 525
    • Shanghai 579 587

The world of work is changing


Skill category


Complex communication


Expert thinking/problem solving


Non-routine manual


Routine cognitive


Routine manual

Which kinds of skill are disappearing fastest from the workplace?

Change 1969-1999






Autor, Levy and Murnane (2003)

Terminal G, MSP airport

Presto tablet from eLaCarte

Meet Maddie Parlier…

Davidson (2012)

Madelyn (Maddie) Parlier runs the laser welding machine at Standard Motor Products, Greenville, SC, welding caps on to fuel injector bodies.

Davidson, A. (2012, January/February). Making it in America. The Atlantic Magazine

Computers in medical diagnosis

  • Pilot study of the use of neural nets to predict biopsy results
  • Sample: 1,787 men with a serum prostate-specific antigen (PSA) concentration > 4.0 ng/ml
  • Data:
    • Age
    • Maximum, average and change in PSA concentration over all visits
    • Maximum digital rectal examination over all visits
    • Maximum transrectal ultrasonography results over all visits
  • Positive predictive value (% of those predicted with positive biopsies)
    • Specialist urologists: 34%
    • Artificial neural nets: 77%


Snow, Smith and Catalona (1994)

Off-shoring and automation


Not off-shoreable



Security analyst

Tax accountant

Surgeon (?)




Food packager

Data entry clerk

Call centre operator

Grocery store clerk


Retail salesperson

Google’s Chauffeur Program

There is only one 21st century skill

So the model that says learn while you’re at school, while you’re young, the skills that you will apply during your lifetime is no longer tenable. The skills that you can learn when you’re at school will not be applicable. They will be obsolete by the time you get into the workplace and need them, except for one skill. The one really competitive skill is the skill of being able to learn. It is the skill of being able not to give the right answer to questions about what you were taught in school, but to make the right response to situations that are outside the scope of what you were taught in school. We need to produce people who know how to act when they’re faced with situations for which they were not specifically prepared. (Papert, 1998)

Successful education?

The test of successful education is not the amount of knowledge that a pupil takes away from school, but his appetite to know and his capacity to learn. If the school sends out children with the desire for knowledge and some idea how to acquire and use it, it will have done its work. Too many leave school with the appetite killed and the mind loaded with undigested lumps of information. The good schoolmaster is known by the number of valuable subjects that he declines to teach.

The Future in Education (Livingstone, 1941 p. 28)

So what should we improve?

Teacher quality vs. teaching quality

  • Teaching quality depends on
    • the amount of time the teacher has to plan teaching
    • the quality of the curriculum
    • the material resources available
    • class size
    • support of colleagues
    • support of the community
    • skill of the teacher


Two key levers for system improvement

  • A broad and balanced curriculum
  • Effective learning environments


21st Century skills?

  • Cognitive competencies
    • Cognitive processes and strategies
    • Knowledge
    • Creativity
  • Intra-personal competencies
    • Intellectual openness
    • Work ethic/conscientiousness
    • Positive core self-evaluation
  • Inter-personal competencies
    • Team-work
    • Leadership


Pellegrino and Hilton (2012)

21st century skills: What are they, really?

  • Many “21st century skills” appear to be universal, generalizable capabilities, such as
    • Creativity
    • Critical thinking
    • Problem-solving
    • Collaboration
    • Communication
  • There is little evidence that they are generalizable
  • They are best thought of as audit tools for the examination of the curriculum in each school subject


Principles for curriculum design

  • Balanced
  • Rigorous
  • Coherent
  • Vertically integrated
  • Appropriate
  • Focused/parsimonious
  • Relevant


Effective learning environments

  • Effective learning environments:
    • Create student engagement (pedagogies of engagement)
    • Are well-regulated (pedagogies of contingency)
    • Develop disciplinary habits of mind (pedagogies of formation)


The evidence base for formative assessment

  • Fuchs & Fuchs (1986)
  • Natriello (1987)
  • Crooks (1988)
  • Bangert-Drowns, et al. (1991)
  • Dempster (1991, 1992)
  • Elshout-Mohr (1994)
  • Kluger & DeNisi (1996)
  • Black & Wiliam (1998)
  • Nyquist (2003)
  • Brookhart (2004)
  • Allal & Lopez (2005)
  • Köller (2005)
  • Brookhart (2007)
  • Wiliam (2007)
  • Hattie & Timperley (2007)
  • Shute (2008)

The formative assessment hijack

  • Long-cycle:
    • Span: across units, terms
    • Length: four weeks to one year
    • Impact: Student monitoring; curriculum alignment
  • Medium-cycle:
    • Span: within and between teaching units
    • Length: one to four weeks
    • Impact: Improved, student-involved assessment; teacher cognition about learning
  • Short-cycle:
    • Span: within and between lessons
    • Length:
      • day-by-day: 24 to 48 hours
      • minute-by-minute: five seconds to two hours
    • Impact: classroom practice; student engagement

Unpacking formative assessment

Where the learner is going

Where the learner is

How to get there




Clarifying, sharing and understanding learning intentions

Engineering effective discussions, tasks, and activities that elicit evidence of learning

Providing feedback that moves learners forward

Activating students as learning

resources for one another

Activating students as owners
of their own learning

And one big idea

Where the learner is going

Where the learner is

How to get there




Using evidence of achievement to adapt what happens in classrooms to meet learner needs

An educational positioning system

  • A good teacher:
    • Establishes where the students are in their learning
    • Identifies the learning destination
    • Carefully plans a route
    • Begins the learning journey
    • Makes regular checks on progress on the way
    • Makes adjustments to the course as conditions dictate

So much for the easy bit

A model for teacher learning

  • Content, then process
  • Content (what we want teachers to change):
    • Evidence
    • Ideas (strategies and techniques)
  • Process (how to go about change):
    • Choice
    • Flexibility
    • Small steps
    • Accountability
    • Support




Supportive accountability

  • What is needed from teachers:
    • A commitment to:
      • The continual improvement of practice
      • Focus on those things that make a difference to students
  • What is needed from leaders:
    • A commitment to engineer effective learning environments for teachers by:
      • Creating expectations for continually improving practice
      • Keeping the focus on the things that make a difference to students
      • Providing the time, space, dispensation, and support for innovation
      • Supporting risk-taking


  • Raising achievement is important.
  • Raising achievement requires improving teacher quality.
  • Improving teacher quality requires teacher professional development.
  • To be effective, teacher professional development must address:
    • What teachers do in the classroom
    • How teachers change what they do in the classroom
  • Formative assessment + teacher learning communities:
    • A point of (uniquely?) high leverage
    • A “Trojan horse” into wider issues of pedagogy,
      psychology, and curriculum

Comments? Questions?

Force-field analysis (Lewin, 1954)

  • What are the forces that will support or drive the adoption of formative assessment practices in your school/district?
  • What are the forces that will constrain or prevent the adoption of formative assessment practices in your school/district?


id_362_how_do_we_prepare_students_for_a_world_we_cannot_imagine - Google Slides