Software Engineering
for Machine Learning Systems
CC BY-SA 4.0 (photos covered separately)
Design
User centered design
A design framework of process in which usability goals, user characteristics, environment, tasks and workflow of a product, service or process are given extensive attention at each stage of the design process.
andreweland.org/swemls a.eland@imperial.ac.uk
The design process
Discovery
Ideate
Prototype
Test
andreweland.org/swemls a.eland@imperial.ac.uk
The design process
Initial ideas
UX research led
Finished features
Visual design led
Early prototypes
Information architecture led
High fidelity prototypes
Interaction design led
andreweland.org/swemls a.eland@imperial.ac.uk
Discovery
The discovery phase involves learning how people currently behave, often in complex environments unfamiliar to the people building a product.
UX researchers will often build personas, descriptions of fictional people who may use our system, including their background and role, to help the team better understand their users.
They’ll use these personas to consider what users says, do, think and feel at many stages throughout a journey of interaction with the system.
andreweland.org/swemls a.eland@imperial.ac.uk
A snapshot of user interaction with the Streams product.
andreweland.org/swemls a.eland@imperial.ac.uk
Initial design prototypes are often sketched on paper for speed.
andreweland.org/swemls a.eland@imperial.ac.uk
Sketches become higher fidelity over time, and more attention is paid to visual design.
andreweland.org/swemls a.eland@imperial.ac.uk
Clarity Focus
Noise Uncertainty Patterns Insights
The design squiggle by Damien Newman
Design for machine learning
Machine learning models are inherently probabilistic, user experiences generally aren’t. It’s very easy to build an interface that overpromises, and hence breaks trust.
andreweland.org/swemls a.eland@imperial.ac.uk
andreweland.org/swemls a.eland@imperial.ac.uk
andreweland.org/swemls a.eland@imperial.ac.uk
Design for machine learning
Machine learning models are inherently probabilistic, user experiences generally aren’t. It’s very easy to build an interface that overpromises, and hence breaks trust.
Machine learning models can produce decisions that are difficult for users to understand. Often we’ll need to explain a decision to build and maintain trust, or comply with convention and regulations in our domain.
andreweland.org/swemls a.eland@imperial.ac.uk
Design for machine learning
Machine learning models are inherently probabilistic, user experiences generally aren’t. It’s very easy to build an interface that overpromises, and hence breaks trust.
Machine learning models can produce decisions that are difficult for users to understand. Often we’ll need to explain a decision to build and maintain trust, or comply with convention and regulations in our domain.
As the decisions made, or answers given, by machine learning systems become more ‘human’, the way in which users react changes. It becomes a new material for design.
andreweland.org/swemls a.eland@imperial.ac.uk
andreweland.org/swemls a.eland@imperial.ac.uk
Clarity Focus
Noise Uncertainty Patterns Insights
The design squiggle by Damien Newman
1984
andreweland.org/swemls a.eland@imperial.ac.uk
1989
andreweland.org/swemls a.eland@imperial.ac.uk
1989
andreweland.org/swemls a.eland@imperial.ac.uk
1990
andreweland.org/swemls a.eland@imperial.ac.uk
1991
andreweland.org/swemls a.eland@imperial.ac.uk
1991
andreweland.org/swemls a.eland@imperial.ac.uk
1995
andreweland.org/swemls a.eland@imperial.ac.uk
1995
andreweland.org/swemls a.eland@imperial.ac.uk
2014
andreweland.org/swemls a.eland@imperial.ac.uk
2014
andreweland.org/swemls a.eland@imperial.ac.uk
Coursework six:
Running in the live environment
Timeline
Lecture: design (you are here - assessment starts now)
Lab
No lecture
Possibly prizes??? (assessment ends 11am)
Coursework 6
andreweland.org/swemls a.eland@imperial.ac.uk
SLA for coursework 6
Maintain a better f3 score than the NHS model.
Alerts delivered within 3 seconds of having the necessary data.
Your mark will be the fraction of time during the testing period that your service meets the SLA.
Adherence to the SLA is recomputed with each new AKI event.
andreweland.org/swemls a.eland@imperial.ac.uk
Coursework 6 simulator
Running now!
env:
- name: MLLP_ADDRESS
value: GROUP_NAME-simulator.coursework6:8440
- name: PAGER_ADDRESS
value: GROUP_NAME-simulator.coursework6:8441
initContainers:
- name: copy-hospital-history
image: ...azurecr.io/coursework6-history
andreweland.org/swemls a.eland@imperial.ac.uk
Alerting
Alerts will be sent by email to each member of your team if:
There’s no pod running in your group’s namespace with the name aki-detection for 10 minutes.
Your pod doesn’t respond with HTTP 200 OK on /metrics on it’s declared port.
Your F3 score drops below the NHS F3 score.
Your latency rises above 3s.
Alerting rules are in gitlab. Feel free to add your own!
andreweland.org/swemls a.eland@imperial.ac.uk
Assessment period
The assessment period starts now. The first AKI event occurs just after 9:10am tomorrow morning.
We will be supporting the lab session on Friday as usual.
andreweland.org/swemls a.eland@imperial.ac.uk
Good luck.
andreweland.org/swemls