Cláudia Pinhão
UX/UI Designer
@Gemeente Amsterdam
Jason Fiawo
Copywriter
@Gemeente Amsterdam
Labeling Guide
Project Sidewalk
The anatomy of this guide
In this guide we want to share with you how to label Amsterdam correctly in Project Sidewalk. The guide is split into three chapters: labeling, validating, and visualizing data—each chapter has visual examples and a video.
Visit Project Sidewalk here.
Labeling data
1
Validating data
2
Visualizing data
3
Video tutorial + guidelines
Video tutorial
Video tutorial
Questions
4
Contact
Labeling data
HOW TO
General Guidelines
How to label?
You should place labels at the center of curb ramps or surface problems, and at the base of obstacles, otherwise it will give the wrong latitude and longitude values.
Try to get as close as possible to the object you are labeling. Labeling from farther away will result in less accurate results.
These are the currently available labels:
Curb Ramp
Missing
Curb Ramp
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Do
Add a curb ramp label in the centre
Do
Add a pedestrian signal label in the centre/base
Do
Add a obstacle label in the centre/base
Don’t
Do not add the curb ramp labels in the ends
Don’t
Do not add the pedestrian signal label on the button or top
Don’t
Do not add the obstacle label on the top or ends of the obstacle
Levels of severity
We use severity levels to get a better evaluation how difficult would be a wheelchair user to pass by in a certain location. The levels can be defined as:
Fully passable. (Curb ramps + Crosswalks) Wheelchair users do not have to reduce their speed or change route. The route is completely fluid.
(Other labels) There is a problem but it should be fully passable for every wheelchair users.
1
2
3
4
5
Not passable. Wheelchair users cannot pass there at all and there is no alternative route.
Almost not passable. Only a few wheelchair users can pass but with some risk or a lot of effort or with help. Alternative route might be present.
Passable. Wheelchair users clearly identify the problem or obstacle but are still capable of overcoming it alone by reducing their speed, stopping or changing route.
Almost fully passable. Wheelchair users have to slightly slow down or deviate a little bit from obstacles or other problems but without thinking too much about it.
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Curb Ramps
What are they?
A curb ramp is a short ramp that cuts through or builds up to a curb. An accessible curb ramp is one that provides an accessible route for people with mobility impairments to safely transition from a curbed sidewalk to a roadway, or vice versa.
Where can we find them?
We can find them often on the both ends of crosswalks, or when a sidewalk continue on the other side of the street, even without crosswalk.
curb ramp
curb
plan
section
At both ends of a crosswalk there should be curb ramps
At street corners or along big paths without crosswalks, there might be curb ramps to cross the street
At both ends of a crosswalk with an island in between there should be curb ramps on all ends of the crosswalk including in the island
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Where shall we label curb ramps?
Curb ramp
Curb ramp
Curb ramp
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Tags
Curb ramps have 8 associated tags further explained below:
The curb ramp has a small width for a wheelchair user
narrow
tactile warning
steep
not enough landing space
not level with street
surface problem
The curb ramp has a textured guide that indicates to blind or visually impaired people there is a crossing
The curb ramp has a sharp angle that leads to a more difficult rising and uncontrolled falling for a wheelchair user
The curb ramp leads to a narrow space in the sidewalk where turning a wheelchair is difficult or impossible
The curb is slightly lowered but not enough to be at the same level of the street, creating a small step
The curb ramp is broken, uneven or has grass, pooled water or debris
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Do
Curb ramp | severity: 1 | No tags
Do
Curb ramp | severity: 1 | No tags
Do
Curb ramp | severity: 2 | narrow
Do
Curb ramp | severity: 2
Do
Curb ramp | severity: 1 | No tags
Do
Curb ramp | severity: 5 | tag: narrow, not enough landing space, not level w/street, surface problem
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Do
Curb ramp | severity: 3 | tag: narrow
Do
Curb ramp | sev: 4 | tags: steep, surface problem
Do
Curb ramp | sev: 4 | tag: not enough landing space
Do
Curb ramp | severity: 3 | description: pooled water
Do
Curb ramp | severity: 1 | tags: tactile warning
Do
Curb ramp | severity: 3 | tag: not level with street
The curb ramp for garages should not be labelled as curb ramp because it is meant for vehicles
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
The curb ramp for driveways should not be labelled as curb ramp because it is meant for vehicles
Where shall we label curb ramps?
Not a curb ramp
Not a curb ramp
At both ends of a crosswalk that is leveled with the sidewalk. It should be labelled instead as crosswalk and tagged as ‘leveled with sidewalk’
Not a curb ramp
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Don’t
The crosswalk is levelled with the sidewalk
Don’t
This ramp is for vehicles not meant for people
Don’t
This ramp is for vehicles not meant for people
Don’t
It is a driveway
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Missing curb ramp at one of the ends of a crosswalk
Curb ramp present on one side of the of the street but missing on the other side
No curb ramp or crosswalk and no alternative route nearby
Where shall we label missing curb ramps?
Missing curb ramp
Missing curb ramp
Missing curb ramp
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Where shall we label missing curb ramps?
Abrupt end of sidewalks even if there is no sidewalk on the other side
Driveways running into sidewalks that cause an high difference
Missing curb ramp
Missing curb ramp
Tags
Missing curb ramps have 3 associated tags further explained below:
There is a visible alternative route nearby such as a driveway
Alternate route present
No alternate route present
Unclear
There is no visible alternative route or it is too far away
Unclear whether there should be a curb ramp here
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Do
Missing curb ramp | sev: 4 | tag: no alternate route
Do
Missing curb ramp | sev: 3 | tag: no alternate route
Do
Missing curb ramp | sev: 2 | tag: alternate route
Do
Missing curb ramp | sev: 5 | tag: no alternate route
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Do
Missing curb ramp | sev: 4 | tag: alternate route
Do
Missing curb ramp | sev: 5 | tag: no alternate route
When it is not a sidewalk it doesn’t need a curb ramp
Speed bumps are not crossings so they don’t need curb ramps
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Where shall we label missing curb ramps?
Not a missing curb ramp
Not a missing curb ramp
When the curb is even slightly lowered it is considered a curb ramp and tagged ‘not level with the street‘
Not a missing curb ramp
Don’t
There is no crosswalk or sidewalk on both sides
Don’t
If it is not a sidewalk it doesn’t need a curb ramp
Don’t
It is a speed bump and not a crosswalk so there is no need for curb ramps
Don’t
This curb ramp creates only a slight height difference in the road
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Obstacles in Path
What is it?
Obstacles are objects that are directly on the path of a pedestrian route, thus blocking the path. In Amsterdam the minimum sidewalk width should be 90cm of clear space. Clear space means the total width of the sidewalk subtracted by the width of obstacles such as bins, traffic signs, poles, between others.
This allows pedestrians, especially those using walkers or wheelchairs, to remain safely on the sidewalk or crosswalk. Moving off the path to avoid an obstacle may be impossible or may cause imbalance, tripping, or other hazards.
Where shall we label obstacles?
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Any obstacle (temporary or permanent) that leaves a remaining width of less than approx. 1,2 m in the sidewalk. Take as reference that a door is normally 80cm wide.
Any construction that blocks the sidewalk leaving less than approx. 1,2 m, should be labelled in the beginning and end (if it is long)
Obstacles that leave more than approx. 1,2 m of sidewalk free or that are in motion
Obstacle in path
Obstacle in path
Not an obstacle in path
Tags
Obstacles in path have 14 associated tags that help identifying their typology. Below we present some examples:
fire hydrant
pole
tree
vegetation
trash/recycling can
parked car
parked bike
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
construction
sign
stairs
height difference
narrow
litter/garbage
parked scooter/ motorcycle
Do
Obstacle in path | severity: 5 | tag: pole
Do
Obstacle in path | severity: 5 | tag: construction
Do
Obstacle in path | severity: 5 | tag: parked car
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Do
Obstacle in path | severity: 5 | tag: stairs
Do
Obstacle in path | severity: 2 | tag: garbage
Do
Obstacle in p. | sev: 5 | tag: parked bike + garbage
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Don’t
It leaves more than 1,2m passage
Don’t
It leaves more than 1,2m passage
Don’t
It leaves more than 1,2m passage
Don’t
It leaves more than 1,2m passage
Don’t
It leaves more than 1,2m passage
Don’t
Car seems to be in movement
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Surface Problem
What is it?
A surface problem is a problem that would cause a bumpy or otherwise uncomfortable experience for someone using a wheelchair or other assistive device. If something on a surface would make it hard or impossible to cross, it should be labeled as a surface problem. For surface problems that cover a large area, you should place a surface problem label at the start of the problem, and then continue placing labels every few meters (about 1 per image) until the end of the problem.
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Where shall we label surface problems?
Cracks, uneven or broken pavement that would be uncomfortable for wheelchair or other assistive devices users
Narrow sidewalks, that have less than 90cm.
Gravel, sand, grass or debris that interfere with an easy use of a wheelchair or assistive device
Surface problem
Surface problem
Surface problem
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Where shall we label surface problems?
Rail and tram tracks that are present in sidewalks
Brick pavement, if it isn’t uneven and does not have cracks, it is not considered a surface problem in Amsterdam
Surface problem
Not a surface problem
Tags
Surface problems have 10 associated tags that help identifying their typology. Below we present some examples:
bumpy
cracks
grass
narrow
construction
uneven/slanted
very broken
height difference
rail/tram track
sand/gravel
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Do
Surface problem | severity: 2 | tag: uneven
Do
Surface problem | sev: 3 | tag: uneven, grass
Do
Surface problem | severity: 3 | tag: tram tracks
Do
Surface problem | severity: 5 | tag: narrow
Do
Surface problem | severity: 2 | tag: grass
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Don’t
This brick work is not uneven or cracked
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
No Sidewalk
What is it?
A sidewalk is a paved path for walking, often alongside a road. As such, sidewalks give a separated path only for pedestrians making it safer to walk or jog.
In the absence of a sidewalk, and if there is no alternative route, pedestrians have to travel on the road risking their safety.
We consider that a no sidewalk label should be placed if there is a missing sidewalk.
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Where shall we label no sidewalk?
Street without any sidewalk. Place labels every other image until you reach a place where the sidewalk begins again
The same pavement is shared with cars and pedestrians
No sidewalk
Sidewalk ends abruptly
No sidewalk
No sidewalk
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Where shall we label no sidewalk?
Median that separates the two directions of traffic, unless the median has a sidewalk that ends abruptly
Not a no sidewalk
Add tag ‘street has a sidewalk’ if there is a sidewalk on the other side of the road
The pavement is for pedestrians only
Not a no sidewalk
No sidewalk
Tags
No sidewalk has 5 associated tags further explained below:
The sidewalk was expected to continue but ends abruptly
ends abruptly
street has a sidewalk
street has no sidewalks
gravel/dirt road
shared pedestrian/car space
The street has a sidewalk on the other side of the road
The street has no sidewalk at all
The street is a secondary road without any planned pavement. It is still in gravel or dirt
The entire street pavement is shared by both pedestrians, cars and other vehicles
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Do
No sidewalk | severity: 5 | tag: ends abruptly
Do
No sidewalk | sev: 5 | tag: street has a sidewalk
Do
No sidewalk | severity: 5 | tag: ends abruptly
Do
No sidewalk | severity: 2 | tag: street has a sidewalk
Do
No sidewalk | severity: 2 | tag: street has a sidewalk
Don’t
Median that separates the two directions of traffic
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Crosswalk
What is it?
A crosswalk is a legally defined space to cross a road, where the pedestrians have priority (providing they have taken due precautions before beginning to cross the road).
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Where shall we label a crosswalk?
Zebra crossing. If it has cracks or rail lines, increase severity and add the corresponding tag
Perpendicular dashed lines are an indication of crossing but pedestrians do not have priority. Add ‘no pedestrian priority’ tag
Speed bumps
Crosswalk
Crosswalk
Not a crosswalk
Tags
Crosswalks have 5 associated tags that help identifying their typology. Below we present some examples:
paint fading
broken surface
uneven surface
bumpy
rail/tram track
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Do
Crosswalk | sev: 1 (left), 2 (right)| tag: tram track
Do
Crosswalk | sev: 3 | tag: paint fading, broken surface
Do
Crosswalk | severity: 1 | tag: leveled with sidewalk
Do
Crosswalk | severity: 4 | tag: no priority
Don’t
It is a speed bump
Don’t
Not a crosswalk
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Pedestrian Signal
What is it?
A pedestrian signal allows a safer way for pedestrians to cross the street at signalized intersections. The pedestrian signal normally when green, provides enough time to the pedestrian to cross the street until it turns red again. Some of this signals are activated by a pedestrian detector push-button, that causes the controller to operate a pre-programmed time sequence.
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Waar moeten we een voetgangerssignaal neerzetten?
Pedestrian signal with push button or waist height push button. Don’t forget to add the label at the base and not at the button level
Pedestrian signal without push button
Push-buttons for bike riders
Pedestrian signal
Pedestrian signal
Not a pedestrian signal
Tags
Pedestrian signal have 2 associated tags further explained below:
The pedestrian signal has a push-button
has button
button waist height
The pedestrian signal has a button at a waist height so it is easier for a wheelchair user to reach it
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Do
Pedestrian signal | tag: button waist height
Do
Pedestrian signal | tag: has button
Do
Pedestrian signal | tag: has button
Do
Pedestrian signal | tag: no button
Do
Pedestrian signal | tag: no button
Don’t
Push-button for bike riders
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Can’t see the sidewalk
When does it happen?
Sometimes the view of the sidewalk is completely blocked. In these cases, you should place a "Can't See the Sidewalk" label so it can be manually inspected later on.
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Where shall we label can’t see the sidewalk?
Car driving or parked in front of the sidewalk, disabling the view
Pixelated or blurry image that disables a proper classification
Stretch of sidewalk is totally blocked from view by a short wall or vegetation between the road and sidewalk
Can’t see the sidewalk
Can’t see the sidewalk
Can’t see the sidewalk
Do
Can’t see the Sidewalk | no severity or tags
Do
Can’t see the Sidewalk | no severity or tags
Do
Can’t see the Sidewalk | no severity or tags
Do
Can’t see the Sidewalk | no severity or tags
Do
Can’t see the Sidewalk | no severity or tags
Do
Can’t see the Sidewalk | no severity or tags
Curb Ramps
Missing Curb Ramps
Obstacle in Path
Surface Problem
No Sidewalk
Crosswalk
Pedestrian Signal
Can’t see the Sidewalk
Validating data
HOW TO DO
Validating Data
Why do we do it?
Data validation allows us to filter wrongly labelled data and thus, increase quality.
In this section you will be shown images labelled by other participants and you will have to decide if you agree, disagree or if you are not sure.
Also, each participant with an account can see their level of accuracy. This is given by the amount of labels that were validated as correct or incorrect by others.
Visualizing data
HOW TO CHECK RESULTS
Visualizing data
Why do we do it?
We can visualize data in the following different ways:
1. Results map - offers a overview map where it is possible to check how far complete are the neighbourhoods and the amount of labels for each category;
2. Label map - offers a overview map of the city, but instead of neighbourhoods it pinpoints all labels by category;
3. Gallery - offers an image gallery where it is possible to filter by city, category, severity level and tag;
4. Leaderboard - Shows the top users, the amount of missions, tags, and distance complete. It also shares the accuracy level (e.g. if you label 500 labels and 250 were validated ‘disagree’, you have 50% accuracy.
1.
2.
3.
4.
Let’s share our ideas!
More info at:
sidewalk-amsterdam.cs.washington.edu
amsterdamintelligence.com