Evaluation Metrics 7.6.22.xlsx

	B	C	D	E	F
1
2	State of good repair
3	Metrics and measures	Service Board	Type	Discussion from 7/6/22	Additional Notes
4	State of good repair: < 1 year beyond useful life, > 1 year beyond useful life or poor condition	Pace	Categorical
5	Extends the useful life of assets: N/A or unsure, yes	Metra	Categorical
6	Condition (FTA TERM Rating): Asset(s) rated 4.5-5, 3.5-4.4, 2.5-3.4, 1.5-2.4, below 1.4	Metra	Categorical	These options are easiest because they align with TAM plans, FTA guidance, facility condition assessments. Distinguish vehicles v. facilities. Pace: Vehicles are done more by age per condition and miles. Project sponsors use TAM data, also subjective rating.
7	Useful life (FTA Benchmark): Not exceeded, less than 6 years, less than 5 years, less than 2 years, less than 1 year or exceeded	Metra	Categorical
8	Specific failures: Failures likely or prevented, occasional failures, several failures at consistent rate, several failures at increasing rate	Metra	Categorical
9	Majority survey response as most suitable type of metric: Quantitative			CTA uses TAM info and project experts with extensive experience. Most of their fleet is beyond a standard lifetime. SOGR form is included with call for projects. This includes asset performing function, safety risk, lifecycle cost met/recovered, condition assessment, accessibility feature included.
10
11	Customer experience
12	Metrics and measures	Service Board	Type	Discussion from 7/6/22	Additional Notes
13	Stay competitive: Improvements to rider experience, technology improvements, and/or ridership increase	Pace	Qualitative	Pace: Qualitative two-point item.
14	Customer facing improvements: Indirect and limited, indirect and minor, direct and major, direct and significant	Metra	Categorical	Metra: Also has ridership quantified (indicating magnitude of impact by station quantile.)
15	Impact to comfort/convenience: No impact, improves or renews the lifecycle on relevant assets, significantly and directly improve rider comfort over current conditions	CTA	Categorical
16	Riders impacted: Project is removed from impacting riders, indirectly impacts a large proportion of daily riders or directly impacts a small proportion, directly impacts and will maintain/improve service for a high proportion of daily riders	CTA	Categorical	CTA: Currently not quantitative, though TAM is being incorporated into cost model.
17	Ridership Quantile (Stations from 2018 On/Off Survey): First quantile (lowest), second quantile, third quantile, fourth quantile, fifth quantile (highest)	Metra	Categorical	Metra suggested including this as a "customer experience" metric.
18	Majority survey response as most suitable type of metric: Qualitative
19
20	Access to key destinations
21	Metrics and measures	Service Board	Type	Discussion from 7/6/22	Additional Notes
22	Access to key destinations: Jobs, retail, healthcare, recreation, etc.	Pace	Qualitative	Metric is evolving - look at associated routes. Evaluators can ask questions of subject matter experts.
23	Access to jobs (regional employment centers): Indirectly improves access to ECs, directly improves access to ECs	Metra	Categorical	Is this based on GIS? Used as a gut check but this is more qualitative by evaluators. Employment Centers have been defined as an internal data product.
24	Majority survey response as most suitable type of metric: Categorical			Note: This is an important factor for IIJA. RTA: Take a regional view in assessing with regional connections? CTA studies this too, in the background. It is part of project planning summary documents after the call for projects. GIS usage is growing, used to map the potential projects. Timing of this is mixed, often after call for projects.
25
26	Racial equity and mobility justice
27	Metrics and measures	Service Board	Type	Discussion from 7/6/22	Additional Notes
28	Equity: Racial equity, factoring demographics of service area or location or project	Pace	Qualitative	Pace: Uses a proprietary tool from low/no and bus/facilities grant applications. This used GIS to study North Division routes vs demographic targets/disadvantaged communities.
29	Equity geography: N/A, project adjacent to equity geography, project in equity geography	Metra	Categorical	Metra: Try to keep it simple, using data from RTA, CMAP, USDOT (Justice40). This is using Census data based on project location in GIS for now.
30	Majority survey response as most suitable type of metric: Categorical			CTA studies this too, in the background. It is part of project planning summary documents after the call for projects. GIS usage is growing to map the potential projects. Timing of this is mixed, often after call for projects. Note that the location of a project may not match the market it serves.
31
32	Economic development
33	Metrics and measures	Service Board	Type	Discussion from 7/6/22	Additional Notes
34	Economic development: Positive economic impact to area/location of project	Pace	Qualitative	This is more qualitative, not a set definition of economic development.
35	Impact to immediately adjacent community (<1/4 mile radius): No/minimal impact on economic development, significant impact on economic development	CTA	Categorical	Data would be through discretionary grant programs - this is more state of good repair focused. Will incorporate going forward.
36	Impact to surrounding community (1+ mile radius): No/minimal impact on economic development, indirect/modest impact on economic development, significant benefit to economic development	CTA	Categorical
37	Majority survey response as most suitable type of metric: Qualitative			A transit center would be a good example. Economic development could mean development near a facility, or new jobs at a rail yard. It could also mean access to work. Different categories of projects may be different. Yes, we should include job creation from investment. Also consider whether the jobs are for the community residents. CTA views the program as a pipeline. Interpret this goal as benefiting economically disadvantaged areas, not specific neighborhoods. Encourage higher skilled trades. Stimulating growth nearby is very complex, not feasible for every project. Can say that a transit center has potential to accelerate growth. (May need catalyst if colder market.) Transit development role?
38
39	Climate impact
40	Metrics and measures	Service Board	Type	Discussion from 7/6/22	Additional Notes
41	Environmental impact: Zero-emission/electrification, CNG, other	Pace	Categorical
42	Impact to environmental sustainability goal: No/minimal direct/indirect impact on resource conservation or waste reduction, moderately/indirectly improves conservation & waste, significantly improves conservation & waste	CTA	Categorical	Federal programs prioritize low emissions, and this guides investments. Climate impact is not in the capital evaluation, more in planning/agency goals. Also CMAQ program guiding candidate projects.
43	Majority survey response as most suitable type of metric: Qualitative			Metra has new measure on emissions: USDOT has a tool (Areas of Persistent Poverty & Historically Disadvantaged Communities) with indicators at the Census tract level. Mobile emissions from vehicles depend on vehicle type, locomotives have more impact than fleet vehicles. (May be more complex for other service boards.) Note that transit ridership also has climate benefit vs driving.
44
45	Reliability improvement
46	Metrics and measures	Service Board	Type	Discussion from 7/6/22	Additional Notes
47	Reliability: Reliability of the capital item or service provided	Pace	Qualitative
48	Impact on service reliability: No direct impact, moderately reduces major service delays (1-10%) and/or indirectly support reliability, significantly reduces major service delays (>10%) and/or required per lifecycle replacement needs	CTA	Categorical
49	Impact to service speeds/travel time: No impact on speed/travel time, indirectly improves, clearly decreases travel times or is necessary to prevent reductions in service speeds	CTA	Categorical
50	No majority survey response for this topic.			Metra has a reliability criteria factor within customer service, based on type of project.
51
52	Capacity needs
53	Metrics and measures	Service Board	Type	Discussion from 7/6/22	Additional Notes
54	Capacity: Expansion of service or seating or facility space	Pace	Qualitative
55	Productivity of service: No/minimal impact on operating capacity, indirect impact on operating capacity, significantly increases operational capacity	CTA	Categorical
56	Majority survey response as most suitable type of metric: Quantitative			Metra: There is a lack of knowledge of project impacts without more design and modeling. Treat this as a judgement call. Rolling stock - bulk purchases of cars.
57
58	Safety
59	Metrics and measures	Service Board	Type	Discussion from 7/6/22	Additional Notes
60	Safety: Security also a factor in addition to safety improvements	Pace	Qualitative
61	Safety benefits for customers and/or employees: Limited and indirect, negligible and indirect, marginal and/or indirect, high and direct benefit	Metra	Categorical
62	Project type: Indirect and limited impact on safety, indirect and minor impact on safety, direct and minor impact on safety, direct and major impact on safety	Metra	Categorical
63	Impact on customer safety: Does not impact and/or assets fail in safe manner, indirect impact or assists employees in monitoring, mitigates risk of catastrophic failure and/or addresses safety hazards	CTA	Categorical
64	Impact on employee safety: Does not impact and/or assets fail in safe manner, indirectly improves employee safety, mitigates risk of catastrophic failure and/or addresses safety hazards	CTA	Categorical
65	Impact on system security: Does not impact, enhances or renews existing security, implements protection and prevention	CTA	Categorical
66	Majority survey response as most suitable type of metric: Qualitative			CTA: Different agencies may evaluate differently. This is easier to measure for roadway projects. The grant program 'Safe Streets and Roads for All' requires an action plan.
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100