1 of 27

Extended Modeling, Calibration and Validity Assessment of Vehicle Models in FASTSim via Real-World Driving Data

Karim Hamza1, Peter Benoliel2, Kang-Ching Chu1, Ken Laberteaux1

1Toyota Research Institute of North America (TRINA)�2Institute of Transportation Studies, University of California at Davis

SAE Paper # 2022-01-0661

2 of 27

Agenda

  • Overview & Motivation
  • Black-Box vs White-Box Approaches for Fuel Economy Estimation
  • Our Previous Work in this Line of Research
  • Extension of the Previous Work
  • Real-World Trips Data
  • Results
  • Summary

3 of 27

Overview & Motivation: Large-Scale�Simulations of Real-World Trips

Initial SOC

Electric Consumption

Gas Consumption

Trip Length

[miles]

[gal-gas]

[kWh]

Index for trip ID

Vehicle Model Parameters

Trip End SOC

Time

Speed

Trip j

Driving�Dataset

e.g. California Household Travel Survey� (CHTS)

Example

Output

Statistical Distributions (Fuel & Electricity Consumption, SOC, …etc.)

Large-scale datasets of real-world driving becoming easier to obtain, many publicly accessible (e.g. NREL TSDC)

4 of 27

Overview & Motivation: Tuning & �Validation of Vehicle Models

Time [s]

Speed [mph]

Rel. Altitude [m]

HVAC [kW]

Real-World Trip(s)

Vehicle Model Parameters

Fuel/Energy Usage

(kWh, gal-Gas/Diesel and/or kg-H2)

Fuel/Energy Usage

(kWh, gal-Gas/Diesel and/or kg-H2)

Measured

Simulated

How Accurate?

  • Ideally, one would want simulation models of vehicles to match reality as closely as possible

5 of 27

“Black Box” (Data-based) �Fuel Economy Estimation

Time [s]

Speed [mph]

Rel. Altitude [m]

HVAC [kW]

Trip or Drive Cycle

Initial Battery State of Charge

Vehicle Model Parameters

Fuel/Energy Usage

(kWh, gal-Gas/Diesel and/or kg-H2)

Feature Extraction

  • Look-up Tables
  • Empirical Formulas
  • Response Surface
  • Neural Networks
  • Deep Learning

Examples

  • MOVES
  • EMFAC
  • Window-Sticker Ratings

Advantages

  • “Grounded” to Real-World Fuel/Energy Consumption
  • Less computational resources needed

Limitations

  • Lags behind latest technology improvements
  • Often less accurate for “non-standard” driving
  • Cannot readily model vehicle designs that don’t exist yet

6 of 27

“White Box” (Physics-based) �Fuel Economy Estimation

Time [s]

Speed [mph]

Rel. Altitude [m]

HVAC [kW]

Trip or Drive Cycle

Initial Battery State of Charge

Vehicle Model Parameters

Fuel/Energy Usage

(kWh, gal-Gas/Diesel and/or kg-H2)

Examples

  • Autonomie
  • FASTSim

Advantages

  • Conceptually easy(er) to justify
  • Can readily model non-standard driving patterns
  • Can readily model new/improved vehicle technologies

Limitations

  • Detailed/high-quality trip data required
  • Larger computational resources needed
  • Accuracy/Validity of Results remains an open question

7 of 27

Our Approach

White-Box Models

Black-Box Models

  • Mostly physics-based, but calibrated with real-world trips data

8 of 27

Our Previous Work / �Deeper Look at FASTSim

Total Instantaneous Power Demand

Time instant i

Acceleration

Rotational Inertia

Wind Drag

Rolling Resistance

Road Slope

Total Moving Mass

Acceleration

Velocity

Eq. Rotational Inertia at Wheels

Wheels Radius

Air Density

Coefficient of Drag

Front Projected Area

Gravity Acceleration

Road Slope

Rolling Resistance Coefficient

Auxiliary

9 of 27

Our Previous Work / �Deeper Look at FASTSim

Noted Modeling Idealizations

  • Input-Output power of Motor, Engine & Fuel Cell utilize Efficiency Curves (not Torque-Speed maps)
  • Hybrid Power Control utilizes Generic Logic (no OEM proprietary data)
  • Regenerative Braking utilizes an Empirical Equation (no OEM proprietary data)
  • Battery & Transmission Efficiency utilize a Constant Parameter value (no operation-point sensitivity)

10 of 27

Our Previous Work / �Deeper Look at FASTSim

Modeling Idealizations in FASTSim

  • Input-Output power of Motor, Engine & Fuel Cell utilize Efficiency Curves (not Torque-Speed maps)
  • Hybrid Power Control utilizes Generic Logic (no OEM proprietary data)
  • Regenerative Braking utilizes an Empirical Equation (no OEM proprietary data)
  • Battery & Transmission Efficiency utilize a Constant Parameter value (no operation-point sensitivity)

It is generally Difficult to Isolate Errors due to:

  • Modeling Idealizations
  • Trip Data Uncertainty, such as:
    • Inaccuracies in road slope
    • Unknown HVAC & Auxiliary power
    • Number of Passengers & Cargo Load
    • Head Wind
    • Road Surface Quality

11 of 27

Our Previous Work / �Deeper Look at FASTSim

Re-Visiting the Power Demand Equation

Time instant i

Acceleration

Rotational Inertia

Wind Drag

Rolling Resistance

Road Slope

Auxiliary

Total Traction Power

Add an Error Term

Expand Error Term as First-Order in Terms of Traction Power

Re-Arrange

Traction Power Scaling Term

Auxiliary Power Correction Term

12 of 27

Our Previous Work / �Deeper Look at FASTSim

Adjusted Power Demand Equation

Traction Power Scaling Term

Auxiliary Power Correction Term

Driven Mass Correction Term

Note that Traction Power is Function of Driven Mass

Three Tuning Parameters

13 of 27

Our Previous Work

Time [s]

Speed [mph]

Rel. Altitude [m]

HVAC [kW]

Real-World Trip(s)

Vehicle Model Parameters

Fuel/Energy Usage

(kWh, gal-Gas/Diesel and/or kg-H2)

Fuel/Energy Usage

(kWh, gal-Gas/Diesel)

Measured

Simulated

How Accurate?

&

14 of 27

Modeling Extension #1

Time [s]

Speed [mph]

Rel. Altitude [m]

HVAC [kW]

Real-World Trip(s)

Vehicle Model Parameters

Fuel/Energy Usage

(kWh, gal-Gas/Diesel and/or kg-H2)

Fuel/Energy Usage

(kWh, gal-Gas/Diesel)

Measured

Simulated

How Accurate?

&

Constant Parameters�(e.g. Drag Coefficient, …etc.)

Engine/Motor�Efficiency Curves

Efficiency

Power Demand

Customizable

Now available in contributed publicly open-�source Java implementation of FASTSim

15 of 27

Modeling Extension #2

Time [s]

Speed [mph]

Rel. Altitude [m]

HVAC [kW]

Real-World Trip(s)

Vehicle Model Parameters

Fuel/Energy Usage

(kWh, gal-Gas/Diesel and/or kg-H2)

Fuel/Energy Usage

(kWh, gal-Gas/Diesel)

Measured

Simulated

How Accurate?

&

Constant Parameters�(e.g. Drag Coefficient, …etc.)

Engine/Motor�Efficiency Curves

Efficiency

Power Demand

Customizable

Re-Tuning the Alpha’s for �partially missing trip data

?

?

16 of 27

(Extended) Framework for Vehicle�Models Tuning & Verification

Adjust Physics-Based Model

Calibration for Real-World Trips

Verification

Initial FASTSim Vehicle Model

Real-World Trips Data

Tuning set of Trips

Verification set of Trips

Check Dyno Coef.

OK?

Adjust Base Vehicle Parameters (Mass, Drag, Tire Coef., …etc.)

Y

Y

Y

N

N

N

Check Catalog Values

OK?

Adjust Efficiency Curves (Engine/Fuel Cell & Motor)

Check Tuning Trips

OK?

Adjust Calibration Parameters (αT, αM, αA) for real-world trips

Check Verification Trips

Output: Tuned FASTSim model & Verification Results

If convergence becomes difficult

If convergence becomes difficult

* In this work, we used ~90% of real-world trips data for Tuning, ~10% for Verification

Stage 1: Adjust Physics-Based Model

Action: Adjust Vehicle Model Parameters Constant Values, as well as Engine/Motor Efficiency Curves

Goal: Dynamometer Coefficients (mass, A, C), Simulated EPA Window-Sticker-Labels within Tolerance (±10%)

Stage 2: Calibration for Real-World Trips data

Action: Adjust

Goal: Minimize Average Error in Simulated Trip Energy Intensity (kWh/mi or gal/mi) w.r.t. Tuning Trips*

Stage 3: Verification

Action: Observe statistics of Error in Simulated Trip Energy Intensity w.r.t. Verification Trips*

17 of 27

Real-World Trips Data: Source

UC-Davis eVMT Survey

  • Includes (among other things) GPS & OBD logging at high sampling rate of all vehicles in participant California households (w/ at least one Plug-in vehicle) for one+ year

Participant Privacy

  • Participants are anonymized & we don’t access GPS Lat/Long data
  • Altitude is analyzed relative to trip start (not actual sea-level altitude)
  • All computations that involve second-by-second trip data are done in a secure environment, only bulk results exported

Current Work

  • Uses trip data from (up to) 10 different individual owners for each of
  • Chevrolet Bolt
  • Nissan Leaf
  • Tesla Model-S
  • Ford C-Max Energi
  • Toyota Prius Prime
  • Chevrolet Volt
  • Chrysler Pacifica Hybrid
  • Toyota Prius (HEV)
  • Honda CR-V (ICE)

18 of 27

Real-World Trips Data: �Statistical Distributions

#1 (328, 57)

#2 (189, 40)

#3 (353, 53)

#4 (95, 20)

#5 (154, 28)

#6 (169, 36)

#7 (360, 58)

#8 (302, 53)

All (1950, 345)

0.1

0.2

0.3

0.4

0.5

Energy Intensity [kWh/mile]

kWh/mile corresponding to �EPA Combined Cycle rating

Individual Vehicles �(Different Owners)

Combined Trips (from�all Vehicle Owners)

Number of Trips in Tuning Set

Number of Trips in Verification Set

There are always variations in the real-world,�thus, we look at the data via Box-Plots

  • Box Limits mark the 25th & 75th Percentiles
  • Middle Line marks the Median value (50th Percentile)
  • Extension Lines mark the 5th & 95th Percentiles
  • Diamond Shape marks the Average Value
  • We also mark the EPA Combined Cycle (window-sticker label) Value and relative difference from it on the second vertical axis

Distribution of�Tuning Set

Distribution of�Verification Set

19 of 27

Real-World Trips Data: �Statistical Distributions

#1 (83, 19)

#2 (45, 7)

#3 (290, 47)

#4 (76, 16)

#5 (67, 13)

#6 (135, 18)

#7 (100, 20)

#8 (156, 27)

All (952, 167)

#1 (142, 22)

#2 (120, 18)

#3 (55, 10)

#4 (89, 19)

#5 (120, 19)

#6 (48, 9)

#7 (50, 9)

#8 (81, 12)

All (705, 118)

#1 (328, 57)

#2 (189, 40)

#3 (353, 53)

#4 (95, 20)

#5 (154, 28)

#6 (169, 36)

#7 (360, 58)

#8 (302, 53)

All (1950, 345)

0.1

0.2

0.3

0.4

0.5

Energy Intensity [kWh/mile]

#1 (403, 60)

#2 (659, 89)

#3 (605, 109)

#4 (205, 42)

#5 (466, 74)

#6 (664, 100)

#7 (375, 57)

#8 (397, 65)

#9 (710, 110)

#10 (349, 63)

All (4833, 769)

#1 (252, 28)

#2 (170, 23)

#3 (369, 63)

#4 (76, 13)

#5 (138, 25)

#6 (436, 77)

All (1441, 229)

0.1

0.2

0.3

0.4

0.5

Energy Intensity [kWh/mile]

#1 (363, 53)

#2 (433, 64)

#3 (197, 26)

#4 (86, 14)

#5 (59, 12)

#6 (363, 53)

#7 (256, 40)

#8 (252, 38)

#9 (352, 59)

#10 (162, 32)

All (2523, 391)

#1 (88, 18)

#2 (217, 34)

#3 (297, 44)

#4 (66, 18)

#5 (66, 11)

#6 (59, 10)

#7 (101, 17)

#8 (355, 37)

#9 (106, 14)

#10 (49, 9)

All (1107, 212)

#1 (223, 31)

#2 (145, 27)

#3 (87, 19)

All (455, 77)

#1 (197, 30)

#2 (70, 15)

#3 (211, 31)

All (478, 76)

0.01

0.02

0.03

0.04

0.05

Gasoline Intensity [gal/mile]

Bolt

Leaf

Model S

C-Max Energi

Pacifica Hybrid

Prius Prime

Volt

Prius (HEV)

CR-V

*https://www.epa.gov/greenvehicles/your-mileage-may-vary

“Your Mileage �Will Vary” – US-EPA*

  • We observed that average of all trips was within ±10% of Window-Sticker Label

20 of 27

Results

We Apply the Vehicle Models Tuning & Show Results* for…

  1. Tuned Vehicle Models** (Physical Parameters, Efficiency Curves, Alpha’s) when both Road Slope & Auxiliary Power are included in the Real-World Trips Data
    • We compare this result (comparable case) to previous result in SAE 2020-01-1441, to show-case the Benefit from Customizable Motor/Engine Efficiency Curves
  2. Tuned Vehicle Models** (same Physical Parameters & Efficiency Curves, different Alpha’s) when Road Slope is included, but Auxiliary Power is not included in the Real-World Trips Data
  3. Tuned Vehicle Models** (same Physical Parameters & Efficiency Curves, yet different Alpha’s) when neither Road Slope nor Auxiliary Power are not included in the Real-World Trips Data

* For the Verification set of Trips

** All tuned FASTSim vehicle models from this work are now publicly accessible

21 of 27

Results for Verification Set of �Real-World Trips

+40%

Relative Error in Simulated Trip Energy [%]

+30%

+20%

+10%

0%

-10%

-20%

-30%

-40%

Leaf

Model S

C-Max Energi

Pacifica Hybrid

Prius Prime

Volt

Prius HEV

CR-V

Bolt

Road Slope & HVAC Power NOT Included in Trips Data

Road Slope Included in Trips Data, but not HVAC Power

Road Slope & HVAC Power Included in Trips Data

22 of 27

Results for Verification Set of �Real-World Trips

+40%

Relative Error in Simulated Trip Energy [%]

+30%

+20%

+10%

0%

-10%

-20%

-30%

-40%

Leaf

Model S

C-Max Energi

Pacifica Hybrid

Prius Prime

Volt

Prius HEV

CR-V

Bolt

Road Slope & HVAC Power NOT Included in Trips Data

Road Slope Included in Trips Data, but not HVAC Power

Road Slope & HVAC Power Included in Trips Data

  • Worst average value of relative error within ±1.5%
  • Better than previous ±4.0% result (SAE 2020-01-1441) 🡪 Custom Efficiency Curves Beneficial

+1.5%

23 of 27

Results for Verification Set of �Real-World Trips

+40%

Relative Error in Simulated Trip Energy [%]

+30%

+20%

+10%

0%

-10%

-20%

-30%

-40%

Leaf

Model S

C-Max Energi

Pacifica Hybrid

Prius Prime

Volt

Prius HEV

CR-V

Bolt

Road Slope & HVAC Power NOT Included in Trips Data

Road Slope Included in Trips Data, but not HVAC Power

Road Slope & HVAC Power Included in Trips Data

+1.5%

  • Worst average value of relative error, also within ±1.5%
  • Effect of HVAC Power “within the noise”?
    • No, different values for
    • Also, trips are in California (relatively mild weather)

24 of 27

Results for Verification Set of �Real-World Trips

+40%

Relative Error in Simulated Trip Energy [%]

+30%

+20%

+10%

0%

-10%

-20%

-30%

-40%

Leaf

Model S

C-Max Energi

Pacifica Hybrid

Prius Prime

Volt

Prius HEV

CR-V

Bolt

Road Slope & HVAC Power NOT Included in Trips Data

Road Slope Included in Trips Data, but not HVAC Power

Road Slope & HVAC Power Included in Trips Data

  • Worst value for average value of relative error within ±4.0%
  • Not having Road Slope reduces accuracy, but still not bad

+4.0%

25 of 27

Results for Verification Set of �Real-World Trips

+40%

Relative Error in Simulated Trip Energy [%]

+30%

+20%

+10%

0%

-10%

-20%

-30%

-40%

Leaf

Model S

C-Max Energi

Pacifica Hybrid

Prius Prime

Volt

Prius HEV

CR-V

Bolt

Road Slope & HVAC Power NOT Included in Trips Data

Road Slope Included in Trips Data, but not HVAC Power

Road Slope & HVAC Power Included in Trips Data

  • We also observe “Less Variation” in Trip-Energy Predictions as more information about the real-world trips become available – but we have not considered quantitative metrics for this yet

26 of 27

Summary

  • Extended Previous Work on Tuning of FASTSim Vehicle Models
    • Added Capability for Custom Engine/Motor Efficiency Curves within the Tuning Framework
    • Considered different levels of information availability in real-world trips data

​

  • Applied the Framework to tune FASTSim models of 3 BEVs, 4 PHEVs, 1 HEV & 1 ICE

​

  • Tuned FASTSim Vehicle models appear capable of predicting real-world trips energy, with average error within:
    • ±1.5% as long road slope information is included in the trips data (i.e. with or without HVAC power, but using the corresponding set of Alpha’s)
    • ±4.0% when second-by-second vehicle speed is the only information available in the real-world trips data

​

27 of 27

Speaker information

  • Thank you!
  • Karim Hamza1, Ken Laberteaux2
  • Toyota Research Institute of North America (TRINA)
  • 1Principal Scientist�karim.hamza@toyota.com
  • 2Senior Principal Scientist�ken.Laberteaux@toyota.com