1 of 33

Software Licenses: �A Bill You Can’t Pay?

Cheryl Jones & Brad Clark

First International Boehm Forum on COCOMO and Systems and Software Cost Modeling

November 2022

1

Nov 2022

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

2 of 33

Problem Statement

  • The Army budget decision makers have been reviewing the projected budget requests for licenses for software maintenance
  • The budget for license costs for software maintenance is increasing exponentially - it is becoming an ever increasing percentage of the Army operational budget
    • We do not have solid data today to defend budgets
  • The Army is considering different approaches to:
    • Better manage software licenses
    • Share the responsibility for these costs, and
    • Reduce the growth in these costs

​

​

2

Nov 2022

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

3 of 33

Topics

​

  • Objectives
  • Software License Analysis
    • Data collection
    • Challenges
    • Demographics
    • Results
  • DevSecOps Implications
  • Conclusions

​

3

Nov 2022

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

4 of 33

Objectives

4

Nov 2022

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

5 of 33

SWS Initiative Objective and Strategy

Accurately estimate Army system Software Sustainment (SWS) costs to:

    • Effectively estimate and justify software and system life cycle costs
    • Objectively evaluate Army system software sustainment execution costs
    • Inform and optimize the allocation of available sustainment resources across the Army

5

Nov 2022

Effective software sustainment cost estimation is the basis for Army system software life cycle cost management

Collect and evaluate SWS cost and technical data for all Army operational systems �(Phase I, II, & III data calls)

Generate and validate cost estimating relationships from data collection

Implement systemic Army SWS data collection via the SRDR-M: Populate cost and technical data repository

Improve Army SWS policy, business, and technical requirements

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

6 of 33

Data Collected During Initiative

6

Nov 2022

System Context

    • System name and description
    • Services involved
    • ACAT level
    • Current phase/milestone
    • MS C date
    • Number of software baselines
    • Number of hardware platforms/number of users
    • C&A types and frequency
    • Release and IAVA rhythm
    • Data rights
    • Pertinent WBS elements
    • Analogous systems
    • Sustainment organization

Annual Funding

    • Annual effort/cost data (total annual plus by WBS elements) broken out by government and contractor
    • Labor rates
    • Hourly basis for FTEs
    • Total cost for software licenses

​

    • For phase 3, LCMC programs requested to provide funding from all sources (not just OMA)

Release Level

Capabilities

    • Release context information
    • Application domain
    • Operating environment
    • Schedule - start and end dates
    • Release effort / cost
    • Size data (those that apply)
      • Software requirements
      • External requirements
      • Source Lines of Code (SLOC)
      • Non-SLOC based size (e.g. RICE-FW, use cases, story points)
      • Software changes counts by priority (e.g. change requests, problem reports, defects)
      • IAVAs

Release Level

IAVAs

    • Release context information
    • Application domain
    • Operating environment
    • Schedule - start and end dates
    • Release effort / cost
    • Size - IAVAs

Software Licenses

    • License name and version
    • License class
    • Company
    • Usage
    • Quantity
    • IAVAs
    • Coverage
    • Cost (if program funded)
    • Type
    • Duration of license

Definitions of data elements available in data collection questionnaire

​

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

7 of 33

Initiative Phases

  • Phase 0 - 1 year of data
    • Selected data from volunteer systems
    • 33 systems (updated in later collection phases)
  • Phase 1 - 3 years of data
    • SES level coordination with G-4, ASAALT, G-6, G-8 and DASA-CE
    • 5 systems from each LCMC and PEO
    • 41 systems
  • Phase 2 - 3 years of data
    • SES level coordination with G-4, ASAALT, G-6, G-8 and DASA-CE
    • Remaining systems from each LCMC and PEO
    • 151 systems
  • Phase 3 - FY18 data - 1 year of data
    • 115 systems
  • Phase 4 - FY21 data
    • License data from ASA/ALT (PEO) programs
    • 70 systems

​

7

Nov 2022

Some systems provided updated data in later phases

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

8 of 33

DASA-CE SWS WBS

8

Nov 2022

Software Sustainment

1.0 Software Change Product

1.1 Capability Changes

Change requirements

Change development

B/L Integration & Test

IV&V

1.2 IAVA Changes

On-Site technical

assistance

Problem Troubleshooting

S/W Installation

Operational Assistance

On-site Training

​

Operations

Organization management

Personnel management

Financial management

Information management

Process management

Change management

3.0 Software

Licenses

4.0 Certification & Accreditation

8.0 Other

​

7.0 Field

Software Eng.

Version 5.0

5.0 System

Facilities

6.0 Sustaining Engineering

Non-System Specific

2.0 Project Management

Planning

Execution management

Configuration management

Resource & team management

Contracting management

Measurement - reporting

System Specific

System Specific

System/Non-System Specific

System Specific

System Specific

System/Non-System Specific

System/Non-System Specific

License - Right to Use� COTS/GOTS

NDI

Open Source� Other

License - Maintenance

​

Security

Safety

Networthiness

Airworthiness

Hardware� Software development� assets/workstations� System integration & test facilities� Test equipment - tools

Facility Operations

6.1 Help desk

6.2 Hosting

6.3 Engineering and User Support

Test Support

Software Delivery

Technical Studies

Training

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

9 of 33

System Demographics

9

Nov 2022

115: PDSS-10; PPSS-105

Phase 3 Systems

151: PDSS-56; PPSS-95

Phase 2 Systems

41: PDSS-26; PPSS-15

Phase 1 Systems

377

Unique Systems

3,835

Data Fields

PDSS (PEO)

Distribution of System Age

PPSS (OMA)

0 - 10 Years

11 - 20 Years

21 - 30 Years

31 - 50 Years

122

71

10

2

15

4

77

25

Number of Systems by Commodity

Systems by Super Domain

Systems by ACAT Level

Fires

C5ISR

Intel

Aviation

Network

SATCOM

Missiles

Comms

Vehicles

Business

MissionCmd

Simulation

ChemBio

DBS

Test

Space

54

48

45

42

34

33

28

22

21

11

11

9

8

6

3

2

70: PDSS-27; PPSS-43

Phase 4 Systems

RT

ENG

AIS

SUP

DBS

193

97

59

22

6

ACAT I

ACAT II

ACAT III

ACAT IV

ACAT UKN

Non-PoR

92

56

138

12

47

32

*51 systems do not have age data

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

10 of 33

Software License Analysis

10

Nov 2022

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

11 of 33

Information Needs

  • License data collection provides:
    • Annual license cost expenditures
    • Change in license costs (with multi-year data)
    • Factors influencing the change in license costs, e.g., license coverage & type
    • Sources of license costs, e.g., vendors
  • Context data for drill-down, e.g.,
    • License cost by system
    • License cost by super-domain
    • License cost by commodity
    • ‘Development’ versus ‘End-User’ license cost

​

11

Nov 2022

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

12 of 33

License Data Fields*

  • Maintenance Organization
  • System Name (if only one)
  • Commodity
  • Super Domain
  • PDSS/PPSS Phase
  • License Name
  • Class (COTS, OSS, etc)
  • Version
  • Vendor Company
  • Usage (Dev., Ops., Both)

​

12

Nov 2022

  • Quantity
  • IAVAs
  • Coverage (Single, Site-Wide)
  • Total Cost
  • Cost per License
  • Type (End-User, Tech Support)
  • Duration
  • License Purchaser
  • Purchase Date
  • Purchase FY
  • Comments

*First five data fields provide contextual data for drill-down analysis

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

13 of 33

License Data Challenges - Government

  • Programs do not always track purchases, support, and service costs
    • About half of the license data had associated cost data
  • Variety of license strategies: enterprise, site, program, supplier purchases
    • Many licenses do not have system costs if they are enterprise or site licenses, or if they are purchased by the supplier
  • Inconsistent information on license purchase approaches
    • The word “Maintenance” could mean maintenance, support, or services
    • Inconsistent knowledge of whether a license is single-use, site-use, or enterprise-use
    • A license that is purchased annually may or may-not be a subscription
  • Inconsistent data / data requires normalization
    • License and Vendor names are reported differently for the same license
    • Use (Development Environment, Operations, Both) not identified
    • Need data for both maintenance environment/facilities (including software factory) and operations
    • Quantity not always known
    • Coverage (Single, Enterprise) not always identified
    • Cost and date formats vary
    • Sometimes people write explanations in the data fields

13

Nov 2022

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

14 of 33

License Data Challenges - Vendor

  • License vendors change sales strategies and costs over time that dramatically increases license costs (purchases to annual subscriptions)
    • Hard to change vendors once committed
  • License vendors update and sunset licenses frequently, driving changes to operational software
    • New products, merge products, repackage products
  • License data changes frequently:
    • Company names (as companies are sold or merged)
    • License names and versions (variations reported and replacement names)
  • Inconsistent data – data requires normalization
  • The word “Enterprise” in a license name is misleading as it could refer to the number of features in a product and not the license coverage

​

​

​

​

​

​

​

  • Normalization is very time-consuming

14

Nov 2022

​

 

Users / Seats / Computers / Processors

​

​

1

~2+

~20+

~100+

Features

Standard

$

$$

$$$

$,$$$

Professional

$$

$$$

$,$$$

$$,$$$

Premier

$$$

$$$$

$$,$$$

$$$,$$$

Enterprise

$,$$$

$$,$$$

$$$,$$$

$,$$$,$$$

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

15 of 33

Software License Demographics

15

Nov 2022

26

9

Licenses by Commodity

Number of Licenses by Super Domain

Average License Cost by Super Domain

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

16 of 33

License Quantity for Top 22 License Vendors

16

Nov 2022

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

17 of 33

Avg License Costs for Top 22 Vendors ($K)

17

Nov 2022

Top 19 Software licenses

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

18 of 33

License Total Costs by Commodity

18

Nov 2022

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

19 of 33

License Cost Analysis Approach

  • All costs were normalized to Base Year 2019 (BY19), effects of inflation were removed
  • Only “Single” or “Site” license coverages that had LIC_Qty were analyzed (BY19 license cost per quantity per year: annual unit cost)
  • Licenses were selected by System with more than 1-year of cost data for a single license
  • License cost change was determined by comparing the ratio of the unit cost for one year to the unit cost of the next year
  • Analysis depended on normalization of License names - provided data grouping
  • After roll-up of same year data, 210 records yielded 81 annual unit cost pairs

19

Nov 2022

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

20 of 33

Pivot Table Example

20

Nov 2022

Systems

2012

2013

2014

2015

2016

2017

LIC_NAME_n

 

18,678,689

20,394,098

7,688,189

18,899,175

​

[ESRI] ESRI Unlimited

​

9,088,193

8,609,505

​

​

​

[ISD] Appraisal Wizard

​

2,585

​

​

​

​

Adobe Acrobat Professional

​

​

​

​

0

​

Adobe LiveCycle Designer

​

​

​

​

0

​

AdobeCLPG 5.x RoboHelp Office MP Rnw 1yr-Lvl 2(EU)

​

1,166

​

​

​

​

Advanced Security

​

​

214,351

​

​

​

Agent Logic 2015

​

​

​

82,130

​

​

Agent Logic RulePoint Base

​

​

​

​

565

​

Agent Logic RulePoint Base Maint

​

311

540

​

​

​

Agent Logic RulePoint Cores for Production Use

​

​

​

​

72

​

Agent Logic RulePoint Cores Maint

​

78

69

​

​

​

Agent Logic RulePoint Development Kit Maint

​

310,605

875

​

​

​

Agent Logic RulePoint SDK

​

​

​

​

92

​

Agent Logic RulePoint Seats for Production

​

​

​

​

21

​

Agent Logic RulePoint Seats Maintenance

​

78

20

​

​

​

AgileGraph

​

​

​

​

158,868

​

AgileGraph 2015

​

​

​

160,775

​

​

AgileGraph Software Maintenance Support

​

164,981

162,543

​

​

​

  • 19 systems had multi-year license data
  • 81 record pairs of two-year data

Note: did not have data for all years for most licenses

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

21 of 33

Same COTS Product Comparison

  • Example: InstallShield
  • Cost per single license varies across systems and Maintenance Organizations

21

Nov 2022

LIC_NAME

LIC_NAME_n

Qty_n

Cover_n

LIC_Cost_Per_BY19

FY_n

Installshield

InstallShield

2

Single

$825

2013

InstallShield

InstallShield

3

Single

$2,316

2014

InstallShield-Anywhere (Renewal)

InstallShield-Pro

5

Single

$942

2014

InstallShield

InstallShield

3

Single

$2,736

2015

InstallShield-Anywhere

InstallShield-Pro

8

Single

$789

2015

InstallShield Professional

InstallShield-Pro

4

Single

$719

2016

Installshield

Installshield

2

Single

$1,041

2017

InstallShield

InstallShield

3

Single

$2,601

2018

InstallShield Professional

InstallShield-Pro

4

Single

$650

2018

  • Shows Opportunity Costs: differences in costs represents money not saved
  • Shows where to negotiate a single product license cost

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

22 of 33

Price Ratio License Analysis Results

  • This approach is based on the Chained Consumer Price Index without the “weighted” price ratio, i.e., all price ratios are treated as equal
  • Two successive time periods were analyzed, p1 and p2

​

​

​

​

​

​

​

​

  • A Price Ratio less than 1.0 means the price was reduced
  • All prices and costs are in Base Year 2019 amounts

​

22

Nov 2022

 

p1 FY

p2 FY

License

P1

P2

p2/p1

15

16

AgileGraph

$160,775

$158,868

0.99

15

16

Viper

$356,920

$385,391

1.08

15

16

Red Hat LINUX Desktop, Basic Support

$2,355

$2,314

0.98

15

16

p2/p1 Mean

 

 

1.02

Example

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

23 of 33

License Cost Change Reasons

23

Nov 2022

Reasons for Major Cost Decreases

  • Initial purchase w/maintenance followed by lower annual maintenance
  • Change from perpetual to an annual subscription license
  • License vendor purchased by another company
  • Large license quantity increase
  • Vendor price decrease

Reasons for Major Cost Increases

  • Custom proprietary products
  • Specialty products, e.g. rule-based logic, imaging, mapping, data comparison, computer memory analysis
  • Vendor price increase

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

24 of 33

Price Ratio License Results

  • The mean price ratio across all year groups was 1.01
    • There was a lower degree of variation
  • Ignoring year groups with less than 5 records, the mean price ratio was 1.08
    • But there are only two selected year groups: FY 13-14 and FY 21-22
  • Across all year groups, the mean price ratio was 1.05

​

24

Nov 2022

Year

Count

p2/p1

12-13

2

1.01

13-14

9

1.08

14-15

2

1.01

15-16

3

1.02

16-17

0

 

17-18

0

 

18-19

1

0.83

19-20

0

 

20-21

3

1.05

21-22

14

1.08

Mean

​

1.01

Selected Mean

1.08

Year

Count

Overall Mean for p2/p1

FY 12 -22

34

1.05

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

25 of 33

Interpolated Data

  • There were 36 licenses that had multi-year data, e.g., FY15 & FY17 but no FY16
  • Data was linearly interpolated for the missing years

25

Nov 2022

p1 FY

p2 FY

License

P1

P2

p2/p1

15

16

AgileGraph

$160,775

$158,868

0.99

15

16

Viper

$356,920

$385,391

1.08

15

16

Red Hat LINUX Desktop, Basic Support

$2,355

$2,314

0.98

15

16

Microsoft VS Enterprise with MSDN (VL)

$1,210

$1,254

1.04

15

16

VBS IG

$459

$583

1.27

15

16

GeoRover Tools for ArcGIS

$1,892,060

$1,870,427

0.99

15

16

TestComplete Enterprise Floating License

$931

$912

0.98

15

16

LiDAR Analyst

$1,081

$864

0.80

15

16

SOCET GXP GeoAnalysis (4004-UE)

$2,278

$2,118

0.93

15

16

Beyond Compare

$34

$91

2.67

15

16

Memory Validator

$229

$347

1.51

15

16

MSDN

$1,072

$969

0.90

15

16

Installshield

$933

$987

1.06

15

16

GEM X Encryptor Manager

$2,145

$2,109

0.98

15

16

SNMPc Enterprise Maintenance

$745

$733

0.98

15

16

SNMPc Workgroup Maintenance

$291

$287

0.98

15

16

p2/p1 Mean

 

 

1.13

Example

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

26 of 33

Interpolation Results

  • The number of records in each year group increased for most groups
  • Ignoring year groups with less than 5 records, the mean price ratio was 1.03
  • Across all year groups, the mean price ratio was 1.03

26

Nov 2022

Year

Count

p2/p1 Mean

11-12

1

 

12-13

4

1.00

13-14

18

1.04

14-15

16

0.90

15-16

16

1.13

16-17

27

1.04

17-18

26

1.01

18-19

1

 

19-20

0

 

20-21

3

1.05

21-22

14

1.08

Mean

​

1.03

Selected Mean

​

1.03

Year

Count

Overall Mean for p2/p1

FY 12 -22

126

1.03

  • Price ratios showed lower variability with interpolated data
  • Probably due to evenly spreading the price across missing year groups

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

27 of 33

Recommendations

  • The collection of data from selected systems could validate a License Price Index - options:
    • Select 12 Systems from different Programs or
    • Evaluate a database of multi-year license costs (if one can be accessed)
    • Collect complete data for FYs 20, 21, and 22 (price and cost)
    • Collect data for FY23 when available
    • The data should be from the same licenses across the FYs

​

  • As with Consumer Price Index, a License Price Index would be a very useful budgeting tool

​

27

Nov 2022

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

28 of 33

DevSecOps Implications

28

Nov 2022

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

29 of 33

DevSecOps Estimation Challenges

  • DevSecOps is based on the principle of Continuous Integration (CI) and Continuous Delivery (CD)
  • Multiple development pipelines create, enhance, and maintain different software products independently and concurrently using a Software Factory construct
  • This construct relies on task automation for repeatable tasks thus reducing workload (and cost), improving quality, and increasing the speed of delivery
  • Automation is enabled with software tools
    • The cost of startup and buildout of a software factory is driven by effort and supporting tool costs
    • The cost will vary with the degree of implementation
  • A Software Factory has an initial start-up and an ongoing cost

29

Nov 2022

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

30 of 33

Task Automation for Repeatable Tasks

30

Nov 2022

  • For non-open source tools, license costs will increase annually
  • The increase should be included in POM and TOC estimates

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

31 of 33

Conclusions

31

Nov 2022

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

32 of 33

License Data Conclusions

  • Costs cannot be controlled until there is a good understanding of the total ownership costs (TOC) of licenses
  • Products will be merged or dropped thus forcing changes to the operational software and impacting TOC
  • The Government needs a method to allocate the license costs down to individual systems to prepare and defend budgets
    • Especially for those paid for by other-than project funds
  • Initial data shows software applications relying on COTS products will experience a rise in cost over the life of the system
  • License costs need to capture complete and standardized descriptive information
  • Associated work in studies, integrated product reports, and working teams show that there is an interest in the Army for a cost factor by which to estimate and manage software license costs
    • Need more and better data before finalizing a factor
  • We need broad executive-level sponsorship to require programs and organizations to regularly collect this data in a consistent manner

32

Nov 2022

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified

33 of 33

Contributors

33

Nov 2022

James Judy

​

NISEC Division Chief

ODASA-CE

703-697-1612

Cheryl Jones

​

Measurement Analyst

US Army AFC

973-724-2644

Dr. Bradford Clark

​

​

Vice President

Software Metrics, Inc.

(703) 402-3576

Ryan Farrell

​

Analyst

Galorath Federal

443-758-4837

Jeramia Poland

​

Senior Operations Research Analyst

ODASA-CE

703-697-1604

Unclassified

Distribution Statement A: Approved for Public Release; Distribution is unlimited.

Unclassified