2023 International Boehm COCOMO® Forum on Systems and Software Cost Modeling with Practical Software and Systems Measurement Users Group

Aerospace Campus, Chantilly, VA

11/13/2023 - 11/16/2023

​

Dr. Kenneth E. Nidiffer

knidiffe@gmu.edu

703 217-0215

Software Engineering Challenges for Integrating Measurement Activities into Defense Systems Viewed Through the Prism of Future Systems

Approved for Public Release

2

Increasing Globalization, Productivity, Complexity and Uncertainty

Increasingly Software-Enable Systems Are a Significant Strategic Resource

Manual

Labor

Water

Steam

Oil

Software

3

 Uncertainties Are The Only Certainties

We Have.

Four Waves of Industrial Revolution

DoD Vision: Deliver Resilient Software At The Speed Relevance*

Source: Department of Defense Software Strategy Coordinator, November 2021

​

Enabling Effective Decision Making

Decision Makers

BANI - How To Make Sense of a Chaotic World? �(Idea: We live in a world of constant disruptive change)

• Brittleness - The world has become brittle or fragile. The new environment has shorter product lifecycle, disruptive change - the transformation to a digital world.
• Anxiety - The situation around us has become more anxious. The current events happening in the world.
• Non-Linearity - Cause and effect relationships are no longer easily predictable in advance – diseconomy of scale.
• Incomprehensibility - It is very difficult to understand and comprehend how events are happening and why they are happening - changing the world and us.

​

5

BANI: facing the age of chaos, by Isabelle Sailer, Linked In 

https://www.linkedin.com/in/jamaiscascio/recent-activity/shares/

​

Twenty- First Century Engineering Imperatives

• Norm Augustine: “One should expect that the expected can be prevented, but the unexpected should have been expected.”
• Engineering resilience is a system property that allows a system to continue providing useful service despite largely unpredictable and disruptive events internal or external to the system.

​

​

6

A Framework for System Resilience Discussions. Sheard 2008

​

Looking Into the Future - Transdisciplinary Systems Engineering with a Focus on Integrating Measurement Activities (1)

* Sheard, Bouyaud, Osaisai, Siviy, and Nidiffer, Finding Your Systems Engineering Role In 21^st Century Software -Dominant Organizations, 2021

** Azad M. Madni ,Transdisciplinary Systems Engineering: Exploiting Convergence in a Hyper-Connected World, 2018

*** Washizaki, H., Sanchez-Segura, M-I., Garbajosa, J., Tockey, S., Nidiffer, K.E.. “Envisioning Software Engineer Training Needs in The Digital Era Through the SWEBOK V4 Prism”, Proceedings of the IEEE International Conference on Software Engineering Education and Training (CSEE&T 2023), August 8-9, 2023, Waseda University, Tokyo Japan.

**** Washizaki, H., Sanchez-Segura, M.I., Garbajosa, J., Tockey, S., Reilly, A.D., Nidiffer, K.E, SWEBOK v4 Article to the Systems Engineering Body of Knowledge (SEBOK v2.8), 2023 – release TBD

​

Finding Your Systems Engineering Role In 21^st Century Software -Dominant Organizations**

*

***

**

****

Looking Into the Future - Transdisciplinary Systems Engineering with a Focus on Integrating Measurement Activities (2)

• Today the network of relationships linking the human race to itself and to the rest of the biosphere is so complex that all aspects affect all others to an extraordinary degrees.*
• Someone should be studying the whole system, however crudely it has to be done, because no gluing together of partial studies of a complex nonlinear system can give a good idea of the behavior of the whole.*

​

*Murray Gell-Mann - physicist

​

8

The illiterate of the 21st century will not be those who cannot read or write but those who cannot learn, unlearn and relearn - Alvin Toffler – Futurist

Grand Challenges/Wicked Problems*�Move to Transdisciplinary Engineering

9

*National Council of Engineering - 2018

​

​

Emerging Ultra-Large Systems of Systems Requiring Transdisciplinary Engineering with a Focus on Integrating Measurement Activities

Source: www.sei.cmu.edu/uls

Trend: Increasing Number of Components, Functions and Intersections*

11

* Roedler G., Systems Engineering of the Future: Shaping the Future of Systems Engineering, Feb 28, 2018

​

Trend: Increasing Number of Connected Platforms*�

12

• An internet of things (IoT) platform is software that enables development, deployment, and management of solutions that connect to and capture data from IoT endpoints.

​

• IoT platforms can be delivered as a hybrid combination of edge platforms and/or cloud IoT platforms as a service.

Source data: ARM Holdings

*AFCEA Technical Vectors, 2021

Increasing Complexity and Interconnectedness of Cyber Physical Systems*�

13

Joint Air Power Competence Centre. Edition, 27, 2018

Engineered Systems Are Distinct But Becoming Intimately Intertwined Transdisciplinary Engineering Disciplines That Need Integrated Measurement Activities

Engineered Systems Are Distinct But Becoming Intimately Intertwined Transdisciplinary Engineering Disciplines That Need Integrated Measurement Activities�

14

Reducing Vulnerabilities and Technical Debt Is Becoming More Important - Need a Focus on Integrating Measurement Activities

15

1–5% Of Defects Are Vulnerabilities; Common Vulnerabilities are Driven By Coding/Design Defects of Known Types – Source SEI Dr. Carol Woody

“Bake In” Data During System Design- Need Integrated Measurement Activities

16

Source: 2022 PSM Workshop, Principal Deputy Assistant Secretary of Defense

Distribution Statement A. Approved for public release. Distribution is unlimited During

​

17

The Result: Software Assurance Is and Has Been a Security Issue CVE 1999 to 2022: Reported Common Vulnerabilities and Exposures (CVEs)

​

Source: Robert A. Martin,�MITRE Corporation, October 2022

CVEs

CVEs

​

Real Data

CWEs

Zero Day

Notional Data

Integrating Measurement Activities

• Software will continue to grow in social technical ecosystems comprising customers, end users’, developers, maintainers, testers and other stakeholders.
• The increasing complexity is becoming apparent that the integrating measurement activities deserves greater attention and more emphasis.
• The people aspect comprises the peoples’ decisions, personal skill sets, training motivation, creativity and talent.
• Principal issue greater than increasing software complexity – more about effectively leveraging integrating measurement competencies.

​

18

So Where Does This Lead Us?

• The scale of complicated and complex software-enable systems & services will continue to increase exponentially with intricate and often hidden interfaces & interrelationships operating in a dynamic and non-deterministic world.
• Increases in information needs, product functionality, and deliveries of overall value at the speed of relevance are dependent on transdisciplinary systems engineering teams working as intimately intertwined disciplines throughout the life cycle.
• Integrating measurement activities among these transdisciplinary systems engineering teams throughout the defense systems life cycle will be increasingly important as we develop and deploy increasingly more new systems in the future.

​

​

19

Example 1: Guide to the Software Engineering Body of Knowledge (SWEBOK)�http://swebokwiki.org

• Objective
• Guiding learners, researchers and practitioners to identify and have common understanding on “generally-accepted-knowledge” in software engineering
• Defining boundary of software engineering and related disciplines
• Providing foundations for certifications and educational curriculum
• Adoption
• IEEE-CS software professional certification programs based on SWEBOK (Associate Software Developer, Professional Software Developer, Professional Software Engineering Master)
• ISO/IEC 24773-4: Certification of software and systems engineering professionals - Part 4: Software engineering
• Software Engineering Competency Model (SWECOM)

20

Activities (and practices)

Body of Knowledge

Islands of Knowledge

Example 2: Integrating Measurement Activities in Software Factory Ecosystem

21

The most dangerous phrase in the language is, "We've always done it this way”.- Rear Admiral Grace Hopper 

�Example:3 AI Technology Transfer Rates

Source: SEI Research Review – October 2023

�Example 4: Language and Image Recognition Capabilities of AI Systems Have Improved Rapidly

Contact Information

Dr. Kenneth E. Nidiffer

• Cell: 703 - 217- 0215
• President and CEO: Ken’s Software-Enabled Systems Company
• Adjunct Professor George Mason University
• Email: knidiffe@gmu.edu

​

​

24

Thank you – Questions/Comments

References

25

References (1of 4)

• Gallagher, B.; Nidiffer, K. & Saga, R. The Ordered Process for Improving Agile Engineering Outcomes, CrossTalk, Nov/Dec 2016
• Alberts, C.; Woody, C.; & Dorofee, A. Evaluating Security Risks using Mission Threads(CMU/SEI-2014-TN-025), 2014.
• Quotes from the Defense Science Board Summer Study on Autonomy, June 2016, p. 45, and Unmanned Systems Integrated Roadmap, FY2013-2038, p. 60
• http://www.gartner.com/technology/research/metodologies/hype-cycle.jsp, web search, 2019
• DoDI 5200.44 Protection of Mission Critical Functions to Achieve Trusted Systems and Networks (TSN), and 2013 NDAA S933
• The Joint Artificial Intelligence Center Overview, AFCEA/GMUC4I Conference, May 2020
• Wilson, Roy. Software Assurance Course (CLE 081). Defense Acquisition University. Preliminary Design Review, July 2017 and Critical Design Review, December 2017.
• Nidiffer, K; Chick, T.; & Woody C.; Program Manager’s Guidebook for Software Assurance, August 2018, CMU/SEI-2018-SR-025
• Miller, C.; Nidiffer, K DoD Software Sustainment Study Phase II: Policy Analysis and Recommendations, Jan 2018, CMU/SEI-2018-SR-001—RESTRICTED
• Sheard, S.: Nidiffer, K.; et al, Finding Your Systems Engineering Role In 21^st Century Software -Dominant Organizations 2021
• Fairley, R., Systems Engineering of Software-Embedded Systems, Wiley, 2019

​

​

​

​

​

​

26

References (2 of 4)

• Design and Acquisition of Software for Defense Systems 2018 Defense Science Board report, http://www.dtic.mil/dtic/tr/fulltext/u2/1048883.pdf
• DoD Digital Engineering Strategy 2018 Department of Defense https://www.acq.osd.mil/se/docs/2018-DES.pdf
• Summary of the 2018 National Defense Strategy for the United States of America https://dod.defense.gov/Portals/1/Documents/pubs/ 2018-National-Defense-Strategy-Summary.pdf
• Summer Study on Autonomy 2016 Defense Science Board, https://www.hsdl.org/?view&did=794641
• Ten Commandments of Software 2018 Defense Innovation Board, https://media.defense.gov/2018/Apr/22/2001906836/-1/-1/0/DEFENSEINNOVATIONBOARD_TEN_COMMANDMENTS_OF_SOFTWARE_2018.04.20.PDF
• National Security Strategy, Office of the President of the United States 2017 https://www.whitehouse.gov/wp-content/uploads/2017/12/NSS-Final-12-18-2017-0905.pdf
• Critical Code: Software Producibility for Defense 2010 National Research Council https://www.nap.edu/catalog/12979/ critical-code-software-producibility-for-defense
• Networking and Information Technology Research and Development (NITRD) Studies https://www.nitrd.gov/

​

27

References (3 of 4)

• DoD Directive 5000.01, “The Defense Acquisition System,” September 9, 2020
• DoD Directive 5135.02, “Under Secretary of Defense for Acquisition and Sustainment (USD(A&S)),” July 15, 2020
• DoD Instruction 5000.02, “Operation of the Adaptive Acquisition Framework,” January 23, 2020
• DoD Instruction 5000.75, “Business Systems Requirements and Acquisition,” February 2, 2017, as amended
• DoD Instruction 5010.44, “Intellectual Property (IP) Acquisition and Licensing,” October 16, 2019
• DOD Instruction 5000.87 Operation Of The Software Acquisition Pathway, October 2, 2020, https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodi/500087p.PDF?ver=virAfQj4v_LgN1JxpB_dpA%3d%3d
• Nidiffer, K. Accelerating Modernization of Software Acquisition to Better Serve the Warfighter … Special Emphasis on Software Assurance and Sustainment (IEEE Published Paper), https:/ieeexplore.ieee.org/xpl/conhome/9171048/proceeding; IEEE/NDIA Systems Security Symposium, July 2020
• Defense Acquisition University (DAU) Acquisition Policies and Guides, October 9, 2020, https://aaf.dau.edu/aaf/policies/
• Williams, C, Why DOD Is So Bad at Buying Software, FCW and Defense Systems Defense Systems, Nov 08, 2021

​

​

​

28

References (4 of 4)

• Azad M. Madni ,Transdisciplinary Systems Engineering: Exploiting Convergence in a Hyper-Connected World, 2018
• , Dr. Sarah Sheard . A Framework for Systems Resilient Discussion, Stevens institute of Technology, Third Millennial Systems, 2008
• Paul Nielsen, and Suzanne Miller, Software and Systems Collaboration in the Era of Smart Systems, March 2022 • PODCAST
• Michael D. Watson, Future of Systems Engineering, First published: 29 May 2019,  https://doi.org/10.1002/inst.12231
• Sheard, Bouyaud, Osaisai, Siviy, and Nidiffer, Finding Your Systems Engineering Role In 21st Century Software -Dominant Organizations, 2021
• ** Azad M. Madni ,Transdisciplinary Systems Engineering: Exploiting Convergence in a Hyper-Connected World, 2018
• *** Washizaki, H., Sanchez-Segura, M-I., Garbajosa, J., Tockey, S., Nidiffer, K.E.. “Envisioning Software Engineer Training Needs in The Digital Era Through the SWEBOK V4 Prism”, Proceedings of the IEEE International Conference on Software Engineering Education and Training (CSEE&T 2023), August 8-9, 2023, Waseda University, Tokyo Japan.
• Washizaki, H., Sanchez-Segura, M.I., Garbajosa, J., Tockey, S., Reilly, A.D., Nidiffer, K.E, SWEBOK v4 Article to the Systems Engineering Body of Knowledge (SEBOK v2.8), 2023 – release TBD

​

​

29