1 of 50

Dr. Kenneth E. Nidiffer

703 217-0215

Research on Challenges for Integrating Measurement and Control Mechanisms into Engineered Systems into the Lifecycle of Future Technology-Centric Systems

Annual Technology Workshop

Boehm Center for Systems and Software Engineering

April 21- 23, 2026

2 of 50

Agenda

Introduction
Problem Space
Need for Solutions
Partial Solutions
Summary
References

The Boehm Center for Systems and Software Engineering (Boehm CSSE) is dedicated to the education and development of the next generation system and

software engineering leaders through coursework,

research, and collaboration with engineering partners in industry, government, and other academic institutions.

Source: CSSE

3 of 50

Part 1: Introduction�Elements of a Cost Estimation Model

Historical

Data

Adjustment

Factors

Future-Product

Attributes/Features

Estimates of

effort

schedule

defects

. . .

Estimation

Procedure

Assumptions and Constraints

A project estimate is a projection from past experiences to the future, adjusted to account for the differences between past and future.

Source: Dr. Richard Fairly

4 of 50

�Introduction�New Technology - Digital Transformation�Changing the Way Work Is Accomplished and How Society Thinks About Work��Metamorphosis: Profound change in body structure during the postembryonic development of an organism�

Source: Stevens Institute of Technology, 2020

5 of 50

Part 2 - Problem Space�System and Software Cost Modeling is Difficult (Partial List)

Requirements Uncertainty: Software requirements are often unclear or change over time, making it hard to estimate accurately. Vague requirements at the beginning can lead to significant estimation errors.
Complexity of Software: Software projects can have complex architectures, dependencies, and integrations, all of which affect the cost and are hard to predict. Larger projects with interdependent modules are particularly hard to estimate.
Human Factors: Productivity can vary widely between developers, teams, and organizations. A highly skilled team might complete tasks faster than a less experienced one which might affect estimates.
Rapid Technology Changes: Technologies, frameworks, and tools change quickly in the software industry. Estimates based on old methods may not apply if new technology is introduced.

6 of 50

Problem Space�System and Software Cost Modeling is Difficult (Partial List)

Unknowns and Risks: Every project has uncertainties, such as, expected bugs, performance issues, or security vulnerabilities. These unknowns can lead to unforeseen delays and cost increases.
Historical Data Limitations: Estimation techniques often rely on historical data, but every project is unique, so past project data isn’t always a reliable indicator for future projects, especially with different requirements or technologies exist.

Scope Creep: Requirements and features often expand as the project progresses. This “scope creep” can make initial estimates inaccurate unless there’s tight control and management.

Stakeholder Expectations: Often, stakeholders might not fully understand the technical complexities and expect lower costs, leading to pressure for optimistic estimates, which may not be realistic.

7 of 50

Problem Space�Scope of Software Project Management Cost Estimation�Source: PMBOK® Guide -2026 V8)�

Project

Company

External

Environment

8 of 50

Problem Space��Estimation techniques often rely on historical data, but every project is unique

9 of 50

Problem Space�Speed of Technology Innovation Cycles

10 of 50

Problem Space

Number and Usage of Software Tools

11 of 50

Problem Space�BANI* - How To Make Sense of a Chaotic World �We Live In A World Of Constant Change, Emerging Technologies and Increasing Complexity

Brittleness - The world has become brittle or fragile. The transformation to a digital world has shorter product lifecycles and disruptive changes e.g., iPhone development since 2007 and usage.
Anxiety - The situation around us has become more anxious. The current events happening in the world e.g., hyper-conductivity
Non-Linearity - Cause and effect relationships are no longer easily predictable in advance e.g., AWS Internet Gateway failure, 2025
Incomprehensibility - It is very difficult to understand and comprehend how and why events are happening e.g., - AI/Machine Learning and Agentic AI Technology are changing the world around us and us.

Source: BANI: Facing the Age of Chaos, by Isabelle Sailer, Linked In

https://www.linkedin.com/in/jamaiscascio/recent-activity/shares/

12 of 50

�Problem Space� Connecting the Dots -The Dallenbach Effect�Source: The MASTER’S AI Course - 2025��

Ref: Karl M. Dallenbach papers

13 of 50

Part 3 - Need for Solutions�Need for Effective and Timely Decision Making in an Increasing Complex Environment��

Decision Makers

14 of 50

Need for Solutions �Need for a New Set of Environmental Factors

Historical

Data

Adjustment

Factors

Future-Product

Attributes/Features

Estimates of

effort

schedule

defects

. . .

Estimation

Procedure

Assumptions and Constraints

Environmental

Factors

15 of 50

Need for Solutions�New Set of Environmental Factors (Partial List)

Faster delivery of software-enabled systems at relevance and scale.
Diseconomy of scale in complexity of future software enabled systems.
Hyperconductivity among future systems.
Increasing requirements/scope of future systems cause unintended issues.
Higher delivery cost of future multinational systems.
Effective and timely decision making in an increasing complex environment.
Rapid innovation and technology transfer.
Increase delivery cost of international systems to address global needs.
Importance of foreign-born talent to the U.S. STEM workforce.
Emergence of transdisciplinary engineered systems.
Equal distribution of intellect.
Lack of relevant estimation data.
Insufficient resources for software cost estimation research.
Need for an international data repository for establishing realistic basis of estimates.

16 of 50

Increasing Change, Hyper-Conductivity, Globalization, Productivity, Complexity and Uncertainty

Need for Solutions

Faster Delivery of Software-Enabled Systems at Relevance and Scale - Systems Are Mostly Software Defined Capacities, Hardware Enabled

Manual

Labor

Water

Steam

Oil

Software

17 of 50

Need for Solutions�Diseconomy of Scale in Complexity of Future Software- Enabled Systems

David Chesebrough, NDIA Systems Engineering Conference, 2022

18 of 50

Need for Solutions�Complexity – Difficult to Define But Must Attempt �Simplicity is not the absence of complexity but a deeper understanding of complexity

First Flight – 17 Dec 1903

19 of 50

��Need for solutions�Diseconomy of Scale in Complexity of Future Software- Enabled Systems – Cynefin Framework��The Cynefin Framework - Five Decision-Making Contexts��

20 of 50

Increasing Complexity and Interconnectedness of Cyber Physical Systems*�

Joint Air Power Competence Centre. Edition, 27, 2018

Need for Solutions

Diseconomy of Scale in Complexity of Future Software- Enabled Systems - JADC2

21 of 50

Part 5 - Partial Solutions�Improved Intersections of Humanity and Innovations

22 of 50

Partial Solutions�Recognition of Three Different Engineering Domains

Information & Software Engineering

Quantum Systems & Software

Engineering

Complex

Systems & Software

Engineering

23 of 50

Partial Solutions�Recognition of Three Different Engineering Domains

Complex Systems & Software Engineering

Math – Continuous
Science - Physical and Biological
Laws - Numerous
Maturity – High
Dimension – Systems & Systems of Systems, Medium Time to Market

Information & Software Engineering

Math – Discrete
Science – Computer Science
Laws - Few
Maturity – Medium
Dimension – Real and Virtual Information, Rapid Time to Market

Quantum Systems & Software Engineering

Math – Probability/Uncertainty
Science – Natural
Laws – Few to Medium
Maturity – Low
Dimension - Matter & Energy, Low Time to Market

24 of 50

Partial Solution� Three Different Engineering Domains

Information & Software Engineering

Quantum Systems & Software

Engineering

Complex

Systems & Software

Engineering

25 of 50

Partial Solutions�Rapid Innovation and Technology Transfer�

26 of 50

The world is still be adjusting to generative AI and large language models (LLMs)
Whereas generative AI is great at following prompts to generate text, audio, imagery and other forms of content by learning from what already exists.
Agentic AI can create and execute its own commands. This shift from content creation to decision and execution machine is huge in many respects, especially project management.

Multiagent

Systems

Partial Solutions�Rapid Innovation and Technology Transfer�

27 of 50

Partial Solutions�Future Air Force Operations Control Center

28 of 50

Partial Solution�Quantum Recognition - Three Different Engineering Domains

Information & Software Engineering

Quantum Systems & Software

Engineering

Complex

Systems & Software

Engineering

29 of 50

Quantum Systems & Software Engineering�

There are several types of qubits that have been proposed or implemented in quantum computing platforms.
Superconducting qubits are the most common type of qubit in current quantum computing systems.
Superconducting qubits are based on the Josephson junction, which is a device that allows the flow of supercurrent without resistance

30 of 50

Partial Solutions�Recognition - Need for the Integration of AI and Quantum Computing into Future Complex Systems

*NP (nondeterministic polynomial time) problems

31 of 50

Integration of Different Engineering Domains Technologies Future of the Complex Systems - Golden Dome

32 of 50

Partial Solutions�Twenty- First Century Imperatives in Transdisciplinary Engineering

Today the network of relationships linking the human race to itself and to the rest of the biosphere is so complex that all aspects affect all others to an extraordinary degree.*
Someone should be studying the whole system, however crudely it has to be done, because no gluing together of partial studies of a complex nonlinear system can give a good idea of the behavior of the whole.*

*Murray Gell-Mann - physicist

The illiterate of the 21st century will not be those who cannot read or write but those who cannot learn, unlearn and relearn - Alvin Toffler – Futurist

33 of 50

Partial Solutions�Integration of AI and Quantum Computing into Future Complex Systems Is Happening �

AI usage in systems is a shift seen worldwide.

Lack to derive metrics to satisfy the decision maker’s information needs.

Data security is a pressing concern as quantum computing threatens existing measurement capabilities.

Need to develop and sustain a measurement capability to support management information needs.

Sectors such as finance, pharmaceuticals, defense, sustainability, and regulated industries are exploring the potential impact of AI and quantum computing in their respective fields.

34 of 50

Partial Solutions�Twenty- First Century Imperatives in Transdisciplinary Engineering

35 of 50

Partial Solutions�Importance of Foreign-Born Talent to the U.S. STEM Workforce�

National (United States) assessments show student mathematics performance from 2019 to 2022. The average mathematics scores of fourth and eighth grade students dropped to levels last measured approximately 20 years ago.
Enrollment of international science and engineering (S&E) graduate students at U.S. institutions has rapidly increased. International students on temporary visas accounted for about a third of S&E master’s and doctoral degree recipients at U.S. institutions in 2021.
Foreign-born individuals made up 24% of all STEM workers and 43% of doctorate-level scientists and engineers.

Source: National Science Foundation, May 2024

36 of 50

��Partial Solutions�Recognition - Transdisciplinary Systems Engineering

* Sheard, Bouyaud, Osaisai, Siviy, and Nidiffer, Finding Your Systems Engineering Role In 21^st Century Software -Dominant Organizations, 2021

** Azad M. Madni ,Transdisciplinary Systems Engineering: Exploiting Convergence in a Hyper-Connected World, 2018

*** Washizaki, H., Sanchez-Segura, M-I., Garbajosa, J., Tockey, S., Nidiffer, K.E.. “Envisioning Software Engineer Training Needs in The Digital Era Through the SWEBOK V4 Prism”, Proceedings of the IEEE International Conference on Software Engineering Education and Training (CSEE&T 2023), August 8-9, 2023, Waseda University, Tokyo Japan.

**** Washizaki, H., Sanchez-Segura, M.I., Garbajosa, J., Tockey, S., Reilly, A.D., Nidiffer, K.E, SWEBOK v4a Article to the Systems Engineering Body of Knowledge (SEBOK V4a, Aug 2025)

Finding the Systems Engineering Role In 21^st Century Software -Dominant Organizations

***

****

37 of 50

Partial Solutions�Twenty- First Century Imperatives in Transdisciplinary Engineering Centers

The most dangerous phrase in the language is, "We've always done it this way”.- Rear Admiral Grace Hopper

38 of 50

Part 4 Summary�The Issues Are Real - Need to Think Differently

39 of 50

Part 4: Summary�International Global Federated System and Software Cost Modeling Center

Supports international systems and software modeling needs:

Identifies and facilitates access to global measurement expertise and capabilities for cost modeling of future and fielded systems.
Assesses modeling gaps and recommends plans to close these gaps.
Develops recommendations for initiatives for international R&D strategy to innovate estimation processes, methods and tools.
Enables efficient collaboration and use capabilities.
Serves as an international point of contact for cost modeling efforts.
Develops and sustains an inventory of modeling data and resources.

Resources initially provided by international countries who provide oversight and leadership.
Provides expert support teams.

40 of 50

SUMMARY��So Where Does� This Lead Us?

We live in a world of constant change and increasing complexity and hyperconductivity - that is not going to change
Meeting current and future challenges for cost estimation of increasing complex and global hyper- connect software-enabled systems are international issues
The scale of complicated and complex software-enable systems & services will continue to increase exponentially with intricate and often hidden interfaces & interrelationships operating in a dynamic and non-deterministic world.
Increases in information needs, product functionality, and deliveries of overall value at the speed of relevance and scale are dependent on transdisciplinary systems engineering teams working as intimately intertwined disciplines throughout the life cycle.
Integrating multinational activities among these transdisciplinary systems engineering teams throughout the systems life cycle will be increasingly important as we develop and deploy new systems.

41 of 50

Contact Information

Dr. Kenneth E. Nidiffer

President and CEO

Ken's Software and Systems Engineering Firm

Adjunct Professor, George Mason University

703-217-0215

Thank you – Questions/Comments

42 of 50

References

43 of 50

References (1of 6)

Gallagher, B.; Nidiffer, K. & Saga, R. The Ordered Process for Improving Agile Engineering Outcomes, CrossTalk, Nov/Dec 2016
Alberts, C.; Woody, C.; & Dorofee, A. Evaluating Security Risks using Mission Threads(CMU/SEI-2014-TN-025), 2014.
Quotes from the Defense Science Board Summer Study on Autonomy, June 2016, p. 45, and Unmanned Systems Integrated Roadmap, FY2013-2038, p. 60
http://www.gartner.com/technology/research/metodologies/hype-cycle.jsp, web search, 2019
DoDI 5200.44 Protection of Mission Critical Functions to Achieve Trusted Systems and Networks (TSN), and 2013 NDAA S933
The Joint Artificial Intelligence Center Overview, AFCEA/GMUC4I Conference, May 2020
Wilson, Roy. Software Assurance Course (CLE 081). Defense Acquisition University. Preliminary Design Review, July 2017 and Critical Design Review, December 2017.
Nidiffer, K; Chick, T.; & Woody C.; Program Manager’s Guidebook for Software Assurance, August 2018, CMU/SEI-2018-SR-025
Miller, C.; Nidiffer, K DoD Software Sustainment Study Phase II: Policy Analysis and Recommendations, Jan 2018, CMU/SEI-2018-SR-001—RESTRICTED
Sheard, S.: Nidiffer, K.; et al, Finding Your Systems Engineering Role In 21^st Century Software -Dominant Organizations 2021
Fairley, R., Systems Engineering of Software-Embedded Systems, Wiley, 2019

44 of 50

References (2 of 6)

Assessment and Program Estimation. Gide. https://www.dmi-ida.org/knowledge-base-detail/dod-cost-estimating-guide, 2026
Madachy, R. Boehm, B Clark, B Tan, T & Rosa. US DoD Application Domain Empirical Software Cost Analysis. ESEM '11: Proceedings of the 2011 International Symposium on Empirical Software Engineering and Measurement Pages 392 - 395. https://doi.org/10.1109/ESEM.2011.56
Software Engeering Economics. Englewood Cliffs, NJ, Prentice-‐Hall, 1981
Boehm B., Abts C., Brown W., Chulani S., Clark B., Horowitz E., Madachy R., Reifer D., Steece B., Software Cost Estimation with COCOMO II, Prentice-‐Hall, 2000
Madachy, R.; Boehm, B.; Clark, B.; Tan, T.; Rosa, W. "US DoD Application Domain Empirical Software Cost Analysis", Empirical Software Engineering and Measurement (ESEM), 2011 International Symposium on, On page(s): 392 – 395
Rosa W., Boehm B., Clark B., Madachy R., Dean J., 2013, “Domain-Driven Software Cost and Schedule Estimating Models: Using Software Resource Data Reports (SRDRs)”, 2013 International Cost Estimating and Analysis Association (ICEAA), Professional Development & Training Workshop
Chatterton, S. D., White, E.D., Ritschel, J.D., Lucas, B.M., Fass, R.D., & Valentine, S.M. (2026). An examination of software cost estimation models for DoW programs. Defense Acquisition Research Journal, 33(1), 30–50. https://doi.org/10.22594/dau.24-935.33.01
Clark, B., Miller, C., McCurley, J., Zubrow, D., Brown, R., & Zuccher, M. (2017). Department of Defense software factbook. Carnegie Mellon University. https://insights.sei.cmu.edu/ library/department-of-defense-software-factbook/ Cost Assessment and Program Evaluation. (2020). DoD cost estimating guide. 1.0. https://

45 of 50

References (3 of 6)

Design and Acquisition of Software for Defense Systems 2018 Defense Science Board report, http://www.dtic.mil/dtic/tr/fulltext/u2/1048883.pdf
DoD Digital Engineering Strategy 2018 Department of Defense https://www.acq.osd.mil/se/docs/2018-DES.pdf
Summary of the 2018 National Defense Strategy for the United States of America https://dod.defense.gov/Portals/1/Documents/pubs/ 2018-National-Defense-Strategy-Summary.pdf
Summer Study on Autonomy 2016 Defense Science Board, https://www.hsdl.org/?view&did=794641
Ten Commandments of Software 2018 Defense Innovation Board, https://media.defense.gov/2018/Apr/22/2001906836/-1/-1/0/DEFENSEINNOVATIONBOARD_TEN_COMMANDMENTS_OF_SOFTWARE_2018.04.20.PDF
National Security Strategy, Office of the President of the United States 2017 https://www.whitehouse.gov/wp-content/uploads/2017/12/NSS-Final-12-18-2017-0905.pdf
Critical Code: Software Producibility for Defense 2010 National Research Council https://www.nap.edu/catalog/12979/ critical-code-software-producibility-for-defense
Networking and Information Technology Research and Development (NITRD) Studies https://www.nitrd.gov/

46 of 50

References (4 of 6)

DoD Directive 5000.01, “The Defense Acquisition System,” September 9, 2020
DoD Directive 5135.02, “Under Secretary of Defense for Acquisition and Sustainment (USD(A&S)),” July 15, 2020
DoD Instruction 5000.02, “Operation of the Adaptive Acquisition Framework,” January 23, 2020
DoD Instruction 5000.75, “Business Systems Requirements and Acquisition,” February 2, 2017, as amended
DoD Instruction 5010.44, “Intellectual Property (IP) Acquisition and Licensing,” October 16, 2019
DOD Instruction 5000.87 Operation Of The Software Acquisition Pathway, October 2, 2020, https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodi/500087p.PDF?ver=virAfQj4v_LgN1JxpB_dpA%3d%3d
Nidiffer, K. Accelerating Modernization of Software Acquisition to Better Serve the Warfighter … Special Emphasis on Software Assurance and Sustainment (IEEE Published Paper), https:/ieeexplore.ieee.org/xpl/conhome/9171048/proceeding; IEEE/NDIA Systems Security Symposium, July 2020
Defense Acquisition University (DAU) Acquisition Policies and Guides, October 9, 2020, https://aaf.dau.edu/aaf/policies/
Williams, C, Why DOD Is So Bad at Buying Software, FCW and Defense Systems Defense Systems, Nov 08, 2021

47 of 50

References (6 of 6)

Azad M. Madni ,Transdisciplinary Systems Engineering: Exploiting Convergence in a Hyper-Connected World, 2018
, Dr. Sarah Sheard . A Framework for Systems Resilient Discussion, Stevens institute of Technology, Third Millennial Systems, 2008
Paul Nielsen, and Suzanne Miller, Software and Systems Collaboration in the Era of Smart Systems, March 2022 • PODCAST
Michael D. Watson, Future of Systems Engineering, First published: 29 May 2019, https://doi.org/10.1002/inst.12231
Sheard, Bouyaud, Osaisai, Siviy, and Nidiffer, Finding Your Systems Engineering Role In 21st Century Software -Dominant Organizations, 2021
** Azad M. Madni ,Transdisciplinary Systems Engineering: Exploiting Convergence in a Hyper-Connected World, 2018
*** Washizaki, H., Sanchez-Segura, M-I., Garbajosa, J., Tockey, S., Nidiffer, K.E.. “Envisioning Software Engineer Training Needs in The Digital Era Through the SWEBOK V4 Prism”, Proceedings of the IEEE International Conference on Software Engineering Education and Training (CSEE&T 2023), August 8-9, 2023, Waseda University, Tokyo Japan.
Washizaki, H., Sanchez-Segura, M.I., Garbajosa, J., Tockey, S., Reilly, A.D., Nidiffer, K.E, SWEBOK v4 Article to the Systems Engineering Body of Knowledge (SEBOK v2.8), 2023

48 of 50

References (5 of 5)

Review Of Cost Estimation: Methods and Models for Aerospace Composite Manufacturing Ch. Hueber, K. Horejsi & R. Schledjewski, 2016
By Edwin Stoop (User:Marillion!!62) -[1], CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=53810658, Web Search, April 2024
David Chesebrough Systems Engineering Conference, 2022
Kenneth E. Nidiffer, Software Engineering Challenges for Integrating Measurement Activities into Defense Systems Viewed Through the Prism of Future Systems, 2023 International Boehm COCOMO® Forum on Systems and Software Cost Modeling with Practical Software and Systems Measurement Users Group, Aerospace Campus, November 12/- 16, 2023, Chantilly, VA,
Reliance on metrics is a fundamental challenge for AI, Rachel L. Thomas, David Uminsky. 2022.
SEI and DOD Center To Ensure Trustworthiness in AI Systems, launched in 2003, CMU News

Three Metrics for Quantum Computing Performance: Scale, Quality and Speed, IBM, 2022
The Biden Administration’s National Security Memorandum on AI, October 2024
Kenneth E, Nidiffer, Research on Challenges for Integrating Measurement and Control Mechanisms into Engineered Systems Viewed Through the Prism of Future Systems – Part II, 2024 International Boehm COCOMO™ Forum on Systems and Software Cost Modeling The Aerospace Corporation, El Segundo, California, 11/12/2024 - 11/14/2024

Cinema Mode

49 of 50

Public and Government Software Measurement Organizations (Partial List)

IEEE (Institute of Electrical and Electronics Engineers): They develop standards for software measurement and metrics, including the IEEE 1061 standard for software metrics.
ISO (International Organization for Standardization): ISO/IEC 25000 series focuses on software product quality and includes measurement frameworks.
SEI (Software Engineering Institute): Part of Carnegie Mellon University, SEI focuses on software engineering and includes measurement as part of their Capability Maturity Model Integration (CMMI).
ACM (Association for Computing Machinery): While ACM covers a broad range of computing topics, many of its publications and conferences address software measurement and metrics.
Agile Alliance: While primarily focused on Agile methodologies, they also discuss measurement techniques to assess software quality and team performance.
PMI (Project Management Institute): PMI incorporates software measurement within its project management standards, particularly in the context of measuring project performance and success.

50 of 50

Public and Government Software Measurement Organizations (Partial List)

ISBSG (International Software Benchmarking Standards Group): Focuses on software benchmarking and measurement to improve productivity and quality in software development.

PSM (Practical Software and Systems Measurements): Focuses on principles and practices for developing, operating, and continuously improving your organization’s measurement program.

Boehm CSSE (Boehm Center for Systems and Software Engineering): Focuses on research in the areas of system and software development practices and the evolution of these practices as well as the estimation of cost/schedule for all things related to software, systems, and system of systems (SoS) engineering and development.

CAIG OSD (Cost Analysis Improvement Group CAIG): assist DoW costs analysts develop and present the results of operating and support cost analyses

GAO (U.S. Government Accountability Office): to meet the demands of today’s changing world by employing effective management practices and processes, including the measurement of government program performance.