MDA Software Acquisition 101

Boehm Center for Systems and Software Engineering Annual Research Review

By: Dan Strickland

MDA/DOC Software SME

Missile Defense Agency

Cost Estimating Directorate

June 7, 2023

DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.

Approved for Public Release 18-MDA-9517 (21 Feb 18)

Overview

• What Is Software?
• Software Development
• Software Artifacts
• Software Acquisition
• Software Maintenance
• Software Estimation

2

What Is Software?

• Software is a collection of data instructions, executed by processes of a computer platform to instantiate pre-defined logic in a system
• Software comprises the development of experimental algorithms, tactical programs, support and simulation software, firmware, commercial off-the-shelf (COTS) products, and the support of the software after delivery
• Software behaves differently than its hardware counterparts
• No weights, materials, or typical dimensions
• Can’t be held, weighed, soldered, or melted
• Defined by the logic, design, and development effort to create it
• Computer Software Configuration Item (CSCI)
• “The lowest level of software development at which formal configuration management is performed by the contractor”
• Defined by the contractor
• AKA Software Item, Computer Program, Software End Item
• This is the level where software oversight and cost estimation occurs
• Examples: Battle Manager, Operator System Interface, Receiver/Exciter
• ISO/IEC 12207 is the standard DoD follows for Software Life Cycle Processes

3

Software Development

• The majority of software development has nothing to do with writing code
• In DoD and MDA, software follows a “V-shaped” process for development, just like systems engineering

System Engineering Processes Using integrated HW and SW as an answer to system-level requirements; SEPM costs

Software Engineering Processes

Developing and testing SW to match the SW Design; SW costs

System Software Engineering Processes

Development and testing of SW Requirements; usually SW costs

CSCI

4

Software Development – Requirements Analysis

• Software Requirements Analysis
• Establish the requirements of the software elements
• Software Requirements are baselined and traceable system requirements
• System/Software Engineers take the system level requirements and decompose them into software and hardware requirements – “What can we accomplish with SW vs. HW?”
• Tested requirements are the true indicator of completed software

​

What Does A Requirement Look Like?

Functional Requirements:

Statements of services the system should provide, how the system should react to particular inputs, and how the system should behave in particular situations

​

“The user shall be able to search either all of the initial set of databases or select a subset from it.”

​

“The system shall provide a unique identifier to every object detected.”

Non-Functional Requirements:

Constraints on the services or functions offered by the system such as timing constraints, constraints on development process, standards, etc.

​

“The system shall not disclose any personal information about users apart from name and reference number to operators of the system.”

​

“The display shall provide a response to user requests for refresh within 0.25 seconds.”

• Notice the use of the word “shall” – indicator of a requirement
• Artifacts: Software Requirements Specification (SRS) and Interface Requirements Specification (IRS)
• Usually maintained in a database tool such as DOORS or RTM

Requirements should be understandable, unambiguous , comprehensive, complete, concise, and testable

CSCI

Software Build

5

Software Development – Software Architectural Design

​

CSC

• Software Architectural Design
• Design for the software that implements and can be verified against the requirements
• Software architects take the software requirements and build a depiction of the system that shows how that system will work and interact– “How am I supposed to break this CSCI into the components and functions to make it work?”
• Also called Preliminary Design or Top-Level Design

​

What Does Top-Level Design Look Like?

Despite significant industrial

and research activity in this area,

there is no single, accepted

framework for codifying

architectural

thinking

For Every CSCI:

• Define the top-level structure and Computer Software Components (CSC)
• Top-level design for interfaces of the CSCI and interaction rules of the CSCs
• Preliminary test requirements for later integration

• Artifacts: Software Design Description (SDD), Software Architecture Description (SAD), Software Product Specification (SPS)
• Sometimes expressed via Unified Modeling Language (UML), a diagram description of the design

12207 Activities

Top-Level Design Examples:

“A single module called ERRMON will handle communication faults.”

“A single module called PRNTMGR will manage communication with the printer.”

CSCI

Software Build

6

Software Development – Software Detailed Design

​

​

CSU

• Software Detailed Design
• Design for the software that implements the software architecture and can easily be transferred to coding and testing
• Software designers take the software architecture and top-level design and decompose the components into units and their interfaces/interactions – “I need to write the algorithms and pseudocode”

What Does Detailed Design Look Like?

For Every CSCI:

• Define the low-level structure and Computer Software Units (CSU)
• Low-level design for interfaces of the CSCs and interaction rules of the CSUs
• Test requirements for the CSUs and code

• Artifacts: Software Design Description (SDD), Software Product Specification (SPS) pseudocode, logic diagrams, algorithms
• Detailed design should be everything a coder needs to implement the logic without the language syntax

12207 Activities

​

CSC

CSCI

Software Build

7

Software Development – Software Construction

​

​

CSU

• Software Construction
• Produce executable software units that properly reflect the software design
• Also called “Coding and Unit Testing” – develop the code and test the CSUs
• Software coders take the software pseudocode and CSUs, write/compile code, and test it against the Unit Tests – “I need to write this code in Ada and run the test on it to make sure it works.”

What Does Code Look Like?

For Every CSCI:

• Code the CSUs, test the coded CSUs, record the results, correct defects
• Develop the test requirements and schedule the integration of the CSUs and CSCs

• Artifacts: Software code, software databases, code walkthroughs
• Unit testing can be done automatically or manually

12207 Activities

​

CSC

CSCI

Software Build

Ada

Java

Assembly

8

Software Development – Software Integration

​

​

CSU

• Software Integration
• Combine the software units and software components into integrated software items
• Also called “Integration and Testing” – combine the CSUs into CSCs and test
• Integrators and testers take the completed software units and bring them together into components and then into CSCIs. The final testing procedures and plans for the final Qualification Testing of the CSCIs – “I need to put all this code together and make sure it works logically”

What Does Integration Look Like?

For Every CSCI:

• Combine the CSUs and CSCs into CSCIs ready for Qualification Testing and consistent with the design
• Develop the test requirements and schedule the Qualification Testing of the CSCIs

• Artifacts: Software Test Description (STD), Software Test Plans (STP), Software Test Report (STR)
• These tests are not specifically against the SW Requirements, so they aren’t “run for record”
• Identifies defects before the critical Qualification Testing

12207 Activities

​

CSC

CSCI

Software Build

Testing examples:

• “Big Bang” – all CSCs tested together
• Top-down – test from the highest level of CSC down its branch
• Bottom-up – test from the lowest level of CSC up step-by-step
• Sandwich – mixture of top-down and bottom-up
• Risky-hardest – testing the risky or hardest components first

9

Software Development – Software Qualification Testing

​

​

CSU

• Software Qualification Testing
• Confirm through testing that the integrated software product meets the defined requirements
• Testers take the integrated CSCIs and test them using the defined test plans that fulfil the software requirements – “I have a collection of CSCIs, but I need to make demonstrate that they meet the requirements in the SRS.”

What Does Qual Test Look Like?

For Every CSCI:

• Demonstrate through testing the fulfilment of the software requirements
• Record the results of the test
• Develop a strategy for regression testing if changes are made – no need to retest EVERYTHING

• Artifacts: Software Test Plan (STP), Software Test Report (STR), Defect Profiles
• The goal of SW Qual Test is to verify the SW Requirements of the CSCI using qualification methods
• “When Is It Over?” – when requirements are met via qualification with no critical, unfixed defects; many times requirements are not fully qualified or have non-critical defects that can be fixed later or mitigated

12207 Activities

​

CSC

CSCI

Software Build

Qualification Methods

• Analysis – using models or simulation
• Demonstration – observing the item in operation
• Inspection – formal examination by a SME
• Test – execution under controlled conditions and inputs

10

Software Development – Support Processes

• Software Support Processes
• Activities that span the entire development of the software and are not restricted by phase
• Many of these activities are rolled up into overall software development effort or in separate lines for Program Management, Quality Assurance. etc.

​

11

Software Artifacts

Describes the developer’s plans for conducting the SW Development effort; includes processes, plans, standards, tools, etc. “One stop shop” for the software development process.

Software Development Plan (SDP)

Virtual or physical container of all the CSCI artifacts (requirements, designs, source code, test plans, trouble reports, reviews, etc.). Find everything about the CSCI here.

Software Development Folder (SDF)

Written description of the architecture and detailed design of the CSCIs. Shows how the code is planned to match the requirements. This is the framework for the code.

Software Design Description (SDD)

Documents the essential requirements (functions, performance, attributes, etc.) for a given CSCI. This is what the software has to do.

Software Requirements Specification (SRS)

Documents the required interfaces for the CSCI inside and outside of the system. This is how the software has to communicate.

Interface Requirements Specification (IRS)

Describes the test preparations, test cases, and test procedures to qualify a CSCI or software system. This is the checklist for making sure the requirements are done.

Software Test Description (STD)

In MDA, the periodic (usually monthly) delivery of contractor software development data for analysis (e.g. Requirements Volatility, Turnover Rate, Computer Network Utilization). Should be compliant with the MDA Assurances Provisions (MAP) Rev B Core Metrics.

Software Metrics

• These are usually contractor deliveries on the Contract Data Requirements List (CDRL) and adhere to published Data Item Descriptions (DID)
• This is not a complete and exhaustive list
• Any individual contract may not have all of these artifacts or they might be called something different

DI-IPSC-81427B

DI-IPSC-81433A

DI-IPSC-81434A

DI-IPSC-81435A

DI-IPSC-81439A

12

Software Development Methodologies

13

Software Acquisition

• Software Acquisition is the process of defining, contracting, overseeing, and procuring a software system
• In MDA, Software Acquisition is under the oversight of Software Engineering (MDA/CES) within MDA/DE
• MDA/CES mission:

The Software Engineering (CES) mission is to provide leadership in the discipline of software engineering within MDA and to define, assess, and support software engineering activities to ensure development of secure, reliable and maintainable MDA software.

CES provides software acquisition expertise in missile defense solicitations; conducts independent assessments of BMDS software; defines MDA policies and establishes processes for software acquisitions; assesses mission critical and support software indicators to include cost, schedule, requirements, quality, and environment; evaluates software independent verification and validation, software assurance, and software reliability efforts; documents lessons learned and best practices for future acquisition models; and provides software acquisition training for MDA.

​

If you need to find answers, data, or artifacts associated with a program, coordinate with CES or their identified program element software leads

14

Software Maintenance

Software Maintenance, or Software Sustainment, is the continual process of modifying existing software without changing its primary functions

Requirements arrive cyclically over time

• Users report problems
• Owner requests enhancements
• The platform (operating system, database, etc.) changes

Not all problems are “software defects”

• Duplicate reports
• Confusing manuals
• Hardware failure
• Suggested enhancements (new features)

The architecture and design exist, but will evolve

• Small staff must be knowledgeable
• Design documentation may be misleading

The owner may decide not to correct all known defects

• Cost and time constraints
• Workarounds may exist
• There could be risk of side effects

The user may not install the new release

• Cost and time constraints
• Workarounds may exist
• There could be risk of side effects

Characteristics of Software Sustainment

​

• Major Release – provides new features and fixes bugs
• Minor Release – primarily only fixes bugs or makes small corrections
• Patch – only corrects a small set of serious bugs or upgrades security

Types of Releases

​

15

Software Estimation

• Software Estimation is the process of turning an estimation of added functionality into effort (e.g. hours, person-months, person-years, etc.)
• The MDA/DOC Software Estimation Handbook details the processes and procedures used to estimate software efforts
• Software Acquisition in Software Estimation:
• Locate the relevant Software Artifacts (e.g. SDP) and review them for context
• Identify the build plan and methodology
• How many formal qualification tests are planned for the SW?
• Waterfall, Incremental, Spiral, Agile?
• Maintenance efforts?
• Identify the number and functionality of each CSCI
• What does it do and what else is like it?
• Is this something that has been done before?
• Confer with Software Acquisition Management personnel
• What do past software metrics tell about the developer?
• Does the developer show process maturity and ability to improve productivity?

​

​

​

​

16

Resources

• Software Acquisition
• MDA/CES Software Acquisition Management portal page:
• MDA Software Acquisition Instruction 8430.01-INS
• MDA Software Acquisition Manual 8430.01-M
• IEEE-12207 Brochure
• Software Acquisition Checklist Brochure
• Software Acquisition Management – John J. Marciniak & Donald J. Reifer
• Software Estimation
• “MDA Software Cost Estimating & Analysis Handbook” – MDA/DOC portal page
• Estimating Software-Intensive Systems – Richard D. Stutzke
• Software Engineering Economics – Barry Boehm
• Software Cost Estimation with COCOMO II – Barry Boehm
• Software Estimation: Demystifying the Black Art – Steve McConnell

​

​

17