1 of 15

Dr. Justine Caylor and Dr. Adrienne Raglin

October 2022

Multi-Criteria Decision-Making for Uncertainty of Information

UNCLASSIFIED

UNCLASSIFIED//FOR OFFICIAL USE ONLY//DRAFT//PRE-DECISIONAL

U.S. ARMY COMBAT CAPABILITIES DEVELOPMENT COMMAND –

ARMY RESEARCH LABORATORY

UNCLASSIFIED//FOR OFFICIAL USE ONLY//DRAFT//PRE-DECISIONAL

2 of 15

OBJECTIVE

There are numerous operational decisions that need to be made and future warfare will have expanding factors and criteria associated with these decisions. However, decision cycles and reaction times will be compressed.

In these decision-making cases the environments will be extremely dynamic.

Thus, the application of an optimal MCDM method is even more crucial to provide solutions to Army capability needs.

This research is exploring augmenting hybrid MCDM with dynamic calculations to enhance automated decision-making and leveraging an Uncertainty of Information concept.

UNCLASSIFIED

UNCLASSIFIED//FOR OFFICIAL USE ONLY//DRAFT//PRE-DECISIONAL

3 of 15

MULTI-CRITERIA DECISION-MAKING (MCDM)

Since as early as the 1950s¹, MCDM methods have been an invaluable tool in the decision-making process, especially in complex and uncertain environments.

¹ [Mardani A, Jusoh A, Nor K, Khalifah Z, Zakwan N, Valipour A, 2015]

UNCLASSIFIED

UNCLASSIFIED//FOR OFFICIAL USE ONLY//DRAFT//PRE-DECISIONAL

4 of 15

ANALYTICAL HIERARCHY PROCESS (AHP)

AHP uses pairwise comparisons and judgments from experts to derive priority measurements

AHP consists of three main parts:

decomposing and breaking down the problem into criteria and sub-criteria in a hierarchical manner;
determining the priorities of the criteria and sub-criteria; and
synthesizing the priorities¹

Matrices of pairwise comparisons are formed to estimate the level of importance using numbers from a 1-to-9 AHP fundamental scale²

¹ [Saaty 1990a; Kivijärvi and Tuominen 1999; Wan et al. 2015]; ² [Saaty 2007]

UNCLASSIFIED

UNCLASSIFIED//FOR OFFICIAL USE ONLY//DRAFT//PRE-DECISIONAL

5 of 15

Technique for Order Preferences by Similarity to Ideal Solutions (TOPSIS)

Based on the idea of minimizing the distance or determining the shortest distance from the positive ideal solution and maximizing the distance or determining the farthest distance from the negative ideal solution¹

The TOPSIS method comprises the following steps² :

Calculate the normalized decision matrix.
Calculate the weighted normalized decision matrix.
Determine the positive ideal solution and negative ideal solution.
Calculate the separation measures using the n-dimensional Euclidean distance.
Calculate the relative closeness to the positive ideal solution.
Rank the preference order.

¹ [Hwang and Yoon 1981; Opricovic and Tzeng 2004; Cui et al. 2011; Thor et al. 2013; Singh and Malik 2014]; ² [Opricovic and Tzeng 2004]

UNCLASSIFIED

UNCLASSIFIED//FOR OFFICIAL USE ONLY//DRAFT//PRE-DECISIONAL

6 of 15

DYNAMIC MULTI-CRITERIA DECISION-MAKING (DMCDM)

DMCDM adapts to different situations using a “retention policy” for historic data/alternatives.^1,2
DMCDM uses aggregation functions to compute final score/rank using the historic and current data.^1,2
DMCDM have a “stopping” criteria.^1,2

¹ [Campanella & Ribeiro, 2011a, 2011b]; ² [Varela & Ribeiro, 2014]

UNCLASSIFIED

UNCLASSIFIED//FOR OFFICIAL USE ONLY//DRAFT//PRE-DECISIONAL

7 of 15

UNCERTAINTY OF INFORMATION (UOI)

Aligning C2 factors

(applied in variable D):

Location of friendly units, Disposition of units
Fidelity or Precision of Information Understanding
Understanding of enemy courses of action
Quality and Availability of Knowledge Products
Time Constraints for Preparedness
Shared Understanding
Communications

General sources of data

(applied in variable S):

Humans (Commanders, Analysts, Soldiers, …)
Agents (Autonomous Systems,…)
Algorithms (various)
Information (data or signal from physical systems)

Text
Audio
Video
Image

Devices (Sensors …)

Raw Sensor Data
Processed

Networks (Enterprise, Research, …)
Visualization Applications (ex. 2D, 3D, XR, Gaming)

Computational model utilizing imperfect information for a variety of data modalities that capture how commanders and analysts can weigh information prior to a decision.

¹ [Raglin, Metu, & Lott, 2020]; ² [Lott, Raglin, & Metu, 2019]

UNCLASSIFIED

UNCLASSIFIED//FOR OFFICIAL USE ONLY//DRAFT//PRE-DECISIONAL

8 of 15

UNCERTAINTY OF INFORMATION (UOI)

Taxonomy of Uncertainty

(applied in variable G):

Corrupt Data that has errors or alterations, including those caused maliciously

Questionable Data where the quality or truthfulness is disputed

Inconsistent Data that does not remain the same or is not compatible

Incomplete Data that has missing parts or is unfinished

Inaccurate Data that is incorrect or has been distorted

Inappropriate Data that is not suitable or not valuable
Disjointed Data that lacks cohesion or organization

¹ [Raglin, Metu, & Lott, 2020]; ² [Lott, Raglin, & Metu, 2019]

UNCLASSIFIED

UNCLASSIFIED//FOR OFFICIAL USE ONLY//DRAFT//PRE-DECISIONAL

9 of 15

ARMY RELEVANT USE CASE

UNCLASSIFIED

First scenario

You are a convoy escort whose task is to protect the supply convoy moving from the FOB to the city. Pre-brief indicated few attacks and threats. To accomplish this task, you will rely on different sources for decisions. For example, Intel and heterogeneous devices or sensors. You will have to be mindful of the Uncertainty of Information (UoI) from those sources. You will be asked to rank how important or concerning the uncertainties from those sources are to you.

Second scenario

You are tasked with defending a valley with several farming villages from surrounding militia. Threat reports are incoming from multiple sources such as intel and heterogeneous devices or sensors. You must determine how to place your defenses and manage patrols to mitigate the risk. You will have to be mindful of the Uncertainty of Information (UoI) from your sources as you make your plans. You will be asked to rank how important or concerning the uncertainties from those sources are to you.

Third scenario

Your Infantry Battalion is tasked with bridging a critical crossing point of the local waterway. The OPFOR has a Company of motorized infantry operating in the area. Threat reports are incoming from multiple sources such as intel and heterogeneous devices or sensors. As the leader of the SCOUT Platoon, you must establish situational awareness of the OPFOR. You will have to be mindful of the Uncertainty of Information (UoI) from your sources as you execute our mission. You will be asked to rank how important or concerning the uncertainties from those sources are to you

UNCLASSIFIED//FOR OFFICIAL USE ONLY//DRAFT//PRE-DECISIONAL

10 of 15

ANALYSIS

UNCLASSIFIED

Using the responses from the subject matter experts, the average ranks were taken for each of the four source categories
For all three scenarios, any uncertainty with Networks was ranked as the least concern
For Scenario 1, the ranks are Networks (2) > Visualization (2.40) > Devices (2.44) > Information (3.3).
For Scenario 2, Network (1.85) > Devices (2.3) > Visualization (2.88) > Information (2.9).
For Scenario 3, Network (1.70) > Devices (2.3) > Information (2.8) > Visualization (3.11).
Three alternatives (ALT 1, ALT 2, and ALT 3) were presented for each scenario and the criteria for choosing amongst the alternatives were the four sources

weights that were produced using AHP

UNCLASSIFIED//FOR OFFICIAL USE ONLY//DRAFT//PRE-DECISIONAL

11 of 15

RESULTS - SCENARIO 1

UNCLASSIFIED

Scenario 1

Scenario 1 Ranking

Dynamic Scenario 1

New Scenario 1 Ranking

The numbers listed for each alternative row are randomly simulated UoI values. The higher the number, the more uncertain the details that the specific source has.

TOPSIS ranking of the three alternatives. Alternative 2 is the most ideal solution to Scenario 1, followed by Alternative 1 and Alternative 3.

For Scenario 1, the UoI values for Alternative 1 were assigned a new simulated value to represent if the details recovered by the sources had experienced a dynamic change.

The new ranks of the alternatives. In this scenario, despite new values, the ranks remain the same as Alternative 2 being the most ideal decision followed by Alternative 1 and Alternative 3.

UNCLASSIFIED//FOR OFFICIAL USE ONLY//DRAFT//PRE-DECISIONAL

12 of 15

RESULTS - SCENARIO 2

UNCLASSIFIED

Dynamic Scenario 2

Scenario 2

Scenario 2 Ranking

New Scenario 2 Ranking

The UoI values of the sources for each of the three alternatives for Scenario 2.

TOPSIS ranking of the three alternatives using the criteria provided. It resulted in Alternative 1 is the most ideal solution to Scenario 2, followed by Alternative 3 and Alternative 2.

To evaluate using the dynamic MCDM method for Scenario 2, new UoI values were assigned for Alternative 1 and Alternative 2.

New ranks of the alternatives. Now Alternative 3 resulted as the most ideal solution rather than Alternative 1, which decreased to rank 2. Alternative 2 remained as rank 3.

UNCLASSIFIED//FOR OFFICIAL USE ONLY//DRAFT//PRE-DECISIONAL

13 of 15

RESULTS - SCENARIO 3

UNCLASSIFIED

Dynamic Scenario 3

Scenario 3

Scenario 3 Ranking

New Scenario 3 Ranking

The UoI values of the sources for each of the three alternatives for Scenario 3.

TOPSIS ranking of the three alternatives using the criteria provided. It resulted in Alternative 1 is the most ideal solution to Scenario 3, followed by Alternative 3 and Alternative 2.

To evaluate using the dynamic MCDM method for Scenario 3, new UoI values were assigned for for all alternatives present.

New ranks of the alternatives. Alternative 1 still remains as the most ideal solution. Alternative 2 and Alternative 3 swapped ranks.

UNCLASSIFIED//FOR OFFICIAL USE ONLY//DRAFT//PRE-DECISIONAL

14 of 15

FUTURE WORK

Utilize MDCM to use the categories of UoI as a key criterion in the process.

Continue to apply MCDM and UoI to Army relevant decision tasks.

Explore integrating additional data sources related to autonomous systems.

Explore leveraging non-kinetic data.

UNCLASSIFIED

UNCLASSIFIED//FOR OFFICIAL USE ONLY//DRAFT//PRE-DECISIONAL

15 of 15

REFERENCES

Campanella G, Ribeiro RA. A framework for dynamic multiple-criteria decision making. Decis Support Sys. 2011a;52(1):52–60.

Campanella G, Ribeiro RA. Multiple attribute decision making in a dynamic environment. World Conference on Soft Computing; 2011 May 32–26; San Francisco, CA; 2011b.

Caylor, J., Hammell, R., and Raglin, A., “Preliminary Evaluation of Multi-Criteria Decision-Making Methodology for Emergency Management”. In Proceedings of the 12th International Conference on Applied Human Factors and Ergonomics (AHFE 2021).

Cui, Z. X., Yoo, H. K., Choi, J. Y., & Youn, H. Y. (2011, February). Multi-criteria group decision making with fuzzy logic and entropy based weighting. In Proceedings of the 5th International Conference on Ubiquitous Information Management and Communication (pp. 1-7).

Hwang, C. L., & Yoon, K., (1981). Multiple attribute decision making methods and applications, Springer-Verlag, New York.

Kivijärvi, H., & Tuominen, M. (1999, August). Integrating AHP and dynamic simulation: experiences, conceptualizations and business experiments. In Proceedings of the 5th Int’l Symposium on the AHP (pp. 386-392).

Lott, D. A., Raglin, A., & Metu, S. (2019, December). On the use of Operations Research for Decision Making with Uncertainty for IoT devices in battlefield situations. In 2019 IEEE 5th International Conference on Collaboration and Internet Computing (CIC) (pp. 266-297). IEEE.

Mardani, A., Jusoh, A., Nor, K., Khalifah, Z., Zakwan, N., & Valipour, A. (2015). Multiple criteria decision-making techniques and their applications–a review of the literature from 2000 to 2014. Economic Research-Ekonomska Istraživanja, 28(1), 516-571.

Opricovic, S., & Tzeng, G. H. (2004). Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS. European journal of operational research, 156(2), 445-455.

Raglin, A. Metu, S. and Lott, S. (2020) Challenges of Simulating Uncertainty of Information, HCII conference

Saaty, T. L. (1990a). Decision making for leaders: the analytic hierarchy process for decisions in a complex world. RWS publications.

Saaty, T. L. (2007). Time dependent decision-making; dynamic priorities in the AHP/ANP: Generalizing from points to functions and from real to complex variables. Mathematical and Computer Modelling, 46(7-8), 860-891.

Singh, A., & Malik, S. K. (2014). Major MCDM Techniques and their application-A Review. IOSR Journal of Engineering, 4(5), 15-25.

Thor, J., Ding, S. H., & Kamaruddin, S. (2013). Comparison of multi criteria decision making methods from the maintenance alternative selection perspective. The International Journal of Engineering and Science, 2(6), 27-34.

Varela, M. L. R., & Ribeiro, R. A. (2014). Distributed manufacturing scheduling based on a dynamic multi-criteria decision model. In Recent Developments and New Directions in Soft Computing (pp. 81-93). Springer, Cham.

Wan, W. M. F. B., Nor, N. M. M., & Jalil, M. A. (2015). Identification of potential crime tactical path-finding using Analytical Hierarchy Process (AHP) in situational crime prevention. In Proc. The 7th Int. Conf. on Information Technology, Jordan (pp. 1-7).

UNCLASSIFIED

UNCLASSIFIED//FOR OFFICIAL USE ONLY//DRAFT//PRE-DECISIONAL