Email: mohamed.abdelazez@forces.gc.ca

Defence Research Development Canada – Centre for Operational Research and Analysis

Improving C2 and Decision Making Using the Joint Operations Fusion Lab

Mohamed Abdelazez and Dr. Ahmed Ghanmi

Centre of Operational Research and Analysis

Introduction – Command and Control (C2)

Image by: Jeff Dahl (File:DesertStormMap v2.svg - Wikimedia Commons)

Introduction – Fusion

Introduction – Joint Operations Fusion Lab (JOpsFL)

The JOpsFL:

• Is conceived by the DND / CAF as an innovation hub for JISR, C2, and Targeting Enterprises to deliver effective operational solutions.
• The lab will encompass JISR and intelligence initiatives from the different CAF branches.

Introduction – JISR

• JISR provides the ‘what’, ‘when’, and ‘where’, while intelligence fuses the information from the JISR to provide the ‘how’ and ‘why’.
• A focus of the JOpsFL will be the ‘how’ and ‘why’; however, the lab will also act as a test bed for new JISR resources to assess their efficacy within the targeting cycle.

Image from: G. W. Jensen and J. MacLennan, "Joint Operations Fusion Laboratory," Canadian Joint Operations Command, Ottawa, 2020.

Introduction – JOpsFL Areas of Interest

• Artificial Intelligence (AI);
• Human-in-the-loop interactive AI systems;
• Multi-spectrum hybrid infrastructure and analysis;
• Integrated tactical networks/Unified networks;
• Flexible data links between sensors, JISR, and Pan-Domain C2;
• Information fusion;
• Fusion at the point of collection;
• Cloud computing;
• Fog computing;
• Quantum computing;
• Virtualization; and
• Cross domain security and sharing

Information Fusion

​

​

The combination of data and information from heterogeneous sources across multiple domains, such as physical sensors and human reports.

M. E. Liggins II, D. L. Hall and J. Llinas, Handbook of Multisensor Data Fusion: Theory and Practice, Second Edition ed., New York: CRC Press, 2017.

Information Fusion – Advantages

• Improved estimation and observation of a phenomenon;
• Reduction of interference, ambiguity, and uncertainty;
• Expanded situational awareness; and
• Improved reliability and robustness

M. E. Liggins II, D. L. Hall and J. Llinas, Handbook of Multisensor Data Fusion: Theory and Practice, Second Edition ed., New York: CRC Press, 2017.

E. Bosse, J. Roy and D. Grenier, "Data fusion concepts applied to a suite of dissimilar sensors," in Proceedings of 1996 Canadian Conference on Electrical and Computer Engineering, Calgary AB, Canada, 1996.

Information Fusion – JDL Framework

• Frameworks are systems for manipulating objects that were defined based on a set of axioms.
• One of the first frameworks developed was by the US DoD Joint Directors of Laboratories (JDL). This was known as the JDL framework and was specifically developed for military applications.

​

Information Fusion – JDL Framework

Fusion Categories

Radar image by: CdnMCG (File:Military Symbol - Friendly Unit (Bichrome 1.6x1 Frame)- Signals - Electronic Ranging (NATO APP-6 & APP-6B).svg - Wikimedia Commons)

LIDAR image by: Andrei Yushchenko (Lidar Icons - Download Free Vector Icons | Noun Project (thenounproject.com))

Satallite image by: Maxxl (File:FP Satellite icon.svg - Wikimedia Commons)

Complementary

Cooperative

Competitive

Fusion Levels

• Signal - raw information from the sources are fused together
• Feature - information from the sources are reduced to a set of features that are then used for the fusion operation
• Decision - independent decisions are fused together to reduce ambiguity and uncertainty
• Assessment:
• Communication Load
• Processing Complexity
• Information Loss
• Performance Loss

Fusion Levels – Signal

• Signal level fusion operates directly on the information reported from the sources.
• Fusion at this level is mainly used for redundancy, calibration, and reduction of interference, ambiguity, and uncertainty by fusing information from homogenous sources.
• Signal level fusion typically takes place in the competitive category to improve the estimation of a phenomenon.
• Assessment:

​

​

Image from: Q. Du, H. Xu, Y. Ma, J. Huang and F. Fan, "Fusing Infrared and Visible Images of Different Resolutions via Total Variation Model," Sensors, vol. 18, no. 11, 2018.

Fusion Levels – Feature

• Feature level fusion is a step higher than signal level fusion as it is operating on features derived from the raw information.
• This level of fusion can occur under any of the fusion categories.
• Assessment:

​

Image from: L. Ting, S. Li, L. Fang, X. Jia and J. A. Benediktsson, "From Subpixel to Superpixel: A Novel Fusion Framework for Hyperspectral Image Classification," IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 8, pp. 4398-4411, 2017.

Fusion Levels – Decision

• Decision level fusion is the highest fusion level and is usually characterized as an expert system.
• The decisions performed by the different sources on a specific phenomenon are fused to reduce ambiguity and uncertainty.
• This level of fusion can occur under any of the fusion categories.
• Assessment:

​

Image from: B. Bigdeli, F. Samadzadegan and P. Reinartz, "A decision fusion method based on multiple support vector machine system for fusion of hyperspectral and LIDAR data," International Journal of Image and Data Fusion, vol. 5, no. 3, pp. 196-209, 2014.

JOpsFL Areas of Opportunities – Information Acquisition

• Internet of Battle Things can be a major driver of agile Pan Domain C2 due to their ability to sense the environment in real-time. A cloud-fog hybrid is needed to alleviate the computational needs required by an increasing number of smart devices.

Image from: macchina.io - Edge and Fog Computing

JOpsFL Areas of Opportunities – Soft and Hard Information Fusion

• Commanders and analysts can provide valuable insight and experience into the fusion process.
• ML algorithms will be used to learn the uncertainty in the soft information and to fuse soft and hard information.

Image from: G. Gross, R. Nagi and K. Sambhoos, "A fuzzy graph matching approach in intelligence analysis and maintenance of continuous situational awareness," Information Fusion, vol. 18, pp. 43-61, 2014.

JOpsFL Areas of Opportunities – Information Visualization

• Allow for commanders to view and process intelligence products efficiently.
• ML based intelligent visualizations that highlight valuable information based on the user’s preferences, while highlighting missing information such as negative information.

Image from: D. L. Hall, S. A. H. McMullen and C. M. Hall, "New perspectives on level-5 information fusion: The impact of advances in information technology and user behavior," in 2015 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI)

JOpsFL Areas of Opportunities – Information Retrieval

• ML algorithms to caption images and potentially answer textual questions about an image.

​

Image from: Q. Wu, P. Wang, C. Shen, I. Reid and A. v. d. Hengel, "Are You Talking to Me? Reasoned Visual Dialog Generation Through Adversarial Learning," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018.

Conclusion

• JOpsFL will be an innovation hub that will investigate information fusion to support command and control.
• Areas of opportunities include but not limited to information acquisition, fusion, visualization, and retrieval.
• Future work includes acquiring the computing infrastructure to support the exploration of the fusion techniques and collating information from different sources for experimentation

Email: mohamed.abdelazez@forces.gc.ca

Defence Research Development Canada – Centre for Operational Research and Analysis

Improving C2 and Decision Making Using the Joint Operations Fusion Lab

Mohamed Abdelazez and Dr. Ahmed Ghanmi

Centre of Operational Research and Analysis