1st AI_INFN Advanced Hackathon 2024

Processing data from

the LHCf detector:

Construction of a multi-layer Neural Network

Rosa Petrini

rosa.petrini@fi.infn.it

Andrea Paccagnella

andrea.paccagnella@unifi.it

2

Subjects

• Dense Neural Network (DNN)
• Convolutional Neural Network (CNN)

Types of data and different analyses

• Sequential model
• Functional API

Why different type of analysis?

Many different types of data must be combined to discriminate events

3

Energy

from calorimeters

Dense

Neural Network

Position

from trackers

Convolutional

Neural Network

Neural Networks

4

Dense Neural Network

Key features:

• Used in various tasks: Classification, regression problems and more.
• Fully connected layers: Every input is connected to every neuron in the next layer.
• Requires training: It learns by adjusting weights and biases through backpropagation.

5

A type of artificial neural network in which every neuron in a layer is connected to every neuron in the next layer.

Example of a neural network with a single neuron

Sigmoid

Dense Neural Network

Composition:

• Input Layer: Takes in the data
• Hidden Layers: Process the data through activation functions (e.g., ReLU, Sigmoid)
• Output Layer: Produces the final prediction (e.g., a classification).

6

Full connectivity ensures that all features from one layer contribute to the next, maximising learning capacity.

Backpropagation

7

• Based on all input data and starting weights calculate the predicted output
• Define some kind of a loss (cost) function that is a function of real output values that are known from train data and predicted output
• Determine how you can change the weights and bias so that the loss function would decrease in value
• Update weights and bias
• Repeat steps 1-4 until loss (cost) function can’t be reduced anymore

​

This procedure is called backpropagation because we are propagating information in the opposite direction of the neural network

​

Dense Neural Network: Problems

8

Problems in image analysis with Dense networks:

• High dimensionality: Images have many pixels, resulting in a huge number of parameters when fully connected to each neuron.
• Loss of spatial information: Dense networks treat all pixels independently, ignoring spatial relationships (e.g. edges, patterns).
• Computational inefficiency: The large number of connections makes training slow and resource-intensive.

​

Same image but with the pixels mixed together

(1588, 1138)

Dense equivalent:

1807144 input feature

football crest

Dense Neural Network: Problems

9

Why CNNs are better:

• Feature extraction: Convolutional layers detect patterns such as edges and shapes, preserving spatial relationships.
• Reduced complexity: Reduce the number of parameters, making CNNs more efficient.
• Hierarchical learning: CNNs learn low-level features in early layers and high-level features in deeper layers.

Problems in image analysis with Dense networks:

• High dimensionality: Images have many pixels, resulting in a huge number of parameters when fully connected to each neuron.
• Loss of spatial information: Dense networks treat all pixels independently, ignoring spatial relationships (e.g. edges, patterns).
• Computational inefficiency: The large number of connections makes training slow and resource-intensive.

​

Dense equivalent:

1807144 input feature

Convolutional Neural Network

A specialized neural network designed for processing structured data like images or videos.

CNN are computationally efficient and can handle spatial hierarchies in data.

​

A neural network capable of processing structured data. The convolution is defined:

​

​

​

where A is an M · N matrix (input image), H is a k · k square matrix. H is called kernel.

C is the Activation Map

​

Key Features:

• Convolutional Layers: Extract features using filters.
• Pooling Layers: Reduce dimensionality while retaining key features.
• Used For: Image recognition, object detection, and video analysis.

​

10

Convolutional Neural Network

11

Convolutional Neural Network: Convolutional Layers

12

The Filter is like a learned matrix multiplication. It is like the weights of Dense NN.

By going over the image, each filter can focus on the local features

We will have filter specializing in recognizing the same element over multiple training images

The result of the convolution operation between kernel and image is called ACTIVATION MAP

Convolutional Neural Network: looking at convolution

13

Thanks to Francesca Lizzi

Convolutional Neural Network: looking at convolution

14

Thanks to Francesca Lizzi

Convolutional Neural Network: Convolutional Layers

15

The Filter is like a learned matrix multiplication

By going over the image, each filter can focus on the local features

We will have filter specializing in recognizing the same element over multiple training images

The result of the convolution operation between kernel and image is called ACTIVATION MAP

Convolutional Neural Network: Pooling

16

• Reduce spatial dimensions of feature maps;
• Decrease the computational power required to process the data;
• Extract dominant features (rotational and positional invariance).

​

Types of Pooling:

• Max Pooling: Returns the maximum value in the kernel region (is noise suppressant and e improved feature recognition).
• Average Pooling: Returns the average value in the kernel region.

Max Pooling is more effective than Average Pooling for noise reduction and feature preservation.

Convolutional Neural Network: Pooling

17

Activation Maps (26x26)

Example Max Pooling (13x13)

Stride = 2 Pixel

Filter Size = 2x2 Pixel

Convolutional Neural Network

The last part of the network is usually a feedforward NN which reasons on such a representation: integrate learned features to make a final prediction.

18

Reduces parameter size compared to dense networks.

Captures spatial hierarchies (e.g., edges, textures, objects)

Design of a model

19

The Sequential model

20

A Sequential model is appropriate for a plain stack of layers where each layer has exactly one input tensor and one output tensor.

The Sequential model is used when the model architecture consists of a simple linear stack of layers.

​

Examples of projects with Sequential model:

• Image classification: For example, classifying images of different animals or recognising handwritten digits.

​

Boundaries:

• No Multiple Inputs/Outputs: The sequential API does not support models with multiple independent inputs or outputs.
• No Shared Layers: It is not possible to share layers across different branch; each layer can have only one input and output.

The Functional API

21

• Is more flexible than the Sequential model.
• Can handle models with non-linear topology, shared layers, and even multiple inputs or outputs.

Examples of projects with Functional API model:

• Recommender system: a movie recommender system where the goal is to predict which movies a user is likely to enjoy based on their past viewing history.
• Image segmentation: For example, segmenting different objects in an image.

Composition of input data from LHCf

• Our goal is to distinguish a hadronic particle from an electromagnetic particle;
• We have multiple inputs in the model from the multiple outputs of the detector;
• We need to use a Functional API to put all these different inputs together and have a one single output.

22

22

Composition of input data from LHCf

23

Energy released in the calorimeter tower

Dense Neural Network

Composition of input data from LHCf

24

Position of particle’s incidence in the last 2 xy planes of the tracer

1D

Convolutional

Neural Network

Composition of input data from LHCf

25

Position of particle’s incidence in the first 2 xy planes of the tracer

2D

Convolutional Neural Network

Creation of a 2D image