1 of 45

Towards AI-driven User Support for CNAF Tier-1

M. Barbetti, A. Trashaj, E. Ronchieri, C. Pellegrino, and the contribution of many others

CNAF seminar | 21 November 2024

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

2 of 45

THE EVOLUTION OF

ARTIFICIAL INTELLIGENCE

A sensationalistic overview of the history of Artificial Intelligence

.1.0.

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

3 of 45

Can you find the (AI)ntruder?

3

3

Credits: Jorg Karg

Credits: Boris Eldagsen

Credits: Aleksei Andreev

Credits: Evgeny Lutsko

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

4 of 45

Can you find the (AI)ntruder?

4

Credits: Jorg Karg

Credits: Aleksei Andreev

Credits: Evgeny Lutsko

Credits: Boris Eldagsen

(by using DALL-E 2)

PSEUDOMNESIA: The Electrician, 2023

WINNER

of the creative open category at the

Sony World Photography Awards

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

5 of 45

History of Artificial Intelligence

5

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

6 of 45

History of Artificial Intelligence

6

First model of an

Artificial Neural Network

by W. McCulloch and W. Pitts

Alan Turing introduced

a test to answer the question

“Can machines think?”

1943

1950

The term Artificial Intelligence was coined for a thematic workshop hosted at Dartmouth

1956

Geoffrey Hinton coined the term

Deep Learning to refer to

deep ANN-powered algorithms

2006

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

7 of 45

History of Artificial Intelligence

7

The modern technique for training MLPs, known as backpropagation, was first published by S. Linnainmaa

P. Werbos and D.E. Rumelhart independently developed and applied backpropagation to MLPs in the way that has become standard

1970

1982

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

8 of 45

History of Artificial Intelligence

8

The birth of the

World Wide Web (WWW)

by Tim Berners-Lee

The launch of Google

by Larry Page and Sergey Brin

1989

1998

The launch of Facebook

by Mark Zuckerberg

The launch of

Amazon Web Service (AWS)

The launch of

Google Cloud Platform (GCP)

2004

2002

2008

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

9 of 45

History of Artificial Intelligence

9

The foundation of NVIDIA by Jensen Huang

The release of Torch, the first open-source machine learning library

The release of CUDA, software that enables parallel computing on NVIDIA GPUs

The publication of Minecraft by Mojang Studios

The release of several deep learning libraries, like Caffe (2013), Keras (2015), TensorFlow (2015), PyTorch (2016)

1993

2011

2013 - 2016

2002

2007

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

10 of 45

History of Artificial Intelligence

10

The defeat of Garry Kasparov by IBM’s Deep Blue, the first computer to win against a world chess champion

The defeat of Lee Sedol by DeepMind’s AlphaGo, the first computer to win against a 9-dan professional Go player

1997

2016

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

11 of 45

History of Artificial Intelligence

11

The first ImageNet Large Scale Visual Recognition Challenge (ILSVRC), an annual contest on image classification and detection via software programs

The victory of the ImageNet 2012 Challenge by a CNN-based model called AlexNet

The invention of Generative Adversarial Nets by Ian Goodfellow and colleagues

The victory of the ImageNet 2015 Challenge by a novel architecture introduced with ResNet

The launch of

DALL-E by OpenAI

2010

2012

2014

2015

2021

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

12 of 45

History of Artificial Intelligence

12

The release of ELIZA by Joseph Weizenbaum, the computer program to engage conversations with humans

The introduction of the Transformer architecture with the paper “Attention is all you need”

The introduction of BERT by Google and GPT-1 by OpenAI, models for NLP problems

The launch of several chatbot platforms, like ChatGPT by OpenAI (2022), Bard/Gemini by Google (2023), Claude by Anthropic (2023)

1966

2022 - 2023

2017

2018

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

13 of 45

The evolution of image generation

13

<PROMPT> A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. She wears a black leather jacket, a long red dress, and black boots, and carries a black purse. She wears sunglasses and red lipstick. She walks confidently and casually. The street is damp and reflective, creating a mirror effect of the colorful lights. Many pedestrians walk about. </PROMPT>

Credits:

  • Alec Radford
  • Luke Metz
  • Soumith Chintala

DCGAN (arXiv:1511.06434), 2015

Sora (OpenAI text-to-video model), 2024

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

14 of 45

Chatbot, chatbot everywhere!

14

Credits: https://yellow.ai/blog/future-of-chatbots

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

15 of 45

EVOLVING THE CNAF USER SUPPORT

THROUGH AI-DRIVEN SOLUTIONS

How AI technologies can help in supporting Tier-1 users

.2.0.

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

16 of 45

Scientific computing at CNAF

16

NON LHC

LHC

As INFN national center dedicated to R&D on Information and Communication Technologies, CNAF host the Italian Tier-1 data center since 2003, providing computing resources to a wide scientific community:

  • collaboration with 60+ experiments (not only the four major experiments of LHC)
  • support to 1500+ active users accessing the Tier-1 resources (including communities different from the High Energy and Nuclear Physics one)

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

17 of 45

Support for experiments at Tier-1

17

User

User Support

Infrastructure

Storage

Network

AAI

Security

Farming

Cloud

+

The Italian Tier-1 ensures support for experiments and users through the dedicated User Support (US) unit:

  • principal contact point between users and other specialized Tier-1 units
  • development of tools/procedures to simplify the access to computing resources
  • preparation and updates of the Tier-1 User Guide (https://l.infn.it/t1guide)

Three levels of support:

  1. User Support unit (entry point)
  2. involvement of other specialized units
  3. involvement of software developers

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

18 of 45

Support for experiments at Tier-1

18

User

User Support

Infrastructure

Storage

Network

AAI

Security

Farming

Cloud

+

The Italian Tier-1 ensures support for experiments and users through the dedicated User Support (US) unit:

  • principal contact point between users and other specialized Tier-1 units
  • development of tools/procedures to simplify the access to computing resources
  • preparation and updates of the Tier-1 User Guide (https://l.infn.it/t1guide)

Three levels of support:

  • User Support unit (entry point)
  • involvement of other specialized units
  • involvement of software developers

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

19 of 45

Support for experiments at Tier-1

19

User Support

Infrastructure

Storage

Network

AAI

Security

Farming

Cloud

+

User

The Italian Tier-1 ensures support for experiments and users through the dedicated User Support (US) unit:

  • principal contact point between users and other specialized Tier-1 units
  • development of tools/procedures to simplify the access to computing resources
  • preparation and updates of the Tier-1 User Guide (https://l.infn.it/t1guide)

Three levels of support:

  • User Support unit (entry point)
  • involvement of other specialized units
  • involvement of software developers

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

20 of 45

Support for experiments at Tier-1

20

User Support

Infrastructure

Storage

Network

AAI

Security

Farming

Cloud

+

Software

developer

User

The Italian Tier-1 ensures support for experiments and users through the dedicated User Support (US) unit:

  • principal contact point between users and other specialized Tier-1 units
  • development of tools/procedures to simplify the access to computing resources
  • preparation and updates of the Tier-1 User Guide (https://l.infn.it/t1guide)

Three levels of support:

  • User Support unit (entry point)
  • involvement of other specialized units
  • involvement of software developers

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

21 of 45

Evolving the User Support using Artificial Intelligence

21

User Support

Infrastructure

Storage

Network

AAI

Security

Farming

Cloud

+

In the context of NRRP and thanks to the action of the ICSC and TeRABIT projects, the Italian Tier-1 will evolve significantly in the next few years:

  • acquisition of more and more users and experiments
  • wide scientific community with various requirements in terms of infrastructure, hardware, and software
  • need for an evolution of the CNAF data center and its units

Employing AI models to build a Digital User Supporter able to help in supporting a growing number of new users and in adopting the latest software technologies

IDEA

User

GenAI

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

22 of 45

Prototyping a Digital User Supporter

22

User request

User Support

Support of 1st/2nd/3rd level?

Which units are involved?

ML multi-label classifier

The answer is available in the Tier-1 User Guide. Can we automate it?

GenAI-powered digital agent

Automated identification of involved units using the first-contact email

Automated answer to the user request relying on the acquired knowledge base

The process of developing an AI-powered digital agent for managing User Support duties has been organized into two parallel and complementary sub-projects:

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

23 of 45

2.1 Multi-label classification

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

24 of 45

Steps for building an email classifier

24

2

1

Preparation of the dataset of emails for training

Mapping of the textual features into a numerical representation (embedding)

Classifier training using differente ML algorithms

Benchmark of the classification performance and results combination (ensemble)

3

4

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

25 of 45

A quick look at the data sample

25

~30k emails processed in 6 years

Extensive interaction between users, US, and specialized units

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

26 of 45

Preparation of the training dataset

26

Data collection

Anonymization

Labelling

Data cleaning

The sample contains ~30k emails received/sent in the period 06/2017 – 05/2023

Each has been saved as an individual JSON file (~260 MB)

No attachments, and (almost) no HTML content

Each file includes:

  • from – email address
  • to – email address
  • date – datetime
  • subject – text
  • content – text
  • parent – email address

All the email addresses have been anonymized by replacing them with UUIDs

Any references to real individuals (e.g., names or usernames) found in the email subject/body have been anonymized and replaced with the placeholder [NAME]

The anonymization process was carried out using automated scripts or by editing manually the email metadata

Each email is assigned a label based on the text contained in its subject/body

The email thread between a user and the various Tier-1 units involved is reconstructed using the information in the parent field

All labels associated to the same thread are propagated to the first contact email as a part of the labelling process

The email body may contain escape sequences (e.g., \n, \t) o HTML tags

The emails are multilingual (English/Italian) and may include special characters (e.g., accented letters, emojis)

The data cleaning process transforms the text to retain only words, placeholder, and punctuation

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

27 of 45

The prepared training set

27

Hello,

​

How can I use Jupyter Notebooks at CNAF?

​

Thank you,

​

[NAME]

[user-support]

[farming]

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

28 of 45

How to represent the textual features

28

Representation for “👩🏻”

(x1, y1)

(x2, y2)

(xR, yR)

Representation for “👑”

Representation for “👸🏻”

+

=

  • The meaning of a text is encoded in word combinations → ML models operate on vectors (features)
    • Word (or token) combinations can be mapped into a “representation space” (a high-dimensional vector space) through an embedding model
  • The embedding model can be designed to preserve semantics, even across multiple languages
    • Sentence-Transformers is a Python framework that provides state-of-the-art models for text embedding

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

29 of 45

How emails look like in the representation space

29

all-mpnet-base-v2

  • base model: MPNet
  • embedding dim: 768
  • max seq length: 384
  • inference speed: 1
  • language: English only

​

all-MiniLM-L6-v2

  • base model: MiniLM
  • embedding dim: 384
  • max seq length: 256
  • inference speed: x5
  • language: English only

paraphrase-multilingual-mpnet-base-v2

  • base model: XLM-RoBERTa
  • embedding dim: 768
  • max seq length: 128
  • inference speed: ∼1
  • language: multiple

​

paraphrase-multilingual-MiniLM-L12-v2

  • base model: MiniLM
  • embedding dim: 384
  • max seq length: 126
  • inference speed: x2.5
  • language: multiple

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

30 of 45

Performance of the trained classifiers

30

Different ML models were trained to perform multi-label classification

  • k-Nearest Neighbors (kNN)
  • Random Forest (RF)
  • Extreme Gradient Boosting (XGBoost)
  • Feed-forward Neural Network (FNN)

Various embedding models were used in combination with these classification models

  • all-mpnet-base-v2
  • all-MiniLM-L6-v2
  • paraphrase-multilingual-mpnet-base-v2
  • paraphrase-multilingual-MiniLM-L12-v2

Different preprocessing strategies → 4 (emb) x 4 (prep) x 4 (clf) = 64 models in total

By combining the outputs of the most promising models using an ensemble approach, the best performance was achieved: ~95% of ROC AUC score on the test set

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

31 of 45

2.2 Digital User Supporter

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

32 of 45

The problem of artificial hallucinations

32

Credits: https://fisicisenzapalestra.com/artificiale-come-lintelligenza-e-gli-allucinogeni.html

One of the major challenges in working with AI chatbots and Computer Vision tools is the issue of artificial hallucinations

  • responses that contain false or misleading information presented as facts

A potential solution to limit the “creativity” of Large Language Models (LLMs) is to integrate them into an architecture designed to retrieve relevant information and compose answers based on that data

The Retrieval-Augmented Generation (RAG) architecture provides a semantic-aware pipeline that retrieves information from a knowledge base and generates text grounded in the retrieved data

​

The first prototype of the AI-powered Digital User Supporter has been developed upon a RAG model, implemented with LangChain and provisioning the LLM relying on ollama

PROTOTYPE

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

33 of 45

How to build a Digital User Supporter

33

Vector database

Knowledge base

Embedding

User query

Similarity search

Embedding

Prompt assembly

Large

Language Models

AI-based answer

Retrieval-Augmented Generation (RAG)

Credits: Alberto Trashaj

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

34 of 45

How to build a Digital User Supporter

34

Vector database

Knowledge base

Embedding

Similarity search

Embedding

Prompt assembly

Large

Language Models

AI-based answer

Retrieval-Augmented Generation (RAG)

User query

Hello,

​

How can I use Jupyter Notebooks at CNAF?

​

Thank you,

​

[NAME]

User ask a question to User Support

of Tier-1

Credits: Alberto Trashaj

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

35 of 45

How to build a Digital User Supporter

35

Vector database

Embedding

User query

Similarity search

Embedding

Prompt assembly

Large

Language Models

AI-based answer

Retrieval-Augmented Generation (RAG)

Knowledge base

https://l.infn.it/t1guide

Credits: Alberto Trashaj

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

36 of 45

How to build a Digital User Supporter

36

Vector database

Knowledge base

User query

Similarity search

Embedding

Prompt assembly

Large

Language Models

AI-based answer

Retrieval-Augmented Generation (RAG)

Embedding

The knowledge base (Tier-1 User Guide) is mapped to a vector space through an embedding model, such as all-mpnet-base-v2

Vector space

Credits: Alberto Trashaj

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

37 of 45

How to build a Digital User Supporter

37

Knowledge base

Embedding

User query

Similarity search

Embedding

Prompt assembly

Large

Language Models

AI-based answer

Retrieval-Augmented Generation (RAG)

Vector database

To store the embedding results, a database optimized for handling large vectors and their metadata (e.g., source text) is required. It should also support efficient searches within the vector space (features like these are at the core of Chroma)

Credits: Alberto Trashaj

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

38 of 45

How to build a Digital User Supporter

38

Vector database

Knowledge base

Embedding

User query

Similarity search

Prompt assembly

Large

Language Models

AI-based answer

Retrieval-Augmented Generation (RAG)

Embedding

As well as the knowledge base, also the user request is mapped to a vector space relying on the same embedding model (all-mpnet-base-v2)

Vector space

Credits: Alberto Trashaj

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

39 of 45

How to build a Digital User Supporter

39

Vector database

Knowledge base

Embedding

User query

Embedding

Prompt assembly

Large

Language Models

AI-based answer

Retrieval-Augmented Generation (RAG)

Similarity search

To retrieve relevant information from the guide to respond to the user request, we can compare the embeddings from both cases based on similarity (e.g., dot product)

Once the guide embeddings that best match the request embedding are identified, the vector database can be queried to retrieve the corresponding metadata

✓

✗

Credits: Alberto Trashaj

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

40 of 45

How to build a Digital User Supporter

40

Vector database

Knowledge base

Embedding

User query

Similarity search

Embedding

Large

Language Models

AI-based answer

Retrieval-Augmented Generation (RAG)

Prompt assembly

The retrieved text segments are used to tailor the template for the textual prompt submitted to the Language Model

Answer the user’s questions based on the below context as you were answering to an email in a professional style.

​

<context>

{context}

</context>

Credits: Alberto Trashaj

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

41 of 45

How to build a Digital User Supporter

41

Vector database

Knowledge base

Embedding

User query

Similarity search

Embedding

Prompt assembly

AI-based answer

Retrieval-Augmented Generation (RAG)

Large

Language Models

The generated prompt can then be submitted to a Large Language Model (LLM), which, even if not an “expert” in a specific subject, is capable of extracting the “relevant” content from {context} and rephrasing it

Since fine-tuning is not required, LLMs are often described as general-purpose technologies and are thus also referred to as Foundation Models

Credits: Alberto Trashaj

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

42 of 45

How to build a Digital User Supporter

42

Vector database

Knowledge base

Embedding

User query

Similarity search

Embedding

Prompt assembly

Large

Language Models

Retrieval-Augmented Generation (RAG)

AI-based answer

Dear User,

​

To answer your question directly, yes, you can use Jupyter notebooks through the JupyterHub service at Tier-1. The service is reachable via browser at the following page: <https://jupyterhub-t1.cr.cnaf.infn.it/>. Please note that you will need to have an account and be logged in to access the notebooks.

​

If you have any further questions or concerns, please feel free to ask.

​

Best regards,

[Your Name]

Note: automated answer generated with Llama 2

Credits: Alberto Trashaj

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

43 of 45

MOVING TO THE CONCLUSIONS

Summary, conclusions, and final remarks

.3.0.

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

44 of 45

Here are my conclusions!

44

  • AI is reshaping the computing landscape, driving technological evolution, influencing market trends, and dominating software development worldwide
  • CNAF should ride the AI wave, exploring and implementing new technologies to drive its evolution within an AI-dominated world
    • the development of a Digital User Supporter is a step in this direction, but also other units can benefit from using AI
  • The results obtained with the first prototype of the Digital User Supporter are promising but there is still room for improvements
    • e.g., multi-expert model, enlarged knowledge base (also including emails)
  • As AI companies develop increasingly large models requiring massive GPU resources, it is essential to implement tailored solutions
    • this is crucial to avoid that AI chatbots monopolize GPU resources needed for scientific research

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing

45 of 45

Thanks!

Any questions or comments?

Matteo Barbetti (INFN CNAF)

email: matteo.barbetti@cnaf.infn.it

Missione 4 • Istruzione e Ricerca 

ICSC Italian Research Center on High-Performance Computing, Big Data and Quantum Computing