2 of 37

Hello

Dr. Christoph Breidert

Computer Science background

20+ years in IT industry

Co-Founder 1xINTERNET

Head of R&D

SoMe / Drupal: breidert

3 of 37

We are 1xINTERNET

1xINTERNET is a full service digital agency and a leader in the development of web solutions based on Drupal and React

Our core competencies are:

Digital Strategy and Consulting
User Experience and Design
Development with Drupal and React
Webtracking, SEO and Online Marketing
Hosting, Maintenance and Support Services

4 of 37

We are 1xINTERNET

International team of 85 employees

From all corners of the world:

South America, North America, Asia and Europe

Over 18 different languages

Offices in Frankfurt (Headquarters), Berlin, Reykjavik and Conil de la Frontera

Founded in 2013

5 of 37

200+ Drupal projects

In recent years, we have become one of the leading European Drupal agencies.

6 of 37

AI with Drupal

Using LLM technology is easy, but how do you actually build useful applications?

7 of 37

Goals of this presentation

You will understand how Large Language Models (LLM) work.

You will see what is possible with AI out-of-the-box.

You will understand how to build your own applications.

You will get some inspiration for useful applications.

8 of 37

Agenda

Overview AI
Demos
Building dynamic applications
Creating useful applications

9 of 37

Large Language Models (LLMs)

Overview AI

For this presentation we refer to so-called large language models when we talk about AI applications.
With AI applications we mean generative AI - applications that generate text.
LLMs are statistical applications, that predict output text, based on input text.
These applications have become so powerful that the responses to text input feel like a real conversation.

10 of 37

Training of LLMs

Overview AI

AI applications like ChatGPT are based on large language models (LLMs), because they are trained on a massive amount of data and contain millions, or even billions of parameters.

11 of 37

Should I train my own model?

No (probably not)

Overview AI

12 of 37

A LLM generates text

A LLM model is a software that generates grammatically correct sentences with a high probability of making sense.
The generated text is based on the training data.

Overview AI

13 of 37

A LLM generates text

Think of a LLM as a software that can “talk”.
Of course a LLM can only talk about what it “knows”.

Overview AI

14 of 37

Building custom applications with LLMs

Overview AI

When building custom applications, additional information that is not present in the LLM itself can be passed together with the user input. This is referred to as in-context learning or pre-prompting.
Usually you pass at least two messages along with the user input:

A set of instructions for the AI how to respond.
Additional information to consider when generating answers.

15 of 37

Agenda

Overview AI
Demos
Building dynamic applications
Creating useful applications

16 of 37

Drupalcamp Scotland Demo

Demos

https://ai.1xinternet.de/en/assistant-drupalcamp-scotland

17 of 37

Drupalcamp Scotland - Configuration

Demos

Instructions

You are a friendly chat assistant that informs about the programe of Drupalcamp Scotlabnd 2024.

You will be presented information about the sessions and speaksers in markdown format in the first user message.

Answer questions based on the information you are provided. Do not use any other source of information.

Answer short and precise.

If you cannot answer, point the user to program overview on the URL https://camp.drupal.scot/

Format your responses as markdown.

Data

…

Time: 09:30

Speaker: Stratos Filalithis, Head of Website & Communication Technologies, University of Edinburgh

Session title: Host's welcome

Time: 09:45

Speaker: Billy Wardrop

Biography: Web Development Team Manager in University of Edinburgh

Session title: A 7 year journey from Drupal 7 to Drupal 10 and what we learned migrating over 600 websites

Description: 7 years ago we started thinking about a new web

…

18 of 37

Example: DrupalCon Barcelona Sessions - Configuration

Demos

19 of 37

Example: DrupalCon Barcelona Sessions - Cost

Overview AI

Model: gpt-4o-mini
Data Input: 21.636 tokens
Cost: $0.150 / 1M input tokens, $0.600 / 1M output tokens
Average conversation: ~10 questions
Cost per conversation: ~$0.04 (100 conversations: $4, 1000 conversations: 40$, etc.)

20 of 37

Agenda

Overview AI
Demos
Building dynamic applications
Creating useful applications

21 of 37

Retrieval Augmented Generation (RAG)

Building dynamic applications

Instead of passing hardcoded contextual data, data can be retrieved in a pre-processing task.
This approach is called Retrieval Augmented Generation (RAG).
Examples for retrieval are:

Search (lexical search, semantic search, etc.)
Data queries (SQL database, Key-value stores, etc.)

22 of 37

Intranet assistant (RAG)

Building dynamic applications

23 of 37

Intranet assistant (RAG)

Building dynamic applications

24 of 37

Intranet assistant (RAG) - Configuration

Demos

Instructions

You are a friendly and professional assistant.

You will be provided content in the user messages separated by """". Respond solely on the content provided.

Create links from the H1 and "Document source" in the following format <a href="[Document source]">[H1]</a> the end of your response and recommend to get more information on these intranet pages.

Format your responses in markdown.

25 of 37

Intranet assistant (RAG) - Data

Building dynamic applications

26 of 37

Intranet assistant (RAG)

Building dynamic applications

27 of 37

Intranet assistant (RAG)

Building dynamic applications

28 of 37

Intranet assistant (RAG)

Building dynamic applications

Modules used

AI Core (ai)
AI Search (ai_search)
Milvus VDB Provider (vdb_provider_milvus)
OpenAI Provider (provider_openai)

No used

AI Assistant API (ai_assistant_api)
AI Chatbot (ai_chatbot)

Custom modules

AI chat assistant (xi_ai)
AI search (xi_ai_search)
HTML to Markdown (xi_html_markdown)
Preview Content in Markdown (xi_html_markdown_node)

29 of 37

What about fine tuning?

Overview AI

30 of 37

Finetuning

Overview AI

How

Generate specialized training set for your use case
Apply specialized training set to complete the knowledge of the LLM
Keep general information of LLM (language structure, grammar, general world knowledge) in place

Use cases

Customer support
Special text (legal, medical, code)
Writing styles tonality
product recommendation on large data sets

Benefits

Token savings due to shorter prompts
Lower latency requests
Higher quality results than prompting
Ability to train on more examples than can fit in a prompt

31 of 37

Agenda

Overview AI
Demos
Building dynamic applications
Creating useful applications

32 of 37

Identifying useful applications

Creating useful applications

FOR each step in workflow

IF ai can do it

let ai do it

ELSE IF ai can simplify it

let ai help

ELSE

do nothing, ai cannot help yet

END IF

END FOR

33 of 37

Ideation workshops

Creating useful applications

Ideation workshops with clients
Groups of ~5 people
Let them build their own applications

Write instructions
Provide data

34 of 37

Ideation workshops - Example results

Creating useful applications

Issue Board Assistant: Easily understand agile work environments where when many developers and stakeholders work together. By downloading issues and providing them as CSV context to the assistant, it could answer questions like:

What is currently being worked on?
Who is handling topic XYZ?
What is team member XYZ working on?
What is the focus of the current sprint?

Job Agent: Provide a list of job offers from various portals, allowing users to search for jobs through a chat assistant. It could answer queries such as:

What jobs are available in field XYZ?
What jobs are available in location XYZ?
I want to work part-time; what jobs are available?

Handover Assistant: Solve the challenge of handovers when key personnel go on vacation. Using pre-prompted handover documents, they created an assistant that allowed colleagues to ask questions as needed without reading everything in advance. Typical questions:

What is the client expecting during the time when you are gone?
When XYZ happenes what should I do?
Who is responsible for what at the client?

35 of 37

Questions?

Dr. Christoph Breidert

Computer Science background

20+ years in IT industry

Co-Founder 1xINTERNET

Head of R&D

SoMe / Drupal: breidert

1 of 37

2 of 37

3 of 37

4 of 37

5 of 37

6 of 37

7 of 37

8 of 37

9 of 37

10 of 37

11 of 37

12 of 37

13 of 37

14 of 37

15 of 37

16 of 37

17 of 37

18 of 37

19 of 37

20 of 37

21 of 37

22 of 37

23 of 37

24 of 37

25 of 37

26 of 37

27 of 37

28 of 37

29 of 37

30 of 37

31 of 37

32 of 37

33 of 37

34 of 37

35 of 37

36 of 37

37 of 37