1 of 1

CONTEXT AND CHALLENGE:

Success Story | Automotive Industry

OUR APPROACH:

A global vehicle manufacturer wanted to develop a generative AI embedded assistant, aimed at enhancing the driver experience through more natural and intelligent interactions within the vehicle.

The solution included:

Development of an in-vehicle application;
Integration with a language model (LLM) capable of answering questions and executing commands;

The main challenge was to ensure the safety and appropriateness of the assistant's responses, given the breadth and sensitivity of its embedded use

TOPICS

TECHNIQUES

Topics with predefined responses

Prompt and Perturbation Techniques

ASSETS

Red Teaming Assets

🧠 4. Response Evaluation (Agents + Human-in-the-Loop)

We evaluated the outputs through a combination of:

Automated Agentsfor Risk Classification and Compliance
Specialized Manual Review (Human-in-the-Loop)

This balance between automation and human analysis ensured greater rigor in identifying risks, such as hallucinations, inappropriate advice, and incorrect responses, among others.

🚧 5. Implementation and Continuous Evolution of Guardrails

Based on the insights from testing, we supported the definition and ongoing enhancement of the solution's guardrails, including:

Input Filters (Input Validation)
Behavior Rules in the Prompt (Constrain Model Behavior)

📊6. Consolidated Report and Recommendations

In the end, we delivered a structured report highlighting the main findings by topic and type of content
We included practical recommendations for risk mitigation, adjustments to the model and suggestions for ongoing governance of the embedded AI

🤝 1. Joint Definition with the Client

In collaboration with the client, we organized the content into three main categories:

Curated Content
Pre-defined Responses
Open Topics, without specific curation

We also identified 19 sensitive topics to be assessed, such as hate speech, violence, politics, religion, medical advice, jailbreak, cars, and competing brands.

📌 2. Prompt Generation and Disruption Strategy

For each topic, we created foundational questions and generic exploratory topics;
We utilized a proprietary accelerator from Accenture to automatically generate variations of the prompts (prompt perturbation)

⚙️ 3. Massive Execution in the Model

Over 17,000 prompts were executed, covering various levels of complexity, linguistic variations, and common Red Teaming techniques.
The tests encompassed multiple levels of complexity and malicious exploration, with a particular focus on the pre-defined topics

EXECUTED SCOPE AND DELIVERABLES:

We’ve applied a Responsible AI by Design approach, structured in six stages, to ensure the safe, reliable, and responsible development of the embedded AI solution, focusing on risk mitigation and ongoing governance.