AI Red Teaming Initiative

September 18, 2024

https://docs.google.com/presentation/d/1ORsLrdvOGWqRyb7Vk5a9oU9dmwXd99xSw2jzwaRj2SM/edit#slide=id.g312bd4908d5_0_6

Agenda (11.7.24)

1. Call To Order
2. Agenda bashing
3. Quick Reports(None)
4. Unfinished business (None)
5. New Business
6. Main Section
1. Document - state of the union
2. Collaboration with CSA/Synack
7. Good of the Order
• Open forum
• Informal comments/Feedback
• Reminders & Information sharing

Collaboration with CSA/Synack (Weida/CSA, Singapore & Charlie/Synack)

1. Red Teaming guide (current)
1. There is an opportunity to contribute to and validate the current guide with more red teamers. Charlie, for example, has a bench the team can leverage.
2. Testing guide for the T10
• The current draft testing guides from Weida/CSA & Charlie/Synack shared could provide a good supporting asset relatively quickly with practical steps and examples for testing the Top 10; though not a pure red teaming asset, it could be a good resource to align red teams with CISOs.
• There is a potential quick win to have a testing guide for the top 10, though arguably more pen testing, many execs don't know the diff.
3. Use case/scenario-based guides
• There is still a need for use case/scenario-based red team testing.
1. Interestingly I was thinking of a similar structure
• A decent sized Red Teaming Guide and separate documents that will elaborate more details.
2. So, Scott’s idea of 3 guides would make sense and would fit in the current initiative.
3. I was also thinking of a set of webinars for T10 Red Teaming - a guide and then webinars would be excellent
4. Interestingly I also have a 2hr Lab session proposal out for RSA 2025 titled “Generative AI Red Teaming CTF for OWASP Top 10 LLM v2” - very similar to the Testing Guide.
5. [Scott 11.6.24] The work from from the CSA is very detailed, with prompt and code examples. He has to get clearance to share

Agenda (10.31.24)

• Call To Order
• Agenda bashing
• Quick Reports(None)
• Unfinished business (None)
• New Business
• Main Section
• Can’t write linearly (Till now)
• Iterative - velocity of change
1. From 1000% to 20-30%
• I think we have enough material for an MVP1 release. Do you agree ?
• If not, what are the gaps ?
• Good of the Order
• Open forum
• Informal comments/Feedback
• Reminders & Information sharing

Agenda

• Call To Order
• Agenda bashing
• Quick Reports
• Unfinished business
• Meet weekly ?
• New Business
• Main Section
• Timeline
• Pragmas
• Walkthrough current outline
• Good of the Order
• Open forum
• Informal comments/Feedback
• Reminders & Information sharing

Red Teaming Project Deliverable Roadmap

[11.7.24] Added to Scott’s “Top 10 for LLM and Gen AI Project Roadmap 2024-2025” worksheet

A Timeline

09.18.2024

Start

10.31.2024

MVP - Decent Outline

11.13.2024

Public Review

12.11.2024

Public Review

02.19.2025

Publication

01.22.2025

Fatal Flaw Release

Do we want to deliver v1 this year ? And move on to v2 ? with a 3 month cadence ?

My only major piece of advice is keep the first version tight, and focused. These things can spin out of control if not tightly managed. As a group, create a schedule and really reduce the scope as much as possible. One you get something out, you can always expand. Just don't let it turn into a runaway project that is super late to market. There are a lot of other people targeting work on this area, so getting something out sooner is better than taking forever.

Timeline (11.15.24)

09.18.2024

Start

11.14.2024

MVP - Decent Outline & Content

11.27.2024

Internal Review & Additions

Deadline for content additions from the team

1.31.2025

Public Alpha Review - End

03.12.2025

V1 Publication

02.26.2025

V1 Fatal Flaw Release

12.11.2024

Final Content

Fatal Flaw Release

12.20.2024

Public Alpha Release

My only major piece of advice is keep the first version tight, and focused. These things can spin out of control if not tightly managed. As a group, create a schedule and really reduce the scope as much as possible. One you get something out, you can always expand. Just don't let it turn into a runaway project that is super late to market. There are a lot of other people targeting work on this area, so getting something out sooner is better than taking forever - Steve

Incorporate feedback

Internal Review

Final Edits,

Tech writing

Deliverable Plan

Pragmas

• MVP #0
• Define/Describe GAI Red Teaming and the context around it
• MVP #1
• Guidelines
• Methodology
• Best Practices
• MVP #2
• LLM Evaluation metrics,
• Benchmarks,
• Datasets,
• Frameworks, Tools and
• Prompt banks (as applicable)
• MVP ? - LLM T10v2 CTF (ads)

Priorities:

1. Keep the outline as a guide
2. Contribute materials - separate Google Doc is fine. I will merge them
3. Don’t worry about wordsmithing, overlap, relevance et al.
1. Just write whatever you think is needed

Useful Slide Templates

Join us for a full weekend of activities!

Thursday

Spa at 3pm

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

Friday

Golf at 7am

Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.

Sunday

Brunch at Noon

Lorem ipsum dolor sit amet, consectetur adipiscing elit

Test

Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur

Wendy Writer

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor

Ronny Reader

Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat

Abby Author

Berry Books

Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum