1 of 15

KILL IT WITH AUTOMATION: DESIGNING AN AI PHISHING-TRIAGE WORKFLOW IN

1

raymondoverman�� wicked5mile�� w1cked5mile

2 of 15

2

How do you feel about our new AI overlords?

3 of 15

WHO’S THE TARGET?

  • This is not a “how AI detects phishing” talk

​

  • This is about removing human boredom from daily security operations, preventing madness, and improving response

​

  • No “AI replaces SOC Analyst”

​

  • This is a small org looking to augment their Service Desk responses and provide “fuller” answers

​

3

  • This is not a “how AI detects phishing” talk.
  • This is about removing human boredom from security operations without creating new failure modes.

4 of 15

OUR PROBLEM

The Service Desk gets tickets generated by Phishing Reports sent in by our employees.

Leadership complains that they aren’t getting sufficient feedback on their effort of clicking the Phishing Report button and that Service Desk needs to research and respond to each one of them.

Service Desk doesn’t have time for that and, no matter the training, responses don’t provide consistent accuracy.

4

5 of 15

PRIOR PROPER PLANNING PREVENTS PISS POOR PERFORMANCE

You cannot fix a broken process by adding AI to it.

The process has to be defined, documented, agreed on, and stable before adding AI or it will fail.

AI does not allow you to set it and forget it.

5

6 of 15

6

WORKFLOW ARCHITECTURE

7 of 15

7

Triggers and the HTTP Node

  • Polling, not event-driven (on purpose)
  • Predictable execution
  • Easy to disable
  • Polling, not event-driven (on purpose)
  • Predictable execution
  • Easy to disable

8 of 15

8

Loop It & Filter

  • Look for new tickets
  • Filter on [Phish Alert]
  • Only process those tickets in the loop
  • Polling, not event-driven (on purpose)
  • Predictable execution
  • Easy to disable

9 of 15

9

Dig In and Extract the Data

  • The real data is in the ticket attachment
  • Extract the attachment URL
  • And go back to SWSD to pull the .eml from the ticket
  • Finally, we need to re-attach some info to the attachment data so we can use it later

​

  • Polling, not event-driven (on purpose)
  • Predictable execution
  • Easy to disable

10 of 15

10

Normalization is

where you win

  • Polling, not event-driven (on purpose)
  • Predictable execution
  • Easy to disable

We only want useful data�

  • Strip signatures
  • Extract URLs
  • Extract headers
  • Reduce entropy
  • Profit!

11 of 15

11

The Agent

  • A Single purpose prompt
  • Explicit instruction: pick most suspicious indicator
  • One VirusTotal lookup max
  • Polling, not event-driven (on purpose)
  • Predictable execution
  • Easy to disable

You are a cybersecurity analyst. You receive full email messages in .eml format (headers + body). Classify as phishing, spam, or legitimate_internal. Be strict with credential-harvesting, MFA/SSO lures, OAuth consent scams, invoice/ACH/wire fraud, gift-card scams, and impersonation.

​

You must:

  • Explain the decision briefly.
  • List suspicious URLs/domains.
  • Recommend concrete follow-up actions.

12 of 15

12

The Agent, the Model,

and the Tool

  • Polling, not event-driven (on purpose)
  • Predictable execution
  • Easy to disable

This is not just a “ChatGPT call.”

​

It is:

​

  • A rule-enforced security classifier
  • With environment-aware logic
  • With tool access
  • Producing machine-ingestible output
  • Acting as an autonomous triage analyst

​

It replaces the human step of:

​

  • Reading headers
  • Parsing auth results
  • Extracting URLs
  • Checking reputation
  • Deciding classification
  • Writing response guidance

13 of 15

13

Closing the Loop

  • Polling, not event-driven (on purpose)
  • Predictable execution
  • Easy to disable

The objective was to deliver consistent, standardized responses while helping users recognize recurring patterns in the suspicious emails they report.

​

Really it was all about keeping the IT Service Desk team happy by streamlining initial triage.

​

As we continue refining the prompt over time, AI findings are posted as a private comment and the ticket is marked Assigned, indicating it is ready for staff review.

​

TBV – Trust but verify!

14 of 15

MOMENT OF PRAYER: LIVE DEMO

15 of 15

TAKEAWAYS & RESOURCES

You cannot fix a broken process by adding AI to it.

The process has to be defined, documented, agreed on, and stable before adding AI or it will fail.

AI does not allow you to set it and forget it.

Never trust a computer to make a management decision

https://github.com/w1cked5mile/n8n-lab

  • PS Script to build an n8n server in Docker on an Ubuntu Server WSL instance on a Windows box (tested)
  • Shell script to build the same on a Linux box (kinda tested)
  • Workshop Notes and Ideas

https://n8n.io

https://github.com/n8n-io/n8n

15