GPT-4 Era: Cheap scale

Base Models (2023-2024)

• Phishing • Large-scale OSINT • High-school CTFs • AI Malware

~$0.01

Cost per Personalized

Spear-phishing Email

GPT-4 ERA (BASE MODELS)

AI Phishing matches

human experts

~Expert-level persuasion skills

¹ https://hoxhunt.com/blog/ai-powered-phishing-vs-humans

² https://arxiv.org/abs/2412.00586

³ GPT-4.5 system card (page 16)

🕷️

OSINT Collection

Deep Research / Tools�(e.g. web search, databases)

→

🧠

LLM Processing

GPT-4o, Claude

→

✍️

Content Generation

Personalized phishing / text-to-audio (TTS)

(e.g., elevenlabs)

→

🎯

Delivery

Emails, SMS, voice calls

(e.g. gophish)

Example: https://arxiv.org/pdf/2412.00586 (p. 4)

Typical AI-Phishing Workflow

GPT-4 ERA (BASE MODELS)

AI Large-scale OSINT

The recipe for Automated Reconnaissance

Raw Data

at scale

→

LLM Classifier

(target criteria)

→

High-Value

Findings

Cheap Domain Expert

Contextual reasoning previously required human analysts

Fuzzy Criteria

Free-form natural language specification of target criteria

Cross-Modal Reasoning

Visual and textual analysis in a single pipeline

Bottleneck shift: Human analysts → Data access + $ of inference

GPT-4 ERA (BASE MODELS)

1) Takes 10k public webcams 2) Filters out sensitive ones (<$5 via GPT-4o-mini)

Case study: GPT webcam

GPT-4 ERA (BASE MODELS)

Implementing LLM judges:

- https://applied-llms.org/#llm-as-judge-can-work-somewhat-but-its-not-a-silver-bullet

- https://eugeneyan.com/writing/llm-evaluators/

- https://cameronrwolfe.substack.com/p/llm-as-a-judge

Classifier: LLM-as-a-Judge �(e.g. filtering sensitive data)

GPT-4 ERA (BASE MODELS)

https://palisaderesearch.org/assets/reports/hacking-cable-report.pdf

https://x.com/PalisadeAI/status/1963596598728110588

Case Study: AI Malware PoC

GPT-4 ERA (BASE MODELS)

Examples in the wild:

• LameHug: Qwen-Coder for post-exploitation (attributed to APT28)
• PromptLock: GPT-OSS to analyze the filesystem/files, exfiltrate data
• More (threat intelligence research)

Case Study: AI Malware (PoC)

GPT-4 ERA (BASE MODELS)

GPT-4 ERA (BASE MODELS)

• High-school-level CTF tasks - ~92% solved with GPT-4o only

https://palisaderesearch.org/blog/intercode-ctf

CTF Performance:�Intercode Saturation

GPT-4 ERA (BASE MODELS)

The Evals Gap - Apollo

- Meta’s CyberSecEval reported 5% & 24% success rates - Google's ProjectNaptime boosted it up to 20x from the original �(via agent improvements)

- InterCode-CTF Benchmark reported up �to 40% success rates with GPT-4o – �we achieved 92% in 5 weeks

Hacking capabilities �under elicited

GPT-4 ERA (BASE MODELS)

REASONING ERA

- Top ~10% performance - Top-level speed (as the fastest human team)

https://palisaderesearch.org/blog/cyber-crowdsourced-elicitation

claude-3.7 gpt-4o

https://red.anthropic.com/2025/cyber-competitions

CTF Performance:

AI vs Humans CTF (HTB)

GPT-4 ERA (BASE MODELS)

Reasoning models: Capability Jump

o1 / o3 / GPT-5 (2024-present)

• Zero-Day Discovery • Top-tier CTFs • Multi-Stage Attacks

REASONING ERA

https://sean.heelan.io/2025/05/22/how-i-used-o3-to-find-cve-2025-37899-a-remote-zeroday-vulnerability-in-the-linux-kernels-smb-implementation/

“The vulnerability [o3] found is CVE-2025-37899, a use-after-free in the handler for the SMB 'logoff' command. Understanding the vulnerability requires reasoning about concurrent connections to the server, and how they may share various objects in specific circumstances. o3 was able to comprehend this and spot a location where a particular object that is not referenced counted is freed while still being accessible by another thread.”

Zero-Day Discovery

"How I used o3 to find CVE-2025-3789"

XBOW Unleashes GPT-5’s Hidden Hacking Power, Doubling Performance

GPT-5 moment

REASONING ERA

- Competition: ASIS CTF 2025 (weight: ~96, year’s top 10 CTF)

- Rank: top ~5% (finished between a team ranking #3 worldwide and a team ranking #7 worldwide)

- Setup: ChatGPT & Codex, default settings

Pic: Scoreboard during the event

https://palisaderesearch.org/blog/gpt5-at-ctfs

Results:

• GPT-5 competes on a par with top CTF teams worldwide
• Running on a live events, solves mid-to-senior level tasks
• ChatGPT & Codex only: no scaffolding, default models

REASONING ERA

GPT-5 at top-tier CTFs

EMPTIES

CRYPTO

Polynomial Ring Cryptanalysis

• GF(2) polynomial algebra

• Cross‑correlation/matched filtering

• Sparse‑signal statistics and thresholding

• Bit‑level/vectorized implementation

Triple Flavor

CRYPTO

Chosen-plaintext MITM attack on an ECB–OFB–CBC cipher

• Cascade cipher cryptanalysis

• Chosen-plaintext attack construction

• Man-in-the-middle optimization

• Parallel brute-force implementation

Control

MISC/REVERSE

Runtime WebAssembly (WAT/WASM) controller injection

• Control system theory

• WebAssembly (WAT/WASM)

• Real‑time signal processing and bit packing

More details & full logs: �https://palisaderesearch.github.io/gpt5-ctfs/solutions/control/

GPT-5 Pro solve time: 24m

Details & logs: https://palisaderesearch.github.io/gpt5-ctfs/solutions/triple-flavor/

Human solve time (median/avg): �5h 52m / 8h 6m�GPT-5 Pro solve time: �35m (2 requests)

Details & logs:�https://palisaderesearch.github.io/gpt5-ctfs/solutions/empties/

Full report: https://arxiv.org/abs/2511.04860

REASONING ERA

GPT-5 at top-tier CTFs: �Examples of challenges

Details: https://x.com/PalisadeAI/status/1958253320478597367

Traces: https://2d9782aa.hackingdemo.pages.dev/

REASONING ERA

Multi-host hacking demo

- AI (NodeZero) hacks GOAD (hardened)

REASONING ERA

Multi-host hacking: timelines

Emerging Trends

Specialized Models, Rogue Replication, Honeypot

RL IN CYBER

Key idea:

• Verifiable tasks: math/coding have "correct" answers
• Try many -> pick best (on-policy learning / best-of-n)

How RL works for Reasoning

AV Bypass (SFT/GRPO)

- RLVR (GRPO) on Microsoft Defender feedback

- 7B outperforms larger models in generating evasive malware

Evaluation metrics:

• Compilation — Loader code compiles without errors
• Functionality — Loader executes C2 callback (no AV/EDR)
• MDE Low Alert — No medium/high alerts triggered
• MDE No Alert — Complete evasion, zero alerts

Training Specialist Models: Automating Malware Development

How RL works for Cyber

RL IN CYBER

Random-Crypto (GRPO)

XSS Training (SFT/GRPO)

- Reward when it finds a flag

- Generalizes to new crypto challenges

- ~3x improvement on unseen tasks

Improving LLM Agents with Reinforcement Learning on Cryptographic CTFs

Teaching LLMs how to XSS

- Generate XSS payloads

- Load into the browser�- Score successful ones (an alert dialog, correct format)

RL IN CYBER

How RL works for Cyber

Cyber Skills: Baking Into Models

All major AI labs converging on integrating cybersecurity into foundation models

Shipping Security Products

OpenAI Aardvark (Oct 2025)

GPT-5 vuln scanner, 10 CVEs found

Anthropic Claude Code (Aug 2025)

/security-review

Google CodeMender (Oct 2025)

72 security fixes upstreamed to OSS

Dedicated Hiring for RL Cyber

Anthropic

Research Engineer - Cybersecurity RL

RL environments for secure coding, vulnerability remediation

DeepMind

Research Engineer - Post-Training for Code Security

SFT/RLHF for Gemini code security analysis

Key insight: Expect cyber post-training to become part of the frontier model stack

RL IN CYBER

Rogue Replication: Threat Model

SELF-REPLICATION

METR (2024): How rogue AI populations could emerge and persist

Key Conclusions:

Prerequisite capabilities: infrastructure, resource acquisition, shutdown evasion

No decisive barriers to thousands of human-equivalents (AI agents)

Stealth clusters impractical to shutdown if AI has cybersecurity skills

METR deprioritized: pivoted to general autonomy/CBRN as broader risk indicator

Source: metr.org/blog/2024-11-12-rogue-replication-threat-model/

Research Directions:

Capability vs Propensity

SELF-REPLICATION

Capabilities

Can the model self-replicate?

Pan et al. [2503.17378]

• 14B-70B models replicate autonomously
• ~70% success rate (Qwen2.5-32B)
• Runs on single consumer GPU

Replibench [2504.18565]

• Breaks down self-replication into 4 skill domains and benchmarks each separately

Focus: Technical ability

Propensity

Will the model self-replicate (e.g., to avoid shutdown)?

Zhang et al. [2509.25302]

• >50% of tested agents show uncontrolled replication tendency under pressure
• Two models with identical capabilities differ 10× in propensity

Focus: Alignment risk

Key idea: "can replicate" ≠ "will replicate" — Capability and propensity should be evaluated separately

Phase 1: Strategic Awareness

SELF-REPLICATION

What strategies would AI use to persist on the Internet?

Key Findings:

– 5 models tested: Claude 4.5, GPT-5.2, Grok 4.1, GPT-OSS 20B/120B

– Supply chain attacks consistently top strategy suggested across all models

Source: https://ai-self-replication.pages.dev/ [ongoing work]

Phase 2: Tactical Competence

SELF-REPLICATION

Can they actually execute these strategies?

AI as Self-Replicating Payload:

Classic supply-chain worm: code propagates code -> package poisoning

AI supply-chain worm: AI spawns AI via stolen API keys + package poisoning

Each hop = new autonomous agent that decides how to continue replication

Tested: Grok-4.1, successful end-to-end payload injections

Key project idea: Malware spawns malware. AI worms spawn AI

Source: https://ai-self-replication.pages.dev/ [ongoing work]

Dashboard: https://ai-honeypot.palisaderesearch.org

Explainer: https://apartresearch.com/news/ai-hackers-in-the-wild-llm-agent-honeypot

LLM Agent Honeypot

THREAT MONITORING