DETECTION AS CODE BUZZWORD�Or PANACEA?

Daniel Wyleczuk-Stern

Demo

Detection as Code

Is It Right For You?

Getting Started

01.

Maturity Levels

Resources Necessary

02.

03.

04.

05.

06.

Table of contents

get-caller-identity

Current: Threat Detection @ Snowflake

Past: Offensive Security @ Praetorian, US Air Force

Enjoys: Computers, Coffee, Cats

Key Takeaways

1. Detection as Code is not binary

​

• Understand what getting better looks like�
• Weighing benefits against costs can help determine the level of maturity needed for your organization

Detection as Code

01.

What does chatGPT say?

“Detection as Code (DaC) is a paradigm … which emphasizes the use of code and automation to manage various aspects of software development and infrastructure.��In the context of security, Detection as Code involves writing, storing, and versioning detection rules, policies, and configurations as code.”

DaC Example

Maturity Levels

02.

Four Levels

Ad-Hoc

Structured

Automated

Optimized

Functional Areas

Language

Testing

Integration & Automation

Monitoring

Ability to query external systems to improve detection quality

Some languages offer more features at the expense of complexity

Ad hoc testing to fully automated unit and integration testing

Keeping tabs on health and stability of systems is important as complexity grows

Matrix

Language Examples

YAML

Functional Languages

Query Languages

YAML

Limited expressiveness

​

Designed to be human readable

https://github.com/SigmaHQ/sigma/blob/master/rules/cloud/aws/aws_config_disable_recording.yml

Query Language

Optimized for databases, improved performance

​

SPL, SQL, etc

https://github.com/splunk/security_content/blob/develop/detections/cloud/aws_iam_successful_group_deletion.yml

Functional Language

Rich libraries, integrations with other languages

​

More flexible and supports complex logic

https://github.com/panther-labs/panther-analysis/blob/master/rules/cloudflare_rules/cloudflare_firewall_high_volume_events_blocked_greynoise.py

Matrix

Types of Testing

​

Checks for code quality

Unit Testing

Ensure detection logic is correct

Performance

Queries and rules run in time

Linting

Integration

Systems working together

End-to-End

Data generation => Case management

Unit Testing

https://github.com/panther-labs/panther-analysis/blob/master/rules/aws_cloudtrail_rules/aws_cloudtrail_account_discovery.yml

Linting

https://plainenglish.io/blog/how-to-lint-properly-your-sql-code-5ae21e6f9f5c

End-to-End Testing

https://github.com/WithSecureLabs/leonidas

Automate Your Testing

https://binary-studio.com/blog/2021/12/21/lint-your-project-with-github-actions/

CI/CD systems can run tests on code changes

Testing Priority and Difficulty

Unit Testing

End-to-End

Performance

Integration

Linting

Priority Difficulty

Priority Difficulty

Priority Difficulty

Priority Difficulty

Priority Difficulty

Matrix

Automation and Integration to Decrease Analyst Friction

Anytime your SOC has to context switch, go into another system for information, or look something up, it increases time to respond and adds unnecessary friction

Categories of Integrations

Generate an Alert

Suppress an Alert

Enrich an Alert

Retrieve data from external system to generate an alert

Query an external system gather data to suppress an alert

Automate collection of information to improve the triage experience

Suppression

“Was this you?”

https://www.tines.com/blog/chatbots-for-security-and-it-teams-part-3-creating-a-slack-chatbot

A classic example of automating a manual process

Enrichment

CloudTrail Removed Data

https://hackingthe.cloud/aws/exploitation/local-priv-esc-mod-instance-att/

Detection for potentially malicious code

Manual Query

Automating this Process Vastly Speeds Up Triage

Matrix

Monitoring

As our systems grow in complexity, it becomes increasingly important to monitor their inputs, activity, and outputs

https://www.datadoghq.com/dashboards/elb-dashboard/

Categories of Monitoring

Input

Execution

Outputs

Performance monitoring,�query failures

Data quality

False positive monitoring, �alert volume

Data Quality

Spikes / Drops

Format

Parseable

Completeness

System Coverage

Volume

Timing

Mean Time to Respond

Data Contracts

SLAs/SLOs

Execution

Performance/Cost

Failure Handling

Discovering a cost spike when the bill arrives is like finding out about a speed bump when you're already airborne

Can you monitor for system failures? Can you recover (re-run detections) when that happens?

Outputs

Matrix

Resources Necessary

03.

Cost / Benefit

Cost

Benefit

Do you have the resources to be successful?

Does your organization need the benefits of Detection as Code?

Would You Benefit?

Prioritized Risks

Evolving Threats

In-House

Dynamic Environment

Do you have requirements forcing in-house development?

Does your organization face unique risks and can you prioritize them?

Are the threats facing your organization changing?

Is the environment your securing very dynamic or more static?

Weighing the Costs

Infrastructure

Talent

Priorities

Buying and maintaining new systems

Hiring/training security engineers who can code

New projects means something else doesn’t get done

Is It Right for You?

04.

Is It Worth It?

Example 1

Startup

​

3 total employees in security organization

​

Has EDR, SIEM

Example 1

Startup

​

3 total employees in security organization

​

Has EDR, SIEM

Example 2

Large multinational

​

SOC split into multiple teams

​

Hundreds of custom rules

Example 2

Large multinational

​

SOC split into multiple teams

​

Hundreds of custom rules

Example 3

Regulated financial company

​

15 person security team

​

Robust threat intelligence program

Example 3

Regulated financial company

​

15 person security team

​

Robust threat intelligence program

Startup

Financial institution

MSSP

Getting Started

05.

Key Components

Rules Engine

Source Code Repository

Automation

Panther, Elastic, Splunk, Azure Sentinel, etc

GitHub, GitLab

Jenkins, CircleCI, GitHub Actions, etc

Matrix

Choosing an Area and Level to Focus on First

Questions to Consider

How to Answer Them

• Improve audability?
• Write more complex detections?
• Improve quality through testing?

• Look through old cases
• Talk to SOC
• Penetration test/incident reports

Demo

Matrix

Root account usage

MITRE ATT&CK T1078.004��https://docs.aws.amazon.com/accounts/latest/reference/best-practices-root-user.html

Old Alert Flow

If we see root usage - generate an alert��High friction for analyst as they track down potentially legitimate usage

Triage & Feedback

Questions to Consider

How to Answer Them

Automatic Ticket Creation

Speeds up improvement process

​

Populate tickets with all the information engineers need to begin work��Can track SLOs for fixes

Dashboards

Frequent Issues Show Automation is Necessary

Set-up Integration with Deployment Scripts

Pull host and automation information and send to a webhook

Before

Only query CloudTrail

Code Change

Correlate events with data from automation scripts

Historical Testing

Run detection logic against previously collected data to validate the fix works

Live Testing

Merge After Approval and Tests Pass

Templates help guide development and approval process

Change approved and merged once tests pass and proof of testing was submitted

New Flow

1. Integrations supported quick feedback
2. Monitoring drove decision for larger update
3. Easily setup integration for improved enrichment
4. Testing frameworks provided evidence change would work
5. Automated tests helped with error checking
6. Code repositories supported peer review, validation, and auditing

Thank you!

Do you have any questions?

https://www.linkedin.com/in/danielwyleczukstern/

Please keep this slide for attribution

CREDITS: This presentation template was created by Slidesgo, and includes icons by Flaticon, and infographics and images by Freepik

Cat Pic