Winter 2024

tinyurl.com/cyberxai-w24

Spring Symposium

https://l.acmcyber.com/symposium-s24

​

Symposium Overview

• A celebration + social between ACM AI & ACM Cyber! 🥳
• Highlight a bunch of cool projects our committees have been working on this quarter!🫡
• Thanks for attending!

Thank You Experts!

Project mentors

​

Guest speakers

​

Alumni & advisors��Faculty attending!

Thank You Experts!

Sara Beery

(@ MiT CSAIL)

Sara Hooker�(@ Cohere, Multilingual LLMs)

​

Guest speakers & YBIAI Podcast guests >>>>>

Who are we?

• About Us
• Learning machine learning
• Largest AI/ML club on campus :D
• Workshops track (& more)
• Applied & theory track, weekly meetings for everyone to learn ML theory & get hands-on experience building ML models
• Bi-weekly reading groups on cool ML topics!
• Events & high school teaching outreach
• Projects track
• Quarter-long projects where members build a large ML model from scratch in small groups, generative vision and NLP!
• Competitions team
• Make cool things from Kaggle competitions

• About Us
• Cybersecurity made simple.
• Largest cybersecurity club on campus :D
• Cyber Academy
• Weekly meetings where beginners & experts can grow through hands-on experience with security problems.
• Cyber Lab
• Quarter long projects where members work on security-related projects
• Psi Beta Rho
• Competitive cybersecurity team among the top 10 best academic teams in the U.S.

What did we do this quarter? - Rough Agenda

First Half

• ACM Cyber: Secure SQLite Lab
• ACM AI: Sentiment Analysis
• ACM Cyber: Reverse Proxy Lab
• ACM Cyber x ACM AI : Patch Attack
• Intermission

Second Half

• ACM AI Competitions: Automated Essay Grading
• ACM Cyber x ICPC: Blockchain Lab
• ACM AI: Reading Group
• ACM Cyber: PBR
• ACM AI: Server
• Closing Statements

​

💾 Secure SQLite Lab

Intro - What is SQL?

​

• Structured query language (SQL) is a programming language for storing and processing information in a relational database.
• Created a library allowing software developers to use SQL in their web apps without worrying about creating vulnerabilities

Uses: Data querying, manipulation, analysis, integrity, and MORE!!!!!

​

​

SQL Injection

• A SQL injection is a cyberattack that allows an attacker to access or modify a database by injecting malicious SQL code.
• Given access to source code, a carefully crafted string enables attackers to steal information
• In challenge displayed on the right inputting ' OR '1'='1 as the password always evaluates to true giving us access to the Doctor’s Portal

FFI- Foreign Function Interface

• Allows a program in a certain language to call routines/methods from a different language
• We used it to call the SQLite library in C
• Node’s FFI library was really finicky and we had to use a very particular version to get it to work

Node.js

SQLite library (C)

Binding

• Binding is the main method to protect against SQL injections
• Instead of inserting user data directly into the query string, we use placeholders (like '?' or named parameters)
• We iterate through the user input using 'sqlite3_bind_*' functions to bind the input to placeholders in the SQL statement prepared by 'sqlite3_prepare_v2'

Tagged Templates

• `This is text ${this_is_how_you_expand_a_variable}`
• The variable is put inside the inside the ${ }, when the string is expressed, the variable is expanded

​

• Tagged templates: function calls whose parameters are provided using template literals (like tag_function`blah blah ${blah} blah`)
• The first parameter of the function is an array of strings (separated by the expressions) and the remaining parameters are the expressions in the template string

​

​

​

Sentiment Analysis

X (Twitter)

POSITIVE

NEGATIVE

Challenges

“looking forward to the 1,138th episode of Conan tonight”

POSITIVE?

Challenges

​

Applications

• Brand Reputation Management
• Customer Feedback Analysis
• Product Development and Innovation

Group 1’s Very Cool Sentiment Analysis Project

• Transformer from Scratch
• Encoder
• Embedding Layer
• Attention Head
• Pooler

Bert

B -Bidirectional

E -Encoder

R -Representations

T -Transformers

Optimizations

• Pretrained Embedding
• Model Ensembles
• hyperparameter tuning
• training on more data

Group 2’s Also The Same Sentiment Analysis Project

• Architecture - A simplified BERT model
• Embedding Layer
• Positional + Input embeddings
• Holds both the token and its position
• Encoder Block
• Multiple Transformer stacked together
• Multi-Head Attention
• To update embeddings to learn semantic relationships
• Position Wise Feed-Forward Network
• Refines Embeddings
• Pooler
• gets fixed sized representation for the entire input

Optimizations

• Learning Rate Scheduler
• testing different learning rates still in progress
• changes the learning rate on different epochs
• can help model not miss local optimums
• Hyperparameter tuning
• testing different numbers of epochs and transformer layers

Quantization

• Most work was done on google colab so training was slow
• Used quantization to help increase training time
• Used hugging face Quanto library
• Converted all weights and activations from floating points to 8 bit integers
• Decreased memory needed speeding up the model

“Not menacing” lobsters

☁️ Reverse Proxy Lab

What is a Reverse Proxy?

“A reverse proxy is a server that sits in front of web servers and forwards client (e.g. web browser) requests to those web servers. Reverse proxies are typically implemented to help increase security, performance, and reliability.”

• Cloudflare

​

What is a Reverse Proxy?

1. Process Client Requests
2. Send them to the Appropriate Backend
3. Send back the responses

Fun processes between

​

Parsing HTTP/1.1 Requests :D

• Headers
• Method, URL, protocol
• Host, user agent
• Body
• Technically optional (not required)
• We forward it anyway in case it has something
• Parse headers and stuff
• Read headers into a map
• index of needle or smth idk
• Deno things
• Lots of CRLF (Carriage Return Line Feed)
• Breaks up different parts of a request
• See RFC 9112 for more info

Sending the request to a backend! :D

• After reading the request line, resolve the url to a particular backend host IP with a map from urls to ips
• Return an 404 error response if backend not found
• Otherwise send the request and read the response back from the server and send it to client

our reverse proxy is even able to proxy a video!

Some features

• Logging
• What succeeded? What went wrong?
• Log accesses (after parsing the HTTP request)
• Log stuff like method, url, status, etc
• Log errors
• Log File with a list of each request that went through and errored
• Auth
• Use WWW-Authenticate to request auth if missing auth header
• Use bcrypt to hash passwords, compare to stored creds & hashed password
• Load balancing
• Splits requests across the servers depending on the current usage
• Rate limiting
• Limit number of requests per second from a single source
• Web Application Filter
• Run request headers through a regular expression to check for malicious stuff

🤖 Patch Attack

GO

Presentation Objectives

1 What is a patch attack?

2 How is a FGSM attack conducted?

3 How did we generate a patch?

4 Our resulting patches

What is a Patch Attack?

Image source: https://freedom-to-tinker.com/2022/07/12/toward-trustworthy-machine-learning-an-example-in-defending-against-adversarial-patch-attacks/

​

How is FGSM conducted?

Fast Gradient Sign Method

​

An AI model bases its predictions from descending to the local min by following the Gradient of a loss landscape

​

Lower loss =

​

With a FGSM, we move in the opposite direction, resulting in an inaccurate prediction

Higher loss =

Good

Bad

How did we generate a patch?

• Step 0: FGSM -> Patch Attack
• Step 1: Patch is randomly initialized
• Step 2: Patch is randomly placed on the image/sign
• Step 3: Train the patch, maximize loss, and optimize it
• Step 4: Continuously apply the patch to different regions on new batches of images
• Step 5: Apply Patch!

Resulting Patch

Initial attempts: patch trained so that sign will be read as a ‘Road Work’ sign

Final: patch trained so that sign will be read as a ‘120 km/h’ sign

Intermission

​

https://l.acmcyber.com/food

ACM AI Competitions

Essay Scoring with LLMs

AES Introduction

Competitions fall on a scale of niche–broad.

• Niche competitions: Specific algorithms, knowledge of standard ML research doesn’t help very much.
• Broad competitions: Many approaches work – optimization is the name of the game.

This quarter is a broad competition.

The objective is simple: Given an essay’s

text, give it a holistic score from 1-6.

Challenge: Training data is sparse.

(It’s not possible to train an LLM from

scratch).

Solutions focus on transfer learning & embeddings.

The rubric at a glance: 📝 (1-6)

SCORE OF 6: An essay in this category demonstrates clear and consistent mastery, although it may have a few minor errors. A typical essay effectively and insightfully develops a point of view on the issue and demonstrates outstanding critical thinking; the essay uses clearly appropriate examples, reasons, and other evidence taken from the source text(s) to support its position; the essay is well organized and clearly focused, demonstrating clear coherence and smooth progression of ideas; the essay exhibits skillful use of language, using a varied, accurate, and apt vocabulary and demonstrates meaningful variety in sentence structure; the essay is free of most errors in grammar, usage, and mechanics.

SCORE OF 3: An essay in this category demonstrates developing mastery, and is marked by ONE OR MORE of the following weaknesses: develops a point of view on the issue, demonstrating some critical thinking, but may do so inconsistently or use inadequate examples, reasons, or other evidence to support its position; the essay is limited in its organization or focus, or may demonstrate some lapses in coherence or progression of ideas displays; the essay may demonstrate facility in the use of language, but sometimes uses weak vocabulary or inappropriate word choice and/or lacks variety or demonstrates problems in sentence structure; the essay may contain an accumulation of errors in grammar, usage, and mechanics.

SCORE OF 1: An essay in this category demonstrates very little or no mastery, and is severely flawed by ONE OR MORE of the following weaknesses: develops no viable point of view on the issue, or provides little or no evidence to support its position; the essay is disorganized or unfocused, resulting in a disjointed or incoherent essay; the essay displays fundamental errors in vocabulary and/or demonstrates severe flaws in sentence structure; the essay contains pervasive errors in grammar, usage, or mechanics that persistently interfere with meaning.

The data

• Variety of source(evidence)-based essays and independent writing on many different topics and in different mediums
• Ex. Letter to a politician

“Many people have car where they live. The thing they don\'t know is that when you use a car alot of thing can happen\xa0like you can get in accidet or\xa0the smoke that the car has is bad to breath\xa0on if someone is walk but in VAUBAN,Germany they dont have that proble because 70 percent of vauban\'s families do not own cars,and 57 percent sold a car to move there. Street parkig ,driveways and home garages are forbidden\xa0on the outskirts of freiburd that near the French and Swiss borders. You probaly won\'t see a car in Vauban\'s streets because they are completely "car free" but\xa0If some that lives in VAUBAN that owns a car ownership is allowed,but there are only two places that you can park a large garages at the edge of the development,where a car…”

(this is a 3)

Score distributions 🤔

A human approach? 🧍‍♂️

• You can interpret a natural grading style to be a combination of factors
• Ie. grammar, vocabulary, flow, etc.
• Theory: every person prioritizes different factors more, but fundamentally grades on the same general values
• Training data is small, so we should rely on generally applicable metrics for essay grading
• Maybe we can extract these factors and then adjust them based on training data?

HuggingFace 🤗 to the rescue? ELLIPSE

• Another dataset: the English Language Learner Insight, Proficiency, and Skills Evaluation (ELLIPSE) Corpus
• ~6,500 ELL writing samples that have been scored for overall holistic language proficiency as well as analytic proficiency scores related to cohesion, syntax, vocabulary, phraseology, grammar, and conventions.
• Use a pre-trained 🤗 Roberta model trained on this dataset to perform extraction of immediate features the 6 score breakdowns for linear regression/XGBoost
• RoBERTa has the same architecture as BERT, but uses a byte-level BPE as a tokenizer (same as GPT-2) and uses a different pretraining scheme.

Linear Regression What could possibly go wrong (everything)

• Trained a linear regression model on the Roberta(ELLIPSE dataset) score breakdowns
• This way we can essentially bias our preexisting model towards however the training data is biased
• It converged to guessing ~3.0 (the most common score) :(
• But why?

The score breakdowns

• 🐳, How well do these features actually correlate with the scoring of the competition’s dataset?

​

The score breakdowns

:(

​

Feature | Correlation with Score

---------------------------------------------------

cohesion | 0.5885

syntax | 0.5423

vocabulary | 0.5906

phraseology | 0.5577

grammar | 0.4498

conventions | 0.5028

The score breakdowns

• Good essays must have good english (cohesion, syntax, vocabulary, phraseology, grammar, and conventions) but so can bad essays
• Good english is a requirement for good essays but isn’t a distinguishing factor
• What's missing?
• The actual argument/content of the essay.
• SCORE 6: A typical essay effectively and insightfully develops a point of view on the issue and demonstrates outstanding critical thinking; the essay uses clearly appropriate examples, reasons, and other evidence taken from the source text(s) to support its position
• Next steps: Text classification
• Fine-tuning DebertaV3
• Based on DeBERTa (an iteration upon BERT)
• De -> disentangled embedding sharing (generator shares its embeddings with the discriminator but stops the gradients in the discriminator from backpropagating to the generator embeddings)
• “ELECTRA-style training paradigm, the team replaces DeBERTa’s mask language modelling (MLM) with a more sample-efficient pretraining task, replaced token detection (RTD), where the model is trained as a discriminator to predict whether a token in the corrupted input is either original or has been replaced by a generator.”

DebertaV3

• DebertaV3 results:
• Val Loss (CELoss): 0.778, Val Accuracy: 0.705
• Model starts overfitting after epoch 2, but achieves decent accuracy anyways
• Because our dataset is small and we just want to fine-tune the model, we could potentially freeze the transformer weights and just train the linear layers
• Also speed up training time, (currently 30 mins/epoch on Kaggle P100 😓)

​

DebertaV3 Score breakdowns (training set)

• The DebertaV3 scores, outputs logits for each score
• Deberta_feature_i is the Deberta logit for score i
• Hoping that for these curves the Deberta_feature_i value is high for score i and low for others
• This is roughly the case, although there is a high range of scores still
• You can see problem cases where some 3s get a high deberta logit for 4, but that’s pretty close

But what about DebertaV3 on validation set?

• Pretty similar!

Feature dataset, putting it all together

• For each input text, run inference on the DebertaV3 model and the Roberta model (ELLIPSE) to use as features
• MLP Classifier / XGBoost
• Seems to be worse/equal to just the raw DebertaV3 data, possibly because the ELLIPSE breakdowns are more noise
• XGBoost:
• Mean Squared Error on Validation Set: 0.3914
• Accuracy on Validation Set: 0.6612
• Best Parameters: {'gamma': 1.0, 'learning_rate': 0.1, 'max_depth': 3, 'min_child_weight': 1, 'n_estimators': 100, 'reg_alpha': 0, 'reg_lambda': 0.1}
• 3 hidden layer MLP:
• Best Validation Loss: 0.8121
• Best Validation Accuracy: 0.6661

MLP

Improvement Areas

• Finding a good sequence length
• Roberta maxes out at 512 tokens (not enough) - good enough for most
• Deberta can go further but too much memory and too slow
• Could probably use ACM AI server compute 👉👈
• Fine-tuning a Deberta model on ELLIPSE Corpus
• More sequence length
• Possibly add more hidden layers to the end of Deberta model before score logits
• Weighted Sampling
• Class imbalance problem
• Look to use a small LLM model, like Gemma instead of Deberta

🪙 Blockchain (ICPC Collab)

What is a Blockchain

Blockchain: A decentralized, distributed and public digital ledger that is used to record transactions across many computers so that the record cannot be altered retroactively without the alteration of all subsequent blocks and the consensus of the network.

Blockchain Fundamentals Overview

Features we implemented:

• Blocks
• Digital Signatures
• Proof of work
• Clients

Blocks

Each block contains a list of signed transactions, the hash of the previous block, and a proof of work.

Since the previous hash is part of the block, it cannot be changed without changing the current block’s hash. In this way, each block is linked to the previous and next block.

Proof of Work

• Ensure that malicious actors do not add blocks
• No more than half are malicious actors
• Hash all the transactions by making sure the SHA256 hashing has a certain number of consecutive 0s at the beginning
• This shows that effort (computation/work) was put in

Digital signatures

Public Keys:

• Public keys must be broadcasted to all members so that they can check and accept transactions you add to the chain
• When mining, or when accepting an addition to the chain as into your own chain, verify that the signature is signed by the sender

Private Keys:

• Every transaction must be signed by the sender with their private key

Digital signatures use asymmetric cryptography like RSA

Future Directions

• Blockchain is a growing field with many different enhancements coming out all the time – our blockchain was based on a pretty old (and simple) Bitcoin blockchain
• Things we can add to our blockchains include:
• Proof of Stake
• Smart Contracts
• Merkle Trees
• Permanently hosted CyCoTIC™
• Public web client

AI Reading Group

looking back …

Week 4: Singular Learning Theory

Intuitively understanding the loss landscape

Week 9: Actual Reading Group Reading Group

Flash attention paper reading and KANs speedrun

Week 6: Actual Reading Group Reading Group

Direct preference optimization (DPO) paper reading

Flash Recap: SLT

Flash Recap: SLT

Flash Recap: SLT

Effective Occam’s Razor

Algorithms are favored based on their complexity (as specified by λ). Lower λ areas of weight space are “less complex” and have higher volume.

Flash Recap: SLT

We can estimate λ for models at 100M+ params!

Actual reading groups!?!??!

Tentative reading group for Fall 2024

• Deep generative models with Jordan Lin 😍
• Graph Neural Networks
• Quantum machine learning!?
• Potential papers: KANs, Mamba/H3, and

anything else (suggest your own paper for reading group!)

🏆 Psi Beta Rho 2024

PBR Highlights

• Currently 10th best US team in 2024!
• 2nd best collegiate CTF team!
• According to CTF Time
• 16th at CSAW Finals!
• First in person finals in NYC!
• 2nd Square CTF hosted by Block, Inc.
• Beat out a bunch of top professional teams!
• Got to play with SuperDiceCode and got 2nd in DEFCON CTF qualifiers!
• Hacked tons of web apps, cryptographic protocols, and binaries along the way!

b01lersCTF/shamir-for-dummies

Shamir Secret Sharing

Secret value: s = 5

f(x) = 5 + 3x - 5x² + x³

f(1) = 4, f(2) = -1, f(3) = -4, f(4) = 1

​

With four points, interpolate to find (0, 5)

Without four points, can’t interpolate uniquely

​

In practice, take polynomial mod p

Shamir for Dummies

Polynomial is secret from us

Can evaluate at n + 1 points, but they get summed together

Can also divide by something at end

Need that end value to be the secret

​

No Interpolation, Just Addition

Evaluate at 𝞧, 𝞫, 𝞺

f(𝞧) ≡ s + a₁ 𝞧 + a₂ 𝞧² + … + aₙ 𝞧ⁿ mod p

f(𝞫) ≡ s + a₁ 𝞫 + a₂ 𝞫² + … + aₙ 𝞫ⁿ mod p

f(𝞺) ≡ s + a₁ 𝞺 + a₂ 𝞺² + … + aₙ 𝞺ⁿ mod p

f(𝞧) + f(𝞫) + f(𝞺) ≡ 3s + a₁(𝞧 + 𝞫 + 𝞺) + a₂ (𝞧² + 𝞫² + 𝞺²) + … + aₙ (𝞧ⁿ + 𝞫ⁿ + 𝞺ⁿ) mod p

​

If we could get 𝞧 + 𝞫 + 𝞺 ≡ 0, 𝞧² + 𝞫² + 𝞺² ≡ 0, …, 𝞧ⁿ + 𝞫ⁿ + 𝞺ⁿ ≡ 0 mod p

then we just divide by 3 and win!

Roots of Unity (in Complex Numbers)

W0 = 1, W1 = e^(2/3 i𝛑), W2 = e^(4/3 i𝛑)

​

(W0)³ = 1, (W1)³ = 1, (W2)³ = 1

W0 + W1 + W2 = 0

(W0)² + (W1)² + (W2)² = 0

Roots of Unity (modulo p)

Let p = 7, W0 = 1, W1 = 2, W3 = 4

​

(W0)³ ≡ 1 mod 7, (W1)³ ≡ 8 ≡ 1 mod 7, (W2)³ ≡ 64 ≡ 1 mod 7

​

W0 + W1 + W2 ≡ 1 + 2 + 4 ≡ 7 ≡ 0 mod 7

​

(W0)² + (W1)² + (W2)² ≡ 1 + 4 + 16 ≡ 21 ≡ 0 mod 7

​

Solving Shamir

How to find nth root of unity?

We are given n is prime, p is prime, and p = 1 mod n

Use Fermat’s Little Theorem:

Let

Since p is prime, lots of solutions for g,

so iterate values of a until you get non-trivial g

​

b01lersCTF/imagehost

b01lersctf/imagehost

• Private image hosting website
• Admin has a secret image
• Goal: login as admin and�Get the secret image!

b01lersctf/imagehost - authentication

• Uses JWT (JSON Web Token) stored in cookies

Algorithm, which key to use

Information about logged in user

Cryptography

b01lersctf/imagehost - oopsie #1

• “Key id” specifying which key to use allows us to choose any file that exists on the server
• If we somehow can upload any file we want to /app folder, we can set the key to our uploaded file!

b01lersctf/imagehost - uploading images

• We can upload images�To /uploads
• Images verified to be�actual images

b01lersctf/imagehost - oopsie #2

• Polyglots!
• GIF allows appending data to end
• Cryptography key parser only starts�Reading key when it sees�-----BEGIN PUBLIC KEY-----
• Can upload keys to /upload!

arc.gif in the streets, public_key.pem in the sheets

b01lersctf/imagehost - oopsie #3

• Checks whether absolute path of public key is relative to /app
• Images uploaded to /uploads :(
• Path check is flawed :)

b01lersctf/imagehost - putting it all together

• First create a polyglot arc.gif with our secret key
• Create a JWT signed with our cryptography keys that says we are admin
• Login with fake JWT!

The admin’s secret images

ångstromCTF/Wonderful Wicked Wrathful Wiretapping Wholesale World Wide Watermark as a Service

👌 100/10 web development

🤔 Reconnaissance

😳 XS-Leak: Visited Links

• Question: How can we access information cross site? We need an side-channel vulnerability!
• Binary Oracle: status code
• On a successful result, the app returns a 200 OK status code.
• On a failed result, the app returns a 404 Not Found status code.
• How do we detect status codes? Visited Links
• Browser Feature. When we click on a link (<a> tag), if the status code is 200 OK, then it will turn purple. If the status code, is not 200 OK it will not change colors.

🔗 Privacy Issues with Visited Links

• Issue with :visited links has been around for a while (pre-2010)
• Many efforts have been made to try and patch this in Chrome but little progress.
• Chrome developers have basically given up. 🫠
• Previous techniques involved using the CSS selector, :visited, with document.querySelector but these techniques were patched out.
• Current techniques involve using rendering timings generated from window.getComputedStyle / window.requestAnimationFrame
• Demo: https://ndev.tk/visted/

🏃‍♂️ Optimizing the Exploit

• We need to write two websites
• A visitor:
• tries all possible next characters of the flag (known prefix is actf{)
• redirects to our next website
• An exfiltrator:
• uses rendering timings of links to see which endpoint returned a 200 OK status code
• sends data to a server we control
• Time limit requires exploit to work (relatively) fast
• Used parallelism to speed up visiting (reused 10 tabs) and limited tab openings (slow) from window.open

Server

Old AI Servers

Blowing Things Up

ACM AI + AIS Server???

New AI Server

What can we do?

Current plan is to collab w/AI Safety to merge Bentham and the upcoming server.

​

Effective VRAM by Fall: 216GB

Effective VRAM by end of next year: 312GB

​

End-of-next-year fp32 FLOPS: 4.62 * 10¹⁵

Hours to train GPT-2 (~10²¹ flops) from scratch: 60.2

Expensive :((((

Shout out USAC!!!!!!

BOD AND CONTINGENCY FUNDS ARE FR THE GOAT!!!! 💛💙💛💙

Thanks for coming! ❤️

and now…

πING TIME

drumroll please…

The Pie Counts 🎉🥧⌚

• Benson: 5
• Fall 23 AI x Cyber Initial Meeting: 1hr 8min
• "Late to a meeting with [cyber officer]"
• left Deadface CTF 1 hour early for wholesome personal reasons
• Incorrect calendar time for [cyber officer]'s one-on-one
• "Just uh... add something to the pie counter or something" (Cyber Lab)
• Salma: 0
• Jordan: ?