SWE Tea

SWE tea is a weekly-ish group that meets to discuss software engineering and computer science papers.

Format

We hang out on Clubhouse and walk through the paper, but the discussion is informal. The meetups are unrecorded, to encourage sharing related stories from our own personal experience, and so we can feel free to explore ideas and be wrong. Think programmer buddies meeting up at a bar.

We are currently aiming for 3 meetups every 2 weeks. We prefer to reschedule meetups and not to cancel, so there may be more meetups in the weeks following those with breaks. Scheduling is dynamic and happens asynchronously over whatsapp or synchronously when we meet. However, there is a

Shared Google Calendar

Paper Selection

We are less interested in the cutting edge and prefer to sample the best papers from a variety of topics across software engineering and computer science. Topics range from high performance computing, big tech infrastructure, data centers, networks, operating systems, storage, machine learning, and algorithms.

We try to select papers in themed batches of 3-6 papers. Themes help us follow the development of ideas across time and amortize the cost of background familiarization. Batches should be small to maintain variety. Themes should only be revisited after a few intervening batches.

These papers should be peer reviewed to ensure quality. In our experience, papers only published on research.facebook.com or arxiv are of inconsistent quality. Good sources of papers include “best paper awards” and “test of time awards” from Top Publication Venues in Computer Science.

Next topics

RDMA

• Meetup 066 - Using One-Sided RDMA Reads to Build a Fast, CPU-Efficient Key-Value Store
• Meetup 067 - Using RDMA efficiently for key-value services
• Meetup 068 - Efficient Memory Disaggregation with Infiniswap

Far Memory

• Meetup 069 - FaRM: Fast Remote Memory
• Meetup 070 - AIFM: High-Performance, Application-Integrated Far Memory
• Meetup 071 - Software-Defined Far Memory in Warehouse-Scale Computers

Huge Pages et al   

• Meetup 072 - Practical, Transparent Operating System Support for Superpages
• Meetup 073 - Making Huge Pages Actually Useful 
• Meetup 074 - Coordinated and Efficient Huge Page Management with Ingens 

Past topics

2025 May

• Meetup 065 - SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training

2025 April

• Meetup 064 - DeepSeekMath - Group Relative Policy Optimization (GRPO)
• Meetup 063 - Proximal Policy Optimization Algorithms (PPO, 2017)
• Meetup 062 - High-Dimensional Control Using Generalized Advantage Estimation (2015)
• Meetup 061 - Trust Region Policy Optimization (TRPO, 2015)

2025 March

• Meetup 060 - Simple statistical gradient-following algorithms for connectionist RL (1992)
• Meetup 059 - Sample Efficient Reinforcement Learning with REINFORCE (2010)
• Meetup 058 - FastText.zip: Compressing text classification models
• Meetup 057 - Bag of Tricks for Efficient Text Classification
• Meetup 056 - Enriching Word Vectors with Subword Information
• Meetup 055 - Detection and Prevention of Silent Data Corruption in an Exabyte-scale Database 
• Meetup 054 - F1: A Distributed SQL Database That Scales 

2025 February

• Meetup 053 - Spanner: Becoming a SQL System 
• Meetup 052 - Spanner: Google’s Globally-Distributed Database 

• And (short article) Spanner, TrueTime & The CAP Theorem 

• Meetup 051 - Practical uses of synchronized clocks in distributed systems (1991)

2025 January

• Meetup 050 - FOCUS – THE ASML WAY 
• Meetup 049 - Fay: Extensible Distributed Tracing from Kernels to Clusters 
• Meetup 048 - Fathom: Understanding Datacenter Application Network Performance
• Meetup 047 - Dhalion: Self-Regulating Stream Processing in Heron 
• Meetup 046 - Profiling a warehouse-scale computer

2024 December

• Meetup 045 - Dapper, a Large-Scale Distributed Systems Tracing Infrastructure
• Meetup 044 - Google-Wide Profiling: A Continuous Profiling Infrastructure for Data Centers (2010)
• Meetup 043 - Beyond Fat-Trees without Antennae, Mirrors, and Disco-Balls 
• Meetup 042 - A Large Scale Study of Data Center Network Reliability (2018)

2024 November

• Meetup 041 - Fast Packet Processing with eBPF and XDP (2020)
• Meetup 040 - Running BGP in Data Centers at Scale - Meta Research
• Meetup 039 - FBOSS: Building Switch Software at Scale - Meta Research
• Meetup 038 - Jupiter Evolving: Transforming Google's Datacenter Network via Optical Circuit Switches and Software-Defined Networking 
• Meetup 037 - Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google’s Datacenter Network

2024 October

• Meetup 036 - TAO: Facebook’s Distributed Data Store for the Social Graph
• Meetup 035 - ClickHouse - Lightning Fast Analytics for Everyone
• Meetup 034 - Gorilla: A Fast, Scalable, In-Memory Time Series Database
• Meetup 033 - Linearizable State Machine Replication of State-Based CRDTs without Logs
• Meetup 032 - Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective 
• Meetup 031 - Borg Autopilot

2024 September

• Meetup 030 - Borg the Next Generation
• Meetup 029 - Large-scale cluster management at Google with Borg
• Meetup 028 - Remus: High Availability via Asynchronous Virtual Machine Replication (2015)
• Meetup 027 - Twine: A Unified Cluster Management System for Shared Infrastructure
• Meetup 026 - Mastering Chess and Shogi by Self-Play with a General RL Algorithm (2017)

2024 August

• Meetup 025 - Mastering the Game of Go with Deep Neural Networks and Tree Search (2016) 
• Meetup 024 - Towards Modern Development of Cloud Application (2023)
• Meetup 023

• Bigtable: A Distributed Storage System for Structured Data
• RocksDB: Evolution of Development Priorities continued, section 4+
• Rescheduled date: August 11, 1pm EST

• Meetup 022

• RocksDB: Evolution of Development Priorities in a Kv Store Serving Large-scale
• Sections 1 to 3
• Moved to Friday, Aug 2nd @ 5pm.

2024 July

• Meetup 021 - Scuba: Diving into Data at Facebook
• Meetup 020 - ThinLTO: Scalable and incremental LTO 
• Meetup 019 - Build systems a la carte (2024-07-10)
• Meetup 018 - CRUSH: Controlled, Scalable, Decentralized Placement of Replicated Data

2024 June

• Meetup 017 - Double ratchet and Sesame
• Meetup 016 - Ceph: A Scalable, High-Performance Distributed File System | USENIX 
• Meetup 015 - Snap: a Microkernel Approach to Host Networking - (2024-06-03)

2024 May

• Meetup 014 - What is Asymmetric Numeral Systems? (May 31st, 2024)
• Meetup 013 - LZ4 compression algorithm on FPGA
• Meetup 012 - Dynamo: Facebook’s Data Center-Wide Power Management System

2024 April

• Meetup 011 - Maglev: A Fast and Reliable Software Network Load Balancer 
• Meetup 010 - Dynamic Instrumentation of Production Systems | DTrace 
• Meetup 009 - Understanding the Mirai Botnet - April 3rd, 2024

2024 March

• Inside the infamous Mirai IoT Botnet: A Retrospective Analysis 

• Meetup 008 - Xen and the art of virtualization (2003), March 27th, 2024

• Quick slides: https://www.cl.cam.ac.uk/research/srg/netos/papers/2004-xen-ols.pdf
• (optional) MirageOS: https://queue.acm.org/detail.cfm?id=2566628
• This paper is a seminal paper in virtualization technology, pointing out the need for software TLB’s, which did not exist in x86 at the time.

• Meetup 007 - Vector similarity search, March 13th, 2024

• Accelerating Large-Scale Inference with Anisotropic Vector Quantization (2019)

2024 February

• Meetup 006 - Facebook’s Tectonic Filesystem: Efficiency from Exascale (2021), Feb 29th, 2024
• Meetup 005 - Percolator, Feb 27th, 2024

• Paper: Large-scale Incremental Processing Using Distributed Transactions and Notifications (2010)

• Meetup 004 - KLL Optimal Quantile Approximation in Streams (2016), Feb 21th, 2024
• Meetup 003 - Operating System Evolution, Feb 7th, 2024

• Perspectives on OS foundations | SOSP History Day 2015 (talk)
• The rise of cloud computing systems | SOSP History Day 2015 (talk)
• Fifty Years of Operating Systems (2016) (short article)
• Parallel computing and the OS | SOSP History Day 2015 

2024 January

• Meetup 002 - Meltdown and Specter, Jan 31st 2024

• Project Zero: Reading privileged memory with a side-channel (2018) (Blog post)
• Spectre Attacks: Exploiting Speculative Execution
• https://meltdownattack.com/meltdown
• The Intel 80x86 Processor Architecture: Pitfalls for Secure Systems (1995)

• Meetup 001 - on Huge Pages, Jan 24th, 2024

• Paper: Adaptive Hugepage Subrelease for Non-moving Memory Allocators in Warehouse-Scale Computers (2021)
• Addon: Beyond malloc efficiency to fleet efficiency: a hugepage-aware memory allocator
• Addon: Unleashing the Power of Allocator-Aware Software Infrastructure
• We’re switching it up a little this week, with one main paper and two other potential papers.

• Meetup 000 - on Slicer, Jan 17th, 2024

• Paper: Slicer: Auto-Sharding for Datacenter Applications (2016)
• Summary: “Slicer: Auto-Sharding for Datacenter Applications” – Aleksey Charapko

Potential topics

Chip-scale Computer Architecture

• The WaveScalar Architecture
• TPUs

• In-Datacenter Performance Analysis of a Tensor Processing Unit  (2017)
• The Design Process for Google's Training Chips: TPUv2 and TPUv3 (2021)
• Ten Lessons From Three Generations Shaped Google’s TPUv4i (2021)
• TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings (2023)

• FACL: A Flexible and High-Performance ACL engine on FPGA-based SmartNIC (2022)
• Putting out the hardware dumpster fire (2023)

Warehouse-scale Computer Architecture

• Research and Technologies for next-generation high-temperature data centers – State-of-the-arts and future perspectives - ScienceDirect (2023)
• Optimizing Resource Allocation in Hyperscale Datacenters: Scalability, Usability, and Experiences  (2024)
• Power oversubscription

• Expanding Datacenter Capacity with DVFS Boosting: A safe and scalable deployment experience 
• Thunderbolt: Throughput-Optimized, QoS-Aware Power Capping at Scale (2020)
• Data Center Power Oversubscription with a Medium Voltage Power Plane and Priority-Aware Capping (2020)
• Prediction-Based Power Oversubscription in Cloud Platforms | USENIX 

Serverless

• What Serverless Computing Is and Should Become (Article 2021) 
• Cloud Programming Simplified: A Berkeley View on Serverless Computing (2021)
• Serverless Computing: A Survey of Opportunities, Challenges, and Applications (2021)
• Sabre: Hardware-Accelerated Snapshot Compression for Serverless MicroVMs (2024)

Networks

• The Click Modular Router (1999)
• Datacenter Traffic Control: Understanding Techniques and Tradeoffs 
• Jellyfish: Networking Data Centers Randomly
• WireGuard: Next Generation Kernel Network Tunnel
• Onix: a distributed control platform for large-scale production networks 
• BwE: Flexible, Hierarchical Bandwidth Allocation for WAN Distributed Computing (2015) 
• [2210.00714] It's Time to Replace TCP in the Datacenter 

Encoding

• Binary formats: Flatbuffers/Protobufs/Captnproto
• Cornflakes

File/block store

• Oxide 445 - Crucible Upstairs Backpressure / RFD / Oxide

• RFD 60 Storage Architecture Considerations

• What's the Story in EBS Glory: Evolutions and Lessons in Building Cloud Block Store | USENIX 

• IPFS - Content Addressed, Versioned, P2P File System 
• The Zettabyte File System 
• Building and operating a pretty big storage system called S3
• A Study of Linux File System Evolution | USENIX 

Databases

• Comparisons

• An Empirical Evaluation of In-Memory Multi-Version Concurrency Control 
• Constructing and Analyzing the LSM Compaction Design Space 

• Write-Ahead Logs (WALs)

• ARIES: A Transaction Recovery Method Supporting Fine-Granularity Locking and Partial Rollbacks Using Write-Ahead Logging (1992) 
• The design and implementation of a log-structured file system

• Specialized databases

• Monarch: Google's Planet-Scale In-Memory Time Series Database
• CliqueMap: Productionizing an RMA-Based Distributed Caching System 
• Druid: A Real-time Analytical Data Store at Scale
• Online, Asynchronous Schema Change in F1 

• Spanner-like

• FoundationDB: A Distributed Unbundled Transactional Key Value Store 
• CockroachDB: The Resilient Geo-Distributed SQL Database 

• Learned

• SageDB: A Learned Database System
• Learned Indexes for a Google-scale Disk-based Database 

• Querying Planning

• BPF-OF Storage Function Pushdown over the network 
• Distributed join algorithms on thousands of cores (2017)
• How to Architect a Query Compiler, Revisited (2018) 

• Media

• Project Silica: Towards Sustainable Cloud Archival Storage in Glass 
• The five-minute rule twenty years later, and how flash memory changes the rules (2007) 
• Kangaroo: Caching Billions of Tiny Objects on Flash 

• General SQL

• Shark: SQL and Rich Analytics at Scale
• The Snowflake Elastic Data Warehouse
• Building an Elastic Query Engine on Disaggregated Storage

• Key Value

• FoundationDB
• Dynamo: amazon's highly available key-value store 
• How we built a general purpose key value store for Facebook with ZippyDB
• SplinterDB: A General-Purpose Key-Value Store for Fast Storage
• Using Lightweight Formal Methods to Validate a Key-Value Storage Node in Amazon S3
• ROLEX: A Scalable RDMA-oriented Learned Key-Value Store for Disaggregated Memory Systems

• Embeddable databases

• DuckDB: an Embeddable Analytical Database
• SQLite: Past, Present, and Future 

• Succinct: Enabling Queries on Compressed Data
• Are you sure you want to use MMAP in your Database Management System?  
• RADOS
• C-Store: A Column-oriented DBMS
• DBSP: Automatic Incremental View Maintenance for Rich Query Languages
• AWS re:Invent 2024 - Deep dive into Amazon Aurora DSQL and its architecture (DAT427-NEW)
• https://www.youtube.com/watch?v=huGmR_mi5dQ 

Operating Systems

• An analysis of decay-usage scheduling in multiprocessors

• kvm: the Linux Virtual Machine Monitor (2007)
• BPF for storage: an exokernel-inspired approach (2021)

• Hubris / Oxide

• Blog post - Hubris and Humility
• Talk - On Hubris and Humility: developing an OS for robustness in Rust :: Open Source Firmware Conference 2021 :: pretalx

• 2017 swampUP Sessions | Distroless Docker: Containerizing Apps, not VMs - Matthew Moore
• Hypervisor-Enforced Kernel Integrity (Heki) for KVM - Mickaël Salaün, Mr Madhavan Venkataraman
• Linux sched-nice-design.txt 
• Everything You Always Wanted to Know About Synchronization but Were Afraid to Ask        
• Debugging in the (very) large: ten years of implementation and experience (Windows PCs)
• Non-linux kernels

• The multikernel: a new OS architecture for scalable multicore systems 
• L4 Microkernels: The Lessons from 20 Years of Research and Deployment 
• SkyBridge: Fast and Secure Inter-Process Communication for Microkernels 
• Understanding Fuchsia Security 

Memory Nonsense

• Survey

• Dynamic Storage Allocation: A Survey and Critical Review         

• Far Memory

• Software-Defined Far Memory in Warehouse-Scale Computers (2019) 
• AIFM: High-Performance, Application-Integrated Far Memory (2020)  
• Can Far Memory Improve Job Throughput? (2020) 
• MemLiner: Lining up Tracing and Application for a Far-Memory-Friendly Runtime (2022)
• A Case Against CXL Memory Pooling (2023)
• SWARM: Replicating Shared Disaggregated-Memory Data in No Time (2024) 

• Compressed Memory

• The Case for Compressed Caching in Virtual Memory Systems (1999)
• Ariadne: A Hotness-Aware and Size-Adaptive Compressed Swap Technique for Fast Application Relaunch and Reduced CPU Usage on Mobile Devices (2025) 

• Non volatile Memory

• NV-Heaps: making persistent objects fast and safe with non-volatile memories (2011)
• Poseidon: Safe, Fast and Scalable Persistent Memory Allocator

• An allocator for non-volatile RAM

• Memory Safety

• The CHERI capability model: Revisiting RISC in an age of risk (2014)
• TikTag: Breaking ARM's Memory Tagging Extension w/ Speculative Execution (2024)

• Armv8.5-A Memory Tagging Extension White Paper (prereq)

• Exploiting Android’s Hardened Memory Allocator

• Sanitizer

• AddressSanitizer: A Fast Address Sanity Checker
• ThreadSanitizer: data race detection in practice
• MemorySanitizer: fast detector of uninitialized memory use in C++ 

• Garbage collection

• A Unified Theory of Garbage Collection (2004) 
• Myths and Realities: The Performance Impact of Garbage Collection (2004)
• Garbage-First Garbage Collection (2004)
• Low-Latency, High-Throughput Garbage Collection
• Shenandoah: An open-source concurrent compacting garbage collector for OpenJDK

• Mallocs

• malloc Implementation for FreeBSD
• Mimalloc: Free List Sharding in Action
• JeMalloc: A Scalable Concurrent malloc(3) Implementation for FreeBSD 
• Learning-based Memory Allocation for C++ Server Workloads

Software Manufacturing (compile/link/build)

• swift/docs/ABI/CallingConvention.rst at main
• Mold Design Document: mold/docs/design.md · rui314/mold · GitHub
• All About Linkers (2017) - Linker overview
• 2016 EuroLLVM Developers' Meeting: R. Ueyama "New LLD linker for ELF" (optional) - LLD talk for LLVM
• Linkers

• Lightweight Feedback-Directed Cross-Module Optimization 
• LWN - 20 part linker essay

• Incremental Computation

Compilers

• Big list of papers: https://github.com/aalhour/awesome-compilers?tab=readme-ov-file 
• Halide: Decoupling Algorithms from Schedules for High-Performance Image Processing
• An Approach to a Compiler-Compiler
• Haskell

• The Glasgow Haskell Compiler
• Secrets of the Glasgow Haskell Compiler inliner
• How to make a fast curry: push/enter vs eval/apply

IRs and assembly

• The Case for the Reduced Instruction Set Computer (1980) 
• ISSCC2018 - 50 Years of Computer Architecture:From Mainframe CPUs to Neural-Network TPUs 

• LLVM/IR, WaSM, Java Bytecode, python bytecode, interpreters, garbage collection
• How exactly does feedback directed optimization (FDO) and profile guided optimization (PGO) work? Are those identical? We may want a survey or intro paper to this
• Lightweight Feedback-Directed Cross-Module Optimization        
• AsmDB: Understanding and Mitigating Front-End Stalls in Warehouse-Scale Computers 
• Machine Learning in Compiler Optimization (IEEE 2018) 
• The Correctness-Security Gap in Compiler Optimization

Differential Privacy

• The Staircase Mechanism in Differential Privacy 

Multi-party computation and homomorphic encryption

• Foundation

• 1992 - How to play ANY mental game (MPC turing machine)
• Somewhat Practical Fully Homomorphic Encryption ? 

• Arithmetic

• Gentry 2010 - Fully Homomorphic Encryption Using Ideal Lattices 
• BGV 2011 - Fully Homomorphic Encryption without Bootstrapping 
• BFV 2012 - Somewhat Practical Fully Homomorphic Encryption 
• CKKV 2016 -  Homomorphic Encryption for Arithmetic of Approximate Numbers

• Boolean Circuits

• 003 TFHE Deep Dive w/ Ilaria Chillotti
• TFHE: Fast Fully Homomorphic Encryption over the Torus? 

• Functions

• Pika: Secure Computation using Function Secret Sharing over Rings 
• Faster Homomorphic Discrete Fourier Transforms and Improved FHE Bootstrapping 

• Applications

• Protecting accounts from credential stuffing with password breach alerting 
• Scalable Private Search with Wally 
• Curl: Private LLMs through Wavelet-Encoded Look-Up Tables

Security

• The First 30 Years of Cryptographic Hash Functions and the NIST SHA-3 Competition
• SoK: Make JIT-Spray Great Again
• Site Isolation: Process Separation for Web Sites within the Browser (2019)
• Twenty years later: Evaluating the Adoption of Control Flow Integrity (2024)

Safety

• spam: The Underground on 140 Characters Or Less (2010)
• Suspended Accounts in Retrospect: An Analysis of Twitter Spam (2011) 

Games / Graphics

• Tomorrow Corporation Tech Demo – Hot reloading code in games
• A Deep Dive into Nanite Virtualized Geometry 
• Vectorized Production Path Tracing

Algorithms

• T-digests and ReqSketch (more approximate quantiles)

• https://arxiv.org/pdf/1902.04023.pdf
• https://arxiv.org/pdf/2102.09299v2.pdf

• Space/time trade-offs in hash coding with allowable errors (1970)
• Summary Cache: A Scalable Wide-Area Web Cache Sharing Protocol (1998)
• Everything you always wanted to know about multicore graph processing but were afraid to ask (usenix best paper ‘17)
• VP8, an open source video codec for the web
• Simple Linear Works Suffix Array Construction
• Five Color Theorem
• Graph Isomorphism in Quasipolynomial Time

• TODO: we need warm up papers to get our graph theory up to par

Machine Learning

• Google Vizier: A Service for Black-Box Optimization 
• Alpa: A System for Training Large Machine Learning Models Efficiently on Distributed Systems
• Unified Declarative Optimization for Machine Learning

Transformer architecture

• LoRA: Low-Rank Adaptation of Large Language Models 
• From Sparse to Soft Mixtures of Experts 
• Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts 

Shadow Realm

These are potential topics that we decided to neither prioritize in the short term, nor to outright delete

• More details about the 2021-10-4 outage - Engineering at Meta
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Kubernetes: The Documentary [PART 1]
• How (and How Not) to Write a Good Systems Paper
• Roblox’s cloud-native catastrophe: A post mortem | InfoWorld
• AWS Post-Event Summaries
• Weaving Relations for Cache Performance 
• Chris's Wiki :: blog/unix/SbrkVersusMmap 

• Prospec Annealing
• Borg Omega and Kuberntes