Section 1: Intro to Labs 0 + 1

CSE 452 Spring 2024

​

Announcements

• Student repos should be available at dslabs-[your CSE NETID], reach out to us if yours is missing
• Lab 1, Lab 1 design doc, Pset 1 due next Wednesday (4/3) [done individually]
• Review syllabus for late day policy
• 48 hr grace period for lab 1
• 48 hr grace period for Pset 1
• NO grace period for design doc
• Please fill out the CSE 452 partner / W credit form by Sunday 5pm (3/31)

Labs (in terms of difficulty from what students have told us in the past)

Difficulty

Lab 1

Lab 2

Lab 3

Lab 4 p1

Lab 4 p2

Lab 4 p3

Time for labs [in hours]

(fairly rough) statistics from Spring 2021

Please report times as a single number (float or int) after the colon. Stats are self-reported, math is hard sometimes and people flipped unions and totals or had unions greater than their total time?

*= some people only did up to part 2

Time with design docs [in hours]

(fairly rough) statistics from Winter 2023

*Design docs were due a week before the lab submission.

​

Doing the design doc in advance of lab 2 saved on average ~6.5 hours of work! Qualitative feedback on design doc usefulness was also very positive.

This quarter, we streamlined the design docs to save student time and still provide the same benefit.

​

Lab Framework Introduction

Framework

• Collection of abstract classes and interfaces that you will extend/implement
• Read the documentation!
• In /doc/dslabs/framework directory
• Read the spec!

Node

• Abstract class that you will subclass
• Basic unit of computation (aka one machine)
• Notable methods:
• set(Timer timer, int timerLengthMillis)
• send(Message message, Address to)
• message and timer handlers (naming is important!)
• Automatically triggers when an XXX message is received/XXX timer goes off
• handleXXX where XXX is name of message (e.g. handlePongReply)
• onXXX where XXX is name of timer (e.g. onPingTimer)
• This is called reflection
• Handlers should be
• Deterministic
• Idempotent whenever possible

Idempotency and Determinism

Determinism: Outcome is only a function of state and message/timer handlers

• This means no timestamps, no random numbers, no UUIDs

Idempotent: When a unique message comes, the actions (e.g. changing state or creating timers) taken by it are only applied once and won’t be applied again for duplicate messages

• Problems come up usually when people create new timers for a duplicate message, which expands the state space and makes exploration towards a goal harder

Types of Nodes:

Client

• interface that client Nodes should implement
• sendCommand(), hasResult(), getResult()
• Called by testing framework

Types of Nodes:

Server

• No interface for server nodes
• Will generally call execute() on an Application

Application

• Where the client wants its RPC (command) to get executed
• Could be anything! Key-Value store, shopping cart, game, bank account information…
• Data structures to store data
• Logic for doing application-specific tasks
• Processes Commands and produces Results
• Each application will define its own set of Commands and Results
• KVStore example: GETS/PUTS/APPENDS
• Interface
• Result execute(Command c)

Message

• send(Message message, Address to)
• Encapsulates data passed between Nodes
• Messages can contain anything (like metadata, Commands, or Results)
• Messages have no methods, but extends Serializable
• Messages are serialized/deserialized for you
• Objects are serialized (copied into a packet) when sent and deserialized (copied out of a packet) when received

Timer

• set(Timer timer, int timerLengthMillis)
• Triggers the timer handler when the timer fires.
• If you have a timer called PingTimer, onPingTimer will be called after the timer has been set and the set duration has passed.
• Will not be automatically reset! Need to set these manually in node.
• Can contain anything, similar to Messages.

Synchronize

• You will see many methods that are synchronized, ex:
• public synchronized Result getResult();
• Reason: Only one thread can execute a synchronized method on an object at a time (all other threads executing synchronized methods on that object will block until the first thread releases the lock)
• We use method synchronization, so the entire object gets locked when a synchronized method gets called, but wait() releases the lock.
• TL;DR: Use synchronized on all Client methods

Wait & Notify

• Think Condition Variable

while (!checkCondition()) wait();

• Re-evaluate condition when notified - notify as a “hint”
• Ex: PingClient.getResult() waits while (pong == null), where pong is set by handlePongReply!

​

Lombok

• @EqualsAndHashCode
• Generates equals and hashCode methods for you
• @Data
• “A shortcut for @ToString, @EqualsAndHashCode, @Getter on all fields, @Setter on all non-final fields, and @RequiredArgsConstructor!” - https://projectlombok.org/features/Data
• All messages and timers should have @Data, will lead to an explosion in state space if you don’t and you may fail some tests
• Very common bug in the labs

If you use Intellij, install the Lombok Plugin: https://projectlombok.org/setup/intellij

​

It’s really just for Boilerplate

Lab 0 Tour + Testing

Lab 0 Demo!

run-tests.py

• ./run-tests.py --lab 0 --debug 1 1 "Hello World,Goodbye World"
• Options
• --debug <# servers> <# clients> <comma-separated list of client arguments>: start the visual debugger with the given arguments
• --test-num, --lab
• --no-run, --no-search: execute only the runtime tests or the graph search tests
• -g FINEST: logs every message
• Use Python 3, Java 17 or higher :)

​

Type of Tests

Regular (Run)

• Runs the Nodes, usually in parallel
• You can use `./run-tests.py --single-threaded` to help debug
• If it doesn’t work in single threaded, it probably won’t work when running in parallel
• “UNRELIABLE” tests means that messages could get dropped

Search

• Checks correctness and liveness
• State search to look for state violations and/or a goal state (BFS, DFS)
• Do not use logging. The logging messages won’t make any sense because of how states are explored, i.e. you might observe they go back on each other.

Common run commands

Running a single test

• ./run-tests.py -l LAB_NUM -n TEST_NUM

Running multiple tests

• ./run-tests.py -l LAB_NUM -n TEST_NUM_1,TEST_NUM_2,TEST_NUM_3

Common run commands

Running with Logging [prints out message receives/timer handles]

• ./run-tests.py -l LAB_NUM -n TEST_NUM -g FINER

Running with Logging [prints out message sends & receives/timer sets and handles]

• ./run-tests.py -l LAB_NUM -n TEST_NUM -g FINEST

Running with Logging and writing to a file (Don’t do this for search tests)

• ./run-tests.py -l LAB_NUM -n TEST_NUM -g FINEST &> output.txt

​

Common run commands -- [SEARCH] tests

Open up the visualizer on a non-search test with a custom workload

• ./run-tests.py -l LAB_NUM --debug NUM_SERVERS NUM_CLIENTS WORKLOAD
• WORKLOAD will look like "PUT:foo:bar,APPEND:foo:baz,GET:foo"

Open up the visualizer on a failed search test �[ONLY STARTS VISUALIZER FOR FAILED SEARCH] �(You’ll want to click “Debug system with trace”)

• ./run-tests.py -l LAB_NUM -n SEARCH_TEST_NUM --start-viz

Equals and Hashcode/Idempotency checks (really stupid reasons for failing search tests)

• ./run-tests.py -l LAB_NUM -n SEARCH_TEST_NUM --checks

​

Note: --checks doesn’t work on run tests.

Saving SEARCH test traces

./run-tests.py -l LAB_NUM -n SEARCH_TEST_NUM --save-trace

• Saves a copy of the trace for the search test if there’s an invariant violation
• Helps with debugging some null pointer exceptions

./run-tests.py --replay-traces [TRACE_NAME...]

• Runs the trace again to see if you get the same error
• (You might need to manage your traces)
• Works with --start-viz

​

General Lab Debugging

Lab 1

Recommendations/General Notes

• Start early!
• Ask questions in lecture or on Ed
• Labs get much, much, much harder
• Know what you’re trying to implement before coding (i.e. do the design docs!)
• Read the spec and reread the spec
• You may find yourself spending time to rewrite your code (to make it cleaner or to trim unnecessary implementation details) → that’s totally ok and expected

RPC - Remote Procedure Call

When a computer program causes �a procedure to execute in a different�address space (perhaps on another�computer)

RPC Semantics

• How many times will an RPC be executed by the server before it returns to the invoker of the RPC?
• At least once
• Client retries
• At most once
• Server prevents non-unique requests from compound
• UID (sequence numbers like 1,2,3,4)
• [Don’t use UUID.randomUUID]
• Idempotent
• Exactly once
• At most once with retries

Application vs. AMOApplication (Lab 1)

Wrapper around original Application

Makes sure requests executed At Most Once

Lab 1 suggestions

• Do not store StringBuilder for the key or value in the KVStore map
• Make sure that alreadyExecuted uses the right comparison operators
• Do not use any static, mutable fields in your classes (constants are fine).
• In Part 3, be sure to integrate the AMO package into SimpleClient and SimpleServer. In SimpleServer, make sure you are constructing an AMOApplication
• Do not use CompleteableFuture
• While the labs recommend IntelliJ, it isn’t necessary to use it. Feel free to use your favorite IDE/text editor.
• There might be some weird saving issues with Intellij where it says it saved some change, but the version saved on the machine doesn’t actually reflect that change, so watch out for that. Try checking with git status/git diff.
• If you have import errors on your IDE, add paths to the JARs provided

Lab 1 suggestions (continued)

• Don’t use a Hashtable, use HashMap instead
• (see framework docs for Node)
• Read the framework docs under docs
• Follow the structure for Lab 0
• If you see effects from previous tests, make sure that you don’t have static variables.

Good Practice for Timers

• If a Node resets a Timer and calls a method to set the same Timer, and this goes on for a while, it may lead to the testing framework halting and you might see the framework make no progress (which is bad).
• To reduce the number of states, when a timer goes off:
1. If you want to reset the timer, make sure to call set with the same Timer object, instead of passing a new Timer object (assuming the state of the Timer does not change. If it does, you will need to construct a new one).
2. private synchronized void onYTimer(YTimer t) {

…

set(t, RETRY_MILLIS);

}

• Recommended: Reset timers at the end of the method/if-statement so that the timer isn’t ticking while still processing the method

​

Gentle Reminders

• Student repos should be available at dslabs-[your CSE NETID], reach out to us if yours is missing
• Lab 1, PSet 1 due next Wednesday (4/3) [done individually]
• Do make submit.tar.gz and submit the tar file to Gradescope (one per group in the future)
• Problem sets: see Gradescope for due dates
• Please fill out the cse 452 partner form by Sunday (3/31) @ 5pm

Submitting Solutions

Submit design document separately in gradescope (one per group)

For later labs, design doc due week before lab, but ok to update

Do make submit.tar.gz and submit the tar file to Gradescope (one per group)

To run the visualizer remotely on attu or CSE lab computer

Linux has a XWindow client installed, then you need a X11 server to get what the client says. (contributor: Michael W. 🦊):

Windows: https://www.howtogeek.com/261575/how-to-run-graphical-linux-desktop-applications-from-windows-10s-bash-shell/

MacOS: https://www.macworld.com/article/1134207/leopardx11.html

Linux: https://www.addictivetips.com/ubuntu-linux-tips/set-up-x11-forwarding-on-linux/

Once X11 server is set up and you are running the search tests with the visualizer on the lab computer, you can ssh into the lab computer and run with �$ google-chrome &�To open up a browser and you should be able to connect to localhost:3000 or something

PuTTy settings if you choose to use PuTTy

It’s kinda slow though. Usually better to run it on your own computer or in person in the lab

Tour of the Visualization Tool

./run-tests.py --lab 0 --debug 1 1 "hi,bye"

Pulling Updates

Updates will be distributed via git (do it now plz).

Do this once at the start of the quarter:

git remote add upstream git@gitlab.cs.washington.edu:cse452-24sp/dslabs/dslabs-handout.git

Do this to pull:

git fetch upstream�git merge upstream/main

​

​