Trap Entry & Exit

ECE 391 · Discussion 9

Traps from U-mode and S-mode, ecall and sret, and how MP3 CP2 changes trap.s

ECE 391 · Discussion 9

1

Announcements

ECE 391 · Discussion 9

2

MP3CP2 Due Next Friday

Error in Discussion Slides from Last Week

Errata Update 10/28

Form for Exam 2 Conflict Due Thurs 11:59pm

Disc 8 Correction

ECE 391 · Discussion 9

2

Before CP2: every trap came from S-mode

sys/trap.s (MP2 release)

_smode_trap_entry:

smode_trap_entry_from_smode:

    addi  sp, sp, -TFRSZ

    sd    a0, A0(sp)

    # ... save GPRs, sstatus, sepc ...

    call  1f

    # ... restore ..., sret

1:  csrr  a0, scause

    mv    a1, sp

    bgez  a0, handle_smode_exception

    slli  a0, a0, 1

    srli  a0, a0, 1

    j     handle_smode_interrupt

Hardware jumps to stvec, set once at boot.

Push a frame on whatever stack the kernel is using.

scause MSB: 0 = exception, 1 = interrupt.

Handler returns, restore, sret back into the kernel.

One path: the trap came from kernel code, so sp was already a kernel stack.

ECE 391 · Discussion 9

3

Why one path was enough

stack pointer

sp

Already a kernel stack.

Just push a frame.

global and thread pointer

gp, tp

saved in trap frame

any exception

panic

Always a kernel bug.

handle_smode_exception gives up.

Interrupts were the only traps the kernel expected to come back from.

ECE 391 · Discussion 9

4

What changes in CP2

The entry code now has to ask first: did this trap come from U-mode or S-mode?

ECE 391 · Discussion 9

5

ecall: U-mode asks the kernel

usr/syscall.S (release)

_write:

    li    a7, SYSCALL_WRITE

    ecall

    ret

a7 = syscall number

a0–a5 = arguments

a0 = return value

what the hardware does

sepc points at the ecall, so the kernel must make the program resume after it.

ECE 391 · Discussion 9

6

sret: going back down

sstatus · the three bits that matter today (riscv.h)

SPP · bit 8

previous mode: 0 = U, 1 = S

…

SPIE · bit 5

previous interrupt enable

…

SIE · bit 1

interrupts on now

what sret does

mode  ← SPP   // 0 → U, 1 → S

SIE   ← SPIE

SPIE  ← 1

SPP   ← 0     // U

pc    ← sepc

ecall saves the mode in SPP.

sret reads it back.

Same sret instruction returns to U-mode or back into the kernel.

sret only follows sstatus and sepc. Set those two correctly and it goes where you want.

ECE 391 · Discussion 9

7

The first sret into a new program

A new program has never trapped, so nothing has set SPP, SPIE or sepc for it.

process_exec fills in a trap frame by hand:

tfr.sepc

ELF entry point

tfr.sstatus

SPP = 0 · SPIE = 1

tfr.a0, tfr.a1, tfr.sp

argc, argv, user stack

sys/trap.h, sys/trap.s (release)

extern void trap_frame_jump(struct trap_frame * tfr, void * sscratch);

// sscratch = initial kernel sp: the thread stack anchor

The second argument becomes sscratch: the kernel’s way back in on the next trap.

ECE 391 · Discussion 9

8

One swap tells us where we came from

sys/trap.s (MP3 release)

_smode_trap_entry:

    csrrw  sp, sscratch, sp

    beqz   sp, smode_trap_entry_from_smode

smode_trap_entry_from_umode:

    # YOUR CODE HERE

smode_trap_entry_from_smode:

    csrrw  sp, sscratch, zero

    addi   sp, sp, -TFRSZ

in S-mode · sscratch = 0

after swap: sp = 0, sscratch = kernel sp

→ beqz taken: S-mode path

in U-mode · sscratch = initial kernel sp

after swap: sp = kernel stack top, sscratch = user sp

→ falls through: U-mode path

sscratch = 0 means “in the kernel”. Non-zero means “in U-mode, and here is your kernel stack”.

ECE 391 · Discussion 9

9

Where the U-mode trap frame lives

While a thread runs in U-mode, its kernel stack is empty.

So the frame always goes in the same slot: right below the anchor.

sscratch = initial kernel sp

= address of the anchor

one thread’s kernel stack · high address at top

thread_stack_anchor

{ ktp, kgp }

← sscratch in U

struct trap_frame

saved U-mode state

allocated on entry

handler frames: handle_umode_exception, handle_syscall, …

nested S-mode trap frame, if an interrupt hits

↓ grows down

The anchor is also how the entry finds the kernel’s tp and gp again.

ECE 391 · Discussion 9

10

U-mode entry: swap to the kernel frame

smode_trap_entry_from_umode · what it must do

1

Swap sp and sscratch (given)

2

Allocate the frame, save user registers

3

Save the user sp, set sscratch = 0

4

Save sstatus, sepc; load kernel gp, tp

5

Dispatch on scause

register state after this step

sp is now a kernel address. The user’s sp is parked in sscratch, not lost.

ECE 391 · Discussion 9

11

U-mode entry: save the user’s registers

smode_trap_entry_from_umode · what it must do

1

Swap sp and sscratch (given)

2

Allocate the frame, save user registers

3

Save the user sp, set sscratch = 0

4

Save sstatus, sepc; load kernel gp, tp

5

Dispatch on scause

register state after this step

Save gp and tp too: the user can change them, and the S-mode path never saves them.

ECE 391 · Discussion 9

12

U-mode entry: save user sp, zero sscratch

smode_trap_entry_from_umode · what it must do

1

Swap sp and sscratch (given)

2

Allocate the frame, save user registers

3

Save the user sp, set sscratch = 0

4

Save sstatus, sepc; load kernel gp, tp

5

Dispatch on scause

register state after this step

We are in S-mode now, so sscratch must say so: any nested trap takes the S-mode path.

ECE 391 · Discussion 9

13

U-mode entry: CSRs, then kernel gp and tp

smode_trap_entry_from_umode · what it must do

1

Swap sp and sscratch (given)

2

Allocate the frame, save user registers

3

Save the user sp, set sscratch = 0

4

Save sstatus, sepc; load kernel gp, tp

5

Dispatch on scause

register state after this step

running_thread() reads tp, so tp must be the kernel’s before any C code runs.

ECE 391 · Discussion 9

14

U-mode entry: dispatch on scause

smode_trap_entry_from_umode · what it must do

1

Swap sp and sscratch (given)

2

Allocate the frame, save user registers

3

Save the user sp, set sscratch = 0

4

Save sstatus, sepc; load kernel gp, tp

5

Dispatch on scause

register state after this step

MSB 0 → handle_umode_exception

MSB 1 → clear it → handle_umode_interrupt

The same test as the S-mode path, but it jumps to the U-mode handlers.

ECE 391 · Discussion 9

15

A trap while already in S-mode

sys/trap.s (MP3 release, given)

_smode_trap_entry:

    csrrw  sp, sscratch, sp   # sp = 0

    beqz   sp, smode_trap_entry_from_smode

    ...

smode_trap_entry_from_smode:

    csrrw  sp, sscratch, zero

    addi   sp, sp, -TFRSZ

    # save GPRs (not gp, tp),

    # sstatus, sepc; call 1f

1:  csrr   a0, scause

    bgez   a0, handle_smode_exception

Kernel sp comes back; sscratch stays 0.

A new frame goes below whatever the kernel was doing.

gp and tp are already kernel values.

Exit: restore, pop the frame, sret with SPP = 1 → back into the kernel.

This is the MP2 path plus the swap-back. It is given; you write only the U-mode path.

ECE 391 · Discussion 9

16

A trap inside a trap

hello prints, and the timer fires mid-syscall

U

printf → _write → ecall

S

U-mode entry: frame in the fixed slot, sscratch = 0

S

handle_syscall runs with interrupts back on

S

timer fires → S-mode entry: new frame pushed below

S

timer handled → sret (SPP = 1) back into the syscall

U

syscall done → U-mode exit → sret (SPP = 0)

kernel stack at the timer

anchor

U trap frame (hello)

syscall frames

S trap frame (timer)

This only works because the U-mode entry zeroed sscratch. Otherwise the timer would overwrite hello’s frame.

ECE 391 · Discussion 9

17

Who handles what

A bad U-mode program dies alone. A bad trap in S-mode takes down the kernel.

ECE 391 · Discussion 9

18

Leaving through the U-mode path

after the U-mode handler returns · in this order

1

Restore the early GPRs

2

Restore sstatus: SIE = 0 from here on, SPP = 0

3

Restore sepc (already moved past an ecall)

4

Re-arm sscratch = initial kernel sp

5

Restore the “late” ones: sp, temp registers

6

sret → U-mode, SIE ← SPIE

why this order

From step 4 on, sscratch says “U-mode” and sp, gp, tp become user values, but we are still in S-mode.

A trap there would be routed wrong. Interrupts are off from step 2, so none can arrive.

trap_frame_jump (given) does these same steps. Read it as your model.

ECE 391 · Discussion 9

21

Leaving through the S-mode path

sys/trap.s (MP3 release, given)

    # ld a0 ... fp (not gp, tp)

    ld    t6, SSTATUS(sp)

    csrw  sstatus, t6

    ld    t6, SEPC(sp)

    csrw  sepc, t6

    ld    t6, T6(sp)

    addi  sp, sp, TFRSZ

    sret

Saved sstatus has SPP = 1, so sret stays in S-mode.

sscratch is never touched: it stays 0.

The frame is popped; the kernel stack is back where it was.

gp, tp were never changed, so nothing to restore.

Same sret instruction, different destination: SPP decides.

ECE 391 · Discussion 9

22

Watch it in GDB

entry and the swap

b _smode_trap_entry

c

info reg sp sscratch

si

info reg sp sscratch

p/x $sstatus & 0x100

# SPP: 0 = from U

cause and handler

p/x $scause

p/x $sepc

p/x $stval

b handle_umode_exception

p cause

p/x tfr->a7

p/x tfr->sepc

exit

b trap_frame_jump

p/x $a1

display/i $pc

si  # up to sret

info reg sepc sscratch

p/x $sstatus

Check sscratch twice: right after the swap, and right before each sret.

ECE 391 · Discussion 9

24