1 of 56

Processes

CS-446/646

C. Papachristos

Robotic Workers (RoboWork) Lab

University of Nevada, Reno

2 of 56

Process Abstraction

 

CS446/646 C. Papachristos

3 of 56

Process Management

Uniprogramming - Simple Process Management: One-at-a-time

A Process runs from start to full completion

    • What the early batch operating system does
    • Load a job from disk (tape) into memory, execute it, unload the job
    • Problem: Low utilization of Hardware
      • An I/O-intensive Process would spend most of its time waiting for punched cards to be read
      • CPU time is wasted
      • Computers were very expensive back then

CS446/646 C. Papachristos

4 of 56

Process Management

Multiprogramming – Multiple Process Management: Multiple at-the-same-time

Modern OSs run multiple processes Concurrently

  • Example:
    • gcc file_A.c – compiler running on file A
    • gcc file_B.c – compiler running on file B
    • vim – text editor
    • firefox – web browser

  • Note:
    • Multiple firefox windows:
      • Implemented as a single Process�(recent versions actually implement splitting into� different Processes, but still the overall collection� serves to manage all of the open window instances)

CS446/646 C. Papachristos

Windows 10 Task Manager

5 of 56

Process Management

Multiprogramming – Multiple Process Management : Multiple at-the-same-time

  • Multiple Processes loaded in Memory and Ready to run
  • If a Process is Blocked in I/O, select another Process to run on CPU
  • Different Hardware components utilized by different Tasks at the same time

Advantages: Increase utilization and overall speed

    • Higher throughput, lower latency

  • Multiple Processes can increase CPU utilization
    • Overlap one Process’ computation with another’s wait

  • Multiple Processes can reduce Latency
    • Running A then B requires 100 sec for B to complete

    • Running A and B Concurrently makes B finish faster�(and more responsive)

Note: A will however run slower than if it had whole machine to itself

CS446/646 C. Papachristos

vim

gcc

wait for input

wait for input

A

B

80s

20s

A

B

6 of 56

Kernel & Processes

Process Components

A Process encapsulates the full State of a Program in execution

  • A (Virtual) Address Space
  • The Code for the executing Program
  • The Data for the executing Program
  • The Stack containing the state of procedure calls
    • Note: Separate User-Space and Kernel-Space Stacks

  • A Stack Pointer (SP) indicating the offset into current Stack
  • A Program Counter (PC) indicating the next Instruction

  • A set of General-Purpose Registers with current values
  • A set of Operating System Resources
    • Open Files, Network Connections, etc.

CS446/646 C. Papachristos

7 of 56

Kernel & Processes

From the Process viewpoint

Each Process has own view of the Machine

  • Its own Virtual Address Space
  • Its own Virtual CPU
  • Its own (virtually-exclusive) open Resources (e.g. Files)

*(char *)0xc000 is different for Process1 & Process2

Note: Above is C code.

  • Simplifies programming model
    • gcc does not care that firefox is running

CS446/646 C. Papachristos

8 of 56

Kernel & Processes

 

CS446/646 C. Papachristos

Process A

Process B

9 of 56

Kernel & Processes

Process Address Space

CS446/646 C. Papachristos

// works with GCC

register void *sp asm ("sp");

printf("%p", sp);

Stack

Pointer

Program

Counter

10 of 56

Kernel & Processes

Process

Address

Space

(Better picture)

Note:

In reality, also Address Randomization takes place…

User Virtual Memory

Kernel Virtual Memory

0xFFFFFFFF

0xC0000000

0x00000000

0x08048000

Grows Downward

Grows Upward

OS Kernel Space

Not accessible by User Mode code

Invalid Virtual Addresses (for this Process)

C. Papachristos

0xBFFFFFFF

Stack (User-Space)

Automatic variables, return Addresses, …

Uninitialized Data (BSS)

Uninitialized (zero-initialized) static variables

Initialized Data Segment (DS)

Initialized static variables

Code / Text Segment

Program binary image

Mapped Memory (also Dynamic)

Heap

(part of) Dynamic Memory

11 of 56

Kernel & Processes

Process Naming

A Process is named using its Process IDentifier (PID)

CS446/646 C. Papachristos

Linux top (Table of Processes)

12 of 56

Kernel & Processes

Inter-Process Communication (IPC)

Interaction between Processes is sometimes desired

  • Conceptually simplest form is through files: vim edits file, gcc compiles it
  • More complicated: shell /command, Window-manager /App

Real-time interaction is usually required. Methods:

  • a) Message-Passing(through the Kernel)

  • b) Shared Memory(sharing a region of Physical Memory)

  • c) Asynchronous SIGNALs or Alerts(again through the Kernel)

CS446/646 C. Papachristos

13 of 56

Kernel & Processes

 

CS446/646 C. Papachristos

14 of 56

Kernel & Processes

Inter-Process Communication (IPC)

Unix PIPE: Creates a one-way communication channel (through the Kernel-Space)

int pipe(int fds[2])

  • fds[2] is used to return two File Descriptors (accessible from User-Space)
    • Bytes written to fds[1] (Write-Endpoint of pipe) can be read from fds[0] (Read-Endpoint of pipe)

How can one Process (e.g. reader) know of another Process’ (e.g. writer) setting-up of a Pipe connection?

  • fork() Process creation mechanism:�(more on this later)

CS446/646 C. Papachristos

int pipefd[2];

pipe(pipefd);

switch( pid = fork() ) {

case -1: perror("fork error"); exit(1);

case 0: close(pipefd[0]); // child process

// write to pipefd[1], close it when done� // (or on cleanup during child process termination)

break;

default: close(pipefd[1]); // parent process

// read from pipefd[0], close it when done � // or child terminates (check with wait())

break;

}

15 of 56

Kernel & Processes

Inter-Process Communication (IPC)

Unix Shared Memory: Direct communication based on Memory operations (from User-Space)

int shmget(key_t key, size_t size, int shmflg);

  • Create a Shared Memory Segment
  • key : Either a unique identifier (e.g. returned by ftok) if the Segment should be accessed by other Processes using this specific (system-wide) key, or IPC_PRIVATE to ignore key and just create the Segment� (Note: In this case returned shmid has to be communicated to other Process through another form of IPC)
  • Returns : shmid identifier of Shared Memory Segment associated with the value of the argument key

int shmat(int shmid, const void *addr, int flg);

  • Attach Shared Memory Segment to Address Space of the calling Process
  • shmid : Identifier (previously returned by shmget())

int shmdt(const void *shmaddr);

  • Detach calling ProcessAddress Space from Shared Memory Segment

Remember: Shared Memory access also requires Synchronization (more on Synchronization Mechanisms later)

CS446/646 C. Papachristos

16 of 56

Kernel & Processes

Inter-Process Communication (IPC)

UNIX Signals (again through the Kernel-Space)

  • A very small payloadan integer Message
  • A fixed set of available OS-defined Signals
    • e.g. 2: SIGINT , 9: SIGKILL , 11: SIGSEGV , 20: SIGSTP , etc.

  • Registering an OS Signal handler in a Process

sighandler_t signal(int signum, sighandler_t handler);

int sigaction(int signum, const struct sigaction *act, struct sigaction *oldact);

  • Send a Signal to a Process / Process Group (typically the Leader Process and any Processes created(/fork-ed) afterwards)

int kill(pid_t pid, int sig);

int killpg(int pgrp, int sig);

CS446/646 C. Papachristos

Note: If pid is positive, then signal sig is sent to the Process with the ID specified by pid.

If pid equals 0, then sig is sent to every Process in the Process Group of the calling Process.

If pid equals -1, then sig is sent to every Process for which the calling Process has permission to send signals, except for Process 1 (init), but see below.

If pid is less than -1, then sig is sent to every Process in the Process Group whose ID is -pid.

Note: typedef void (*sighandler_t)(int);

i.e. Pointer to void function with a single int argument.

17 of 56

Kernel & Processes

Process Implementation

A data structure for each process: Process Control Block (PCB)

    • Contains all information about a Process
    • PCB is also maintained when the Process is not in Running Execution State (why?)

  • Process Execution State
    • Running, Ready(/Runnable), Waiting, etc.
  • CPU Register (last updated) values
    • %sp, %eip, %eax, etc.
  • Scheduling information
  • Virtual Memory mappings
  • I/O status information
  • Credentials (User/Group ID), Signal mask, Priority, Accounting, etc.

The Process is a heavyweight Abstraction!

CS446/646 C. Papachristos

18 of 56

Kernel & Processes

struct proc (Solaris OS)

CS446/646 C. Papachristos

/*

* One structure allocated per active process. It contains all

* data needed about the process while the process may be swapped

* out. Other per-process data (user.h) is also inside the proc structure.

* Lightweight-process data (lwp.h) and the kernel stack may be swapped out.

*/

typedef struct proc {

/*

* Fields requiring no explicit locking

*/

struct vnode *p_exec; /* pointer to a.out vnode */

struct as *p_as; /* process address space pointer */

struct plock *p_lockp; /* ptr to proc struct's mutex lock */

kmutex_t p_crlock; /* lock for p_cred */

struct cred *p_cred; /* process credentials */

/*

* Fields protected by pidlock

*/

int p_swapcnt; /* number of swapped out lwps */

char p_stat; /* status of process */

char p_wcode; /* current wait code */

ushort_t p_pidflag; /* flags protected only by pidlock */

int p_wdata; /* current wait return value */

pid_t p_ppid; /* process id of parent */

struct proc *p_link; /* forward link */

struct proc *p_parent; /* ptr to parent process */

struct proc *p_child; /* ptr to first child process */

struct proc *p_sibling; /* ptr to next sibling proc on chain */

struct proc *p_psibling; /* ptr to prev sibling proc on chain */

struct proc *p_sibling_ns; /* prt to siblings with new state */

struct proc *p_child_ns; /* prt to children with new state */

struct proc *p_next; /* active chain link next */

struct proc *p_prev; /* active chain link prev */

struct proc *p_nextofkin; /* gets accounting info at exit */

struct proc *p_orphan;

struct proc *p_nextorph;

*p_pglink; /* process group hash chain link next */

struct proc *p_ppglink; /* process group hash chain link prev */

struct sess *p_sessp; /* session information */

struct pid *p_pidp; /* process ID info */

struct pid *p_pgidp; /* process group ID info */

/*

* Fields protected by p_lock

*/

kcondvar_t p_cv; /* proc struct's condition variable */

kcondvar_t p_flag_cv;

kcondvar_t p_lwpexit; /* waiting for some lwp to exit */

kcondvar_t p_holdlwps; /* process is waiting for its lwps */

/* to to be held. */

ushort_t p_pad1; /* unused */

uint_t p_flag; /* protected while set. */

/* flags defined below */

clock_t p_utime; /* user time, this process */

clock_t p_stime; /* system time, this process */

clock_t p_cutime; /* sum of children's user time */

clock_t p_cstime; /* sum of children's system time */

caddr_t *p_segacct; /* segment accounting info */

caddr_t p_brkbase; /* base address of heap */

size_t p_brksize; /* heap size in bytes */

/*

* Per process signal stuff.

*/

k_sigset_t p_sig; /* signals pending to this process */

k_sigset_t p_ignore; /* ignore when generated */

k_sigset_t p_siginfo; /* gets signal info with signal */

struct sigqueue *p_sigqueue; /* queued siginfo structures */

struct sigqhdr *p_sigqhdr; /* hdr to sigqueue structure pool */

struct sigqhdr *p_signhdr; /* hdr to signotify structure pool */

uchar_t p_stopsig; /* jobcontrol stop signal */

19 of 56

Kernel & Processes

struct proc (Solaris OS)

CS446/646 C. Papachristos

/*

* Special per-process flag when set will fix misaligned memory

* references.

*/

char p_fixalignment;

/*

* Per process lwp and kernel thread stuff

*/

id_t p_lwpid; /* most recently allocated lwpid */

int p_lwpcnt; /* number of lwps in this process */

int p_lwprcnt; /* number of not stopped lwps */

int p_lwpwait; /* number of lwps in lwp_wait() */

int p_zombcnt; /* number of zombie lwps */

int p_zomb_max; /* number of entries in p_zomb_tid */

id_t *p_zomb_tid; /* array of zombie lwpids */

kthread_t *p_tlist; /* circular list of threads */

/*

* /proc (process filesystem) debugger interface stuff.

*/

k_sigset_t p_sigmask; /* mask of traced signals (/proc) */

k_fltset_t p_fltmask; /* mask of traced faults (/proc) */

struct vnode *p_trace; /* pointer to primary /proc vnode */

struct vnode *p_plist; /* list of /proc vnodes for process */

kthread_t *p_agenttp; /* thread ptr for /proc agent lwp */

struct watched_area *p_warea; /* list of watched areas */

ulong_t p_nwarea; /* number of watched areas */

struct watched_page *p_wpage; /* remembered watched pages (vfork) */

int p_nwpage; /* number of watched pages (vfork) */

int p_mapcnt; /* number of active pr_mappage()s */

struct proc *p_rlink; /* linked list for server */

kcondvar_t p_srwchan_cv;

size_t p_stksize; /* process stack size in bytes */

/*

* Microstate accounting, resource usage, and real-time profiling

*/

hrtime_t p_mstart; /* hi-res process start time */

hrtime_t p_mterm; /* hi-res process termination time */

hrtime_t p_mlreal; /* elapsed time sum over defunct lwps */

hrtime_t p_acct[NMSTATES]; /* microstate sum over defunct lwps */

struct lrusage p_ru; /* lrusage sum over defunct lwps */

struct itimerval p_rprof_timer; /* ITIMER_REALPROF interval timer */

uintptr_t p_rprof_cyclic; /* ITIMER_REALPROF cyclic */

uint_t p_defunct; /* number of defunct lwps */

/*

* profiling. A lock is used in the event of multiple lwp's

* using the same profiling base/size.

*/

kmutex_t p_pflock; /* protects user profile arguments */

struct prof p_prof; /* profile arguments */

/*

* The user structure

*/

struct user p_user; /* (see sys/user.h) */

/*

* Doors.

*/

kthread_t *p_server_threads;

struct door_node *p_door_list; /* active doors */

struct door_node *p_unref_list;

kcondvar_t p_server_cv;

char p_unref_thread; /* unref thread created */

20 of 56

Kernel & Processes

struct proc (Solaris OS)

CS446/646 C. Papachristos

/*

* Kernel probes

*/

uchar_t p_tnf_flags;

/*

* C2 Security (C2_AUDIT)

*/

caddr_t p_audit_data; /* per process audit structure */

kthread_t *p_aslwptp; /* thread ptr representing "aslwp" */

#if defined(i386) || defined(__i386) || defined(__ia64)

/*

* LDT support.

*/

kmutex_t p_ldtlock; /* protects the following fields */

struct seg_desc *p_ldt; /* Pointer to private LDT */

struct seg_desc p_ldt_desc; /* segment descriptor for private LDT */

int p_ldtlimit; /* highest selector used */

#endif

size_t p_swrss; /* resident set size before last swap */

struct aio *p_aio; /* pointer to async I/O struct */

struct itimer **p_itimer; /* interval timers */

k_sigset_t p_notifsigs; /* signals in notification set */

kcondvar_t p_notifcv; /* notif cv to synchronize with aslwp */

timeout_id_t p_alarmid; /* alarm's timeout id */

uint_t p_sc_unblocked; /* number of unblocked threads */

struct vnode *p_sc_door; /* scheduler activations door */

caddr_t p_usrstack; /* top of the process stack */

uint_t p_stkprot; /* stack memory protection */

model_t p_model; /* data model determined at exec time */

struct lwpchan_data *p_lcp; /* lwpchan cache */

/*

* protects unmapping and initilization of robust locks.

*/

kmutex_t p_lcp_mutexinitlock;

utrap_handler_t *p_utraps; /* pointer to user trap handlers */

refstr_t *p_corefile; /* pattern for core file */

#if defined(__ia64)

caddr_t p_upstack; /* base of the upward-growing stack */

size_t p_upstksize; /* size of that stack, in bytes */

uchar_t p_isa; /* which instruction set is utilized */

#endif

void *p_rce; /* resource control extension data */

struct task *p_task; /* our containing task */

struct proc *p_taskprev; /* ptr to previous process in task */

struct proc *p_tasknext; /* ptr to next process in task */

int p_lwpdaemon; /* number of TP_DAEMON lwps */

int p_lwpdwait; /* number of daemons in lwp_wait() */

kthread_t **p_tidhash; /* tid (lwpid) lookup hash table */

struct sc_data *p_schedctl; /* available schedctl structures */

} proc_t;

21 of 56

Kernel & Processes

Process Execution State

A Process has an Execution State to indicate what it is doing

Running

Executing Instructions on a CPU

    • It is the Process that has control of a CPU

Ready

Waiting to be assigned to a CPU

    • Ready to execute, but other Processes are executing on the system’s CPU(s)

Waiting

Waiting for an Event, e.g., I/O completion

    • It cannot make progress until an Event is signaled (e.g. Disk completes)

CS446/646 C. Papachristos

22 of 56

Kernel & Processes

Transition of Process Execution State

As a Process executes, it moves from Execution State to Execution State

  • In Unix ps: STAT column indicates Execution State
    • Maximum number of Processes in OS: “In the Linux Kernel Space context, a Process and a Thread are one and the same. They're handled the same way by the Kernel. They both occupy a slot in the task_struct. A Thread, by common terminology, is in Linux a Process that shares Resources with another Process (they will also share a Thread Group ID).”
      • /proc/sys/kernel/threads-max
      • /proc/sys/kernel/pid_max

General

Execution State Graph:

CS446/646 C. Papachristos

23 of 56

Kernel & Processes

Transition of Process Execution State

As a Process executes, it moves from Execution State to Execution State

  • In Unix ps: STAT column indicates Execution State
    • Maximum number of Processes in OS: “In the Linux Kernel Space context, a Process and a Thread are one and the same. They're handled the same way by the Kernel. They both occupy a slot in the task_struct. A Thread, by common terminology, is in Linux a Process that shares Resources with another Process (they will also share a Thread Group ID).”
      • /proc/sys/kernel/threads-max
      • /proc/sys/kernel/pid_max

Linux

Execution State Graph:

CS446/646 C. Papachristos

Sleep

States

Stopped State

(via SIGNAL)

24 of 56

Kernel & Processes

(Execution) State Queues

How the OS keeps track of Processes

Simple 1st Idea:

  • List of Processes
  • How to find out ones in the Ready Execution State?
    • Iterating through the list is slow

Improvement:

  • Partition List of Processes based on type of Execution State
  • OS maintains a collection of Queues that represent the Execution State of all Processes
    • A typical implementation is to have one Queue for each type of Execution State: Ready, Waiting, etc.
  • Each Process Control Block (PCB) is queued on a State Queue according to its current Execution State
    • When a Process changes Execution State, its PCB is moved from one State Queue into another

CS446/646 C. Papachristos

25 of 56

Kernel & Processes

(Execution) State Queues

CS446/646 C. Papachristos

Note: There may be many Wait State Queues, one for each

type of Wait (Disk, Console, Timer, Network, etc.)

26 of 56

Kernel & Processes

Scheduling

How the OS determines which Process to run

    • If 0 Runnable, run idle loop (or Halt CPU), if 1 Runnable, run that one
    • If >1 Runnable, must make Scheduling decision

Simplistic 1st Idea: Scan Process Table for first Runnable one

    • Expensive & Unfair (smaller PIDs prioritized)

Better Idea: First-In First-Out (FIFO) Scheduling

    • Put Tasks on back of Queue, pull them from front:
      • Pintos does this – see ready_list in thread.c

Even Better: Priority Scheduling

    • More on this in Scheduling Lecture…

CS446/646 C. Papachristos

27 of 56

Kernel & Processes

Process Dispatching Mechanism

OS Process Dispatching loop

CS446/646 C. Papachristos

WHILE TRUE:

Run Process for some time

Save Process State

Next Process = Schedule (Ready-State Processes)

Load next Process State

II. Context Switching

I. Regaining Control

28 of 56

Kernel & Processes

I. How to Regain Control

  • First of all, must switch from User Mode to Kernel Mode

Cooperative Multitasking:

Processes voluntarily yield control back to OS

  • How / When? With System Calls that relinquish CPU
    • Trustworthiness: OS needs to trust User Processes

True Multitasking:

OS “PreemptsProcesses by periodic Interrupts

  • Processes are assigned Time Slices (/“Quanta”) of execution
  • Dispatcher counts Timer Interrupts before context switch
    • OS needs to trust no one

CS446/646 C. Papachristos

29 of 56

Kernel & Processes

I. How to Regain Control

Preemption

  • Temporarily Interrupting an executing Task, with the intention of resuming it later
  • Taking over performed by an external Preemptive Scheduler with no assistance/cooperation from the Task

Process Scheduling decision triggering

  • Cooperative Multitasking: Yielding control of CPU

    • Voluntarily, e.g. sched_yield

    • Forced, e.g. on a System Call, Page Fault, Illegal Instruction, etc.

  • True Multitasking: Preemption
    • Periodic Timer Interrupt
      • Indicates that Running Process used up Time Slice (/“Quantum), Schedule another
        • Time Slice: The period of time for which a Process is allowed to run in a Preemptive Multitasking system
    • (Device) Interrupt
      • Disk request completed / Packet arrived on Network
        • Process previously Waiting now becomes Runnable

CS446/646 C. Papachristos

Note: Intended for use with a specific class Scheduling policies (i.e. Real-Time). Causes the calling Thread to relinquish the CPU. The calling Thread is moved to the end of the Queue for its Static Priority and a new Thread gets to run.

30 of 56

Kernel & Processes

II. How to Switch Context

Context Switching

  • The procedure of storing the State of a Process or Thread, so that it can be restored and resume execution at a later point, and then restoring a different, previously saved, State

Changing over the running Process

  • CPU (& MMU) Hardware state is changed from�one to another
    • Can happen thousands of times each second

CS446/646 C. Papachristos

31 of 56

Kernel & Processes

II. How to Switch Context

Context Switching

Changing over the running Process

  • Save P0’s User Mode CPU Context and switch from User to Kernel Mode (HW)
  • System Call / Interrupt Handler Routine (OS)
  • Save P0’s Kernel Mode CPU Context and switch to the Scheduler CPU Context (OS + HW)
  • Scheduler selects another Process P1 (OS)
  • Switch to P1’s Address Space (OS + HW)
  • Save Scheduler CPU Context and switch to P1’s Kernel Mode CPU Context (OS + HW)
  • Switch from Kernel Mode to User Mode and load P1’s User Mode CPU Context (HW)

CS446/646 C. Papachristos

32 of 56

Kernel & Processes

II. How to Switch Context

Context Switching is very Machine-dependent. Typical things include:

    • Save Program Counter (PC) and Integer Registers (always)
    • Save Condition Codes, i.e. the CPU Status Register (e.g. EFLAGS Register in x86 Architecture)
    • Save Floating Point Registers or other Special Registers (conditionally optimizable)
    • Change Virtual Address Translations (load Page Directory/Master Page Table, but possibly also flush TLB)

Tricky:

    • Need to save Program Counter (PC) to the Process Control Block (PCB) in Memory
      • But that would require to load the PC of the code that saves the original PC, which would now be lost
    • Need to save all Registers to the Process Control Block (PCB) in Memory
      • But we have to actually run code to save Registers, and running code changes Register values
    • Need combined Software/Hardware support
      • When an Interrupt occurs, the CPU will automatically save the Program Counter (PC), the Status Register, and active data Registers, at a known location�(Remember: typically pushes them onto the Stack)

CS446/646 C. Papachristos

33 of 56

Kernel & Processes

II. How to Switch Context

Context Switching has a non-negligible cost

  • It represents pure overhead: The system does no useful work while Context Switching
    • Save/restore Floating Point Registers expensive
      • Optimization: only save if Process actually used floating point arithmetic
    • May require flushing / Invalidating part of the MMU’s Translation Lookaside Buffer (TLB) cache
      • Memory cache that stores the recently utilized translations of Virtual Memory to Physical Memory.�Used to reduce the time taken to access a user Memory location.
  • The OS must balance Context Switching frequency with Scheduling requirements

Context Switching usually causes more CPU Cache Misses

    • Due to frequent switching between Working Sets
      • Working Set: The (Memory) Pages most frequently accessed at some point
      • Potentially causing Cache Pollution
    • Also, in the extreme case of overcommitted resources: Thrashing

CS446/646 C. Papachristos

34 of 56

Next Lecture Reading Preparation

Operating Systems Three Easy Pieces (https://pages.cs.wisc.edu/~remzi/OSTEP/)

Virtualization

  • 5. Process API
    • Beginning of Chapter
    • 5.1 The fork() System Call
    • 5.2 The wait() System Call
    • 5.3 Finally, The exec() System Call
    • 5.4 Why? Motivating The API
    • 5.5 Process Control And Users
    • 5.6 Useful Tools

CS446/646 C. Papachristos

35 of 56

Process-related System Calls

Allow one Process to create another Child Process

A Process is created by another Process

    • Parent is creator, Child is created (Unix: ps “PPID” field / pstree -p command)
    • Very first Process (Unix: init / Linux: systemd (PID 1 and PPID 0))�“Once the Kernel has started, it starts the init process, a “Daemon” which then bootstraps the User Space, for example by checking and mounting Filesystems, and starting up other Processes. The init system is the first Daemon to start (during booting) and the last Daemon to terminate (during shutdown).”�Note: init is by definition a User Space Task

Parent defines Resources and Privileges for its Children

    • Unix: Process User ID is inherited – Children (Processes) of your Shell (Process) will execute with the same User Privileges

After creating a Child Process

    • Parent Process may either Wait for it to finish its work, or Concurrently continue�its own work

CS446/646 C. Papachristos

36 of 56

Process-related System Calls

Process Creation in Windows

1. Creates and initializes a new PCB

2. Creates and initializes a new Address Space

3. Loads the program specified by prog into the Address Space

4. Copies args into Memory allocated in Address Space

5. Initializes the saved hardware Context to start execution at main (or as specified)

6. Places the PCB on the Ready Queue

Note: Returns BOOL. If CreateProcess succeeds, it returns a PROCESS_INFORMATION structure that contains Handles and Identifiers for the new Process and its primary Thread. The Thread and Process Handles are created with full�access rights, although you can restrict access if you specify Security Descriptors.

CS446/646 C. Papachristos

CreateProcess( prog, // Module to load (e.g. "c:\\winnt\\notepad.exe“ or NULL – use command line)

argv[1], // Command line args

NULL, // Process handle not inheritable

NULL, // Thread handle not inheritable

FALSE, // Set handle inheritance to FALSE

0, // No creation flags

NULL, // Use parent's environment block

NULL, // Use parent's starting directory

&si, // Pointer to STARTUPINFO structure

&pi ) // Pointer to PROCESS_INFORMATION structure

37 of 56

Process-related System Calls

Process Creation in Unix

1. Creates and initializes a new PCB

2. Creates a new Address Space

3. Initializes the Address Space with a copy of the Address Space of the Parent

4. Initializes the Kernel Resources to point to the Parent’s Resources (e.g. open Files)

5. Places the PCB on the Ready Queue

fork() returns twice:

(???)

Because if it’s successful, we now have 2 Processes

  • Returns the Child’s PID to the Parent Process
  • Returns 0 to the Child Process

CS446/646 C. Papachristos

pid_t fork (void)

38 of 56

Process-related System Calls

Process Creation in Unix

CS446/646 C. Papachristos

pid_t fork (void)

39 of 56

Process-related System Calls

Process Creation in Unix

Expected program output ($ gcc –o fork_proc fork_proc.c && ./fork_proc)?

I’m the parent and my child’s pid is: 486

I’m the child of ./fork_proc and my pid is: 486

CS446/646 C. Papachristos

#include <stdio.h>

#include <unistd.h>

int main(int argc, char *argv[])

{

char *name = argv[0];

int fork_retval = fork();

if (fork_retval == 0) {

printf("I’m the child of %s and my pid is: %d\n", name, getpid());

return 0;

} else {

printf("I’m the parent and my child’s pid is: %d\n", fork_retval);

return 0;

}

}

pid_t fork (void)

40 of 56

Process-related System Calls

Process Creation in Unix

  • After duplicating Address Spaces

CS446/646 C. Papachristos

pid_t fork (void)

fork_retval

fork_retval

fork_retval

fork_retval

fork_retval

fork_retval

41 of 56

Process-related System Calls

Process Creation in Unix

  • Divergence between different Processes

CS446/646 C. Papachristos

pid_t fork (void)

fork_retval

fork_retval

fork_retval

fork_retval

fork_retval

fork_retval

42 of 56

Process-related System Calls

Process Creation in Unix

Program output (continued)

$ ./fork_proc

My child is 486

Child of ./fork_proc is 486

$ ./fork_proc

Child of ./fork_proc is 486

My child is 486

Note: Notice possible change of output order between subsequent runs of the same executable

CS446/646 C. Papachristos

pid_t fork (void)

#include <stdio.h>

#include <unistd.h>

int main(int argc, char *argv[])

{

char *name = argv[0];

int fork_retval = fork();

if (fork_retval == 0) {

printf("Child of %s is %d\n", name, getpid());

return 0;

} else {

printf("My child is %d\n", fork_retval);

return 0;

}

}

43 of 56

Process-related System Calls

Process Creation in Unix

Running a Program

v : Takes an argv array of C-strings populated with the Program, its Arguments/Flags, and a NULL-pointer at the end

e : Takes an envp array of C-strings populated Environment variables.

p : Inherits the Parent Process Environment.

execXX(…)

1. Stops the Program executing under the current Process

2. Loads the new Program prog into the calling ProcessAddress Space

3. Initializes Hardware Context and Args for the new Program

4. Places the PCB onto the Ready Queue

Note: It does NOT create a new Process. “… it replaces the entire current Process with a new Program image”.� It loads the Program into the current Process’ Address Space and runs it from the entry point.

(Also, in PintOS, exec is more like a combined fork/exec)

  • What does it mean for execXX() to return?

CS446/646 C. Papachristos

int execv(char *prog, char *const argv[]);

int execve(const char *prog, char *const argv[], char *const envp[]);

int execvp(const char *prog, char *const argv[]);

44 of 56

Process-related System Calls

Process Creation in Unix

Most calls to fork() will be followed by execXX(…)

    • (Can always be combined in a single spawn System Call)

Very useful when

  • Child is working together with Parent
  • Child relies on Parent’s data to accomplish its task

Example: Web Server

CS446/646 C. Papachristos

int server_sock = socket() … bind() … listen() …

while (1) {

// Accept connection request from client in addr

int client_sock = accept(server_sock, addr, addrlen);

// accept() returned: incoming connection - Create a Child Process

if ((fork_retval = fork()) == 0) { // Child Process

close(server_sock);

// Use client_sock (e.g. send() client a handshake msg)

} else { // Parent Process

// back to listening for more incoming, no need to wait on Child

}

}

Note 1: The accept() Syscall creates a new SocketDescriptor with the same properties as server_sock and returns it to the caller. If the queue has no pending connection requests, accept() Blocks the caller unless server_sock is in Nonblocking mode. If no connection requests are queued and server_sock is in Nonblocking mode, accept() returns -1 and sets the error code to EWOULDBLOCK.

Note 2: The argument server_sock is a Socket that has been created with socket(), bound to a local Address with bind(), and is listening for Connections after a listen().

45 of 56

Process-related System Calls

Process Creation in Unix

Example: Simple Shell

CS446/646 C. Papachristos

pid_t pid; char **av;

/* ... main loop: */

for (;;) {

parseInput (&av, stdin); //read shell commands from stdin

switch (fork_retval = fork ()) {

case -1: //Parent Process – fork error

perror ("fork error"); break;

case 0: //Child Process

execProc (); break;

default: //Parent Process

waitpid (fork_retval, NULL, 0); break;

}

}

/* ... rest of main */

void execProc () { //inside Child Process

execvp (av[0], av);

perror (av[0]);

_exit (1);

}

$ gcc -o simpleshell simpleshell.c

$ ./simpleshell

$ date

Wed Sep 6 09:20:53 PDT 2023

$ /usr/bin/gcc --version

gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0

Copyright (C) 2017 Free Software Foundation, Inc.

This is free software; see the source for copying conditions. There is NO

warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

46 of 56

Process-related System Calls

Process Creation in Unix

Majority of calls to fork() are followed by execXX(…)

    • (Can always be combined in a single spawn() function)

Very useful when

  • Child is working together with Parent
  • Child can rely on Parent’s data to accomplish its task

Real win is in the simplicity of its interface

  • Many things we might want to do to the Child Process
    • Manipulate File Descriptors, set Environment Variables, reduce Privileges, etc.
  • Yet fork() requires no arguments at all
    • Remember: Windows CreateProcess(…) System Call required a lot of different options�(e.g. function call arguments) to cover the possibilities for the new Program we intend to run

CS446/646 C. Papachristos

47 of 56

Process-related System Calls

Process Creation in Unix

Example: Simple Shell with Redirection

CS446/646 C. Papachristos

void execProc () { //inside Child Process

/*...handle case for infile→fd:0...*/

if (infile) { /* set non-NULL when "cmd < infile" */

int fd_in;

if ((fd_in = open (infile, O_RDONLY)) < 0) {

perror (infile);

_exit (1);

}

if (fd_in != 0) {

dup2 (fd_in, STDIN_FILENO);

close (fd_in); //don’t keep around extra

} //reference to opened infile

}

/*...handle case for outfile→fd:1...*/

if (outfile) { ... }

/*...handle case for errfile→fd:2...*/

if (errfile) { ... }

execvp (av[0], av);

perror (av[0]);

_exit (1);

}

$ gcc -o redirshell redirshell.c

$ ./redirshell

$ ls > list.txt

$ sort < list.txt > sorted_list.txt

$ cat sorted_list.txt

a.c

b.c

cs446.txt

Note:

0: STDIN_FILENO

1: STDOUT_FILENO

2: STDERR_FILENO

Note:

System Call – Duplicate File Descriptor

int dup2(int oldfd, int newfd);

(In short:) Makes newfd be the copy of oldfd, closing newfd first if necessary.

48 of 56

Process-related System Calls

Process Creation in Unix

Why does Windows use CreateProcess while Unix uses fork/execXX?

    • Different OS design philosophy

  • What happens if you run exec csh in a Shell?

  • What happens if you run exec ls in a Shell?

  • fork() may return an error. Why might this happen?
    • Note: There exists an explain_fork() to obtain explanations for errors returned by the fork() System Call

CS446/646 C. Papachristos

49 of 56

Process-related System Calls

Process (Normal) Termination

In Unix: exit (int status) (In Windows: ExitProcess (int status))

Frees Resources and terminates “normally”, returns status & 0xFF to Parent wait()

1. Terminate all Threads of calling Process (more in Threads Lecture …)

2. Close open Files, Network Connections

3. Allocated Memory (mark Virtual Memory Pages as Free)

4. Remove PCB from Kernel data structures, delete

Note: All functions registered with atexit() and on_exit() are called, in the reverse order of their registration. … All open stdio streams are flushed and closed. Files created by tmpfile() are removed.

exit() is a System Call, i.e. a Process does not just simply clean up after itself

  • The OS has to do it (Why?)
    • Virtual Memory Mappings, Resource allocation, etc. falls under Kernel Space responsibilities

CS446/646 C. Papachristos

50 of 56

Process-related System Calls

Process (Immediate) Termination

In Unix: _exit (int status)

Frees resources and terminates “immediately”, returns status & 0xFF to Parent wait()

Note: Terminates the calling Process “immediately”. Any open File Descriptors belonging to the calling Process are closed; any Children of the calling Process are inherited by Process 1, init, and the calling Process’s Parent is sent a SIGCHLD Signal.

I.e. calling the “normal” exit() from a forked Child that hasn’t successfully exec’d something (to replace the original Process image it inherits), will interfere with the Parent Process’ external data (Files) via calling its atexit() handlers, calling its Signal handlers, and/or flushing Buffers (i.e. print out whatever exists in stdio of the newly forked Child Process which is an exact User Space duplicate of the Parent – “double-flushing”)

Rule-of-Thumb:

  • Use _exit() to abort the Child when the exec fails
  • Use _exit() to terminate a Child that performs no exec (more rare)

CS446/646 C. Papachristos

51 of 56

Process-related System Calls

Waiting on a Process

In Unix: pid_t wait (int *status) (In Windows: WaitForSingleObject(eHandle,millis))

  • Suspends current Process until any Child Process changes State (Terminated, or Stopped/Resumed by a Signal)

pid_t waitpid (pid_t pid, int *status, int options)

  • Suspends current Process until the pid - specified Child Process changes State

Return value of wait/waitpid :

  • wait(): �On success, returns the PID of the State-changed (e.g. terminated) Child; on error, -1 is returned.

  • waitpid():On success, returns the PID of the Child whose State has changed; if WNOHANG was specified and one or more Child(ren) specified by PID exist, but have not yet changed State, then 0 is returned.�On error, -1 is returned.

CS446/646 C. Papachristos

Note:

pid > 0: wait for the Child whose Process ID is equal to the value of pid.

pid = 0: wait for any Child Process whose Process Group ID is equal to that of the calling Process.

pid = -1: wait for any Child Process.

pid < 0: wait for any Child Process whose Process Group ID is equal to the absolute value of pid.

52 of 56

Process-related System Calls

Waiting on a Process

The purpose of the wait() Syscall is to allow the Child Process to report a status back to the Parent Process

  • Child Process is not completely cleaned up when it exit()s�It “dies” as a Running Process, most Resources are released, but it still remains in the Process Table
    • That's where its Exit Status is stored, so that the Parent can retrieve it by calling one of the wait() variants
  • It will remain & keep consuming that Process Table slot until it is “Reaped” (by being wait()ed on)

  • Unix: Every Process must be “Reaped” by a Parent

Zombie Process: Child that terminates, but has not been wait()ed for yet (by the/a Parent). The Kernel maintains a minimal set of information about the zombie (PID, termination status, resource usage information) in order to allow the Parent to later perform a wait() to obtain information about the Child. As long as a Zombie is not removed from the system via a wait(), it will consume a slot in the kernel Process Table, and if this table fills, it will not be possible to create further Processes. If a Parent Process terminates, then its Zombie Children (if any) are adopted by init(), which automatically performs a wait() to remove the Zombies.

Note: Every Process spawned after init has a Parent Process

    • Each Parent can have many Children and those can have their own Children
    • Zombie Processes are eventually “adopted” by init, which will routinely�wait() on and Reap any Zombie Children

CS446/646 C. Papachristos

A Process Tree

Note: Usual convention is that the Parent that forked the Children waits on�them, because typically they are doing part of the job it was expected to do.

53 of 56

Process-related System Calls

Waiting on a Process

The purpose of the wait() Syscall is to allow the Child Process to report a status back to the Parent Process

  • Child Process is not completely cleaned up when it exit()s�It “dies” as a Running Process, most Resources are released, but it still remains in the Process Table
    • That's where its Exit Status is stored, so that the Parent can retrieve it by calling one of the wait() variants
  • It will remain & keep consuming that Process Table slot until it is “Reaped” (by being wait()ed on)

  • Unix: Every Process must be “Reaped” by a Parent

Orphan Process:When a Parent Process dies before a Child Process

  • In this case, the Kernel knows that it's not going to get a wait() call
  • The Kernel will immediately put Orphan Processes under the care of init, which will routinely wait on and Reap them eventually
    • Try it:

CS446/646 C. Papachristos

Note: Usual convention is that the Parent that forked the Children waits on�them, because typically they are doing part of the job it was expected to do.

ping -D localhost >/dev/null 2>&1 & # Terminal 1: Start looping ping as daemon (background)

pstree -p # Terminal 2: Find PID of ping’s shell parent process

kill –SIGTERM [PING_PARENT_SHELL_PID] # Terminal 2: Kill ping’s shell parent process (T1 dies)

pstree –p # see how ping has been reparented to systemd (init)

54 of 56

Process-related System Calls

Process Miscellanea

  • A Process can be kill()ed
    • Not just by a Parent Process
    • Any Process running under the same User ID or as the root User can kill() any other Process

    • Terminal command sequence Ctrl-C can be used to send SIGINT Signal to the active Process (in that terminal)
      • Will (by default – unless otherwise handled) Terminate that Process
    • Terminal command sequence Ctrl-Z can be used to send SIGTSTP Signal to the active Process (in that terminal)
      • Will (by default – unless otherwise handled) Stop (/Pause) that Process and move it to Background
      • Command jobs can be used to see such stopped (alive but not Running) background Processes
      • Command fg in shell will Resume it in Foreground, bg will resume it in Background

  • When a Parent Process is kill()ed, it doesn’t by default kill() the Children Processes
    • You can do that by kill()ing a Process Group (Process Group: Way to manage SIGNAL delivery to a set of Processes)
      • use with (negative) Process Group ID (PGID)
      • Or use killpg()

    • As mentioned if the Parent Process is kill()ed first, it will leave Orphan Processes out of any Children Processes

CS446/646 C. Papachristos

55 of 56

Next Lecture Reading Preparation

Operating Systems Three Easy Pieces (https://pages.cs.wisc.edu/~remzi/OSTEP/)

Intro

  • 2. Introduction
    • 2.3 Concurrency

Concurrency

  • 26. Concurrency and Threads
    • Beginning of Chapter
    • 26.1 Why Use Threads?
    • 26.2 An Example: Thread Creation
  • 27. Thread API
    • Beginning of Chapter
    • 27.1 Thread Creation
    • 27.2 Thread Completion
    • 27.5 Compiling and Running

CS446/646 C. Papachristos

56 of 56

Time for Questions !

CS-446/646

CS446/646 C. Papachristos