CS 423 – Operating Systems Design

Lecture 4 – UNIX History

and Architecture

Klara Nahrstedt/Sam King

Fall 2007

CS 423 - Fall 2007

Overview
 Administrative Issues
 History of UNIX (T.10.1)
 Overview of UNIX (T.10.2)

CS 423 - Fall 2007

Administrative
 MP1 posted, deadline September 10, 8am

CS 423 - Fall 2007

UNICS
 1940-1950 – one person machines – huge
 1960 – batch systems
 Time-consuming process of entering punch card, debugging,
etc.
 Timesharing – invented at Dartmouth and MIT
 MIT system – CTSS
 MIT&Bell Labs&GE -> MULTICS (MULTiplexed Information and
Computing Service)
 Bell Labs pulled out of MULTICS, Ken Thompson stripped
MULTICS on PDP-7
 UNICS (UNiplexed Information and Computing Service)
 Name later changed to UNIX

CS 423 - Fall 2007

PDP-11 UNIX
 1970s - UNIX moved to PDP-11/20 and then PDP-11/45
and PDP-11/70 (powerful machines at the time – 256K
and 2 MB main memory space, 16-bit machines)
 Memory hardware protection, hence multiple users could be run
 Limitation: individual processes were limited to 64KB instruction
and data space
 Problem: language in which UNIX was written
 Thompson rewrites UNIX in high-level language, called B
(simplified version of BCPL)
 BCPL (Basic Combined Programming Language), developed by
Martin Richards at U. Cambridge in 1966
 B – lack of structures – attempt unsuccessful
 Ritchie designed successor to B – language called C and
compiler
 Thompson and Ritchie rewrote UNIX in C

CS 423 - Fall 2007

PDP-11 UNIX
 1974 – landmark paper by Ritchie and
Thompson about UNIX (for this paper they got
ACM Turing Award)
 PDP-11 was computer of choice at nearly all
universities at the time and UNIX got deployed
quickly
 Version 7 UNIX was the first portable version of
UNIX
 1980s – UNIX was widely spread on
minicomputers and engineering workstations

CS 423 - Fall 2007

Portable UNIX
 First port beyond PDP-11 was to Interdata 8/32
minicomputer
 Porting required:
 write a C compiler for the new machine,
 write device drivers for the new machine’s I/O devices such as
printers, terminals, disks
 Small amount of machine-dependent code such as the interrupt
handlers, memory management routines must be rewritten,
usually in assembly code
 Problem: Ritchie’s compiler – good, but produced PDP-
11 object code.
 Steve Johnson at Bell Labs implemented portable C compiler,
called also Johnson’s compiler
 1984 – AT&T released UNIX as a commercial product,
first System III, then System V.
CS 423 - Fall 2007
Berkeley UNIX
 UNIX Version 6
 First produced on PDP-11, called 1BSD (First Berkeley Software
Distribution)
 Most famous - 4BSD for VAX (Similar to AT&T Version 7)
 Virtual memory, paging
 File names longer than 14 characters
 File system implementation changed (faster)
 Signal handling more reliable
 Introduction of networking, using TCP/IP
 Berkeley UNIX added substantial number of utility
programs
 Editor – vi
 Shell – csh
 Pascal and Lisp compilers
 Last Berkeley version – 4.4BSD – later FreeBSD
CS 423 - Fall 2007
Standard UNIX
 Late 1980s – two version of UNIX
 4.3BSD
 System V Release 3
 Standardizing Efforts
 Initial efforts failed
 Successful effort – POSIX (Portable Operating System)
 POSIX committee produced standard 1003.1
 Set of library procedures – open, read, fork, …
 Committee took intersection of System V and BSD
 Industrial Split
 IBM, DEC, HP – consortium OSF (Open Software Foundation) met
IEEE standard and included also windowing system X11, graphical
user interface MOTIF, distributed computing DCE and others
 AT&T – consortium UI (UNIX International) was based on System V
 Multiple UNIX implementations: HP-UX, AIX, Solaris
CS 423 - Fall 2007
MINIX
 1987 - New UNIX-like system
 Small enough and open with source code available
 Functionally almost equivalent to Version 7 UNIX
 Based on micro-kernel design, i.e., minimal
functionality in the kernel to make it reliable and
efficient
 Memory management and file system in user processes
 Easy to understand
 Easy to maintain due to their highly modular structure
 Other Micro-kernels
 Mach – CMU – 1986
 ChorusOS – INRIA - 1988 (for embedded systems)
CS 423 - Fall 2007
LINUX
 1991 - Linus Torvalds – Finland
 UNIX clone – Linux
 Linux 0.01
 came from MINIX ideas, ranging from structure of the source tree to
the layout of the file system
 Monolithic (not micro-kernel design)
 Originally run on 386 architecture
 1994 – Linux 1.0
 New file system, memory-mapped files, BSD-like networking
with sockets and TCP/IP
 New device drivers
 1996 – Linux 2.0
 Support for 64-bit architectures, symmetric multiprogramming,
new networking protocols
 Business Model – free software
 80% of ca. 150 Linux system calls are exact copies of the
corresponding system calls in POSIX, BSD or System V.
CS 423 - Fall 2007
Linux Version 2.6
 Compliant with IEEE POSIX standard
 All features of a modern UNIX OS
 Virtual memory, virtual file system, lightweight processes, Unix
signals, SVR4 (System V, Release 4) inter-process
communications, support for Symmetric Multiprocessors (SMP)
 Monolithic kernel
 Most commercial UNIX variants are monolithic
 Exceptions: Apple MAC OS X and GNU Hurd OS (both derived
from CMU Mach OS) follow microkernel approach
 Support for modules on demand – automatic load and
unload of modules on demand
 Only SVR4.2 and Solaris kernel have similar feature
 Kernel threading
 Kernel organized as a set of kernel threads
 Only Solaris and SVR4.2 are organized like this
CS 423 - Fall 2007
Linux Version 2.6
 Multithreaded application support
 Multi-threaded user application could be composed of many light-weight
processes (LWP) which are processes that can operate in common
address space, common physical memory pages, common opened
files, etc.
 Linux defines its own LWP different from SVR4 and Solaris
 Preemptive kernel
 If kernel 2.6 is compiled with “preemptible kernel” option, kernel can
arbitrarily interleave execution flows while they are in privileged mode
 Only Solaris and Mach 3.0 have preemptible kernels
 Multi-processor support
 Support of SMP for different memory models
 File System
 Linux has several standard file systems
 Old Ext2 file system (if no specific needs)
 Ext3 file system (if one wants to avoid lengthy file system checks after
system crash)
 ReiserFS file system (for small files)
 STREAMS
 No analog to STREAMS I/O subsystem introduced by SVR4
CS 423 - Fall 2007
UNIX Overview
 Designed as an interactive OS
 Handles multiple processes and multiple
users at the same time
 Designed by programmers and for
programmers with assumptions that
 Majority of users are relatively sophisticated
 Majority of users are engaged in software
development

CS 423 - Fall 2007

What do good programmers want
from a system?
 Simple, elegant, consistent
 Example
 Simple format: File at the lowest level as a collection
of bytes
 Having different classes of files for sequential access,
random access, keyed access, remote access, etc just gets
in the way (this is the approach mainframes take)
 Consistent commands:
 ls A* means list files beginning with ‘A’

 rm A* means remove files beginning with ‘A’

 Principle of least surprise

CS 423 - Fall 2007

What do good programmers want
from a system?
 Power and Flexibility
 Small number of basic elements that can be
combined in an infinite variety of ways to suit the
application
 Every program should do just one thing and do it well
 Remove useless redundancy
 Example
 Why type copy when cp is enough?

 Efficient extracting all lines containing the string “ard” from

file f
> grep ard f

CS 423 - Fall 2007

UNIX Shell
 Graphical User Interface at MAC and Windows
 UNIX – command line interface, called shell
 UNIX Graphical environment – X Windows
 Bourne shell (sh) Protocol
 Shell starts up and initializes itself
 Shell types a prompt character (percent or dollar sign) and waits
for user to type a command line
 User types a command line
 Shell extracts first word and assumes the name of a program to
be run
 Shell searches for this program
 If it finds it, runs the program
 Shell suspends itself until the program terminates
 Shell waits for another command
CS 423 - Fall 2007
UNIX Shell
 Shell is an ordinary user program – needs the ability to
read from and write to the terminal, and execute other
programs
 Commands may take arguments, which are passed to
the called program as character strings
 Example
 cp src dest
 Shell accepts magic characters, called wild cards
 Example: ls *.c
 Shell does not have to open terminal, but it has access
automatically to a file ‘standard input’ (for reading), file
‘standard output’ (for writing normal output), and file
‘standard error’ (for writing error messages)
 Shell can redirect standard input/output to files
 Example: sort <in >out
CS 423 - Fall 2007
UNIX Shell
 Pipe symbol
 Example: grep ter *.t | sort >out
 Possible pipeline – sequence of pipe symbols

 Single user can run several programs at once

 Shellsyntax for running a program in the background
– use &
 Example: wc –l <a >b &

 Shell scripts – files containing shell commands

CS 423 - Fall 2007

UNIX Interfaces
The layers of a UNIX system.
User
Interface

CS 423 - Fall 2007

UNIX Interfaces
 There are really 2 means of connecting to UNIX
computers :
 You can be sitting in front of a dual-bootable PC that you
have booted into Linux and logged onto. All of your
commands are then being run locally on that computer.
When you logon in this manner you have a full GUI
environment.

 You can connect remotely to one of the UNIX servers

(whether from home or at the labs). This is often how
your projects are suggested to be developed as they are
graded on those same servers. When you logon in this
manner you have a command line (or text based)
environment. You can also open up a command line on
local lab machines as well.
Graphical User Interfaces
(GUIs)
 When you logon locally, you are presented with
graphical environment.
 You start at a graphical login screen. You must
enter your username and password. You also
the have the option to choose from a couple
session types. Mainly you have the choice
between Gnome and KDE.
 Once you enter in your username and
password, you are then presented with a
graphical environment that looks like one of the
following...
Gnome
KDE
Command Line Interface
 You also have access to some UNIX servers as
well.
 You can logon from virtually any computer that has
internet access whether it be Windows, Mac, or UNIX
itself.
 In this case you are communicating through a
local terminal to one of these remote servers.
 All of the commands actually execute on the remote
server.
 It is also possible to open up graphical applications
through this window, but that requires a good bit more
setup and software. (Time permitting, we may cover
how to do this.)
The Terminal
Common UNIX Utilities
Functional List
 Files and Directories
 Information
 Mail
 Miscellaneous
 Printing
 Programming Tools
 Shells, Job Control and Access
 Terminal Related
 Text Formatting and Editing
 User Information and Communications

CS 423 - Fall 2007

Files and Directories
 ar - create library archives, and add or extract files
 basename - display portions of pathnames and filenames
 cat - concatenate and display
 cd - change working directory
 chgrp - change the group ownership of a file
 chmod - change the permissions mode of a file

Information
 uptime - show how long the system has been up
 whatis - describes what function a command performs
 whereis - locates source, binary, or manual for program
 which - locates a program file, including aliases and paths

CS 423 - Fall 2007

Mail
 biff - give notice of incoming mail messages
 mail, Mail - read or send mail messages
 vacation - reply to mail automatically

Miscellaneous
 banner - display a string in large letters
 cal - display a calendar
 calendar - a simple reminder service
 date - display or set the date
 join - relational database operator
 echo - echo arguments to the standard output

CS 423 - Fall 2007

 Printing
 cancel - cancel requests to a printer
 lp - send requests to a printer
 lpq - display the queue of printer jobs
 lpr - send a job to the printer
 lprm - remove jobs from the printer queue
 pr - prepare files for printing, perhaps in multiple columns

Programming Tools
 ar - create library archives, and add or extract files
 cb - a simple C program beautifier
 cc - C compiler
 cpp - the C language preprocessor

CS 423 - Fall 2007

Shells, Job Control and Access
 at - execute a command or script at a specified time
 batch - runs jobs when the system load level permits
 expr - evaluate arguments as a logical, arithmetic, or string
expression
 false - provide truth values (see true also)
 kill - send a signal to a process, or terminate a process
 login - log in to the system

Terminal Related
 clear - clear the terminal screen
 mesg - permit or deny messages on the terminal
 stty - sets, resets, and reports workstation operating parameters

CS 423 - Fall 2007

Text Formatting and Editing
 ctags - create a tags file for use with ex and vi
 edit - provides a simple line editor for the new user
 ex - edits lines interactively, with a screen display

User Information and Communication

 finger - display information about users
 groups - display a user's group memberships
 last - indicate last logins by user or terminal
 rwall - write to all users over a network
 rwho - who's logged in on local machines

CS 423 - Fall 2007

UNIX Utility Programs

CS 423 - Fall 2007

Traditional
Unix kernel

CS 423 - Fall 2007

Flexibility
 Traditional Kernel: file, scheduling,
executable file formats
Modern Unix system
UNIX Kernel

Approximate structure of generic UNIX kernel

CS 423 - Fall 2007
Kernel Structure
 All UNIX drivers are classified as either
character device drivers (mouse,
keyboard) or block device drivers (disk)
 Network device drivers (possible character
devices, but handled very differently)
 Process dispatching – when interrupt
happens, context switch happens between
processes

CS 423 - Fall 2007

Kernel Structure
 Editors such as vi/emacs need raw tty
 Shell is line oriented, hence uses cooked mode
and line disciplines
 Networking software is layered with
MAC/Routing/Transport layers and Socket
interface
 On top of buffer cache sits file system
 Most UNIX systems support multiple file systems
 Berkeley fast file system

 Log-structured file system

 Various System V file systems

CS 423 - Fall 2007

Kernel Structure
 On top of file system comes file naming, directory
management, hard link and symbolic link management,
etc
 On top page cache sits virtual memory (VM)
 All paging logic is here, such as replacement algorithm
 On top of VM is mapping files onto VM and high level
page fault management code
 Figure out what to do when a page fault occurs
 On top of process dispatching is process management
 Process scheduler to choose which process to run next
 If threads are managed in the kernel, thread management is also
here
 Note in some UNIX systems, threads are managed in user space
 On top of scheduling we have signal processing and process
creation and termination
CS 423 - Fall 2007
Kernel Structure
 Top layer has the system call interface
 Allsystem calls are directed to one of the
lower modules, depending on the nature of
the call
 Top layer also has entrance to interrupts
and traps
 Entrance to signals, page faults, processor
exceptions of all kinds, I/O interrupts

CS 423 - Fall 2007

Summary
 Most UNIX kernels are monolithic: each kernel layer is integrated
into the whole kernel program and runs in Kernel Mode on behalf of
current process
 Some kernels use micro-kernel approach
 Very small set of functions from the kernel
 Few synchronization primitives
 Simple scheduler
 Inter-process communication mechanism
 Several system processes run on top of micro-kernel
 Memory allocators, device drivers, system call handlers
 Disadvantage: Microkernels are generally slower than monolithic
ones due to explicit message passing between different layers of the
OS
 Advantage: microkernels force system programmers to adopt a
modularized approach since OS layer is relatively independent
program that must interact with the other layers through well-defined
and clean software interfaces
 Advantage: Microkernels are easily ported to other architectures
CS 423 - Fall 2007
Summary
 Linux – wants to achieve theoretical advantages
of micro-kernels without introducing
performance penalties
 Linux kernel offers modules
 Module is an object file whose code can be
linked to kernel at runtime
 Advantages of modules:
 Modularized approach
 Platform independence
 Frugalmemory usage
 No performance penalty

CS 423 - Fall 2007