Introduction_print

Introduction_print
9/19/2012
Advanced Operating Systems Course aims and goals
(263‐3800‐00L)
Timothy Roscoe Herbstsemester 2011
http://www.systems.ethz.ch/courses/fall2012/AOS
1
2
© Systems Group | Department of Computer Science | ETH Zürich
Writing real OS code
It’s not Unix
• There’s a limit to how much you can learn about an OS by reading books.
• Solve common OS problems for yourself
The course avoids using a Unix‐like OS:
• Gives a broader outlook on what an OS can be
• Encourages a critical approach to OS designs
• Better understanding of the structure of Unix / Linux / Windows / Android / iOS / …
• Clearer picture of ideas (finally) appearing in older OS designs
– Dealing with concurrency
– Dealing with asynchrony
– Tailoring code to specific hardware
– Architecting complete system of processes
– Or not yet…
3
What is OS research?
4
Key ideas in OS research
We’ll be working with a research OS
• You’ll get a sense of how OS research is done
• The OS we use includes a selection of key ideas in OS research
– Read research papers
– Relate them to implementations
– How to evaluate such ideas
– You get hands‐on experience with the ideas
– We’ll go through the papers on the topic
– We’ll put them into historical context
• See what constitutes an OS research topic
• Many are not well‐known and/or unimplemented in the wider community
– What problems do these ideas solve?
– How are they motivated, where do they come from?
5
6
1
9/19/2012
Why teach non‐Unix OS design?
• This is an interesting time for OS research:
– Multicore
– Heterogeneous processing
– System complexity
– System diversity
Motivation
• The basic assumptions of Unix/Windows/etc. are being re‐evaluated. 7
8
Moore’s law: the free lunch
Intel
Pentium
Intel 486
286
8086
Intel
Pentium 4
The power wall
Intel
Itanium 2
Intel
Pentium II
Intel 386
4004
AMD Athlon 1500 processor
http://www.phys.ncku.edu.tw/~htsu/humor/fry_egg.html
9
10
Source: table from http://download.intel.com/pressroom/kits/events/moores_law_40th/MLTimeline.pdf
The power wall
The memory wall
• Power dissipation = Capacitive load  Voltage2  Frequency
10000
CPU
– Increase in clock frequency  mean more power dissipated  more cooling required
– Decrease in voltage reduces dynamic power
but increase the static power leakage
Operations / µs
1000
• We’ve reached the practical power limit for cooling commodity microprocessors
100
10
1MHz CPU clock, 500ns to access memory in 1985
Memory
– Can’t increase clock frequency without expensive cooling
1
11
12
2
9/19/2012
End of the road for serial hardware
The ILP wall
• ILP = “Instruction level parallelism”
• Implicit parallelism between instructions in 1 thread
• Processor can re‐order and pipeline instructions, split them into microinstructions, do aggressive branch prediction etc.
• Power wall + ILP wall + memory wall = brick wall
– Power wall  can’t clock processors any faster
– Memory wall  for many workloads performance dominated by memory access times
– ILP wall  can’t keep functional units busy while waiting for memory accesses
– Requires hardware safeguards to prevent potential errors from out‐of‐order execution
• Increases execution unit complexity and associated power consumption
• There is also a complexity wall, but chip designers don’t like to talk about it…
– Diminishing returns
• Serial performance acceleration using ILP has stalled
13
Multicore processors
14
Implications for software
The things that would have used this “lost” perf must now be written to use cores/accel
• Multiple processor cores per chip
– This is the future (and present) of computing
Log (seq. perf)
• Most multicore chips so far are shared memory multiprocessors (SMP)
– Single physical address space shared by all processors
– Communication between processors happens through shared variables in memory
– Hardware typically provides cache coherence
Historical 1‐thread perf
gains via improved clock rate and transistors used to extract ILP
#transistors still growing, but delivered as additional cores and accelerators
Year
15
16
And the others?
Trends towards:
• Heterogeneous processing
– Different types of core on a chip
Structure and Logistics
• System complexity
– Devices, interconnects, memory system
• System diversity
– No two systems are alike
– Very different performance tradeoffs
We’ll see this all very shortly…
17
18
3
9/19/2012
ETH courses
Assumptions
(or similar elsewhere)
• Parallel programming
• C Programming
– Concurrency, locks, synchronization, …
– Practical labs are mostly in C. – You will be writing an OS. • Digital Design
– Basics of hardware
• Computer Architecture and Systems Programming
• Computer Architecture
– C and assembly
– Caches, devices, I/O, virtual memory, traps, …
– Need to know about hardware, registers, DMA, – You will be writing a device driver, pager, etc.
• Networks and Operating Systems
– Memory management, protection, scheduling, device drivers…
• Command line development
• Systembau: alternative/complementary
– Our emphasis: microkernels (practical) and advanced/research OS topics (theoretical)
– gcc, emacs/vi, make, etc. – Debugging facilities are primitive!
19
Structure of the course
20
Lectures
Project
• Cover topics related to current
project milestone.
Project
– Key ideas
– History / influences
Lectures
Readings
• Explanation / elaboration of readings
• Also Q&A if required
Lectures
Readings
21
Readings
22
Project
Project
• Most of our material is not (or insufficiently) covered in textbooks
• Readings will be research papers
Lectures
Project
• Build (most of) an OS
Readings
– based on Barrelfish research OS
– PandaBoard hardware
Lectures
Readings
– Seminal papers in the field
– Background to the concepts you will deal with in the weeks’ project milestone
– Context for the material covered in the lecture
23
24
4
9/19/2012
Project Milestones
Evaluation: Project (65%)
Project
• Demonstrate new functionality
– Important to keep up with the schedule!
• Milestone 0: this week
Lectures
– Familiarization, building, booting
Readings
• Each milestone covers one or more research concepts
– E.g. self‐paging, DSLs, multikernel architecture, URPC, …
• Final codebase: complete OS
• Final milestone: documentation for the system
• There are marks for each milestone
• Late milestones incur a penalty
• Beware milestones which build on previous ones
– Don’t slip behind 25
26
Logistics
Evaluation: Exam (35%)
• Date(s)
• Initial milestone is done individually
• Project is undertaken in teams of 2
• Everyone gets a PandaBoard + cables each
– To be determined – sometime in February
• Material – Lectures, readings, and project material
– Take care of them – they are fragile!
• Format
• So:
– Oral exam (schedule will be arranged)
– 15 minutes per student
– Please find a team partner before tomorrow’s session
– You MUST attend tomorrow’s session to sign out your PandaBoard
– If you can’t find a partner, you will be assigned one at the session. • We are reasonable people!
– Test understanding and principles, not fiddly details
27
28
Dates, times, and people
• Lecture:
– Thursday 10:00 ‐ 12:00 in HG F.26.5
Hardware
• Consultations and marking:
– Friday 10:00 ‐ 12:00 in CAB H.57
• People: – Timothy Roscoe, Pravin Shinde
29
30
5
9/19/2012
Hardware: PandaBoard ES
PandaBoard block diagram
Many thanks to ARM Ltd and Texas Instruments for their generosity in providing the hardware! 31
32
The TI OMAP 4460 SoC
USB booting
“USB On‐The‐Go” port:
• Provides power from the host PC
• Host copies code over USB into PandaBoard memory
• Jumps to start of boot image
• No need for network, SD card, etc.
USB OTG port
Reset button
33
34
Serial console
– Very simple console output
– Flash indicator LED
RS‐232 interface
Indicator LEDs
• One byte at a time, bidirectional serial line
• Still the best low‐level console interface for real OS hacking
• First milestone includes:
ARM CPU cores
•
•
•
•
2 x ARM Cortex A9 cores
SMP configuration
We’ll start with just one core
We’ll bring up the other one later
There are, in fact, two more ARM Cortex M3 cores. We’ll ignore these.
(unless you want to play with them in your spare time )
35
36
6
9/19/2012
ARM architecture overview
ARM Processor modes
• Data Sizes:
Mode
Exception modes
• Note: different from Intel/AMD names!
Byte
Halfword
FIQ
Fast interrupt
IRQ
Normal interrupt
Abort
Memory access violations
Undef
Undefined instructions
System
Privileged mode, same registers as user mode
User
User mode (regular processes)
Word
Doubleword
32
64
Description
Supervisor (SVC) Entered on reset or SWI instruction
Privileged modes
– 16 bits: Halfword
– 32 bits: Word
– 64 bits: Doubleword
Each mode has its own stack!
16
8
37
User/
Supervisor
r0
r1
r2
r3
r4
r5
r6
r7
r8
r9
r10
r11
r12
r13 (sp)
r14 (lr)
r15 (pc)
38
ARM register sets ARM condition codes
(all 32 bits)
(some of them)
IRQ
FIQ
Undef
Abort
SVC
Extra (banked) registers
r13 (sp)
r14 (lr)
r8
r9
r10
r11
r12
r13 (sp)
r14 (lr)
r13 (sp)
r14 (lr)
r13 (sp)
r14 (lr)
r13 (sp)
r14 (lr)
spsr
spsr
spsr
spsr
spsr
Flag
Description
N
Negative result
Z
Zero result
C
Carry from ALU
V
Overflow from ALU
I
Disable IRQ
F
Disable FIQ
mode
Current mode (4 bits)
Found in the CPSR/SPSR registers
cpsr
39
40
Exception handling
1. CPSR → SPSRmode
2. Set CPSR bits
– New mode
– Disable interrupts (if appropriate)
3. Return addr → LRmode
4. Vector addr → PC
ARM instruction set
0x0C
0x08
FIQ
IRQ
reserved
Data abort
Prefetch abort
Software interrupt
0x04
Undefined instruction
0x00
Reset
0x1C
0x18
0x14
0x10
• Classic RISC load/store architecture
– ALU operations are register‐register
•
•
•
•
All instructions 32 bits long
Most instructions are conditional
On‐die barrel shifter
LDR
r0,[r1]
Branches are PC‐relative STRNEB r2,[r3,r4]
B
Base address
= 0x00 or 0xFFFF0000
41
<label>
SUB
r0,r1,#5
ADD
r2,r3,r3,LSL#2
ADDEQ
r5,r5,r6
42
7
9/19/2012
What is Barrelfish?
• Open source research operating system
– Fairly influential in research community
OS: Barrelfish
• ETH Zurich + Microsoft Research
• Currently supports:
– 32‐ and 64‐bit x86 AMD/Intel – Intel Single‐chip Cloud Computer
– Intel MIC (Knight’s Ferry)
– ARMv5, XScale, ARMv7
– Beehive (experimental softcore)
43
What things run on it?
•
•
•
•
•
•
HPI
44
44
Why use Barrelfish?
Many microbenchmarks
Parallel benchmarks: Parsec, SPLASH‐2, NAS
Webserver: http://www.barrelfish.org/
Databases: SQLite, PostgreSQL
Openssh, other utils
Virtual machine monitor • Very different to Linux, BSD, MacOS, Windows
– The shock of the new
• Smaller OS
– Easier to understand, easier to hack
• Multikernel approach (see later)
– Simplifies multicore issues
– Linux kernel binary
• Includes lots of other people’s research ideas
• Microsoft Office 2010!
– Good illustration of any concepts
– via Drawbridge
11/11/2011
11/11/2011
HPI
45
45
46
Some of the (non‐original)
ideas in Barrelfish
•
•
•
•
•
•
•
•
•
•
Capabilities for resource management (KeyKOS, seL4)
Minimize shared state (Tornado, K42)
Upcall processor dispatch (Psyche, Sched. Activations)
Push policy into user space (Exokernel, Nemesis)
User‐space RPC decoupled from IPIs (URPC)
Lots of information (Infokernel)
Single‐threaded non‐preemptive kernel per core (K42)
Run drivers in their own domains (µkernels, Xen)
Fast interprocess communication (LRPC, L4)
Specify device registers in a little language (Devil)
Toolchain
We’ll see many of these later in the course
47
48
8
9/19/2012
Toolchain
•
•
•
•
•
C compiler: gcc
Haskell compiler: ghc
Building Barrelfish
$ mkdir ~/build_milestone_0
$ cd ~/build_milestone_0
$ /hake/hake.sh -s ~/barrelfish -a armv7
Huh?
make
Terminal emulator: picocom
Boot utility: usbboot
• Builds a Makefile for Barrelfish in the build_milestone_0 directory, based on the source tree in ~/barrelfish, for just the ARMv7 architecture.
• You can look at ./Makefile to see the result (but it can be very large…)
• We provide a VirtualBox image with all this
• You are welcome to use your own platform
49
What’s this Hake thing?
Building Barrelfish
$
$
$
$
$
50
(for the curious)
Scattered through the source tree are files called “Hakefile”
• Hake concatenates all of these and interprets them as a single, large Haskell expression
• This expression evaluates to the contents of ./Makefile
mkdir ~/build_milestone_0
cd ~/build_milestone_0
/hake/hake.sh -s ~/barrelfish -a armv7
make milestone01
ls -l milestone0_1_image
Why?
• Allows us to build multiple architectures from one source tree at the same time
• Power of a full programming language in expressing configurations
• Still get the efficiency of make for building
• Make builds its own directories, dependencies, and other tools as well.
• Result ends up in the top of the build directory.
51
52
Important advice
• You will be graded on the design of your code
– Does it work?
– Handle corner cases, errors, invalid inputs, etc?
– An operating system runs for a long time. Do you leak memory? Etc. – Have you thought about issues not explicit in the milestones, but important to a real OS?
And finally, some advice
• Not all these criteria can be well‐specified in advance
– Use common‐sense in system design
– You will be graded on issues you can think of and deal with
53
54
9
9/19/2012
Important advice
Important advice
You will be graded on the quality of your report
• You will be examined on the depth of your understanding
– Doxygen and other tools document lines of code, not the design of a system! – Don’t submit generated documentation in place of a report. – Describe the choices you made (and didn’t make)
– Talk about the tradeoffs
– Mention the difficulties and challenges, and how you overcame them.
– Show you understand how to build a system.
– Design principles
– Why do techniques work?
– When do they work (or not)?
– What are the tradeoffs in a problem?
– What factors affect the tradeoffs?
– How might this change? How has it changed?
55
56
Final advice
• This course is a lot of fun, but a lot of work
– and we are aware of this!
• It is important not to fall behind
– If your team is struggling, ask for help.
• If you're good, it's tempting to be clever and cool
– Resist this temptation!
– Get the required work done before freestyling. • We are here to help you!
Good luck and have fun!
57
10
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement