Hijack: Taking Control of COTS Systems for Real ... - Computer Science

1 downloads 0 Views 234KB Size Report

Apr 5, 2007 - provide many advantages for RT/Embedded systems ... QoS, predictability, policies absent for satisfying ... on a 2.4 GHz Pentium 4 processor.

Hijack: Taking Control of COTS Systems for Real-Time User-Level Services Gabriel Parmer and Richard West Computer Science Deparment Boston University Boston, MA 02215 {gabep1, richwest}@cs.bu.edu

April 5, 2007

COTS in RT/Embedded Systems

Commodity Off The Shelf (COTS) general purpose systems provide many advantages for RT/Embedded systems Tested and widely deployed code-base Established development tools/environments Developer familiarity → faster time to market/smaller development costs

Parmer, West, BU CS

Hijack

2/33

COTS in RT/Embedded Systems (2)

General purpose systems have a number of disadvantages General-purpose policies are often insufficient/awkward for needs of RT applications QoS, predictability, policies absent for satisfying app-specific requirements, i.e. EDF Semantic gap between the requirements of the application and the functionality/guarantees of the system

Parmer, West, BU CS

Hijack

3/33

Shrinking the Semantic Gap

Domain-specific OSs created with a focus on one class of applications (RTOSs) Extensible systems allow the modification of system policies in an application-specific manner Generally either not COTS, or not isolation preserving Developing extensions requires skill/experience Goal: provide app-specific policies using a COTS base in a safe and predictable manner

Parmer, West, BU CS

Hijack

4/33

Hijacking your COTS system

Efficient interposition on service requests from specific applications allows the definition at user-level of application-specific policy

Parmer, West, BU CS

Hijack

5/33

Hijack Mechanism Hijack execution environment Guest

Guest

... Background process

Executive

Schedule / dispatch Syscall interception Unintercepted syscalls

IDT

Host Kernel

Kernel module Interrupts

Hijack module receives specific events system calls page faults possibly device interrupts Vector guest service requests to executive

Hardware (I/O devices)

executive controls execution context of guests create/switch address spaces access guest registers event-triggered executive scheduler Parmer, West, BU CS

Hijack

6/33

Hijack Mechanism (2) Hijack execution environment Guest

Guest

... Background process

Executive

Schedule / dispatch Syscall interception Unintercepted syscalls

IDT

Host Kernel

Kernel module Interrupts

executive isolated at user-level executive harnesses base system functionality where appropriate

Hardware (I/O devices)

Does not require changes to the COTS system source-code (no kernel recompilation) One (2000 LOC) hijack module enables flexibility in the definition of user-level app-specific services Parmer, West, BU CS

Hijack

7/33

Case Study: Guest System Call Interposition 1 Guest

Guest

syscall

...

2

saved guest state

Executive executive state (to be restored)

Kernel module

3

4 Host Kernel

5

guest service request intercepted by Hijack module executive region mapped into current guest address space guest registers saved into executive region executive registers restored executive executed

executive not present while guest is executing – mapped in dynamically executive isolated from guests Parmer, West, BU CS

Hijack

8/33

Case Study: Guest System Call Return 1

Guest

Guest

2

...

3 saved guest state (to be restored)

saved executive state

Kernel module

Executive

4

Host Kernel

5

6

executive returns to kernel module executive registers saved in module guest registers restored from executive region executive region unmapped from guest address space executive’s mappings evicted from TLB guest executed

Can use global bits to avoid flushing guest pages from TLB set all guest pages as global Parmer, West, BU CS

Hijack

9/33

Experimental Setup

All experiments conducted on a 2.4 GHz Pentium 4 processor on Linux 2.6.13 with a clock tick every 10 milliseconds

Parmer, West, BU CS

Hijack

10/33

nanosleep Experiments

A goal of Hijack is to offer the ability to enhance default system functionality in an application-specific manner nanosleep: yield for at least a specific number of nanoseconds used in multimedia apps such as mplayer

Wake up time variability/unpredictability clock granularity COTS CPU scheduler

Parmer, West, BU CS

Hijack

11/33

nanosleep Experiments (2)

Hijack-provided extensions: 1 Hijack: Executive can give scheduler preference to tasks waking from nanosleep 2 Hijack Extended: Executive can busy wait for periods less than a clock tick

Parmer, West, BU CS

Hijack

12/33

nanosleep Experiments (3) 100000

Jitter (Tens of Microseconds)

Hijack Linux Task Hijack Extended 10000

1000

100

10

1

0

1

2

3

4

Number of Background CPU Bound Tasks Parmer, West, BU CS

Hijack

13/33

QoS for Packet Stream Delivery Scheduling of Tasks dependent on I/O availability with QoS constraints: models traffic shapers, QoS aware stream processing, etc. . . Four streams of 42,000 16 byte packets/second from separate hosts over GigE Single host with four tasks, each receiving a stream QoS constraints: Task 0: 35,000 p/s

higher QoS

Task 1: 20,000 p/s Task 2: 10,000 p/s



lower QoS

Task 3: best effort

Start tasks every 5 seconds from Task 3 to Task 0 Parmer, West, BU CS

Hijack

14/33

QoS for Packet Stream Delivery (2) Three scenarios: 1 Linux, tasks with same priority 2 Linux, tasks with different priority 3 Hijack, Executive using policy similar to proportional-share Tasks assigned tokens proportional to QoS select used to probe for I/O activity Task with tokens and available I/O executed Tokens refreshed every given period When guest make system call to read data read data into guest buffer until no tokens, or no data

Parmer, West, BU CS

Hijack

15/33

Number of packets delivered to a task

Packet Delivery QoS Results: Linux Same Priority 45000 40000 35000 30000 25000 20000 15000 Task 0 Task 1 Task 2 Task 3

10000 5000 0

0

5

10

15

20

25

30

Time (seconds) Parmer, West, BU CS

Hijack

16/33

Number of packets delivered to a task

Packet Delivery QoS Results: Linux Increasing Priority 45000 40000 35000 30000 25000 20000 15000 Task 0 Task 1 Task 2 Task 3

10000 5000 0

0

5

10

15

20

25

30

Time (seconds) Parmer, West, BU CS

Hijack

17/33

Number of packets delivered to a task

Packet Delivery QoS Results: Hijacked Linux 45000 40000 35000 30000 25000 20000 15000 Task 0 Task 1 Task 2 Task 3

10000 5000 0

0

5

10

15

20

25

30

Time (seconds) Parmer, West, BU CS

Hijack

18/33

Related Work Related work includes: RTLinux Separate system into two functional domains for Hard-RT predictability Focus is on interrupt latency, not app-specific resource management policies VMs Interface provided to guest OSs (executives) is identical to the hardware itself Focus is on HW virtualization, not on providing app-specific services

Parmer, West, BU CS

Hijack

19/33

Conclusions

Hijack enables app-specific, user-level RT policies using a general purpose computing base Use interposition on system service requests to redefine policies executive defined at user-level can leverage underlying system functionality where appropriate Demonstrated that complex policies can be introduced A useful approach towards shrinking the semantic gap

Parmer, West, BU CS

Hijack

20/33

Limitations

global bit trick not ideal for all workloads can revert to simply flushing whole TLB or use other techniques

Certain aspects of the system that cannot be hijacked using these techniques If utilize functionality in base system, generally cannot Hijack that functionality COTS system interrupt handling behavior (prototype limitation)

Parmer, West, BU CS

Hijack

22/33

Using Global-bit Trick to Avoid TLB Flushes Study the effect of TLB flushes on Executive ↔ Guest communication

Parmer, West, BU CS

350

Hijack Guest -> Executive RPC Linux Pipe System Call

300 250 # iTLB Misses

Vary working set size (WSS) of guest by touching data/instruction pages then making system call instruction-TLB has 128 entries data-TLB has 64 entries Global-bit trick avoids TLB flush, thus avoiding misses

200 150 100 50 0

Hijack

0

50

100

150 200 Instruction WSS

250

300

23/33

Using the Global-bit Trick to Avoid TLB Flushes (2) 35000

30000

Hijack Guest -> Executive RPC Linux Pipe

30000

20000

20000

Cycles

Cycles

25000

15000

15000 10000

10000

5000

5000 0

Hijack Guest -> Executive RPC Linux Pipe

25000

0

Parmer, West, BU CS

50

100

150 Data WSS

200

250

300

0

Hijack

0

50

100

150 200 Instruction WSS

250

300

24/33

Timer interrupts in Executive synthesized with signals Predictable notification Executive can define customizable policy for scheduling beyond what is present in the COTS system (EDF, PFAIR, DWCS, etc. . . )

Parmer, West, BU CS

Average Signal Interarrival Time (milliseconds)

Asynchronous Event Notification Experiments

30.0

Hijack Linux Task

25.0

20.0

15.0

10.0

5.0

0.0

0

1

2

3

4

Number of Background CPU Bound Tasks

Hijack

25/33

Hijack Execution Environment Address Space

sigaltstack

read-writable

4KB guard page executive stack executive

4KB guard page signal_handler

Parmer, West, BU CS

0x3FC00000 read-only

Hijack

26/33

QoS Expts. Executive Algorithm main_event_loop () { next = NULL; select on the file descriptors for each task; if (timing period has expired) for (each task in tasks) curr_tokens(task) = init_tokens(task); for (each task in tasks) if (select indicated that task has data && curr_tokens(task) > 0) { next = task; break; } if (next == NULL) next = best_effort_task; execute next; } Parmer, West, BU CS

Hijack

27/33

QoS Expts. Executive Algorithm (2)

guest_syscall_read(guest_fd, guest_buf, guest_size) { fd = translate_to_host_fd(guest_fd); loop until (read doesn’t return data || curr_tokens(task) == 0) { read(fd, guest_buf, guest_size); //nonblocking curr_tokens(task)--; } }

Parmer, West, BU CS

Hijack

28/33

Max. Jitter QoS Results: Linux Same Priority

Maximum stream jitter (cycles)

1e+09

1e+08

1e+07

1e+06

100000

Task 0 Task 1 Task 2 Task 3 0

5

10

15

20

25

30

Time (seconds) Parmer, West, BU CS

Hijack

29/33

Max. Jitter QoS Results: Linux Increasing Priority

Maximum stream jitter (cycles)

1e+09

1e+08

1e+07

1e+06

100000

Task 0 Task 1 Task 2 Task 3 0

5

10

15

20

25

30

Time (seconds) Parmer, West, BU CS

Hijack

30/33

Max. Jitter QoS Results: Hijacked Linux

Maximum stream jitter (cycles)

1e+09

1e+08

1e+07

1e+06

100000

Task 0 Task 1 Task 2 Task 3 0

5

10

15

20

25

30

Time (seconds) Parmer, West, BU CS

Hijack

31/33

Suggest Documents