Tuesday, November 25, 2008

Reading for Lecture Nov 25

A Policy-aware Switching Layer for Data Centers

In today’s data centers, a variety of middleboxes are deployed to protect, manage and improve the performance of the applications and services they run. Administrators often have to overload layer-2 path selection mechanisms to coerce traffic through the desired sequence of middleboxes placed on the network path. This process is inflexible and prone to misconfiguration.

The Player is a policy-aware switching layer consisting of inter-connected policy aware switches or pswitches. Based on policies specified by administrators at a centralized controller, pswitches explicitly forward different types of traffic through different sequences of middleboxes.

A pswitch consists of two independent parts - the Switch core and the policy core. The switch core provides the forwarding functionability of standard Ethernet switch. The Policy core redirects frames to the middle boxes dictated by policy. The policy core redirects a frame by encapsulating it in a new Ethernet-II frame , identified by a new EtherTupe code.

Only switch connecting to the external networks requiring middlebox traversal guarantees are plugged in, need to be converted to pswitches. Other switches need not be converted if they can be configured or modified to treat encapsulated frames with the new EtherType as regular Ethernet frames.

Improving MapReduce Performance In Heterogeneous Environments

Hadoop is an open source implementation of MapReduce enjoying wide option and is often used for short jobs where low response time is critical. Hadoop’s performance is closely tied to its task scheduler, which implicitly assumes that cluster nodes are homogeneous and tasks make process linearly.

The above assumptions in Hadoop are used to run a speculative copy of a task that is performing poorly on another machine so that the task can finish faster. Hadoop’s scheduler start a speculative task based on a simple heuristic comparing each task’s progress to the average progress. This heuristic works well in homogeneous environments but can lead to severe performance degradation in other heterogeneous environments.

The LATE scheduler estimate the progress rate of each task as ProgressScore/T where T is the amount of time the task has been running for. Then the finish time estimate is (1 – ProgressScore)/ ProgressRate. This assumes that tasks make progress at a roughly constant rate.

The advantage of LATE is that it robust to node heterogeneity because it relaunches only the slowest task and just a number of tasks. LATE prioritizes between slowest tasks based on how much they hurt the task response time. It also caps the number of speculative tasks to limit the contention and share resources.

No comments: