NSF: CSR: Medium:
Limiting Manipulation in Data Centers and the Cloud

P.I.s
Ion Stoica, ICSI and EECS UC Berkeley
Eric Friedman, ICSI and EECS UC Berkeley
Ali Ghodsi, ICSI and EECS UC Berkeley

Graduate Students
Peter Bailis, EECS UC Berkeley

Arka Bhattacharya, EECS UC Berkeley

Alex Psomas, EECS UC Berkeley

Summary

In recent years, datacenters and clouds have become the main compute platform for many large scale corporations. Petabyte-scale datasets are stored throughout the datacenter and different jobs are scheduled to collect business intelligence, gather statistics, or to compute essential data, such as a large scale index or a list of top users or their message posts. How- ever, a significant challenge to these centers arises from user manipulation (both intentional and unintentional) of the underlying resource allocation mechanisms through misreporting of true job characteristics.

This project will develop new methods of analysis and implement new mechanisms to reduce the manipulability of these mechanisms. First, it will extend the analysis of resource allocation protocols which prevent manipulation in resource allocation. Recent results have developed non-manipulable allocation mechanisms under the assumption of continuously divisible resources. This project will develop fine grained extensions of these mechanisms to deal with the discreteness issues in real data centers and clouds. Second, it will study the allocation of machines in clouds, as in both private clouds (e.g. Facebook cluster) and public clouds (e.g. Amazon EC2). Recent work has suggested abstractions for reducing manipula- tion and this project will develop practical algorithms that implement this abstraction. Lastly, this project will apply recent results in algorithmic mechanism design and economics to de- velop general procedures for converting current manipulable protocols into non-manipulable ones, often freeing designers from having to explicitly build manipulation limiting features into their mechanisms.

 

Publications