NSF: CSR: Medium:
Limiting Manipulation in Data Centers and the Cloud
Arka Bhattacharya, EECS UC Berkeley
Alex Psomas, EECS UC Berkeley
In recent years, datacenters and clouds have become the main compute platform for many large scale corporations. Petabyte-scale datasets are stored throughout the datacenter and different jobs are scheduled to collect business intelligence, gather statistics, or to compute essential data, such as a large scale index or a list of top users or their message posts. How- ever, a significant challenge to these centers arises from user manipulation (both intentional and unintentional) of the underlying resource allocation mechanisms through misreporting of true job characteristics.
This project will develop new methods of analysis and implement new mechanisms to reduce the manipulability of these mechanisms. First, it will extend the analysis of resource allocation protocols which prevent manipulation in resource allocation. Recent results have developed non-manipulable allocation mechanisms under the assumption of continuously divisible resources. This project will develop fine grained extensions of these mechanisms to deal with the discreteness issues in real data centers and clouds. Second, it will study the allocation of machines in clouds, as in both private clouds (e.g. Facebook cluster) and public clouds (e.g. Amazon EC2). Recent work has suggested abstractions for reducing manipula- tion and this project will develop practical algorithms that implement this abstraction. Lastly, this project will apply recent results in algorithmic mechanism design and economics to de- velop general procedures for converting current manipulable protocols into non-manipulable ones, often freeing designers from having to explicitly build manipulation limiting features into their mechanisms.
1. A. Ghodsi, M. Zaharia, S. Shenker and I. Stoica (2013). Choosy: Max-Min Fair Sharing for Datacenter Jobs with Constraints. EuroSys 2013. Prague, Czech Republic.
2. Arka A. Bhattacharya , David Culler , Eric Friedman , Ali Ghodsi , Scott Shenker , and Ion Stoica (2013). Hierarchical Scheduling for Diverse Datacenter Workloads. SOCC ‘13 Proceedings of the 4th annual Symposium on Cloud Computing. Santa Clara, California, USA.
3. Eric Friedman , Ali Ghodsi, Christos-Alexandros Psomas (2014). Strategyproof Allocation of Discrete Jobs on Multiple Machines. 15th ACM Conference on Economics and Computation ( EC 2014 ). Palo Alto, CA USA.