ABSTRACT

The emergence of large-scale web services, smartphones, social media, and high throughput scientific instruments within the last decade or so has resulted in a deluge of data. For example, Facebook has an incoming daily data rate of 600 TB with data warehouses storing upwards of 300 PB of data [1]. The amount of data generated by DNA sequencing centers is many terabytes every day with the amount doubling every 7 months [2]. The data problem is expected to worsen with the roll out of the Internet-of-Things (IoT) [3]. A key feature of IoT is equipping physical infrastructure with sensing and information transmission capabilities with the goal of realizing intelligent transportation, energy management, environmental monitoring, manufacturing, and health care systems.