Tech Talk Video: Prof. John Ousterhout on RAMCloud

A couple weeks ago, Professor John Ousterhout visited LinkedIn to give a tech talk called "RAMCloud: Scalable High-Performance Storage Entirely in DRAM". In case you missed it, you can check out the recorded version below.

Video

Abstract

The performance of disk based systems has not kept pace with the ever increasing scale of modern web applications. Developers have to rely on aggressive caching which creates its own set of problems. In his talk, Prof Ousterhout talked about an entirely new approach of dealing with this problem: shift the focus on online data storage from disk to DRAM. While DRAM is still more expensive than disk, for lot of online data, DRAM is cheap enough to be practical.

RAMCloud is a new general purpose storage system being developed at Stanford that keeps all data in DRAM and uses disk only for backup. It is built to scale to 1000+ servers holding 100+ TB data. While each DRAM based server is able to deliver much higher throughput than a disk based one, the biggest impact comes from the latency, which is many orders of magnitude lower. For web applications, this translates into ability to perform many more lookups per page, which means a richer user experience. During the rest of his talk, Prof. Ousterhout explained how crash recovery is done in RAMCloud which has some very innovative ways of speeding up recovery while keeping the memory footprint as low as possible.