Spyglass: Metadata Search for Large-Scale Storage Systems

Appeared in ;login: — The USENIX Magazine 34(3).

Abstract

Are our growIng storage systems a blessing or a curse? As storage systems expand to billions of files they become increasingly difficult to manage. Effective management requires quickly searching the metadata of the files being stored. Unfortunately, tools such as find and grep and off-the-shelf databases cannot easily scale to billions of files. Spyglass is a new approach to metadata search, which leverages file and query properties to improve performance and functionality and to allow people to focus on using rather than merely managing their data.

Publication date:
June 2009

Authors:
Andrew Leung
Minglong Shao
Timothy Bisson
Shankar Pasupathy
Ethan L. Miller

Projects:
Scalable File System Indexing
Ultra-Large Scale Storage

Available media

Full paper text: PDF

Bibtex entry

@article{leung-login09,
  author       = {Andrew Leung and Minglong Shao and Timothy Bisson and Shankar Pasupathy and Ethan L. Miller},
  title        = {Spyglass: Metadata Search for Large-Scale Storage Systems},
  journal      = {;login: — The USENIX Magazine},
  volume       = {34},
  number       = {3},
  month        = jun,
  year         = {2009},
}