Paper Details

Reference:

Xu Zhao, Yongle Zhang, David Lion, Muhammad FaizanUllah, Yu Luo, Ding Yuan, and Michael Stumm,
"lprof : A non-intrusive request flow profiler for distributed systems",
In Proceedings of the 11th USENIX conference on Operating Systems Design and Implementation (OSDI'14) Proceedings of the 11th USENIX conference on Operating Systems Design and Implementation (OSDI'14), Broomfield, CO, USA, USENIX Association, October, 2014.

Download:

PDF ; Talk Slides ; Talk

Abstract:

Applications implementing cloud services, such as HDFS, Hadoop YARN, Cassandra, and HBase, are mostly built as distributed systems designed to scale. In order to analyze and debug the performance of these systems effectively and efficiently, it is essential to understand the performance behavior of service requests, both in aggregate and individually.

lprof is a profiling tool that automatically reconstructs the execution flow of each request in a distributed application. In contrast to existing approaches that require instrumentation, lprof infers the request-flow entirely from runtime logs and thus does not require any modifications to source code. lprof first statically analyzes an application's binary code to infer how logs can be parsed so that the dispersed and intertwined log entries can be stitched together and associated to specific individual requests.

We validate lprof using the four widely used distributed services mentioned above. Our evaluation shows lprof's precision in request extraction is 88%, and lprof is helpful in diagnosing 65% of the sampled real-world performance anomalies.

Keywords:

distributed system, nonintrusive request flow profiler, distributed application, distributed service, runtime log, performance behavior , execution flow

Reference Info:

DOI: 10.5555/2685048.2685099
ISBN: 9781931971164
OCLC: 8876979613

BibTeX:

@inproceedings(Zhao-OSDI-14,
    author = {Xu Zhao and Yongle Zhang and David Lion and Muhammad FaizanUllah and Yu Luo and Ding Yuan and Michael Stumm},
    title = {lprof : {A} non-intrusive request flow profiler for distributed systems},
    booktitle = {Proceedings of the 11th USENIX conference on Operating Systems Design and Implementation (\textbf{OSDI'14}) Proceedings of the 11th USENIX conference on Operating Systems Design and Implementation (\textbf{OSDI'14})},
    location = {Broomfield, CO, USA},
    publisher = {USENIX Association},
    month = {October},
    year = {2014},
    doi = {10.5555/2685048.2685099},
    isbn = {9781931971164},
    keywords = {distributed system, nonintrusive request flow profiler, distributed application, distributed service, runtime log, performance behavior , execution flow}
)