Change the future

Sunday 2:30 p.m.–3 p.m.

Server Log Analysis with Pandas

Taavi Burns

Audience level:
Intermediate
Category:
Other

Description

Use iPython, matplotlib, and Pandas to slice, dice, and visualize your application's behaviour through its logs.

Abstract

  • Will include live Notebooks and actual server logs!
  • Introduction to iPython Notebook (3 minutes)
    • Code and prose and matpolotlib interleaved
    • matplotlib -pylab inline extension
  • Pandas DataFrame (3 minutes)
    • How indexing works (to filter data)
  • Loading data (including datetime indexes) (3 minutes)
    • You have to get your data into CSV or Python tuples yourself
  • Histograms to show request size distribution shape (3 minutes)
  • Scatterplots to show changes in proportions over time (5 minutes)
    • e.g. Comparing CPU vs request size, before and after a release
  • resample of discrete events to create a line graph (5 minutes)
    • Using percentiles to show service levels. Your slowest requests may not be common, but they're very annoying for users and could cause an unintentional DoS.

Sponsors