Intended Audience

Python programmers with intermediate knowledge.

Tutorial Format

Class

This is a hands-on tutorial. Each topic is introduced through a short lecture. There are numerous exercises for nearly all points of the outline. All source codes are provided on CD before the course. Questions are encouraged throughout the course. There is a comprehensive handout detailing all presented topics including in-depth discussions of all code examples.

Class Size

This tutorials has been successfully given to 70+ people at PyCon 2010. While this might not be the ideal size for a class, it still worked reasonably well.

Requirements

All participants should bring laptops with Linux, Windows, or Mac OS. Python 2.6 or 2.5 as well as an editor or IDE will need to be installed. It seems unlikey that Python 2.7 will work at the time of the tutorial. So far none of the packages listed below is availbale as a release for this version.

The following third-party packages are needed:

• psyco (version 1.5.2 or higher)
• NumPy (version 1.2 or higher)
• pyprocessing (2.5 or lower only)
• Guppy_PE framework

Testscript

A test script will be provided by the instructor. If it runs through without complaining all necessary packages are installed. Otherwise it will prompt to install the missing package(s).

Notes for Reviewers

This tutorial builds on a tutorial I gave from 2007 to 2010 at PyCon. Every year I rework it thoroughly and incorporated suggestions from the tutorial participants. I've alos given this tutorial at PyCon Asia-Pacific 2010 in Singapore as well at OSCON 2010 in Portland.

The tutorial was always very well filled, often the biggest none-Django one. Somtimes pretty close to Django.

There still seems to be demand for it.

Outline for Review

• How Fast is Fast Enough? (5 min)
  • Short intro
  • Development speed vs. execution speed
  • Resources
• Optimization Guidelines (15 min)
  • Premature optimization
  • Optimization rules
  • Seven steps for incremental optimization
• Optimization strategy (20 min)
  • Measuring in stones
  • Profiling CPU usage
  • Profiling memory usage
• Algorithms and Anti-patterns (35 min)
  • String Concatenation
    • Constructing a large string from substrings with `+=` and a list
    • Exercise
  • List and Generator Comprehensions
    • Intro to list and generator comprehensions with examples
    • Exercise
  • The Right Data Structure
    • list vs. set
    • list vs. deque
    • dict vs. defaultdict
    • Big-O notation and data structures
    • Exercise
  • Caching
    • Deterministic caching
    • Non-deterministic caching
    • Memcachaed
• The Example (10 min)
  • Calculation of pi with the Monte Carlo method
• Testing Speed (10 min)
  • Small speed testing program for the course based on `timeit`
  • Explanation of source code
• Pure Python (15 min)
  • Two implementations for the example with just the standard library
  • Exercise
• Meet Psyco, the JIT (15 min)
  • Intro of Psyco using `full`, `log`, `profile`, and `bind` with examples
  • Exercise
• Numpy for Numeric Arrays (20 min)
  • Short intro to Numpy
  • Using Numpy for our example
  • Exercise
• Using multiple CPUs with pyprocessing/multiprocessing (20 min)
  • Short intro to pyprocessing/multiprocessing
  • Using pyprocessing/multiprocessing for our example
  • Exercise
• Combination of optimization strategies (15 min)
  • Psyco and Numpy
  • Psyco and pyprocessing/multiprocessing
  • Psyco and numpy and pyprocessing/multiprocessing
• Results of Different Example Implementations (10 min)
  • Compilation of all speed measuring results
  • Discussion of the ranking
  • Exercise

Outline for Website

• How Fast is Fast Enough?
• Optimization Guidelines
  • Premature optimization
  • Optimization rules
  • Seven steps for incremental optimization
• Optimization strategy
  • Measuring in stones
  • Profiling CPU usage
  • Profiling memory usage
• Algorithms and Anti-patterns
  • String Concatenation
  • List and Generator Comprehensions
  • The Right Data Structure
  • Caching
• The Example
• Testing Speed
• Pure Python
• Meet Psyco, the JIT
• Numpy for Numeric Arrays
• Using multiple CPUs with pyprocessing/multiprocessing
• Combination of optimization strategies
• Results of Different Example Implementations