Using Lazy Evaluation to Optimize Python Programs
Joe Jevnik
Description
Lazy evaluation, also known as call by need, is an evaluation strategy which
defers all computations until a result is actually required. This is the
opposite of eager evaluation, what Python normally does, where functions are
executed as seen by the interpreter. Lazy evaluation gives the executor more
context about how any given expression will be used, opening the door for
interesting optimizations like intermediate object sharing or parallel
execution.
Python has some tools for emulating lazy evaluation, namely closures and
generators; however, these objects cannot be used interchangably with non-lazy
values. `lazy_python` is a library which allows users to automatically translate
standard, eager Python functions into a lazy equivalent. This allows users to
write or use standard Python code while getting the benifits of lazy
evaluation.
This poster will cover different ways to implement lazy evaluiation, focusing on
the techniques used in `lazy_python`. It will show how to use the added
execution context to implement optimizations. Finally, we will show a real
example of using `lazy_python` with `dask` to automatically parallelize the
execution of Python programs.