Change the future

Mutation Testing in Python

Sebastian Kreft

Audience level:
Novice
Category:
Testing

Description

The elcap tool, a nose plugin, presents a flexible mutation testing tool for python. This poster introduces common testing terms related to mutations, basic examples, test statistics for some of the major python packages as well as some key aspects we need to think about when writing tests.

Abstract

This poster will cover the following topics:

  • What is mutation testing and why is it useful?

  • Statistics for projects: coverage vs real coverage after mutations in projects like Python itself, Django, sympy, etc.

  • Advice for better tests: By analysing major python projects we will show how this tool finds fragile tests and how can we write better ones.

  • Common patterns for testing: When we have mutation testing in mind it's easier to think what kind of data we need to test.

  • Developing a custom mutation: The tool already comes with a bunch of meaningful mutations however one of the main goals of this tool is to be flexible enough for anyone to write their own extensions.

Sponsors