#### AUTOMATED TESTING: PROBLEMS FACED BY THE TESTING COMMUNITY Automated testing often requires a user to write a test harness —- essentially a program that defines (or generates) the set of valid tests for the Software Under Test (SUT). Such harnesses are common to random testing, many kinds of model checking, and various machine-learning influenced approaches. Unfortunately, these harnesses themselves are complex software artifacts and it is all too easy to spend valuable testing time hunting down bugs in the test harness and not the System Under Test. Harness code is often highly repetitive (choosing between a set of available API calls to make, and assigning values to parameters in those calls, for example) and is almost always tightly coupled to one particular test generation method. #### SOLVING THE PROBLEM: A LITTLE LANGUAGE FOR TESTING WRITTEN IN PYTHON TSTL is a language written in Python, embedding Python, intended to make these testing difficulties less onerous and decouple testing technology from the System Under Test. This will enable reuse of advanced testing techniques and tools without rewriting test harnesses. TSTL allows a system under test to be defined in declarative style, in files with a .tstl extension. TSTL (for Python) consumes this .tstl file and generates a Python harness in the sut.py file. This sut.py file is a TSTL-generated Python interface for testing that can be tested by any Python test generator that uses its interface to choose actions, restart tests, and check properties. Some of these generators are bundled with TSTL in the tstl/generators directory. The sut.py interface is discussed in more detail in an ISSTA 2015 paper [1]. It essentially transforms the set of valid tests into a graph to explore by choosing valid actions. The interface also includes support for automated reduction of the length of test cases, collection of data on code coverage, and novel methods for making test cases easier to understand and reducing the redundancy of test cases. Generators are tools for generating tests for any sut.py. Generators distributed with TSTL include a heavily-developed random testing tool, and simple BFS and DFS based explicit-state model checkers. ##### More information about TSTL can be found here: [1] "TSTL: A little language for testing (demo)" - International Symposium on Software Testing and Analysis 2015 http://dl.acm.org/citation.cfm?id=2784769 [2] "A little language for testing": NASA Formal Methods: 7th International Symposium, NFM 2015 https://books.google.com/books?id=u9LzBwAAQBAJ&pg=PA204#v=onepage&q&f=false [3] Source code: https://github.com/agroce/tstl