HackerPapers

arXiv:2007.14049cs.SE

Automated Unit Test Generation for Python

Stephan Lukasczyk, Florian Kroiß, Gordon Fraser

Abstract

Automated unit test generation is an established research field, and mature test generation tools exist for statically typed programming languages such as Java. It is, however, substantially more difficult to automatically generate supportive tests for dynamically typed programming languages such as Python, due to the lack of type information and the dynamic nature of the language. In this paper, we describe a foray into the problem of unit test generation for dynamically typed languages. We introduce Pynguin, an automated unit test generation framework for Python. Using Pynguin, we aim to empirically shed light on two central questions: (1) Do well-established search-based test generation methods, previously evaluated only on statically typed languages, generalise to dynamically typed languages? (2) What is the influence of incomplete type information and dynamic typing on the problem of automated test generation? Our experiments confirm that evolutionary algorithms can outperform random test generation also in the context of Python, and can even alleviate the problem of absent type information to some degree. However, our results demonstrate that dynamic typing nevertheless poses a fundamental issue for test generation, suggesting future work on integrating type inference.

arXiv comments
15 pages, to be published in Proceedings of the 12th Symposium on Search-Based Software Engineering (SSBSE 2020)
Journal reference
DOI
doi:10.1007/978-3-030-59762-7_2
Subjects
cs.SESoftware Engineering

On Hacker News

Linked by 3 identifiable commenters in 3 comments across 1 thread, from Jun 2021 to Jun 2021.

Comments linking this paper, per year.

Discussed in comments1 thread

  1. Pynguin – Generate Python unit tests automatically 3 comments

    • preprint available for those curious about how this works: https://arxiv.org/abs/2007.14049

      khimarosView on HNParent context

    • There wasn't much info on the GitHub repo about how it actually works. Here's the paper from the authors: https://arxiv.org/abs/2007.14049 From what I gather from the paper, they frame the problem of test generation as a search problem. An evolutionary algorithm randomly mutates…

      mmmkkaaayyView on HNParent context

    • Just skimmed through the paper: https://arxiv.org/pdf/2007.14049.pdf, it seems like they are using some sort of heuristically mutating a test suite until it's fully branch covered, based on calculating `branch distance` for each predicate. Why isn't a SMT solver like Z3-solver…

      usagitoneko97View on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.