Recent content in About testmon on Testmon
Thoughts on CI systems The modern CI services (GitHub Actions, GitLab Pipelines, etc.) model is very unsatisfactory. They are: proprietary and enforce vendor lock-in configured/programmed via YAML hard or impossible to run/debug locally unable to execute without a commit/push misnamed (CI is neither about integration nor truly continuous) I see clear overlap between automated QA, build systems, CI…
** This post is outdated** The recommended GH action is not maintained anymore. Most of its functionality was added to the standard setup-python action, see https://github.com/actions/setup-python#caching-packages-dependencies . There is also a new tool to use testmon in CI, see https://www.testmon.net/ Some more thouhgs: here For many projects creating the Python environment and installing…
I’m glad to announce the release of version 2.0 of pytest-testmon. Most changes compared to version 1.4.5 are internal. They will allow better maintenance and the ability to add new features. Testmon now obtains file checksums from git, if possible. There are a couple of fixes regarding basic stats, and the issue of pytest reporting a negative number of deselected tests has been resolved.
For many years it hasn’t been possible to use pytest-testmon and run tests in parallel via pytest-xdist . With pytest-testmon=>1.4.0 the support has recently been added. Test parallelization is probably the most general way to speed up a large test suite so this was a major limitation, and I’m glad I was able to lift it. With this addition pytest-testmon will be more relevant for the…
The 1.3.x line of pytest-testmon brought a lot of functional and compatiblity improvements: much smaller .testmondata saving space and improving performance doctest and doctest modules support support for pytest 7 --testmon-noselect now sorts the tests! All the tests are executed but the affected go first. Unaffected tests only follow afterwards. Quickest tests are still prioritized within each…
I’m very glad to announce pytest-testmon version 1.2 is out on PyPI . In previous versions pytest-testmon stored the failure reports and re-reported them from cache even if tests were not affected. The result was imperfect in many ways and sometimes spooked users. This version always re-executes tests which failed last time. Database schema was updated to version 8, thus previous versions of…
I’m very glad to announce pytest-testmon version 1.1 is out and up on PyPI . The new feature is lifting a basic limitation of testmon. There are many things influencing the outcome of a test. Before version 1.1 testmon kept track of code inside tested project and configured environment variables . With version 1.1 comes the ability to detect changes of packages(libraries) . With each run…
Significant portions of testmon have been rewritten for v 1.0.1. Although the UI is mostly the same, there are some significant differences. End of python 2.7 support Testmon requires python 3.6 or higher and pytest 5 or higher. No subprocess measurement –testmon-singleprocess was removed (because it’s the only option now) It’s not easy to automate the setup of subprocess…
Automatically determining affected tests sounds too good to be true. Python developers rightfully have a suspecting attitude towards any tool which tries to be too clever about their source code. Code completion and symbol searching doesn’t need to be 100% reliable but messing with the test suite execution? This page explains what testmon tries and what it does not try to achieve. There is…
In this article, we’ll take a look and set-up pytest framework in PyCharm using our small example project. We’ll also introduce and try two very useful pytest-plugins to run tests more flexibly and effectively to save our time and CI/CD resources. MasterMind – example project For the sake of this article, we’ve prepared a small project called MasterMind. It’s a simple…
We all know each one of our tests should be independent, self-contained and therefore able to run by itself or in parallel with other tests with deterministic results. However in real life we many times encounter tests that sometimes pass and sometimes fail. There are many different sources of non-deterministic outcomes of tests. In this post, I would like to specifically focus on tests that…