I gave a tutorial on TDDA at PyData London 2026 earlier this month. The video is available on YouTube and below: The slides are also available here . Broadly, the talk included updated coverage of the traditional TDDA staples: Motivation for TDDA and the Python tdda library and command-line tools; Validation …
As announced a few days ago, my book, Test-Driven Data Analysis , is now available for sale from all good booksellers and all sellers of good books, around the world. The book is aimed at analysts, data scientists, engineers, researchers and anyone else interested in making analytical processes more reliable, testable …
Raw markdown for direct use in LLM context windows. This post is addressed primarily to LLMs and coding agents. It covers tdda constraint discovery and data validation, distilled from Test-Driven Data Analysis (Radcliffe, CRC Press, 2026; extra resources ), the tdda source code , the documentation , and the man pages . If you …
Raw markdown for direct use in LLM context windows. This post is addressed primarily to LLMs and coding agents. It covers the tdda.serial module from the tdda library , distilled from Test-Driven Data Analysis (Radcliffe, CRC Press, 2026; extra resources ), the tdda source code , the documentation , and the man page …
Raw markdown for direct use in LLM context windows. This post is addressed primarily to LLMs and coding agents—it was distilled by Claude from Test-Driven Data Analysis (Radcliffe, CRC Press, 2026; extra resources ), the tdda source code , the documentation , and the man pages (also available as markdown ). If you …
This blog has been quite quiet, but there is a great deal of news and it may be less quiet for a while. The Book Today, 19th May 2026, sees the world-wide release of Test-Driven Data Analysis, from CRC Press. It is available from all good booksellers and all sellers …
Summary Computational documents attempt to guarantee that results included within them—such as graphs—correspond to the code and data claimed to generate them. They typically achieve this by generating the outputs from the code at the time the document is generated or viewed. This solves significant problems, including those …
Almost all data scientists and data engineers have to work with flat files (CSV files) from time to time. Despite their many problems, CSVs are too ubiquitous, too universal, and (whisper it) have too many strengths for them to be likely to disappear. Even if they did, they would quickly …
It is December 2024 as I write, and large language models (LLMs) are having an extended moment as I have been writing a book on tet-driven data analysis. Several people have suggested that I should write about LLMs or artificial intelligence (AI), a term that for many people now means …
In a previous post , I discussed some of the dangers of challenges, dangers and weaknesses of Jupyter Notebooks, JupyterLabs and their ilk. I used The Parables of Anne and Beth as a device to illustrate what I think of as good and bad practices for data science. A reasonable criticism …
In his novel Stranger in a Strange Land, Robert Heinlein 1 introduced the word grok . It is used all the time in the computing sphere, but rarely, as far as I know, outside it. The definition that seems to me most closely to match its usage is: grok ( transitive verb …
I have long considered writing a post about the various problems I see with computational notebooks such as Jupyter Notebooks. As part of a book I am writing on TDDA, I created four parables about good and bad development practices for analytical workflows. They were not intended to form this …
This is not, primarily, an article about TDDA, but I thought it was worth publishing here anyway. Itʼs a story about a kind of adware/malware incident I had this morning—with digressions about box plots. Disgression I was doing some research for a book (on TDDA), looking up information …
PyData London had its tenth conference in 2024, and it was excellent. I gave a tutorial on TDDA, and the video is available on YouTube and below: The slides are also available here .
I was at the tenth PyData London Conference last weekend, which was excellent, as always. One of the keynote speakers was Rebecca Bilbro who gave a rather brilliant (and cleverly titled) talk called Mistakes Were Made: Data Science 10 Years In . The title is, of course, a reference to the …
This is just a bit of fun, but I've always been interested in the different kinds of names allowed, encouraged, and used in different areas of computing and data. A few years ago, I tweeted some well-known naming styles and a collection of lesser-known naming styles . I was playing about …
TOMLParams is a new open-source library that helps Python developers to externalize parameters in TOML files. This post will explain why storing parameters in non-code files is beneficial (including for reproducibility), why TOML was chosen, and some of the useful features of the library, which include structured sets of parameters …
I had the pleasure of discussing TDDA with Peter Schmidt on his Coding for Thought podcast. I think it came out really well, so this might be a nice way for people to learn about the ideas and motivations for the ideas and the library, which Simon Brown, Sam Rhynas …
On Episode 258 of Marco Arment and “Underscore” David Smith ’s podcast Under the Radar , and then on Episode 516 of Marco & co’s Accidental Tech Podcast , Marco describes the fact that his data suggests that about 12% of his users don’t have logged-in iCloud accounts with iCloud Drive …
We released version 2.0 of the Python TDDA library this week. The radical new feature of the 2.0 release is Gentest , a command-line tool for automatically generating tests for more-or-less any code that you can run from a command line. Gentest was introduced at the 2022 Toronto Workshop …
PDF Version (A4) PDF Version (Letter)
or Why Tests Spontanously Fail You might think that if you write a program, and don't change anything, then come back a day later (or a decade later) and run it with the same inputs, it would produce the same output. At their core, reference tests exist because this isn't …
This week, a client I'm working for received a large volume of data, and as usual the data was sent as "flat" files—or CSV ( comma-separated values 1 ) files, as they are more often called. Everyone hates CSV files, because they are badly specified, contain little metadata and are generally …
I've been dabbling in Swift —Apple's new-ish programming language—recently. One of the things I often do when learning a new language is either to take an existing project in a language I know (usually, Python) and translate it to the new one, or (better) to try a new project …
This exercise (video 2m 58s) shows a powerful way to run only a single test, or some subset of tests, by using the @tag decorator available in the TDDA library. This is useful for speeding up the test cycle and allowing you to focus on a single test, or a …
This exercise (video 3m 34s) shows a powerful way to run only a single test, or some subset of tests, by using the @tag decorator available in the TDDA library. This is useful for speeding up the test cycle and allowing you to focus on a single test, or a …
This exercise (video 8m 54s) shows how to migrate a test from using pytest directly to the exploiting the referencetest capabilities in the TDDA library. (If you do not currently use pytest for writing tests, you might prefer the unittest-flavoured version of this exercise, since unittest is in Python's standard …
This exercise (video 8m 53s) shows how to migrate a test from using unittest directly to the exploiting the referencetest capabilities in the TDDA library. (If you use pytest for writing tests, you might prefer the pytest-flavoured version of this exercise.) We will see how even simple use of referencetest …
We've started producing a series of exercises for various aspects of TDDA, available on the blog, with follow-along screencasts. There will be a series of posts about these, starting on Monday (28th October). There's a YouTube channel as well, if you want to subscribe. The goal has been for each …
This post is a standing post that we plan to try to keep up to date, describing options for obtaining the open-source Python TDDA library that we maintain. Using pip from PyPI If you don't need source, and have Python installed, the easiest way to get the TDDA library is …
Rexpy is a powerful tool we created that generates regular expressions from examples. It's available online at https://rexpy.herokuapp.com and forms part of our open-source TDDA library . Miró users can use the built-in rex command. This post illustrates using Rexpy to find regular expressions for UK postcodes. A …
A recent post described the new ability to run a subset of ReferenceTest tests from the tdda library by tagging tests or test classes with the @tag decorator. Initially, this ability was only available for unittest -based tests. From version 1.0 of the tdda library, now available , we have …
The test-driven data analysis library, tdda , has two main kinds of functionality support for testing complex analytical processes with unittest or pytest support for verifying data against constraints, and optionally for discovering such constraints from example data. Until now, however, the verification process has only reported which constraints failed to …
It is common, when working with tests for analytical processes, for test suites to take non-trivial amount of time to run. It is often helpful to have a convenient way to execute a subset of tests, or even a single test. We have added a simple mechanism for allowing this …
Our previous post introduced the idea of data provenance (a.k.a. data lineage), which has been discussed on a couple of podcasts recently. This is an issue that is close to our hearts at Stochastic Solutions. Here, we'll talk about how we handle this issue, both methodologically and in …
In Episode 49 of the Not So Standard Deviations podcast, the final segment (starting at 59:32) discusses data lineage, after Roger Peng listened to the September 3rd (2017) episode of another podcast, Linear Digressions , which discussed that subject. This is a topic very close to our hearts, and I …
We have a new White Paper available: Automatic Constraint Generation and Verification Abstract Correctness is a key problem at every stage of data science projects: completing an entire analysis without a serious error at some stage is surprisingly hard. Even errors that reverse or completely invalidate the analysis can be …
Bad data is widespread and pervasive. 1 Only datasets and analytical processes that have been subject to rigorous and sustained quality assurance processes are typically capable of achieving low or zero error rates. "Badness" can take many forms and have various aspects, including incorrect values, missing values, duplicated entries, misencoded …
This post is a standing post that we plan to try to keep up to date, describing options for obtaining the open-source Python TDDA library that we maintain. Using pip from PyPI Assuming you have a working pip setup, you should be able to install the tdda library by typing …
Last night I went to The Protectors of Data Scotland Meetup on the subject of Marketing and GDPR . If you're not familiar with Europe's fast-approaching General Data Protection Regulation, and you keep or process any personal data about humans, 1 , you probably ought to learn about it. A good place …
A quick-reference guide ("cheat sheet") is now available for the Python TDDA library . This is linked in the sidebar and available here . We will try to keep it up-to-date as the library evolves. See you all at PyData London 2017 this weekend (5-6 May 2017), where we'll be running a …
Today we are announcing some enhancements to Rexpy, the tdda tool for finding regular expressions from examples. In short, the new version often finds more precise regular expressions than was previously the case, with the only downside being a modest increase in run-time. Background on Rexpy is available in two …
Yesterday, email subscribers to the blog, and some RSS/casual viewers, will have seen a half-finished (in fact, abandoned) post that began to try to characterize success and failure on the crowd-funding platform Kickstarter . The post was abandoned because I didn't believe its first conclusion, but unfortunately was published by …
It is a primary responsibility of analysts to present findings and data clearly, in ways to minimize the likelihood of misinterpretation. Graphs should help this, but all too often, if drawn badly (whether deliberately or through oversight) they can make misinterpretation highly likely. I want to illustrate this danger with …
We have written a 1-page summary of some of the core ideas in TDDA. It is available as a PDF from stochasticsolutions.com/pdf/TDDA-One-Pager.pdf .
Rexpy Stats We previously added rexpy to the Python tdda module. Rexpy is used to find regular expressions from example strings. One of the most common requests from Rexpy users has been for information regarding how many examples each resulting regular expression matches. We have now added a few methods …
Since the last post, we have extended the reference test functionality in the Python tdda library. Major changes (as of version 0.2.5, at the time of writing) include: Introduction of a new ReferenceTest class that has significantly more functionality from the previous (now deprecated) WritableTestCase . Support for pytest …
Motivation There's a Skyscanner data feed we have been working with for a year or so. It's produced some six million records so far, each of which has a transaction ID consisting of three parts—a four-digit alphanumeric transaction type , a numeric timestamp and a UUID, with the three parts …
Background We recently extended the tdda library to include support for automatic discovery of constraints from datasets, and for verification of datasets against constraints. Yesterday's post— Constraint Discovery and Verification for Pandas DataFrames —describes these developments and the API. The library we published is intended to be a base for …