I created a new mailing list to get email notifications when a buildbot starts failing (state change from green/success to red/failure).
It seems like the buildbot config already sends email, but I don't see them in the archives of the python-checkins list?
Config:
https://github.com/python/buildmaster-config/blob/master/master/master.cfg#L820-L827
python-checkins archives:
https://mail.python.org/pipermail/python-checkins/2017-May/thread.html

On the buildbot server, I see the following code in lib/python/buildbot/status/mail.py:
        # GB: NEW: add test logfile excerpts
        if "test" in t: # only if the test went wrong
            for log in build.getLogs():
                if "test" in log.getStep().getName():
                    text += "\n"
                    text += "Excerpt from the test logfile:"
                    text += interpret_test_logfile(log.getText().splitlines())
                    text += "\n"
                    break
But I'm unable to find this code in the upstream code:
https://github.com/buildbot/buildbot/blob/v0.8.12/master/buildbot/status/mail.py#L742
Do we have downstream changes on buildbot? I'm asking because it seems like emails cannot be send because of these lines. Extract of logs:
2017-05-05 11:55:42+0000 [-] Exception caught notifying <buildbot.status.mail.MailNotifier instance at 0x7f9512f3add0> of buildFinished event
2017-05-05 11:55:42+0000 [-] Unhandled Error
        Traceback (most recent call last):
          File "/data/buildbot/lib/python/buildbot/status/build.py", line 320, in buildFinished
        (...)
          File "/data/buildbot/lib/python/buildbot/status/mail.py", line 792, in buildMessage
            if "test" in t: # only if the test went wrong
        exceptions.NameError: global name 't' is not defined
Note: It seems like 0.8.14 is deployed, but I don't see any v0.8.14 tag on GitHub, the last one of the 0.8 branch is v0.8.12. https://pypi.python.org/pypi/buildbot/0.8.14 wasn't tagged in Git?

It seems like the code is copied by rsync from a server called dinsdale:
https://github.com/python/psf-chef/blob/master/cookbooks/buildmaster.txt#L13

To rebuild the lib/python/buildbot/ directory, you can start from buildbot 0.8.14 and then apply attached buildbot-0.8.14py1.patch diff.

Ok, Zachary gave me a clue, the code comes from https://github.com/python/buildbot/
It seems like our server is outdated, since buildbot/status/mail.py now contains:
        # GB: NEW: add test logfile excerpts
        # AP: disabled (code is broken)
        if 0 and "test" in t: # only if the test went wrong

Zachary updated the buildbot Git repository to get the mail.py fix.
I changed the email address to buildbot-status@python.org:
https://github.com/python/buildmaster-config/pull/5
Let's see how things are going.

Email notifications is now working properly, I close the issue.

stage: resolved