It's that time of the year again, a new version of Python is just around the corner. With the Python 3.15.0b1 feature freeze, we know what's coming to Python later this year. There are so many big features coming including lazy imports and the tachyon profiler which I …
In November of 2023 cpython core developer Anthony Shaw wrote about sub-interpreters in his post: Running Python Parallel Applications with Sub Interpreters , where he previewed the ongoing development for subinterpreters and highlighted his attempt to run web applications in a subinterpreter. Anthony concluded his post having adapted hypercorn to run …
I recently saw Trey Hunner's take blog post on 'switch case' like statements in Python. He makes a lot of good points and it's definitely worth a read. What I want to talk about is his last point Using a dictionary instead of switch-case: Sometimes you can replace long if …
Despite optimisations we explored in my previous post , there are still many obstructions to achieving free-threading performance. One specific issue is that: often benign code abstractions can hurt multi-threading performance. This is something we explored a while ago in How free are threads in Python now? : The following code end …
I posted a lot about free threading in the past year: How free are threads in Python now? Python 3.14: State of free threading Over all I had 3 issues with free-threading: Reference count contention hurts performance on even immutable objects. Often benign code abstractions can hurt multi-threading performance …
Working with datetime is hard and error prone, but Python actually provides the right tools with timedelta to work with them. It's a shame that most developers are not making the best use of it. Basic Arithmetic Let's first catch up what the basic operations are: Advanced Arithmetic Now, let's …
If you've used asyncio for some time you've probably noticed a few things that work differently to the synchronous counter parts. I recently had to add some caching to an asyncio function. Let's say something like: async def get_user ( user_id : UUID ) -> User : ... Of course naturally we want to use functools …
I've been on vacation in China leading up the lunar new year and I'd rather not spend all my time researching some heavy topic. However, I thought I might instead spend a few minutes combining my previous posts on marimo notebooks and the Lunar calendar .
Recently I've been updating some of my libraries to Python 3.10+ after Python 3.9 has finally reached end of life. The upgrade to Python 3.10 is a relatively simple one, but I thought it be a good idea to run tests on different versions of Python. Existing …
I recently published a project webasgi a static site allows you run FastAPI and other asgi apps (such as litestar ) right in the browser. As you can imagine, Python web apps are not designed to run statically in the browser, and this project has been particularly challenging to me, in …
Recently I've been working on framework to run LLM tasks using AWS's excellent SQS . And I made the decision to write my own task framework/library as opposed to using a pre-exiting framework. I thought this would be a great opportunity to discuss the considerations and levels of abstractions involved …
Don't listen to random benchmarks.. I recently came across an article benchmarking Python performances in web frameworks, comparing asyncio and sync performance. The author sets out to measure performance of FastAPI / Django web servers running with postgresql comparing async and non-async workloads. The methodology is pretty reasonable, the following is …
... but that's actually a good thing I've already written a ton about subinterpreters since a year ago: How good are sub-interpreters in Python now? Python 3.14: State of free threading Subinterpreters and Asyncio Recently I had some time to dig a bit deeper into subinterpreters and check my understanding …
As we approach the holiday seasons, we get to "enjoy" another 12 days of Advent of Code . Every year I try to do something a little different, last year I had a lot of fun solving it in Ocaml, prior years I've tried the latest Python features. This year I've …
So I've just released a package called lazy-helper . This comes as lazy loading has been a hot topic once again due to the proposal PEP-810 . Using my package we can define lazy loaded dependencies nicely: Coincidentally Brian Okken has recently written a blog post covering the topic of lazy loading …
Python 3.14 was officially released on October 7th. There are a lot of new features and I've covered some of them before in: Python 3.14: 3 smaller features t-strings: the good and the ugly Subinterpreters and Asyncio What I haven't covered here are any of the asyncio changes …
Mojo has been on my radar for a while. In fact, I heard about it on launch back in May 2023. It was touted as a super set of Python with 10000s of times the performance. It's main focus is on AI related workloads with strong built in support of …
It's been just over a year since I started my blog. In fact, this is my 27th blog post. I thought it'd be a good time for me to share some insights on blogging. Motivation Recently I've been asked by some people on why I do this. I think for …
Previously when discussing asyncio backpressure I've made some claims that were not necessarily complete. I said: It works well for 100s of urls but when we hit a big number like 10000s we have a problem. The program seemingly hangs. This is because all the tasks are being created first …
The python packaging landscape and developer experience has shifted dramatically in the past year or so with uv 's launch marked a pivotal moment. But behind the scenes many PEPs have worked to get us to this point. One such PEP was PEP 723 – Inline script metadata which we discussed …
Feature flags or dynamic configuration is something that I find very useful, however I've never had the chance to use them. This is for a lack of options launchdarkly , flagsmith and unleash to name a few. SaaS options can be amazing with a full array of features, but I would …
PEP-734 subinterpreters in the stdlib has officially been included in the Python 3.14 as a very late addition . subinterpreters now has a new home in the standard library module called concurrent.interpreters . If you've been following my blog posts you'll know that I'm particularly excited about this feature. How …
Python 3.14 is just around the corner and it's jampacked with huge updates: Free threading and multiple interpreters? Template strings But as with any release, there are many nice smaller and less noticeable features. Features you won't see unless you comb through the entire release notes . Luckily I am …
I grew up in both the UK and China. So I'd like to think I have a little understanding of both cultures. China uses both the western gregorian calendar and a version of lunar calendar called 农历. This means I have a Chinese lunar birthday as well as a …
There's mixed feedback to Asyncio in the community. Some people passionately hate it whilst others believe "writing async make program go fast". This debate is way too much for me to cover here right now. Though maybe I'll look at it in the future. For me I use asyncio a …
Short update since First Look at MCP . Docker has released it's MCP toolkit which includes a catalog of MCP servers as well as helpers to connect to the MCP clients. It is exceeding easy to use it, docker is registered as a single MCP server for your client. And tools …
Previously I've played with tool calling in Langchain and Python sandboxes . But recently MCP (Model Context Protocol) is front and center. So I tried to create my own github MCP in Python to integrate cursor with the github cli. What is MCP? Before we get started, its good to familiarise …
In my posts earlier this year I talked about the parallelism performance on 3.13 free-threaded builds. In particular I looked at solving an advent of code problem . In How free are threads in Python now? I discovered significant performance penalties for using free-threading and a lack of tooling available …
This one's hot off the press as the first beta for Python 3.14 (aka. π-thon) has hit. We're looking at a chunky release with a lot of new features. But all I can think about are these new template strings (officially t-strings ). PEP-750 officially introduces the concept. The idea …
You're probably thinking that there are already plenty of tools written in Python. But I see that most of the popular tools like mypy and flake8 are built for development environments. In contrast, general purpose cli tools tend to be built in other languages, for example most of the docker …
I've been a little obsessed with operator overloading lately. First using |= in sqlalchemy-builder and then using | and @ in better-functools . I didn't actually know that these qualify as DSLs, specifically what's known as "internal DSLs". Funnily enough, I'm usually not a fan of DSLs. A few reasons come to mind: DSLs …
I recently put some effort into creating better-functools . It's a package that adds some tooling for functional programming in Python. And allows us to write some unique looking code in a manner similar to functional languages: Try in sandbox Why? At a glance this doesn't look very "Pythonic". More complex …
This one has frustrated me for a while. It starts off with a REST API route. For example in fastAPI @app . get ( "/" ) def search_users ( session : Session ) -> list [ User ]: """Finds users optionally filter by user_id""" statement = select ( User ) . order_by ( User . name ) return session . execute ( statement ) . scalars () . all () Then we get asked …
Structural pattern matching is probably the coolest new syntax introduced to Python. Added in 3.10, it's been a few years now and more people are writing apps in 3.10 * than any other version now. Though even with the wide adoption of 3.10 and more people being exposed …
Introduction I wrote about free-threading recently in How free are threads in Python now? . Where I found some unexpected difficulties parallelising my solution. In the conclusion section I wrote: I'm starting to think that the approach sub-interpreters is taking, with brand new concurrency primitives and shared memory data structures has …
Free-threaded Python ( PEP-703 ) was released in October 2024. It enables true multi-threaded execution without the restriction of the GIL . I previously covered this in Free Threaded Python With Asyncio , the example used there was selected because it very clearly demonstrates the performance increase, or moreover, the threads "running free". But …
I've been doing advent of code again this year. There are two Python features I always rely on, iterators and pattern matching. Iterators allow for operations on each of its elements without allocating memory for a collection. Ever since pattern matching was introduced in Python 3.10, it's been particularly …
I've been experimenting with Langchain for GPT based queries. One problem we often encounter with GPT is hallucinations. This makes certain classes of problems unsuited to GPT, one example is maths and statistics. Whilst there are improvements for recent models often the maths cannot be trusted. When I try to …
I've worked with SQLAlchemy for a while now, and in my opinion it's the best ORM in Python. It's feature rich with strong support for all major databases. And it maintains the SQL feel without losing things like typing. However there are some challenges here. Despite having very nice documentation …
TypedDict was introduced in PEP-589 which landed in Python 3.8. The primary use case was to create type annotations for dictionaries. For example, class Movie ( TypedDict ): title : str movie : Movie = { "title" : "Avatar" } I remember thinking at the time that this was pretty neat, but I tend to use dataclass …
I recently came across a video by mcoding about Python iterators' unfortunate closing behaviour. To sum up the video, when an iterator is interrupted we intuitively we would expect the exit of a context manager or a finally block to be called but that's not necessarily the case. For example …
With the imminent release of Python 3.13, I wanted to look at the biggest changes coming to Python. I think by far the most exciting feature is free-threaded Python from PEP-703 . As I'm quite late to the party, there's already a lot of articles talking about it. I came …
My name is Jamie Chang, I'm a software engineer in London. I like all things Python, but I'll try to mix it up sometimes. I decided to start this blog to get some of the ideas out from my head and to practice communicating technical topics. For this site, I'm …