I've seen a few discussions around lack of non-blocking file io support in asyncio. Normally, we assume that disk io is fast and doesn't block, but there are use cases where it can block for a long time:
- HDDs are still very common and widely used. Huge ones can be quite slow at random access.
- Network filesystems - sshfs, nfs, aws ebs etc.
A workaround is to use a threadpool for file io, but that's suboptimal since it's hard to interact with asyncio code.
libuv library (the one that NodeJS is built on) uses threadpools internally, to provide non-blocking file io.
I see three options for asyncio:
- Keep the status quo. Advise users to either assume that disk io never blocks, or use threadpools.
- Provide a function that will return a pair of
(StreamReader, StreamWriter). Loop will use a threadpool to do the actual IO. - Provide a file object with a similar API to FileIO, but with coroutines for most of its methods. Loop will use a threadpool to do the actual IO.
If (2) or (3) is acceptable, I volunteer to draft the initial implementation.