This post is based on my JupyterCon 2020 talk . I decided to write it 5 years later, because I think that notebook-based data platforms are relevant, now more than ever as AI helps us do more things, to enable citizen data science and seamless collaboration in large orgs. It's a …
J'ai besoin de présenter un exposé sur un sujet peu connu de mon pays dans mon cours de français cette semaine. J'ai choisi comme sujet : le vol des cerfs-volants dans le ciel pour célébrer la fête de l'indépendance à Delhi, d'où je viens. J'ai essayé de faire des recherches mais …
Recently, I've been exploring geospatial data about Paris. Although I've seen others work with this type of data and know basic terms like shapefiles and polygons, this is my first time working with it myself. I've been looking up terms and going through docs to learn how various geospatial data …
Last month, I saw Ferruccio with a TRMNL and it picqued my interest because I'd been thinking about putting some things that I always wanted top of mind right in front of me. For example, my running goals that I track using the Strava widget on my phone. After some …
Originally published on the June blog . At June, we use ClickHouse as our analytics database to process billions of user events and generate real-time insights for our customers. While ClickHouse is incredibly powerful for analytical workloads, managing database schema changes has been a pain in our dev workflow. Unlike traditional …
Cap d'Antibes In May 2022, I swam in the sea for the first time in my life. Before that, I'd only swam in pools ( except for that one time in a piscine naturelle in Nongriat), so this experience was both exhilarating and scary at the same time. Exhilarating because I …
I'm happy to announce that Camelot v0.10.0 is out! tl;dr You can now choose between two image conversion backends , or supply your own. pip install camelot-py[base] instead of pip install camelot-py[cv] Background Camelot uses ghostscript to convert a PDF page into a PNG so that …
I've been using Fraidycat as an RSS reader for some time. The project puts out an AppImage as a release asset , which is a neat way for distributing software on Linux because it does not need root privileges to install things in system directories. After making a Software.AppImage file …
Friday was the last day of my batch :( I've been writing these posts a day or two after the actual day, but this one is slightly late because my laptop charger gave up on me while I was hanging out with Edith and Adam after the Halloween party on Friday …
Today I continued working on translating cutypr to Rust. I was able to serialize and sign messages, and send them to the kernel, but the kernel kept saying that the message has an invalid signature :( I found it difficult to debug and check if the (message signature) bytes sent from …
Today I worked on translating cutypr to Rust. I found that I could use pyo3 to start the IPython kernel from Rust! So I wrote a start_kernel() Python function to start the IPython kernel, returned some useful information (the key required to sign messages + ports for all kernel channels ) from …
Today I read the first chapter of the zeromq guide and looked at the client-server and pub-sub examples! I love this paragraph from the preface (sounds so similar to Derek Sivers's CD Baby email !): We took a normal TCP socket, injected it with a mix of radioactive isotopes stolen from …
This week I'm planning to build a terminal frontend for Jupyter (in Rust!) so today I started looking into how Jupyter works! I found this doc which shows how messaging works in Jupyter. When we start a jupyter_console (like IPython), it starts a Python kernel in the "backend". The kernel …
Today I read up on structs , enums, and pattern matching in the Rust book. I also did some rustlings , and read a lot of Rust blog posts to figure out what I should work on in my last week at RC! I want to build a small project to get …
Today I read the chapter on ownership in the Rust book. In this post, I'll try to summarize what I learned for future me! Ownership in Rust Ownership is Rust's central feature and the chapter explains how it allows Rust to make memory safety guarantees without needing a garbage collector …
Today I read up on some numpy docs to try and find how they bundle the OpenBLAS DLL with the numpy wheel. I've updated the day 51 post with the things that I found. I also packaged the find_dll_dependencies function from wheel_repair.py into a convenient CLI tool which can …
Yesterday, after reading about the DLL Search Order on Windows and DLL Hell , I got concerned about shipping DLLs as package_data , because they might clash with other DLLs with same names that are shipped by other Windows wheels. This is the gist of the whole issue: If a DLL with …
Last week, I was able to build pdftopng on Windows , but the extension worked only when all the DLLs it depended on were kept in the same directory as the PYD ( which is basically a DLL ). At the end of that post, I had some questions around bundling DLLs with …
Today I created a test PR to replace ghostscript with pdftopng , and most tests passed ! The ones that failed have coordinates hardcoded in them. I'll just need to add a small tolerance in that float comparison and everything should be good to go! Not quite, I need to build Windows …
For the past 4 days, I've been spending some time at JupyterCon! It's been great listening to talks and looking at all the things people are building with Jupyter! All of the chat has been happening on the JupyterCon Mattermost instance. Thank you to the organizers for all their work …
I made Linux and MacOS wheels for curlyboi and uploaded them to PyPI! You can now install curlyboi using: $ pip install curlyboi Since I used pybind11 (which works with C++ code) to wrap curlyboi , I had to rename all the .c files to .cpp . I ran into an error when …
Recently I found this snake game by Julia Evans and thought that it might be fun to go through it to learn more about large-ish C projects and ncurses ! I also paired with Edith (who is writing her own text editor from scratch!) to walk through the code and understand …
Today I read through some open issues on camelot , and found a bug for when you install it from conda-forge . I'd assumed that installing ghostscript from conda-forge installs all of its dependencies. It does, but looks like all the those depedencies are statically linked into one gs executable. This would've …
I looked into how to build C extension wheels on Windows over the weekend. Since there isn't a fastmac equivalent to get a Windows machine for debugging, I booted up Windows on my laptop after a really long time! I need to find a fastwin or winfast ! Installation Visual Studio …
Today I continued my quest of packaging my Python C extension for multiple OSes. Yesterday while doing a packaging "test run" with the curses "hello world" program, I found that curses is not supported on Windows (it should work with WSL, but not with the "default" Windows terminal I guess …
Today I started looking into building cross-platform wheels for my Python C extension . I've never built binary extension wheels so I decided to start simple by building wheels for this snake game I want to write in C. Right now it's just a "hello world" ncurses program. I started looking …
I've been reading (and forgetting about) a lot of PEPs lately. When you read a PEP, it mentions how "this" thing was defined in "this" PEP, and how "that" thing was defined in "this other" PEP. So I thought that it would be cool to look at all the PEPs …
Yesterday I paired with Ilia to walk through the pdftoppm code , and we identified some code that wasn't required to wrap pdftoppm . I also wrapped a "hello world" ncurses game with pybind11 ! Today I started looking into pdftoppm again and commented some code (like command-line argument parsing and JPEG support …
Today I paired with Ilia to walk through the poppler codebase. This was of great help as I don't have a lot of experience navigating through large C/C++ codebases. I got to learn about how cmake and C++ preprocessor directives work! We also walked through the pdftoppm code to …
Today I replaced some ghostscript code in camelot with a pdftoppm subprocess call just to see if the tests pass. They did! I think poppler could be the ghostscript alternative that I'm looking for! pdftoppm -r 300 -png -singlefile foo.pdf foo That got me all excited to look into …
Today I watched these awesome talks on wheel building by Elana Hashman and Paul Kehrer. I also read PEPs 513 , 571 , 599 , and 600 where I learned about manylinux , and how it solved a problem I used to face a lot! It's the problem of installing Python packages that contain …
Today I read PEP 427 (and other PEPs mentioned in there) to learn more about Python wheel format! There are two types of distribution formats in Python - a source distribution, and a built distribution. The wheel format belongs to the second category in which things are already built , ready to …
When I was trying to learn Rust (I need to get back :( ) some weeks ago, I really liked how I was able to install every tool I needed to write Rust with just one curl command! I also loved the rustup doc command which opened the docs for my installed …
Today I spent most of my time watching some talks on writing Python C extensions. Every talk mentioned that the main reasons for writing an extension are: Interfacing with C/C++ (I'm in this camp!) Improving performance of Python code by rewriting it in C/C++ I need to try …
I saw "ELF" being mentioned a lot while reading about linkers yesterday so I thought about looking into it. Turns out it's a file format for executables. Just hearing the word "executable" used to give me the chills because I couldn't look at what's inside one. (One time I ran …
Continuing from the day before yesterday , I started looking into linkers and loaders. Till now, I haven't had experience writing multiple C files that work together. And up until yesterday , my only other experience with .so files has been knowing that if I put opencv's .so file in my Python …
Using strace to spy on ghostscript 's system calls made me wonder if there's a way to print the names of all function calls (along with their file paths) that a compiled executable makes after you invoke it on the command-line. This would be awesome to navigate large C codebases …
Today I did some Yak shaving and made opep because opening/reading each PEP in its own browser tab (with all the extra information on the webpage) was getting tiring both for me and the browser. I'd used the man command a lot yesterday to read some manual entries, so …
I recently (at the start of my batch) found out about strace from this zine by Julia Evans! Today I used it to look at things ghostscript does under the hood (system calls!) to do a PDF to PNG conversion. I also read through the manual page for strace which …
Today I spent a lot of time upgrading my Ubuntu from 18.04 to 20.04 , just to get a new version of gnome-shell which was needed by this cool window manager I wanted to try out. The whole process reminded me of 2014 when I was literally trying out …
Today I paired with Andrew to try out EasyOCR since we thought that it could be useful in both chronicle (to extract text out of camera images) and camelot (to extract text out of image-based PDFs). Andrew suggested that we could use images from duetocovid19.com , another cool project that …
This is not something new, almost every video in the world is recorded this way! At first, I fought against this strategy by doing just one take, thinking that it would save time. I couldn't be more wrong! Disclaimer: I'm a new speaker, and I'm not very good at speaking …
Today (plus yesterday, and the whole weekend!) I focused all my energy on writing (and trying to record ) my talk for JupyterCon. Now I just need to record it.
Oh no, I'm almost at the midpoint of the batch! Half of my batch is almost over :( I wish I could be in batch forever, and work on learning and building new things every day! I took some time to reflect on the past 5 weeks. These are the goals …
Welcome to my blog. ICYMI, I'm using it to post about how I spend each day at RC, so that I can look back upon the things I learned, and the interactions I had, things that can otherwise be so ephemeral! This post got some replies on Zulip, which means …
Some days ago I set up minikube to put together a small demo (or at least some screenshots) for a talk. After a minikube start , it has been taking a long time to come up for me (maybe because of my smol laptop), and it also throws some errors before …
tl;dr a.out is short for "assembler output", the filename of the output of Ken Thompson's PDP-7 assembler. It remains the default output filename for executables created by compilers like gcc , even though the created files actually are not in the original a.out format! [ source ] Today I paired …
Today I attended the "Intro to WebAssembly" session by Ezzeri. He talked about some useful performance benchmarks that he's been working on. I went on to play with wasmer-python which is a WebAssembly runtime for Python! It lets you execute WASM binaries with an API similar to how you'd execute …
For the past month, I've been working in Eastern Time , while being physically located in Indian Standard Time . Which means I wake up around 12-2pm IST and go to sleep around 5-6am IST. Timezones are weird. Over the weekend I attended PyConline AU in Australian Central Standard Time ( Curlyboi Standard …
Today I paired with Ezzeri to look at some of his Exercism Rust track solutions. The discussion moved to WebAssembly and we went through some minimal examples he has been working on. He's also written a nice blog post which has some beginner level WebAssembly resources. The whole exercise filled …