Recent content on Yuvi
I’m really excited to be one of the co-chairs of the Maintainers track at SciPy 2026 . It’s happening July 13-19 2026, and I’d love for you to submit a talk. In particular, I want to list some talks that I wish existed, but don’t right now. There are many people qualified to give these talks, but may not think they are. I know many, many people (including me!) would love to…
What is this? During the JupyterHub leadership meeting (North America) of June 2025, the breakout group comprising of Yuvi, Martha Cryan and Ryan Lovett came up with the following diagram to explain the aspirational mission of JupyterHub: This document is Yuvi’s interpretation of one of the technosocial changes required to achieve this aspiration. It’s also heavily influenced by prior…
As many of you know, I have been doing a lot of intense therapeutic work over the last few years, with the primary goal of being more consistently emotionally regulated under a variety of circumstances. I have observed a few hard fought but helpful changes in my behavior, and wanted to explicitly write about them. The primary audience is people who may have known me for a long time in specific…
Recently, I started becoming aware that I hold opinions and desire choices outside what is considered “mainstream” in many circumstances. Some examples are: Motorcycle safety Circumstances under which I will be wearing an N95 mask (far more often) Social Media use (A few more that you already know about if you know me well enough) When I was younger, this used to bother me. Observing my behavior,…
Jeff Nippard (the science communicator you watch if you want to spend your evening reading scientific papers and going to the gym) wrote a beautiful post recently about handling harassment and bullying at Gyms. One part that really stuck out to me was: Also, every gym has at least one “big [person]”. Almost every big [person] I know is a good person. I’m sure big [person] already all know this,…
If I want to share a YouTube video with someone while on my phone, here is what I need to do: Click the ‘Share’ button on the YouTube app Click ‘copy link’ Paste it into a textbox where I can edit the URL (so if I am using iMessages on the iphone, I have to use a text editor app in the middle) Delete the ?si=ajflsdkjf at the end of the URL added by Google so it can…
Julia pre-compiles packages on first load, allowing them to deeply optimize the generated code for the particular CPU architecture the user is running on for maximum performance. However, it does take some time, so there’s a startup latency penalty here. If you make docker images with precompiled Julia packages, you can pay this pre-compilation penalty at image build time , rather than at…
Why write this? This August / September, I had a lot of life changes in one go: Moving from an apartment i disliked to one I absolutely love, and maybe even feel at home in, for the first time in my life Officially switch jobs, becoming an employee of a non-profit I had co-founded Catch COVID for the first time and recover from it, with appropriate MH breakdowns from the isolation Get on…
Trying to start these back up, as a list of the kind of ‘work’ I have done the last few weeks. Let’s see if this works! Last 2 weeks Spent a good chunk of time investigating and fixing issues with AzureAD auth for the UToronto hub. “Fixing” it meant moving off AzureAD, so we only have CILogon & GitHub in the 2i2c infrastructure as authentication providers! This is now…
I’ve had a bit of a slump the last few months, working through what feels like deep mental fog for totally unclear reasons. The lack of clarity on causation is one of the most frustrating parts of this, and one of the most familiar too - this isn’t the first time I’ve felt like this, and I know (rationally at least, if not emotionally) it won’t be the last. Probably a…
I walked into a local hardware store, with the following thought in my mind: I need a 5mm metric hex key And I wandered around aimlessly as I do in stores, almost bought some double sided tape for a different project (decided against it), and eventually made it to the place with all the sockets, hex keys, screwdrivers, etc. Words that previously meant nothing to me (Torx, SAE, metric) made sense .…
If you are running a JupyterHub on Kubernetes and use NFS for home directory storage (a common occurance), I highly recommend the following settings: singleuser : extraEnv : # notebook server writes secure files that don't need to survive a # restart here. Writing 'secure' files on some file systems (like # Azure Files with SMB or NFS) seems buggy, so we just put runtime dir on # /tmp. This is ok…
Alex Merose asked me on Mastodon how to best set up a conda environment on GitHub Actions. I thought I’d write a short blog post about it! The short version is, use the conda-incubator/setup-miniconda GitHub action instead of the official setup-python action. Specifically, try out these options for size: steps : - uses : conda-incubator/setup-miniconda@v2 with : # Specify python version your…
I am not smart enough to consistently write and debug shell scripts that use any conditional or looping constructs. So as soon as I’m writing something in bash that requires use of those, I switch to python. This works fine when writing scripts, but what to do when writing GitHub Actions workflows? You can only write bash in run: stanzas in your step s, right? Not at all! You can set the…
I decided to write publicly and publish my motorcycle riding trip notes now. Montarey Trip notes Need to learn to plan when to pee . This is important to do! Can’t just get off any exit, as that often doesn’t actually lead to wilderness but to more concrete jungles. Need to learn more about lane splitting and how and when to do it safely. I did it for a little bit while stuck on CA-1S,…
I wanted a single keyboard shortcut that would let me switch between the built-in terminal and the editor in vscode. I couldn’t find any, so I made a short macro using the VSCode Macros Extension First, you define your macros in settings.json . You can open up vscode settings with Cmd+, , search for macros , and edit this under Macros: List . "macros.list" : { "showTerminal" : […
Sometimes you want to temporarily disable a JupyterLab extension on a JupyterHub by default, without having to rebuild your docker image. This can be very easily done with z2jh’s singleuser.extraFiles , and JupyterLab’s page_config.json JupyterLab’s page_config.json lets you set page configuration by dropping JSON files under a labconfig directory inside any of the directories…
This post is inspired by a slack conversation with Ryan Abernathey and Sarah Gibson . It is very important to me that we don’t exist in a world where the Internet is just centralized and controlled by a few huge, hypercapitalist players. I’ve written about ways the open source community can help here ( by not tying ourselves to a single provider ), and helped draft the right to…
Sharing notebooks is harder than it should be . You are working on a notebook (in Jupyter, RStudio, Visual Studio Code, whatever), and want to share it quickly with someone. Maybe you want some feedback, or you’re demonstrating a technique, or there is a cool result you want to quickly show someone. A million reasons to want to quickly share a notebook, but unfortunately there isn’t a…
(Leave comments or discuss this post on the jupyter discourse ) nbgitpuller is my most favorite way to distribute content (notebooks, data files, etc) to students on a JupyterHub. The student mental model is ‘I click a link, and can start working on my notebook’, which is as close to ideal as we have today. That is possible since all the information required for this workflow is…
I’m paying money to learn datascience with R via this John Hopkins course on Coursera. Hopefully I can spend about 4h a week on it. Why? I try to have a ‘T’ shaped set of competencies - some competence in a lot of areas, and deeper knowledge in some. Having some competence in many areas lets you notice where a particular skill can be very helpful in solving a problem, and either…
JupyterHub was designed to provide multi-user access to Classic Jupyter notebook and JupyterLab, running on machines far away with access to data & compute your local machines don’t have. However, there are a lot of users this leaves out - R users on RStudio, folks who prefer ssh based interfaces, people on local IDEs like vscode or pycharm, etc. We trust that these users know better than…
Submitting ’non-interactive’ jobs that can run independent of a user’s current session on a JupyterHub is a pressing problem. There are many many solutions to this, but I think there’s a lot of value in having an extremely simple solution that solves exactly one extremely well defined problem. We will only consider JupyterHubs running on top of Kubernetes, since that has…
The Littlest JupyterHub is an extremely capable hub distribution that I’d recommend for situations where you expect, on average, under 100 active users. Why not Kubernetes? The primary reason to use Zero to JupyterHub on k8s over TLJH in cases with smaller number of users would be to reduce costs - Kubernetes can spin down nodes when not in use. However, you’ll always have at least one…
You can run your Jupyter Notebook locally, and connect easily to a remote dask-kubernetes cluster on a cloud-based Kubernetes Cluster with the help of kubefwd . This notebook will show you an example of how to do so. While this example is a Jupyter Notebook, the code will work any local python medium - REPL, IDE (vscode), or just plain ol’ .py files Latest executable version of this notebook…
I needed to find out if an organization was using GSuite for their emails, so I can allow their users to login to a JupyterHub I was setting up. I use Google Auth in other places, so I wanted to find out if this organization was using GSuite. I could’ve just asked them, but where’s the fun in that? Instead, you can use dig to find out! For example, if I were to test berkeley.edu , I…
Entitled users burn out maintainers. While my massive burning out wasn’t directly related, it didn’t help either. For a lot of other maintainers, it has been a primary source of burnout. This is an unfortunate reality of our time. There are a ton of stories from maintainers out there -see this for a recent example. Not going to rehash that. However, I’ve recently found myself on…
Talking to Ankur Sethi made me think more clearly about why I want note-taking , bookmarking setups . I am trying to stop feeling so overwhelmed by moving things out of my puny brain . The solution to this is two fold. Commit to and do fewer things. Already working on this. Be more efficient in the things you are doing. A few years ago I gave up on chasing efficiency gains because they were all so…
In conversation with Nirbheek about my blog post on note taking , he said something like ‘my system of knowledge has holes, and I lose track of stuff I learnt because of that’. This was slightly complementary to my earlier quest - the outliner helped me in structuring how I thought and produced content. However, a bookmarking system will help me consume and retain knowledge. Both need…
This is a page with links to people’s blogs I like.
I am trying to find a nice way to keep thoughts, ideas and notes outside my head for a few months now. I worked my way through many applications, trying to use the system they promote as the basis for my own. Along the journey, I got a better sense of what was important to me in such an application. I’ll walk through the various applications I’ve tried, along with the criteria…
This person misses blogs a lot . So do I. I made many frients, built myself a platform, and expressed myself in ways I do not feel like I can anymore on my various blogs. Some of them are lost to time, but some are there if you know how to find it. Part of it was where my life was - I had time, nobody physically nearby I jived with, and a lot of angst. But now, I have too little time, and too many…
Physical activity I walked for about 6 miles yesterday evening, after doing 3-4 miles each day for about 2-3 days before. The roads were empty, and it was lovely. I’ve listened to maybe 6-10 hours of Deborah Frances-White in the last week or so, split between The Guilty Feminist and Global Pillage . Next time though, I’m going to try listen to podcasts less & observe my surroundings…
Gracefully exiting asyncio application Continuing yesterday’s work on my simple supervisor library , I continued trying to propagate signals cleanly to child processes before exiting. I remembered that it isn’t enough to just propagate signals - you also have to actually reap them. This meant waiting for wait calls on them to return. I had a task running concurrently that is waiting on…
I have enjoyed keeping running logs of my coding work (devlogs) in the past, and am going to start doing those again now. This ‘holiday’ season, I am spending time teaching myself skills I sortof know about but do not have a deep understanding of. JupyterLab extension I spent the first part of the day (before I started devlogging) working on finishing up a jupyterlab extension I…
This post is from conversations with Matt Rocklin and others at the PANGEO developer meeting at NCAR Today, almost all of ’the cloud’ is run by ruthlessly competitive hypercapitalist large scale organizations. This is great & terrible. When writing open source applications that primarily run on the cloud, I try to make sure my users (primarily people deploying my software for their…
I’m writing up monthly ‘work plans’ to plan what work I’m trying to do every month, and do a retrospective after to see how much I got done. I work across a variety of open source projects with ambiguous responsibilities, so work planning isn’t very set. This has proven to be somewhat quite stressful for everyone involved. Let’s see if this helps! JupyterCon…
Inspired by conversations with Nick Bollweg and Matt Rocklin , I experimented with using conda constructor as the installer for The Littlest JupyterHub . Theoretically, it fit the bill perfectly - I wanted a way to ship arbitrary packages in multiple languages (python & node) in an easy to install self-contained way, didn’t want to make debian packages & wanted to use a tool that people in…
This idea comes from brainstorming along with Lindsey Heagy , Carol Willing , Tim Head & Nick Bollweg at the Jupyter Team Meeting 2018. Most of the good ideas are theirs! The name is inspired by one of favorite TV series of one of my favorite people. I really love the idea of JupyterHub distributions - opinionated combination of components that target a specific use case. The Zero to JupyterHub…
Recently I had to write some code that had to call the kubernetes API directly, without any language wrappers. While there is pretty good reference docs , I didn’t want to go and construct all the JSON manually in my programming language. I discovered that kubectl ’s -v parameter is very useful for this! With this, I can do the following: Perform the actions I need to perform with just…
I haven’t done a ‘year in retrospective’ publicly for a long time, but after reading Alice Goldfuss’ 2017 year in review decided to do one for me too! This is a very filtered view - there are lots of important people & events in 2017 that are not contained here, and that is ok. Professional New Job I finished around 6-ish years at the Wikimedia Foundation, and joined UC…
The wonderful Graham Dumpleton asked on twitter why we built an entirely new tool ( repo2docker ) instead of using OpenShift’s cool source2image tool. This is a very good question, and not a decision we made lightly. This post lays out some history, and explains the reasons we decided to stop using s2i. s2i is still a great tool for most production use cases, and you should use it if…
I was at maintainerati today, which was super fun & quite intense! I highly appreciate GitHub & the individuals involved in making it happen! Here’s my key takeaways from this (and several other conversations over the last few weeks leading up to this): I am now a maintainer , which is quite a different thing from a core contributor or just a contributor . The power dynamics are very…
I’ve been reading Designing Data Intensive Applications book & am using this post to keep notes! I’ve picked up ideas on scaling systems through the years, but never actually sat down to actually study them semi-formally. This seems like a great start to it! It’s a pretty big book, and it’s gonna take me a while to go through it :) Will update these notes as I go! Trying to…
Following a trail from a wonderful Julia Evans post led me to Allen Downey’s nice textbook manifesto . Also led me to the nice Think OS book, which seems like a super nice introduction to Operating System principles. It is short enough (~100 pages) that I wanted to read through it. I’ve spent a good chunk of time absorbing how Operating Systems work by dint of diving into things and…
I am trying to understand SELinux and AppArmor, and collecting resources here as I learn. k SELinux for mere mortals (2014) This was the first video I watched, and it helped me understanding what SELinux does at a fundamental basic level. It’s probably useless in a container-filled world (where I doubt Fedora shipes pre-configured SELinux rules for my containers), but it helped me think I…
This is a running list of things I want to build! There’s an analogous running list of things I want to learn . Things move between them :) I also have higher standards of documentation (other people should be able to use it) before marking these as complete. kubernetes-login A helper to openssh that allows users to log in to a configurable user pod running on a kubernetes cluster. Should…
I was in a conversation about the Python GIL with friends a few days ago, and realized that my understanding of the specifics of the GIL problem were super hand-wavy & unstructured. So I spent some time collecting resources to learn more, and now have a better understanding! Python’s Infamous GIL (Larry Hastings) This was a great introduction to the history of the GIL, why it was necessary &…
I earlier had a vaguely working setup for making sure browsers, shells and other applications don’t eat all RAM / CPU on my machine with systemd + sudo + shell scripts. It was a hacky solution, and also had complications when used to launch shells. It wasn’t passing in all the environment varialbes it should, causing interesting-to-debug issues. sudo rules were complex, and hard to do…
Update : There’s a follow-up post with a simpler solution now. Ever since I read Jessie Frazelle ’s amazing setup ( 1 , 2 , 3 ) for running GUI applications in docker containers, I’ve wanted to do something similar. However, I want to install things on my computer - not in docker images. So what I wanted was just isolation (no more Chrome / Firefox freezing my laptop), not…