We have had our heads down building and working toward launching Kenn Software ’s product offerings later this year, but in the meantime, I wanted to give some insight into how our agentic engineering process and culture have evolved since the beginning of the year, and what a work day for us looks like. We merge hundreds of pull requests per week into our projects with a team of three people, and…
Getting coding agents (we say “clankers”: “agents” gives them too much credit) to behave in a reasonable manner is a full time job even with the latest frontier models. At Kenn , we spend so much time tuning their behavior across different repositories and projects that we started developing a “constitution” of sorts based on how we like to build software. We hope you find it useful: you can…
Like a lot of people , I’ve found that AI is terrible for my sleep schedule. In the past I’d wake up briefly at 4 or 4:30 in the morning to have a sip of water or use the bathroom; now I have trouble going back to sleep. I could be doing things . Before I would get a solid 7-8 hours a night; now I’m lucky when I get 6. I’ve largely stopped fighting it: now when I’m rolling around restlessly in bed…
I’m thrilled to announce msgvault , a new side project I have been building over the last two months with the assistance of many coding agents and some new agentic engineering tools I created (like roborev , a continuous agentic code review system). TL;DR msgvault is a local-first storage and retrieval engine for slicing, dicing, and querying a lifetime of email and messaging data in milliseconds.…
I have been building a lot of new software in Go. Except that I’ve never actually written a line of Go in my life. What is going on? Anyone following my LinkedIn or Twitter/X feed has seen that I have been down a coding agent rabbit hole in recent months. This has included: moneyflow , a TUI accounting system (written in Python) roborev , a continuous background code review system (written in Go).…
We are officially now in the era of “why not?”. Up until about 10 months ago, it was easy to come up with reasons not to build things. Coding agents have changed all that. I remember what we used to say: too expensive, too much time, not worth it, too little marginal benefit, does not benefit enough people . That’s over now. A side effect of this is that everyone should be asking “why not?” all…
This text of this post was written without AI assistance or editing. Wes does vibe coding Up until about March of this year, I regarded AI-assisted development with skepticism, avoiding LLM-powered autocomplete or “AI IDEs” like Cursor and Windsurf. Most of my career’s work from 2008 onward was done in emacs without even the assistance of LSP. In retrospect, a modest investment in developer…
Summary TL;DR I am joining Posit today as a Principal Architect where I will advocate for the needs of the PyData ecosystem in Posit’s work as well as continue advancing critical open source initiatives to accelerate progress in the polyglot “Data Science without Borders” mission . In this post, I will review some of the back story of how I got involved with Posit (known formerly as RStudio) and…
Summary TL;DR I am transitioning out of my full-time CTO role at Voltron Data so that I can expand my portfolio of entrepreneurial and open source data projects. While no longer serving in a full-time operational role, I will remain engaged as a Senior Advisor and will continue driving forward work to enable fast and interoperable analytical data systems. In this post I will offer some highlights…
A new joint VLDB paper on Composable Data Management Systems with Meta, Databricks, Sundeck, and others at is out! This post is a reflection on how I arrived at thinking about these problems and what the future might look like. Enjoy. Getting Started: 2008 to 2015 I started building data analysis tools a little over fifteen years ago, in April 2008. The world has changed a lot since then. Going…
Joint Post from Wes McKinney and Josh Patterson Allow us to reintroduce ourselves Too often people say “let’s do something together” in passing, and don’t. There’s the occasional inter-project collaboration, but rarely will people take that next step. There are countless reasons why this happens, and aligning goals is challenging to say the least. But after spending the last several years working…
The first quarter of 2019 has now wrapped up. In March we spent a good amount of time focused on getting the 0.13.0 Apache Arrow release out of the door. I will mention a few development highlights from the month and provide the full changelog of patches later in the post. Development Highlights We are continuing to set up our physical build and test cluster which we’ll use to run integration…
The team had a busy 28 days this February. The Apache Arrow community is discussing a 0.13 release toward the end of March, so we spent February helping the project toward the next release milestone. We have been pushing projects on multiple fronts and discuss some of those here. The Apache Arrow project just had its 3rd birthday , and we are pleased to report that the community is thriving and…
Ursa Labs had a busy January that went by too quickly. After a high-intensity 3 months of development, we helped release Apache Arrow 0.12 on January 20th. A good chunk of our time was spent fighting fires (in packaging and builds) related to the continued expansion of the project in recent months. The 0.12 release contains a new merged documentation site where you can expect more project-level…
For ten out of the last eleven years, I’ve lived in two places: New York City and San Francisco. The last two years have been in NYC. After founding Ursa Labs , a not-for-profit open source development group, I felt it was time to make my home somewhere that isn’t either of those places. After some contemplation and consulting many friends, I decided on Nashville, Tennessee. This blog explains…
I’m excited to announce that NVIDIA AI Labs has signed on as a supporter of Ursa Labs . NVIDIA’s new open source RAPIDS data science platform uses Apache Arrow for an interoperable representation of tabular data (data frames). We are looking forward to collaborating on our respective development roadmaps and growing the ecosystem of projects that use Arrow. This new financial support will enable…
Funding open source software development is a complicated subject. I’m excited to announce that I’ve founded Ursa Labs ( https://ursalabs.org ), an independent development lab with the mission of innovation in data science tooling. I am initially partnering with RStudio and Two Sigma to assist me in growing and maintaining the lab’s operations, and to align engineering efforts on creating…
Well-known database systems researcher Daniel Abadi published a blog post yesterday asking Apache Arrow vs. Parquet and ORC: Do we really need a third Apache project for columnar data representation? . Despite the somewhat confrontational title, based on his analysis, the answer is “Yes, we do” , but I have a number of issues to discuss, including in part the premise of the article. Storage and…
Earlier this year, development for the Feather file format moved to the Apache Arrow codebase. I will explain how this has already affected Feather and what to expect from the project going forward. Feather: How’s it related to Arrow? Shortly after we announced the formation of Apache Arrow in February 2016, Hadley Wickham and I met up and discussed how we could foster more collaboration in the…
This post is the first of many to come on Apache Arrow, pandas, pandas2, and the general trajectory of my work in recent times and into the foreseeable future. This is a bit of a read and overall fairly technical, but if interested I encourage you to take the time to work through it. In this post I hope to explain as concisely as I can some of the key problems with pandas’s internals and how I’ve…