Recent content in Home on Peter Baumgartner
tl;dr: Evaluate agentic tooling in realistic end-to-end agentic tasks, not in isolation. At the task level, costs are driven more by the persistence of context through many turns than by the token count of any single turn. Intro Organizations are entering the tokenmaxxing hangover stage. And with that, lots of tooling is popping up claiming to reduce token usage. I’m not an organization, but…
Recently I’ve been using Quarto to generate HTML reports to share with stakeholders and I’d like to share a workflow and configuration that has worked for me. Background Most of my programming I do in VSCode. A typical workflow involves using an IPython REPL for exploratory coding, organizing commonly used code into a package, and creating CLI functions with typer as an abstraction for…
In this blog post I’ll walk through step-by-step how I wrapped the voronoice Rust crate and created a Python package with it. It’s written as a development journal, where I walk through my thought process and document the code and errors we’re getting along the way. My hope is that this makes the process a little more approachable for beginners and adds some transparency to the…
This year I decided to participate in Advent of Code (AoC) and use it as an opportunity to learn Rust. Since I was learning a new language, I also decided to try and use GitHub Copilot within VSCode. AoC is a series of 25 daily puzzles that typically need to be solved through programming. Each daily puzzle consists of two parts. This year I fully solved 18 of the 25 daily puzzles and solved part…
Note: If you haven’t read my prior post on the bootstrap and inter-rater reliability, this post probably won’t make sense. Go read that first. After I had began discussing my last post with a few people, they had begun to replicate my analysis. Their replications raised an important issue I hope to address here: estimation is dependent on the initial sample of annotations that we have.…
One of the most frequent questions that arises when doing applied ML projects is “How much data to I need to label?” When I get asked this question, I usually ask a few questions in return: what’s the base rate of the outcome that you’re labeling? Are you experimenting or building a production-ready system? How ambiguous or well-defined is your annotation task? Is all of…
“Writing has so much to give, so much to teach, so many surprises. That thing you had to force yourself to do—the actual act of writing—turns out to be the best part. It’s like discovering that while you thought you needed the tea ceremony for the caffeine, what you really needed was the tea ceremony. The act of writing turns out to be its own reward.” - Anne Lamott
This week my pal Vicki tweeted this, which I disagreed with: The funny part is that every single response to this is correct. Should you learn Python? Yeah. Should you learn K8s? Yeah. Should you focus on SQL? Yeah. https://t.co/i9KHigqLXL pic.twitter.com/ZCOvI9PIVI — Vicki (@vboykis) April 20, 2022 I responded that beginners shouldn’t learn K8S, and Vicki said they might not have a…
Since starting work at Explosion, I’ve been trying to learn more about Cython. About 16% of spaCy’s codebase is Cython, so I decided to pick up a book and learn from that. I did a few example projects and started thinking: now that types are cool in python, why don’t more people use Cython? In case you’re unfamiliar with Cython, here’s my incremental and…
In this blog post I’ll walk you through the workflow I use for managing virtual environments and creating python packages. I have a long list of criteria I’ve used to develop this workflow and have honed it over time since the start of my career as a data scientist. The factors that I’ve determined are critically important for choosing these tools are as follows: They work for me…
I’m joining Explosion AI as a Machine Learning Engineer. This is my first career move in 6 years and I thought I’d take some time to reflect on my personal experience in data science and natural language processing. Since I’ve been in data science, I’ve been working in professional services/consulting environments. My last job was working mostly with social scientists and…
This is a short blog post on two things I’ve found helpful when I’m not using notebooks and running python code that I want to log things in. Loguru Here’s an example of how I use loguru. Typically I don’t include all the boilerplate at the top, but for illustration of some of the functionality I’m including it. In this snippet I’m changing the default logger…
In my work, writing tests serves three purposes: making sure things work, documenting my understanding, preventing future errors. When I was starting out with testing, I had a hard time understanding what I should be writing tests for. As a beginner, I just assumed my code worked–I was staring right at the output in a notebook and visually inspecting that the output was correct. After…
I’ve worked on several projects where we’re applying some natural language processing techniques on responses to open-ended survey items. Typically this means putting them into categories—either following a pre-existing coding scheme or by creating a new one with unsupervised learning. Through these experiences, I’ve developed a few principles for working on these types of…
Context: I’ve recently been experimenting with porting portions of a simulation codebase from python to Julia. Setting up a productive development environment, using the packages (PyJulia & PyCall) that allow for communicating between python and Julia, and familiarizing myself with Julia enough to use those packages took quite a bit of time and experimentation. Here’s my collection of…
tl;dr: Load data up to 77x faster with django-postgres-copy and an in-memory csv. Go to results. When starting a new Django project often my first step is to create a model and bulk load in some existing data. As I’ve learned more about Django and databases, I’ve learned a few ways to speed up the data loading process. In this post I’ll walk through progressively more efficient…
We work on a lot of projects where we need to iterate quickly on visualizations generated from a flat file of data. Tableau is a great tool to get started with this, but its ease of use can quickly lead people towards illusions of progress, misunderstandings of the underlying data, or technical debt. Usually this comes up in a conversation like this: Them: I want to build this specific type of…
TL;DR The Rule-Based Matcher in spaCy is awesome when you have small datasets, need to explain your algorithm, locate specific language patterns within a document, favor performance and speed, and you’re comfortable with the token attributes needed to write rules. I created a notebook runnable in binder with a worked example on a dataset of product reviews from Amazon that replicates a…
Link to Notebook What’s in this notebook? This is an updated version of my TSNE to Bokeh Scatterplot workflow. I found that UMAP is faster and is able to handle larger datasets where TSNE would previously fail, so I’ve switched over to UMAP as my dimensionality reduction default. I’ve been building a lot of dashboards and visualizations in the plotly ecosystem, so I’ve also…
These are the main points and resources for my talk, Applied NLP: Lessons from the Field, delivered at spaCy IRL 2019. Summary Natural Language Processing projects often fail in their conception, in their delivery, or in their impact. To identify good candidate problems for NLP, talk to the client presenting you with a problem and first discuss how you would solve the problem without NLP. To…
Why did the chicken cross the road? Because it had no legs. These are the types of hilarious jokes the gpt-2 small model can generate for you. After reading a few blog posts here and here, and playing around with gpt-2 small myself, I thought I would write up the full process I used to fine-tune and produce generative text. For this example, we’ll use a dataset of jokes pulled from the…
Summary: I wrote this document based on my experience validating a risk assessment instrument. These were some of the issues, rewritten for generalizability, that we encountered. Accessible, quality data is often a project bottleneck, and I’ve found these helpful to consider before working on a project with criminal justice data. Comments: I wrote these notes in the context of evaluating a…
Summary: These are some notes, combined with my own experience and commentary, derived from Matthew Honnibal’s PyData Berlin 2018 talk: Building new NLP solutions with spaCy and Prodigy. I intended to use these as a reference when starting new NLP projects. In NLP and ML we talk a lot about models and optimization. But this isn't where the battle is really won! I've been trying to explain my…
Links Link to Notebook What’s in this notebook? This is the notebook behind my blog post Holy NLP! Understanding Part of Speech Tags, Dependency Parsing, and Named Entity Recognition. It’s an exploration into three common NLP tasks applied on the Bible as a corpus, with some pandas aggregations and seaborn plotting to round things out.
Introduction When we think of data science, we often think of statistical analysis of numbers. But, more and more frequently, organizations generate a lot of unstructured text data that can be quantified and analyzed. A few examples are social network comments, product reviews, emails, interview transcripts. For analyzing text, data scientists often use Natural Language Processing (NLP). In this…
What are word embeddings? Imagine if every word had an address you could look up in an address book. Now also imagine if words that shared meaning lived in the same neighborhood. This is a simplified metaphor for word embeddings. For a visual example, here are simplified word embeddings for common 4- and 5-letter english words. I’ve drawn 3 neighborhoods over this embedding to illustrate the…
TL;DR If you want to test ipython magics you can do the following: Import the global ipython app with from IPython.testing.globalipapp import get_ipython Crete an object with the global ipython app with ip = get_ipython() Load your magic with ip.magic('load_ext your_magic_name') Run your magic with ip.run_line_magic('your_magic_function', 'your_magic_arguments') (Optional) Access results of your…
This post will be an exploration into Earth Mover’s Distance as well as its application to NLP problems through Word Movers Distance. To get started, we’ll follow the benign pedagogical path of copying the Wikipedia definition: The earth mover’s distance (EMD) is a measure of the distance between two probability distributions over a region D. In mathematics, this is known as the…
Links Link to Notebook What’s in this notebook? This is the notebook behind my blog post An Exploration in Earth & Word Movers Distance. It’s an exploration into Earth/Word Movers Distance algorithm that includes a there’s a lot of great matplotlib plots and some pandas-fu.
Links Link to Notebook What’s in this notebook? This is a worked example derived from my blog post on Making the Most of spaCy’s Rule-Based Matcher. It works through developing a matching algorithm to identify reasons people purchase products from a dataset of Amazon product reviews. For example, you can automatically generate a list like this for any product: Customers buy this…
Risk Assessment tools are currently used to assist in decision-making at several points in the criminal justice system. These tools take in some data about an individual and to provide a ‘risk score’ for an individual that’s reflective of their likelihood of committing a specific behavior in the future. A standard outcome of interest is recidivism, or a person's relapse into…
Links Link to Notebook What’s in this notebook? This is the notebook behind my blog post The Impact of Model Output Transformations on ROC. It contains some seaborn plots, some pandas-fu with method chaining, a simulation of analyzing model results, and some plots with seaborn.
Links Link to Notebook What’s in this notebook? The Receiver Operating Characteristic (ROC) curve is helpful in evaluating model performance, especially since Area Under the Curve (AUC ROC) has a several friendly interpretations. I use ROC curves in evaluating models I have to explain the model performance to non-technical folks. I was reading through Machine Learning: The Art and Science of…
Links Link to Notebook What’s in this notebook? Biclustering is an unsupervised learning algorithm that clusters both the rows and columns. This notebook contains a workflow for completing biclustering in a network analysis context as well as a clean export of the data to an excel spreadsheet.
Link to Notebook What’s in this notebook? This is a workflow I use often in data exploration. TSNE gives a good representation of high-dimensional data, and Bokeh is helpful in creating a simple interactive plots with contextual info given by colors and tooltips. This workflow has been extremely helpful for: text analytics/NLP tasks if text data is passed through a TfidfVectorizer or similar…
Note (2018-04-09): I no longer use Pelican as the engine to build my blog, so you want be able to see parts of this workflow in this blog's repository. The insipiration for ideas in this post is captured in this notebook from Chris Albon. Switching static site generators is a great way to kill a few hours on the weekend. I was previously using Jekyll because it works seamlessly with Github Pages,…
PyData Carolinas Recap & Presentation Reflection I was fortunate enough this year to attend the first PyData Carolinas conference, though my attendance was only made possible with the development and delivery of a tutorial talk. My colleague Rob and I proposed and then delivered a 90m tutorial talk on using NetworkX to do Social Network Analysis in Python (repo, video coming soon). The talk was…
Note (2018-04-09): Slack's API has changed since I wrote this article. I also never wrote a part 2. If you want an up to date tutorial this blog from DigitalOcean is good. Part 1: Setting Up Our Workflow and a Simple Application A few weekends ago my pet project was to set up a drive time slash command in Slack. Searching through our organization’s Slack conversation history, on top of…