Academic and journalist
I first need to apologize to and thank a guy named Chuck Morris. When we were students at the University of Pittsburgh in the previous century, Chuck and I had several things in common. We both worked at WPTS, the campus radio station. We both were geeks. And, perhaps most unusually given those two factors, we liked sports. OK, we loved sports, and in particular, basketball. I can’t remember…
I began collecting congressional press releases more than a dozen years ago for the reasons I usually start collecting anything: I was interested in them and there wasn’t a freely-available dataset I could find. At the time, I was writing code in Ruby since I was working at The New York Times, and so the library to scrape those releases, called Statement , was in that language. Writing scrapers in…
Of nearly all of the potential uses for Large Language Models, perhaps the best and most defensible is using them to write code. During the recently-completed fall semester, I co-taught a Computer Science class with Bill Pugh that explored some good ways to do that. We used a lot of Claude Code for that class, and it quickly has become my favorite coding model. And then, in the middle of November,…
Today I’m releasing two datasets on women’s college basketball, one an update and the other a brand new collection, both tied to the 2025-26 season. The first is roster data for 1,074 NCAA teams across all three divisions , and the second is coaching history data for as many NCAA coaches as I was able to scrape. I’ll describe them in turn, starting with the roster data. Roster Data This is the…
A few years ago I started publishing roster data for women’s college basketball , thanks to the assistance of students in my Sports Data Analysis and Visualization class at Maryland. The first few times were a painstaking and lengthy process, mostly due to slight but substantive variations in how colleges presented roster information on their websites. While many NCAA teams use SIDEARM Sports to…
When I started teaching at Maryland in 2021, one of the first things I did was pull up the university’s Schedule of Classes , something I still do all the time. It’s a (relatively) quick way to see what classes are being offered and how many seats are available in them. As browsable web apps go, it’s not too bad. You can start by looking through a single department/college’s offerings, or you can…
Today I’m releasing a new Python command-line library that allows users to ask questions of federal campaign finance filings by leveraging large language models, with some built-in help. This project, llm-fecfile , also builds on previous work by data journalists that helps make these filings easier to parse and use. The combination of LLMs and domain expertise is a powerful one that has some real…
In the 12-plus years that we’ve been turning official precinct election results into data at OpenElections , the single biggest problem has been converting pictures of results into CSV files. Many of the precinct results files we get are image PDFs, and for those there are essentially two options: data entry or Optical Character Recognition. The former has some advantages, but not many. While most…
As the students drifted away from the Zoom call one-by-one, one young woman stuck around. This was during the pandemic, and who wanted to be on Zoom more than they already had to be? I was wrapping up a talk about data journalism to a group of them, and there’s a line I use about the importance of data in understanding what’s really important to people and institutions. I usually phrase it like…
The first thing you should know about me is that I’ve been programming computers for at least two decades. The second (and more important) thing you should know is that I’m not very good at it. I like to tell people that my programming skills are good enough for journalism, which is to say that I’ve never received any interest from companies that do actual software development, the kind that…
I had been staring at my Political Email Extraction Leaderboard since I first published it, and was disappointed by some of the performances I found in extracting committee names from the disclaimers in fundraising messages. Some of the commercial models did worse than I expected, below 90% in some cases, and I was surprised. I should have been more skeptical. Once I dug into how I was compiling…
As a former Hill reporter, some of the best never-published stories revolved around the treatment (usually, mistreatment) of congressional staff. The people who actually make Congress work mostly don’t have a lot of power and are subject to the demands, schedules and whims of the lawmakers who employ them. It’s also tough to get a real understanding of how lawmakers actually understand and value…
What kind of sicko signs up for political fundraising emails from just about every committee? Oh, right, that’s me. Sure, there’s a certain masochism to this, but I’m genuinely interested in seeing how campaigns communicate their messages to prospective donors and the distance between what they are willing to say in an email to their supporters compared to other contexts. But, as with nearly every…
The following is based on a talk I gave at American University on Oct. 25, 2024. Since this is about Artificial Intelligence and Large Language Models, let me offer this initial baseline: I need to emphasize this not just as a journalism educator but as a participant in our shared civic life. There are many, many examples of people using LLMs as reference tools, and even some products that…
Whenever American federal officials - from the President to members of Congress and executive branch employees - meet with foreign government representatives, they often are offered gifts. Some of them are pretty weird , like crocodile insurance. Others are very pricey, such as gold medallions. Or paintings, like one given to Dwight Eisenhower from Konrad Adenauer, then the chancellor of Germany.…
If there’s anything that I’ve developed an expertise in in the past 25 years, it’s campaign finance data. And if there’s anything I’ve been really into during the past year, it’s large language models (LLMs). You might think this is a story about how I’ve combined the two. But it’s not, because mostly I don’t, and I’d like to explain why. Because most campaign finance data is structured - in rows…
I called it my “running with scissors” class, because after I offered to teach a course on using artificial intelligence for journalism, the thought dawned on me that I had very little idea how to actually design and teach such a class. Running with scissors, a terrible image from DALL-E Here’s what I eventually settled on: give students access to various AI services - ChatGPT, Claude, Gemini and…
When it comes to turning official precinct election results into data, there are few states like Mississippi. The Magnolia State isn’t like New York, where there’s no centralized statewide reporting system for precinct results (not yet, at least). But the 82 counties in Mississippi participate in a frustrating system: they produce a precinct report in the same electronic format, then print out…
Last year some students of mine at Maryland and I worked to produce a comprehensive set of data on women’s college basketball rosters , and while that process involved a lot of manual effort on top of a base of Python scrapers, it also got me thinking: how hard would it be to extend that system to other sports? As it turns out, it wasn’t that hard, which is why today I’m happy to announce the…
If there’s one state that justifies the need for OpenElections, it’s Mississippi. Unlike some other states that don’t have a centralized election results system, Mississippi does. Each of the state’s 82 counties uses the same form to report official results to the Secretary of State. But that form is quite a thing in terms of organization: Statewide Results There’s a lot going on here, but the…