Recent content in Posts on Potentially Useful
I was proposing my doctoral dissertation fifteen years ago when neural network machine learning models entered a second Renaissance. Over the ensuing decade they eclipsed other techniques for classifying and generating many kinds of digital material—including images, sound, and text. The technical details aren’t important here, but the text-generating models support a neat trick: they…
They say it takes six weeks to form a habit, but as far as I can tell, the research suggests an average closer to two months—depending on the habit being undertaken and the individual being studied. Nevertheless, I find myself six weeks into an attempt to build a new habit to help with my writing, and some links have come my way that feel relevant to the effort. Specifically, I’ve…
I’m not making as many data graphics at work as I had in previous positions, and I sometimes miss it. I’m hoping to convert an infographic that my team produces collaboratively with a designer (who uses InDesign) into an entirely code-based process, but it’s a low-priority project and I need to make sure nobody’s feelings would be hurt about the transition. Helena Jambor kindly put together a data…
At work I’m helping my team transition a large code base (really, a collection of code bases) from a “legacy” closed-source data analysis tool to a popular “big data platform” which presents an interface based on computational notebooks . I’ve had reservations about notebooks for a while, and this work has sharpened those frustrations. So much so, that I considered submitting a talk to the DataBS…
Most folks who would ever read this weeknote probably already have a good understanding of how LLMs work, but this blog post from Josh Sharp is a great explainer I foresee sharing with others. It reminds me of the first half of this presentation that Emily Bender gave almost exactly two years ago, but which I only came across recently: ChatGP-why: When, if ever, is synthetic text safe,…
I learned a lot from this piece on “Expert Generalists” , certainly (at least in part) because I identify as such. At every step in my career, I’ve gotten a lot of mileage out of being my organization or group’s “best X at doing Y” (where X and Y are variously related to different science, engineering, or programming tasks), without being distinguished at X or Y…
Microsoft will stop supporting Windows 10 after October this year. This has been well-publicized in some corners of the internet, but I suspect that not enough folks know that their computers will soon become security hazards (as newly discovered exploits fail to get patched), and upgrading them to Windows 11 won’t be possible due to hardware limitations. These are perfectly good computers…
This will be brief, but I’m proud of this one. I’m away on vacation for a long weekend and thought I’d treat myself by leaving my laptop at home. I accepted that I’d have to miss another weeknote, but that seemed like a small price to pay for freedom. Until… I had some quiet time while my kiddo napped, and finally decided to give Termux a go. I only needed to add openssh, git, and vim to…
I have become obsessed with this puzzle website: Every 5x5 Nonogram . I was unfamiliar with nonograms when this came through my feed, but have since learned that the genre originated in Japan and is probably best associated with Nintendo’s Picross games. A 5×5 nonogram is pretty easy, but that’s the secret to their addictiveness; once you learn the rules most of them require no thought at all, but…
Just a QR Code It does exactly what it says on the tin. This is an in-browser QR code generator that doesn’t add any tracking info to your links, or track you when you use it. Just a QR Code “The most radical thing you can do is care” A lot of folks in my network are sharing Dan Sinker’s reaction to the news about the sloppy LLM-generated “summer reading supplement” that was published in a couple…
It’s all stats and data science for this post folks! Weird ad-hoc analysis work vs. repeatable infrastructure work I enjoy reading how people with jobs similar to mine approach their work 1 , and here Au explains his workflow as a “mostly ad-hoc” data scientist. For example: [T]here’s a very strong tension between how much I leverage pre-existing code and my effectiveness writing new code.…
I broke my streak of “weeknotes” 1 and missed last week’s update. Many people will describe their lives as “busy” when pressed—I’m no different—but I was juggling a few extra responsibility and chose to let this one slide. If you’re gonna drop a ball, make sure it’s made of rubber, right? An easy to follow WASM primer I harbor some ambivalence about WebAssembly, like many of the…
Making more informative maps in R I love making maps in R—I was the go-to map-maker a couple jobs ago, and I believe that making a decent-looking map in a take-home coding assignment helped me land my current gig. But the histogram legend showcased in this tutorial really kicks things up a notch. Andrew Heiss — How to use a histogram as a legend in {ggplot2} How smart people get fooled…
Managing reproducible environments in R I saw a toot on Mastodon bemoaning the experience of using renv to manage “reproducible environments” in R. I’ve had the exact experience the author describes: More than half the time when I actually have to use it to restore a project that hasn’t been touched in more than a year or two renv::restore() fails with tons of errors, I struggle for hours to…
FFmpeg’s ASM lessons Assembly language (specifically the x86 instruction set) was actually one of the first “programming languages” that I learned, having come to computers through engineering. I don’t have any excuse to write assembly code these days, but if I wanted to now, I would definitely brush up by using these instructions from the folks behind the FFmpeg project (and of which I learned…
The whys and hows of alt text This piece nominally targets journalists posting to social media, but it does a great job of explaining why alt text is important and has guidelines for composing good descriptions. I could do a better job myself of composing alt text for the data graphics I’m called to produce 1 , so I’ll be referring back to this for guidance. Joe’s a pro at this game! Joe Murphy…
This post is a major departure from what I usually write. I’m feeling inspired to branch out by the April Cools’ Club (which I learned about one day late), but I also want to be less precious about what I post here in general. I played D&D online with friends during the Covid lockdowns, and since moving I’ve picked up playing—and more recently DMing—at a game shop in my neighborhood.…
pyOpenSci statement on DEIA A couple weeks back I shared a statement from rOpenSci reaffirming their commitment to diversity, inclusivity, equity, and accessibility. I’m glad to see similar values articulated by the folks behind pyOpenSci. Reaffirming pyOpenSci’s Commitment to Inclusion Typst and Pandoc I had been meaning to check out Typst for a little bit, but I never got around to it before…
Multiprocessing in Python is pretty broken A common scenario: you have some code Python that’s taking too long. But the algorithm is embarrassingly parallel and your computer has all these cores, so maybe you can speed it up by using them all. So you spend a day refactoring things to use multiprocessing , and… it doesn’t help at all. Congratulations, you just got bitten by the…
How to move R projects to Codeberg Last week I shared a link collaborating with Git and no code forge, but now I want to share a new (to me) resource for doing R on Codeberg, “a democratic community-driven, non-profit software development platform.” I have yet to use Codeberg, but if I ever develop an R package outside of work, that’s where I’d host the project. Resources…
rOpenSci statement on DEIA A thing that drew me to switch to R as my primary programming language for data analysis 11 years ago was the community of R enthusiasts that I found online 1 ; people were friendly and inclusive, and the vibes—in general—were excellent. This statement from rOpenSci on Diversity, Equity, Inclusion, and Accessibility reminds me that this community is still out…
Personal websites are in This week’s zeitgeist brought a bunch of new (to me) stuff about indie and personal websites to my attention. This page is under construction: A love letter to the personal website Why Personal Websites Matter More Than Ever Good Internet , a new magazine about the web that’s launching soon smallweb , “a curated collection of cool indie websites”…
Purrr goes parallel Literally the only thing I miss from my days of programming MATLAB is parfor , which made it dead simple to massively hasten slow but parallelizable algorithms. I’ve written parallel code in other languages, but nothing has been as easy as replacing for with parfor . The new .parallel option in purrr’s map functions looks to change that. This feature is only…
Astroterm: a terminal-based star map I walk my dog after dark most evenings, and I enjoy the excuse to look at the sky and try to identify the planets and constellations I can. Sometimes I check my work when I get back inside by spinning up Stellarium , a wonderful—but fairly heavyweight—open source planetarium. I learned about astroterm shortly after its recent 1.0.7 release, and…
Tinyplot 0.3.0 is on CRAN Tinyplot is “a lightweight extension of the base R graphics system, with support for automatic grouping, legends, facets, themes, and various other enhancements.” I consider myself to be something of a ggplot2 “power user”, but I see the appeal of a lightweight alternative and plan to try this; the examples on the home page look nothing like what…
Constructive pessimism beats false hope Do you feel like the state of the world is bad, is nearly certain to get worse, and that there’s virtually nothing you can do to change any of it? If you answered yes to all three questions, I think you have a more realistic understanding of our circumstances than many people. But that doesn’t mean you’re off the hook! Research described in this…
uBlock Origin is more than an ad-blocker This post from my friend @slgr@scholar.social does a great job explaining how useful uBlock Origin is for more than just blocking ads. In some cases, it makes the web usable on older hardware that couldn’t otherwise keep up with modern front-end web stacks. using old devices as a means to not doomscroll (and using ublock origin to help give life to those…
Fight Flash Fraud I came into possession of a fraudulent USB “thumb” drive, fully aware of what I was getting. The device claims to have a capacity of 2 TB, which is nearly impossible, but I wanted to keep it out of a landfill and I didn’t know its true capacity. I figured that one could find the last good sector of such a drive, create a partition that went up to that, and have a perfectly…
1001 Album Generator In 2005, a panel of music critics published the first edition of 1001 Albums You Must Hear Before You Die . Recently a web forum friend introduced me to 1001 Album Generator, which selects one random album (without replacement) each weekday for you to listen to and rate. As I write this, I’ve listened to 12 records, and while the list has all the problems you might expect 1 ,…
Happy Public Domain Day! January 1 marked the introduction of many significant works into the Public Domain. This is only the fifth year to see meaningful copyright expirations following the 20-year freeze legislated by Congress, so I still get excited about these. PDD pairs well with the resource Standard Ebooks , which is driven by a group of volunteers who clean up, beautify, and republish…
Henry Leach — Lego Trains on Brio Track My youngest child is three and obsessed with the toys in question, so when Henry posted a picture of a train made of LEGO-style plastic bricks running on Brio-style wooden tracks, I had to know his secret. Fortunately for me, he had already blogged about it a whole year ago! Brandon Rohrer — Write an HTTP client in Python Sometimes you need to…
Yesterday was dark and chilly, demanding a one-pan casserole for dinner. I sauteed the filling in my skillet, and once the gravy was ready and the flavors had married, I topped it with potatoes and moved the pan to a hot oven to finish. I cleaned the pan before work this morning. Hot water leftover from making coffee softened the dried residue, then I put a plastic bowl-scraper to task. I finally…
When I present data to non-technical stakeholders, I sometimes express differences (typically changes over time) in terms of percentage change . This indicator has the advantage of being unitless and normalized 1 , and is familiar to broad audiences (although some aspects are counter-intuitive—see below). The ratio of the difference between two observations ( x₀ and x₁ ) and the starting…
I have added a now page to my site and a link to it in the navigation footer below. This isn’t particularly noteworthy—I’m pretty boring, so its content will probably only interest a few of my friends and family members (… maybe ). However, because I have enough self-awareness to realize that I will “occasionally” let this become out of date, I devised a workaround that feels worth…
The complexity of our world is beyond the limits of human comprehension. In spite of this, we generally feel like we understand what’s going on around us. Each of us achieves this feat of self-deception by cobbling together an assortment of abstractions, mental models, schemas, and metaphors 1 . When life confronts us with yet another task that demands attention, we select the most appropriate of…
To view a list of vignettes associated with a package (for example, dplyr ), enter the following command in any R console: vignette (package = 'dplyr' ) You can also use: browseVignettes ( 'dplyr' ) To see a specific vignette, use: vignette ( 'dplyr' , package = 'dplyr' ) You can omit the package option when a vignette is uniquely named, and many packages (by convention) contain an introductory…
This is just a collection of interesting (and not necessarily recent) Git posts from around the web. I will update this as I find new things. Introductions Mikkel Paulson — Git is my buddy: Effective Git as a solo developer I love reading about how other people use Git, and like Mikkel I am usually using it as a “solo developer”. It’s interesting how different his approach is from my own. I…
SourceHut , a hacker 1 -oriented code forge , recently suffered a couple days’ downtime due to a sustained DDoS attack , and parts of it are still coming online. I’m impressed by the way Drew and SourceHut are handling the situation, but it’s a good reminder that “Git is a distributed version control system”. Code forges are useful tools—and I’m really…
The R package ecosystem has several packages 1 that were created to help with the design and layout of tables in documents. condformat ( Code forge ) DT ( Code forge , Website ) flextable ( Code forge , Website ) ftextra ( Code forge , Website ) formattable ( Code forge , Website ) gt ( Code forge , Website ) gtsummary ( Code forge , Website ) gtExtras ( Code forge , Website ) pointblank ( Code…
Today, I followed these instructions and “hibernated” my LinkedIn account. I had to take an extra step to disconnect my account from a “LinkedIn Learning Enterprise license”, which is two jobs old now (I hope it wasn’t costing them anything), but otherwise it was a painless process. Unless I find myself really missing access to the LinkedIn network, I expect to close…
I was wondering how to incorporate a test for equality into a pipeline using R’s new( ish ) native forward pipe operator 1 , so I asked the Fediverse and got great advice 2 . Why would I do this? I approach scripts differently than notebooks (whether using Jupyter or RMarkdown) since they’re usually meant to be run non-interactively. I try to keep the list of packages (or modules)…
I participated in December Adventure this year and posted nearly daily updates on this page . My project was written for uxn , as a follow-up to my recent exploration in PostScript , and heavily inspired by this amazing web tribute to Forth . My goal is to eventually build a game to help me learn Morse Code 1 , but for now I’m happy to have accomplished simple things like printing text to…
I needed a few pages of graph paper, and it seemed easier and more economical 1 to print some at home than to buy a whole pad of it. There are plenty of websites offering PDF files with grids on them, but I didn’t like what I found 2 and decided to make my own. I thought that the process was fun enough that I have since made a few different kinds: 5 mm × 5 mm grid (pdf) ( postscript ) 5 mm ×…
This is a departure from the things I normally post here, but I want to put an important PSA on the internet… and hopefully prevent some folks from getting ripped off. I may be the only “Eamon Caddigan” there is, and I’m almost certainly the only one in Southern California My house is not for rent Scammers are using readily available information 1 to pose as landlords and…
One of my top pet-peeves on internet discussion and help boards (dating back to the Usenet days, covering the web forum era, and continuing in this current age of Stack Overflow hegemony) occurs when a person replies to a request for help or information with a smarmy and dismissive, “have you tried searching the internet?” 1 . It’s rude and unhelpful, but there are plenty of rude…
I have a Python script that needs to be run from within a specific Anaconda environment (which I’ll call environment_name ) for reasons that aren’t relevant here. It was tricky to get this working, so I figured it was worth a post. Here’s what made this challenging: The script only works if it’s actually run in a specific environment (accessed by executing conda activate…
Andy Bell from Set Studio shared the outcome of a study of browser viewport sizes ; the area of the screen (measured in pixels) available for web pages. They shared the data so I thought I’d take a quick look at it. First let’s check the distribution of aspect ratios. With mobile browsing more popular than desktop, I expect tall displays to dominate the wide ones, but what do the data…
Every nerd knows that ISO 8601 is the “correct” way to display dates, but those who dig into the standard know that it encompasses more than YYYY-MM-DD format 1 . One of the stranger conformant formats uses week number; today’s date in this format is 2023-W31-3; i.e., today is the third day of the 31st week of 2023. I’ve been using this format for note-taking for a couple…
Google is getting worse 1 . Twitter is imploding 2 . Reddit is at war with its volunteer moderators . Facebook is doing… whatever Facebook is doing . It feels like the major hubs for “internet culture”, if such a thing can be said to exist, are reaching the later stages of “ enshittification ” at the same time. In my corner of the internet, the reaction to this…
A smart and talented friend, who’s a fellow data scientist by profession, came across a probability puzzle that’s supposedly given in some job interviews for data science roles. The problem asks the following: Assume the distribution of children per family is such that the probability of having 0 children is 0.30, 1 child is 0.25, 2 children is 0.20, 3 children is 0.15, 4 children is…