I’ve been watching AI development for a long time. I found LessWrong around 2012-2013, and managed to get myself worked up about the oncoming singularity. I managed to chill out about it, but interest and excitement for AI remained. The initial Deep Dream image generation, Alpha Go, etc, were all so exciting. And then GPT-2 came out. Over the last five years, people have been making wild claims…
How coarse-grained STM containers can livelock under load Edit history: 12-19: @teofilC on the Haskell Discourse remarked a place where TVar (Map _ _) may be appropriate - I’ve modified the post to incorporate this. Software Transactional Memory (STM) is one of Haskell’s crown jewels. The promise is easy, lock-free concurrency with guaranteed transactional semantics and great performance. Used…
Lucas Escot wrote a good blog post titled “Making My Life Easier with GADTs” , which contains a demonstration of GADTs that made his life easier. He posted the article to reddit . I’m going to trust that - for his requirements and anticipated program evolution - the solution is a good one for him, and that it actually made his life easier. However, there’s one point in his post that I take issue…
The Haskell persistent library provides a QuasiQuoter syntax for defining a Haskell datatype, along with code to convert it into a database table. However, there’s a bit of a subtlety here. Here is the documentation for the syntax on the QuasiQuoter . I’ll refer to it throughout this blog post. The conventional use of this library is to define a bunch of tables that represent the complete table.…
At work, someone noticed that they got a compiler warning for a derived instance of MonadTrans . newtype FooT m a = FooT { unFooT :: StateT Int m a } deriving newtype ( Functor , Applicative , Monad , MonadTrans ) GHC complained about a redundant Monad constraint. After passing -ddump-deriv , I saw that GHC was pasting in basically this instance: instance MonadTrans FooT where lift :: Monad m => m…
I’ve been learning cello again recently. The omnipresent advice for novices is to rent a cello from a reputable violin shop, which is great - my cello would be $2,700 new, but I’m renting it for $50/mo. And I can tell I’ll outgrow it, and I don’t know how to properly evaluate an acoustic cello for great suitability yet. I’ve got a practice mute, but it’s still rather loud - I can’t practice at…
GHC Haskell provides a type CallStack with some magic built in properties. Notably, there’s a constraint you can write - HasCallStack - that GHC will automagically figure out for you. Whenever you put that constraint on a top-level function, it will figure out the line and column, and either create a fresh CallStack for you, or it will append the source location to the pre-existing CallStack in…
I’ve been pondering a smart watch for tracking more of my daily activity and health, as well as getting some metrics on my non-cycling workouts. I had narrowed the field down to the Fitbit Charge, Apple Watch SE, and Garmin Fenix 6. The Fenix seemed like the winner - awesome battery life, navigation, and even music on board. However, even at a discount, it’s $450, so it must be awesome to be…
I’m happy to announce that my book “Production Haskell” is complete. The book is a 500+ page distillation of my experience working with Haskell in industry. I believe it’s the best resource available for building and scaling the use of Haskell in business. To buy the ebook, go to the Leanpub page - the price is slightly lower here than on Amazon. To buy hard copies, go to Amazon . Thanks to all of…
I was recently made aware that haddock hasn’t been working, at all, on the Mecury code base. I decided to investigate. Watching htop , haddock slowly accumulated memory, until it exploded in use and invoked the OOM killer. My laptop has 64GB of RAM. What. I rebooted, and tried again. With no other programs running , haddock was able to complete. I enabled -v2 and --optghc=-ddump-timings , which…
This is a really brief post to call out a nice trick for providing users a nice migration message when you delete a constructor in a sum type. The Problem You have a sum type, and you want to delete a redundant constructor to refactor things. data Foo = Bar Int | Baz Char | Quux Double That Quux is double trouble. But if we simply delete it, then users will get a Constructor not found: Quux . This…
Everyone loves Haskell because it makes concurrent programming so easy! forkIO is great, and you’ve got STM and MVar and other fun tools that are pleasant to use. Well, then you learn about asynchronous exceptions. The world seems a little scarier - an exception could be lurking around any corner! Anyone with your ThreadId could blast you with a killThread or throwTo and you would have no idea…
Apparently, I’ve spent my entire life with a condition called “femoracetabular impingement.” The bones in my hips are deformed - the femoral neck is too thick and mis-shapen, and I have a “pincer” on my acetabum which restricts range of motion even further. As a result, I wasn’t able to internally rotate my hips almost at all - I had a single degree range of motion (normal for the population is 45…
Exceptions kind of suck in Haskell. You don’t get a stack trace. They don’t show up in the types of functions. They incorporate a subtyping mechanism that feels more like Java casting than typical Haskell programming. A partial solution to the problem is HasCallStack - that gives us a CallStack which gets attached to error calls. However, it only gets attached to error - so you can either have…
I’m moving the programming stuff over to https://overcoming.software . Well, I will at some point in the future. But I don’t want to break links. So I need to setup a server at this domain which has a permanent redirect to the relevant overcoming.software domain. That’s a decent amount of work, which I don’t have time for, so this probably won’t happen any time soon.
RankNTypes is a language extension in Haskell that allows you to write even more polymorphic programs. The most basic explanation is that it allows the implementer of a function to pick a type, rather than the caller of the function. A very brief version of this explanation follows: The Typical Explanation Consider the identity function, or const : id :: a -> a id x = x const :: a -> b -> a const…
justifiably lazy orphans (alternative subtitle: “I used the TemplateHaskell to destroy the TemplateHaskell ”) (EDIT: 2021-11-05 - Having actually tried this approach in production, I now have an experience report and a warning. Scroll to the bottom for the details!) At the day job, we use the aeson-typescript library to generate TypeScript types from our Haskell types. One problem is that the…
I wrote a big thread on the company Slack to compare type families: open vs closed vs associated. I also ended up discussing data families, as well, since they are a good complement to type families. I’ll probably edit this further and include it in my book, Production Haskell, but here’s the Slack transcript for now: An associated type family is an open type family, but with the requirement that…
I want a better way of constructing Haskell records. Let’s compare and contrast the existing ways. We’ll be using this datatype as an example: data Env = Env { accountId :: String , accountPassword :: String , requestHook :: Request -> IO Request , responseHook :: Response -> IO Response } This type is an Env that you might see in a ReaderT Env IO integration with some external service. We can…
With the new OverloadedRecordDot language extension, we can use the . character to access stuff on records. {-# language OverloadedRecordDot #-} data User = User { name :: String } main :: IO () main = do let user = User { name = "Matt" } putStrLn user . name This is syntax sugar for the following code: import GHC.Records data User = User { name :: String } instance HasField "name" User String…
The hspec testing library includes many useful facilities for writing tests, including a powerful “hooks” capability. These hooks allow you to provide data and capabilities to your tests. SpecWith The typical hspec test suite looks like this: main :: IO () main = hspec specs specs :: Spec specs = do describe "math" $ do it "1 + 1" $ do 1 + 1 ` shouldBe ` 2 it "3 * 2" $ do 3 * 2 ` shouldBe ` 6…
TemplateHaskell is a powerful feature. With it, you can generate Haskell code using Haskell code, and GHC will compile it for you. This allows you to do many neat things, like quoted type safe literals , database entity definitions , singletonized types for type-level programming , automatic Lens generation , among other things. One of the main downsides to TemplateHaskell is that it can cause…
I’m investigating a way to speed up persistent as well as make it more powerful , and one of the potential solutions involves persisting some global state across module boundaries. I decided to investigate whether the “Global IORef Trick” would work for this. Unfortunately, it doesn’t. On reflection, it seems obvious: the interpreter for Template Haskell is a GHCi-like process that is loaded for…
This post is an investigation of persistent issue #1199 where an asynchronous exception caused a database connnection to be improperly returned to the pool. The linked issue contains some debugging notes, along with the PR that fixes the problem . While I was able to identify the problem and provide a fix, I don’t really understand what happened - it’s a complex bit of work. So I’m going to write…
Haskell’s type classes and deriving facilities are a killer feature for type safety and extensibility. Over nearly 30 years they’ve acquired quite a bit of cruft and language extensions. With DerivingVia , we now have the ability to dramatically simplify the deriving story. This post outlines a change to the language that would hopefully be adopted with the next version of the language standard.…
In plucking constraints , I talked about a way to shrink a set of constraints by partially concretizing it. At the end of the article, I show how to use it for errors. The plucky package documents the technique, and my upcoming library prio embed plucking into run-time exceptions, effectively solving the trouble with typed errors . Figuring that out has been bothering me for two and a half years!…
I recently wrote a library prairie to have “First Class Record Fields.” The overall gist is that I wanted a pair of functions: diffRecord :: Record rec => rec -> rec -> [ Update rec ] updateRecord :: Record rec => [ Update rec ] -> rec -> rec I want to be able to send these [Update rec] over the wire, so they needed to be serializable. The design choice I ended up with borrowed the EntityField…
I’m thrilled to announce that my book Production Haskell is released in alpha version. The first release has 240 pages of content, with much much more to come. If you want to buy the book or sign up to receive updates, click here .
So, Haskell has an amazing potential for writing correct code, but sometimes it doesn’t leave an obvious way to remember things. With record or product types, we don’t usually write direct pattern matches to access fields. You’re much more likely to see field labels as accessor functions, or an extension like RecordWildCards or NamedFieldPuns , or occasionally a field match. data User = User {…
The following discussion occurred on the FPChat Slack (invite link) . It was pasted into a gist by Emily Pillmore . However, the gist was mysteriously deleted. I offered to host the interesting discussion on my blog, which will hopefully preserve it. The gist was recovered from Google Cache and is reproduced here. Some formatting and emojis are probably wrong because I don’t have the original…
Raise your hand if you’ve been annoyed by imports in Haskell. They’re not fun. Imports are often noisy, lists are often huge, and diffs can be truly nightmarish to compare. Using a term often requires modifying the import list, which breaks your workflow. Fortunately, we can reduce some of the pain of these problems with a few choices in our stylish-haskell configuration and a script that…
You’re convinced that Property Based Testing is awesome. You’ve read about using PBT to test a screencast editor and can’t wait to do more. But it’s time to write some property tests that integrate with an external system, and suddenly, it’s not so easy. The fantastic hedgehog library has two “modes” of operation: generating values and making assertions on those values. I wrote the compatibility…
Mirror, mirror, on the wall, where is the skolem that escapes the forall ? This post is about reflection, reification, and (to get to the pragmatism) the use of the new DerivingVia mechanism to provide awesome codecs. What does reflection and reification have to do with any of this? Well, we’ll see, but first let’s dig into some code. Encoding and decoding JSON is a common problem, and you very…
There’s a Haskell trick that I’ve observed in a few settings, and I’ve never seen a name put to it. I’d like to write a post about the technique and give it a name. It’s often useful to write in a type class constrained manner, but at some point you need to discharge (or satisfy?) those constraints. You can pluck a single constraint at a time. This technique is used primarily used in mtl (or other…
A plea to Haskellers everywhere. Haskell has a hiring problem. There aren’t many Haskell jobs, and there aren’t many Haskell employees. Haskell employees tend to be senior engineers, and the vast majority of job ads want senior-level Haskell candidates. The vast majority of Haskell users do not have any professional production experience, and yet almost every job wants production Haskell…
Reddit user /u/qseep made a comment on my last blog post , asking if I had any advice for splitting up persistent model definitions: A schema made using persistent feels like a giant Types module. One change to an entity definition requires a recompile of the entire schema and everything that depends on it. Is there a similar process to break up a persistent schema into pieces? Yes! There is. In…
You’re a Haskell programmer, which means you complain about compilation times. We typically spend a lot of time waiting for GHC to compile code. To some extent, this is unavoidable - GHC does a tremendous amount of work for us, and we only ever ask it to do more. At some point, we shouldn’t be terribly surprised that “doing more work” ends up meaning “taking more time.” However, there are some…
Alternative, less click-baity title: Names Do Not Transmit Meaning People often complain about the names for concepts that are commonly used in Functional Programming, especially Haskell. Functor, monoid, monad, foldable, traversable, arrow, optics, etc. They’re weird words! Functor comes from category theory, Monoid comes from abstract algebra. Arrow comes from – well it’s just kind of made up!…
Haskell’s persistent database library is convenient and flexible. The recommended way to define your database entities is the QuasiQuoter syntax, and a complete module that defines some typical entities looks like this: -- src/Models.hs {-# LANGUAGE DeriveGeneric #-} {-# LANGUAGE EmptyDataDecls #-} {-# LANGUAGE FlexibleContexts #-} {-# LANGUAGE FlexibleInstances #-} {-# LANGUAGE GADTs #-} {-#…
To help write robust, reliable, and easy-to-test software, I always recommend purifying your code of effects. There are a bunch of tricks and techniques to accomplish this sort of thing, and I’m going to share one of my favorites. I have implemented a pure data pipeline that imports records from one database and puts them in another database with a slightly different schema. Rather than implement…
Algebraic datatypes are a powerful feature of functional programming languages. By combining the expressive power of “and” and “or,” we can solve all kinds of problems cleanly and elegantly. SQL databases represent product types – “and” – extremely well - a SQL table can correspond easily and directly to a product type where each field in the product type can fit in a single column. On the other…
We use the SQL UNION operator at IOHK in one of our beam queries, and esqueleto does not support it. To make porting the IOHK SQL code more straightforward, I decided to implement UNION . This blog post series will delve into implementing this feature, in a somewhat stream-of-thought manner. Background esqueleto is a SQL library that builds on the persistent library for database definitions and…
2018 was a bit of a rollercoaster. Like last year , I kept a detailed todolist in Workflowy . As a result, I can look back at my goals for the year and see how I worked through them and what I accomplished. One thing that I noted in my previous year’s retrospective was a desire to focus on non-software stuff, and I think I did fairly well on that. Physical Health Weight My goal was to get to…
Do you understand laziness? It’s okay if you don’t. Most people don’t. It can be somewhat surprising when something actually gets evaluated in Haskell, even when you’re using bang patterns. So, here is a quick quiz on laziness in Haskell! If it makes you feel better, I didn’t get it right either on my first try. You can copy and paste this into a source file and run it in GHCi. {-# LANGUAGE…
You, like me, program in either Haskell, or Scala, or F#, or Elm, or PureScript, and you don’t like runtime errors. They’re awful and nasty! You have to debug them, and they’re not represented in the types. Instead, we like to use Either (or something isomorphic) to represent stuff that might fail: data Either l r = Left l | Right r Either has a Monad instance, so you can short-circuit an Either l…
Gary Bernhardt has a fantastic talk on Capability vs Suitability , where he separates advances in software engineering into two buckets: Capability: The ability to do new things! Suitability: The ability to do things well. Capability is progressive and daring, while suitability is conservative and boring. Capability wants to create entirely new things, while suitability wants to refine existing…
I always forget the difference between a TChan and a TQueue . They appear to have an almost identical API, so whenever I need a concurrent message thing, I spend some time working out what the difference is. I’ve done this a few times now, and it’s about time that I write it out so that I don’t need to keep reconstructing it. Aside: Please don’t use TChan or TQueue . These types are unbounded ,…
… and your bugs smaller In my post “Type Safety Back and Forth” , I discussed two different techniques for bringing type safety to programs that may fail. On the one hand, you can push the responsibility forward. This technique uses types like Either and Maybe to report a problem with the inputs to the function. Here are two example type signatures: safeDivide :: Int -> Int -> Maybe Int lookup ::…
Supercharge your Haskell development experience with ghcid! ghcid is – at the current moment – the most important tool for Haskell development environments. It is fast, reliable, works on all kinds of projects, and is remarkably versatile. You can use it with any editor workflow, primarily by not integrating your editor! (though there are integrations available if you’re brave) For these reasons,…
There’s a kind fellow named lunaris on the FPChat slack channel that shares exceptionally good advice. Unfortunately, due to the ephemeral nature of Slack, a lot of this advice is lost to history. I’ve been pestering him to write up his advice in a blog so that it could be preserved. He hasn’t posted it yet, so I’m going to start posting his rants for him ;) lunaris works with a company called…