This post is about open source burnout, but it's not about adopters feeling entitled to my time, or disagreements between maintainers, or anything like that. My personal experience of working on open source Scala projects for the past decade has been almost entirely positive, with only one or two exceptions that I can think of off the top of my head, out of dozens of thousands of…
This document (originally published on GitHub ) contains additional supporting material for the claims in this post (published 1 September 2019), in response to a cease and desist letter from John A. De Goes that I received on 6 July 2020. De Goes is the CEO of a Scala consultancy named Ziverge , the founder of a project named "ZIO" (not to be confused with the racist slur ), and the organizer of…
The Fantasyland Code of Professionalism (FCoP) is a code of conduct developed by the Fantasyland Institute of Learning , an organization that was founded by John A. De Goes and is responsible for LambdaConf , a functional programming conference. Many other people have written about the shortcomings of the FCoP as a code of conduct, including Christie Koehler , who calls it "beyond mediocre" and…
I might try to fill in micro-reviews at some point, but right now this is just a list, in approximately descending order of how important the book was to me. The Accidental by Ali Smith (2005) 🔗 A Little Life by Hanya Yanagihara (2015) 🔗 Hotel World by Ali Smith (2001) 🔗 1Q84 by Haruki Murakami (2009) 🔗 Drive Your Plow Over the Bones of the Dead by Olga Tokarczuk (2009) 🔗 Invisible Man by…
Update: the experiment described in this post is now available in the Cats 2.2.0 pre-releases, starting with 2.2.0-M1 , which was published in March 2020. This post is an attempt to provide some additional context and justification for an experiment that I've been working on as a proposal for a future version of Cats (probably 3.0). The argument is that by moving Cats's type class…
Scala provides a handful of language features that are designed to make it easy for users to define and work with algebraic data types (ADTs). You write some case classes that extend a sealed trait, you write some functions that pattern match on those case classes, and you're done: you have a nice linked list or rose tree or whatever. Sometimes you can't use case classes to implement…
I've written a couple of blog posts about how the Parallel type class has changed in Cats 2.0 , but those posts don't really say much about why someone using Cats should care about Parallel in the first place. The name suggests that it has something to do with running computations at the same time, and while that's one of things you can do with it (via the instance for IO in…
Typelevel has just published Cats 2.0.0 , and while the core modules are guaranteed to be binary compatible with 1.x, there are some changes that break source compatibility. Most of these changes are unlikely to affect users, but a few will, and the goal of this post is to point out which those are and what you can do about them. Note that while some of the stuff below is pretty intense, it's…
(Apologies for the title—after a lot of time on Twitter this week I've been feeling nostalgic for things like Tumblr c. 2010.) This post is an attempt to answer a question Baccata64 asked on Reddit yesterday afternoon: how does the Parallel change not break bincompat ? Is it that type parameters and type members are encoded the same way at the bytecode level ? The context is that Cats…
Update (25 July 2020): John De Goes has hired a lawyer to send me a cease and desist letter demanding that I delete this post. See my responses here and here for more details. This post is a collection of links about John De Goes that show some clear patterns of behavior: De Goes defending white supremacists and misogynists. De Goes attacking critics and accusing them (especially women) of lying.…
Last Sunday I got a notification from GitHub about an issue someone had opened against an open source project I maintain. The issue description was one line: Publish for Scala 2.13.x please. Its a dependency of Eff I couldn't remember ever having interacted with the author, and the phrasing seemed kind of weirdly demanding coming from a total stranger, so I did a Twitter search for the…
I had a lot of fun doing this thing a couple of years ago , so I thought I'd go ahead and give up the pretense of this being a tech blog and do it again. I don't drink now so it wasn't as fun as last time, but if I convince one person to read Suttree or The Little Friend (or to skip Lincoln on the Bardo) it'll have been worth it. I've included Amazon links for convenience…
At least as far as my reading went, 2016 wasn't a great year for fiction, but I did have more time for reading fiction than I've had since I was a grad student and that was my job. This isn't the usual kind of thing I do with this blog, but here are thirty or so one-or-two-sentence reviews of some of the novels that I read for the first time this year (sorted roughly in decreasing…
Yesterday morning John De Goes published a blog post explaining why he and the other LambdaConf organizers decided not to uninvite an outspoken defender of slavery and lots of other vile stuff from their conference. If you think that keeping Yarvin on the program was the right choice, then this post isn't for you. I think you're wrong, and it's pretty likely I also think your…
I'm publishing this article as a blog post (rather than as part of the circe documentation) because it's intended to be a discursive overview of the representation of JSON numbers in circe 0.3.0 , not as an up-to-date description of the implementation in whatever the current circe version happens to be when you're reading this. For information about JSON numbers in circe versions…
This post is a kind of sequel to my previous article on type classes and generic derivation , although unfortunately there's a lot of intermediate content that should go between there and here that I haven't written yet. This post introduces a new feature in circe that I'm pretty excited about, though, so I'm not going to worry about skipping over that stuff for now. One of the…
I'll start with the story of how I got saved, since it's kind of relevant. Back when I was an English Ph.D. student, I worked on a number of projects that involved natural language processing, which meant doing a lot of counting trigrams or whatever in tens of thousands of text files in giant messy directory trees. I was working primarily in Ruby at the time, after years of Java, and at…
Suppose we're writing a service that accepts JSON requests and returns some kind of response. If there's a problem with a request—it's not even valid JSON, it doesn't match the schema we expect, etc.—we want to return an error, and of course it'd be nice if these errors were actually useful to the caller. Unfortunately "useful" in this context can mean lots of different…
Yesterday I wrote a Stack Overflow answer about using Shapeless for generic derivation of type class instances, and this morning I started putting together some new documentation for circe 's generic derivation, and after a few paragraphs I decided that it might make sense to write a blog post that could serve as a bridge between the two—between simple examples like the one in my answer…
Today is my last day at Twitter (well, kind of—my email access was shut off this morning, so I've been spending the day reading a novel). I'm not sure how I feel about this yet. After some very recent changes in the Twitter Open Source team, I'd been vaguely thinking about leaving the company anyway, and since I'm not a proud person (at least I don't think I am), I have no…
I've always really liked this passage from On the Genealogy of Morals : [T]here is a world of difference between the reason for something coming into existence in the first place and the ultimate use to which it is put, its actual application and integration into a system of goals… anything which exists, once it has come into being, can be reinterpreted in the service of new intentions,…
Suppose we've got a simple representation of a user: case class User ( id : Long , name : String , email : String ) Now suppose we're writing a web service where we allow clients to post some JSON to a resource to create a new user. We get to pick the id , not the client, so we might accept something like this: { " name ": " Foo McBar ", " email ": " foo@mcbar.com " } If we're using…
Tomasz Nurkiewicz recently published an article arguing that the fold on Option (new in Scala 2.10) is unreadable and inconsistent and should be avoided. I personally disagree about the unreadability part and the should be avoided part, which isn't too surprising, since I generally disagree with Tomasz . I have a lot of respect for him, though, and I can actually get on board with a good…
It's not unusual in Scala to want to take a collection with items of some algebraic data type and partition its elements by their constructors. In this Stack Overflow question , for example, we're given a type for fruits: sealed trait Fruit case class Apple ( id : Int , sweetness : Int ) extends Fruit case class Pear ( id : Int , color : String ) extends Fruit The goal is to be able to…
This post is another entry in my series on stuff you should never do with macros in Scala, but that you could do with macros in Scala, if you really wanted to, and if you'd picked up a bottle of Macallan on the way home from work and were willing to waste half an evening doing something ridiculously useless. It's specifically a response to this Stack Overflow question , which asks if…
Scala provides lexicographic Ordering instances for TupleN types when all of the element types have Ordering instances. It doesn't provide the same instances for case classes, however—probably just because lexicographic order isn't what you want for case classes as often as it is for tuples. Sometimes you actually do want a lexicographic ordering for your case classes, though, and Scala…
This Stack Overflow question is interesting—it asks whether we can use Scala macros to create a value class for positive integers where the positiveness is checked at compile-time, and where it's not possible to create an invalid instance. I'm pretty sure it's not. My first thought was to turn PosInt into a sealed universal trait with a private value class implementation in the…
Quasiquotation is an old idea (Miles Sabin notes the term in a 1937 Quine paper, for example) that's now available in Scala (thanks to the efforts of Eugene Burmako and Den Shabalin ), where it allows us to avoid the nightmarishly complicated and verbose code that's required to construct abstract syntax trees manually in our macros. Quasiquotes are a little like reification, but much…
My attempt to sneak the terms vampire and zombie into the Scala vernacular seems to be succeeding , so here's a new one: Potemkin definitions : definitions in a macro-constructed structural type that are intended only to make an expression passed as an argument to another macro method typecheck before that macro rewrites it. I came up with the trick to support this horrible abuse of Scala…
I've written several times about vampire methods , which are macro methods inside a macro-defined type, where the macro method's implementation is provided in an annotation. Normally when we define a type in a def macro, it looks like a structural type to the outside world, and calling methods on a structural type involves reflective access in Scala. Vampire methods allow us to avoid…
When macros first showed up in Scala as an experimental language feature last year, many Scala developers responded with skepticism or distaste . They argued that macros were a distraction from work on more urgent problems with the language, that they would lead to even more complex and reader-unfriendly code, etc. After a year and a half I think these arguments have less weight, as macros have…
This is a quick follow-up to my post last night about stream processing with iteratees. It's worth pointing out that we can accomplish much the same thing even more concisely using Haskell's lists: import Data.Char ( isSpace ) import Data.List ( mapAccumL ) import Data.List.Split ( splitWhen ) nextPage (page, _) = (page + 1 , 0 ) nextLine (page, line) = (page, line + 1 ) locator = snd .…
This blog post is a short response to my MITH colleague Jim Smith , who several weeks ago published a blog post about a stream processing language that he's developing. His post walks through an example of how this language could allow you to take a stream of characters, add some location metadata to each, and then group them into words, while still holding onto the location metadata about…
I wish I could take credit for what I'm about to show you, because it's easily the cleverest thing I've seen all week, but it's Eugene Burmako's trick and I've only simplified his demonstration a bit and adapted it to work in Scala 2.10. First for the setup. Start your REPL like this to have it print the tree for every expression after the compiler's cleanup…
I like writing code. I also like not writing code, especially when I'm writing code. Type providers are a particularly nice way not to write code. They let you take some kind of schema (for a relational database, RDF vocabulary, etc.) and turn it directly into binding classes at compile time—with no worrying about managing generated code, etc. I've wanted type providers in Scala for a…
It's sometimes useful in Scala to have a type with a single value. These are called singleton types , and they show up most easily in the context of Scala's objects. For example, if we have the following definition: object foo { def whatever = 13 } We can refer to a type foo.type that is the singleton type for foo —i.e., the type that contains nothing except foo . We can use this type to…
Suppose we want to define an HListable trait in Scala that will add a members method returning an HList of member values to any case class that extends it. This would let us write the following, for example: scala> case class User ( first : String , last : String , age : Int ) extends HListable defined class User scala> val foo = User(" Foo ", " McBar ", 25 ) foo: User = User(Foo,McBar, 25 )…
We've recently started using the W3C 's banana-rdf library at MITH , and it's allowing us to make a lot of our code for working with RDF graphs both simpler and less tightly coupled to a specific RDF store. It's a young library, but also very clever and well-designed, and it does an excellent job of exploiting advanced features of the Scala language to make its users'…
I began writing this post as an answer to this Stack Overflow question about learning how to use Shapeless , but it ended up a little long for a Stack Overflow answer, so I'm posting it here instead. When I first started teaching myself type-level programming in the context of Shapeless last year, I spent a lot of time working through simple problems with heterogeneous lists of type-level…
I just noticed that the Lawrence Berkeley National Laboratory's Nux library provides streaming XQuery functionality that makes it very easy to do the kind of XML processing that I described in this post last week. Using Scala, for example, we can start with some imports: import nu.xom.{ Builder, Element, Nodes } import nux.xom.xquery.{ StreamingPathFilter, StreamingTransform, XQueryUtil }…
People say that Validation is Scalaz's gateway drug, which might be accurate if you ignore the suggestion that there's anything even remotely fun about validation. In my book, making sure that your program doesn't choke on bad input is always a chore. Applicative validation is at least a step in the right direction—it makes it easier to write less code, introduce fewer bugs, and…
In my field (computational humanities), people like to distribute databases as enormous XML files. These are often very flat trees, with the root element containing hundreds of thousands (or millions) of record elements, and they can easily be too big to be parsed into memory as a DOM (Document Object Model) or DOM-like structure. This is exactly the kind of problem that streaming XML parsers are…
I chose Hakyll over other static site generators for this blog in part because Hakyll is built on Pandoc , John MacFarlane's fantastic document conversion library. Unfortunately Pandoc's syntax highlighting for Scala is incredibly bad: import scala. language . experimental . macros import scala. reflect . macros . Context object TupleExample { def fill[A](arity: Int)(a: A): Product =…