I would like to talk to you about black-market, performance-enhancing automotive accessories. I don't know much about them, to be honest, besides a few acronyms that I can't define, but a few hours on the reddits has convinced me that this is more than enough to qualify as an expert. It certainly qualifies me for blog-writing, which Aristotle famously called the "lowest form of art" 1 ;…
Stupidly Obscure Programming in a Troubled Time Since obsessively underlining passages in a tattered copy of Goodbye to Berlin hasn't proven to be the uplifting diversion I was hoping for, I resolved to bash my head against some really complicated scala code that I'm not qualified to write and that nobody is asking for either. So that this exercise in self-abuse could pass for a reasonable use of…
Donald Knuth famously opined: “The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming.” I broadly agree. It's incredibly difficult to guess which parts of our code will turn out to be performance bottlenecks, and, conversely, which…
Select, few Recently, there's been a lot of interest in DSLs to address the so-called N+1 selects problem , which describes the very common situation where you Do one database query to get a list of things, say user IDs; for each, call some sort of processing function that happens to do another query, say for each user's name. It can be even worse than that, as in this example, where (for some…
(See Updates section at end for latest notes.) In the middle of the journey of our life I came to myself within a dark wood where I was no longer comfortable identifying myself as a Mac user. Superficially, the relationship with my MacBook air had been rewarding. It lasted all day, ran the necessary few bits of commercial software I needed, and otherwise did a great job of pretending to be running…
Thou whoreson zed! thou unnecessary letter! - King Lear, 2.2.61 Partial functions are great in theory. By "great in theory", I mean that the basic idea is pretty simple but with very little additional research you can make weighty pronouncements involving the words "in" and "theory", as in, "in category theory, something something surjective algebra manifold." The simplest example of a partial…
Recently, I came across some horrible, horrible code. I immediately pasted it into a messaging application and quickly received an expression of solidarity. With what though? My indignation? Grief? Amusement? I find it suspicious that code I consider horrible tends to have been written by people I already disliked for some reason. Here are some things I have complained about recently: Code that…
Holy cow! Can you believe your luck? What better way to spend some portion of whatever day of the week it is than to think about parallelizing reduction algorithms? Really? You can't think of one? In that case, I'll kick off our inevitable friendship by contributing a clojure implementation of a fancy algorithm that asynchronously reduces arbitrary input streams while preserving order. Then I'll…
Who knew that paranoia could be so generous? Not only does it allow you to worry about a larger fraction of the world's hazards, it can even increase the number of potential hazards for you to worry about. I read ads on the subway, so I know that you can't be too careful these days. There are no bounds, for example, to the treachery my dog might be perpetrating while I'm hard at work, and it is…
Zoltar I took an "Impostor Syndrome" quiz consisting of the following 6 statements When people praise me for something I've accomplished, I'm afraid I won't be able to live up to their expectations of me in the future. At times, I feel my success has been due to some kind of luck. Sometimes I'm afraid others will discover how much knowledge or ability I really lack. When I've succeeded at…
Synecducers Great computer languages, like monarchs and planets, become emblems of what surrounds them. The greatest computer languages are barely there, as nearly everything we file under their names could be described as a library or other customization. It's not unusual and not even absurd to find a question about socket() on a C language forum: Linux is arguably a morphogenic implication of C…
Mutt: What will be the appropriate designation for the upcoming period of solar rotation in the Anno Domini system as devised by St. Dionysius Exiguus? Jeff: Why don't you just look at the pie? Mutt: Couldn't you just answer the question? Why do you always have to be a sarcastic asshole? Jeff: Seriously, look at the pie. Mutt: Huh. Yeah. I see what you mean. Sorry dude. Jeff: No problem, my…
In the alternate universe where I am very honest, my résumé contains a line item for the aggregate year I've spent fiddling with init.el . It does not, however, list emacs lisp among the languages I know, because (in this alternate universe) I have too much self respect to flaunt cut and paste skills. In the present universe, this is going to be awkward, because I will have to present really…
To everyone who attended yesterday's NYC Clojure Meetup: thanks for listening to me, asking good questions and providing some pretty great answers as well. Here are the slides. For more detail on nearly everything, see previous posts. (Navigate using the compass arrows. Up/Down within a section; Left/Right betwen sections; ESC for overview.)
Were you at Clojure/conj in Washington last week? If so, hello again. Wasn't that a great conference? If not, head to Clojure TV , where all the talks are ready for streaming. Assuming some moderate level of Clojure obsession on your part, I couldn't recommend skipping any of them, so the full catch-up might take you a while, but there are two in particular that I strongly recommend. Avoiding…
How strange to think that, a mere week ago, the world had not yet heard my public pronouncement that transducers ought to be stateless . True, the media frenzy has died down a bit, but in its stead comes the quiet awareness that life will never be the same. Anyway, that's the way I like to think about it. TL;DR Storing state in the transducer makes it mutable, which might be unfortunate on general…
After my recent attempt to provide type annotations for transducers, several people pointed out that I wasn't accounting for state. The signature of a pure function transformation, whether in Clojure ( t/defalias Transducer ( t/TFn [[ a :variance :covariant ] [ b :variance :contravariant ]] ( t/All [ r ] [[ r a -> r ] -> [ r b ->r ]]))) or Haskell type Transducer a b = forall r . ( r -> a -> r )…
TL;DR As noted earlier, transducers can be properly annotated in Clojure using core.typed and they probably should be. But... there are a few tricks necessary to make it work. Transducers in Scala require tricks too, but different ones. Oh, but they're so lovely in Haskell. Update 2015-01-12 Were you led here by Clojure Gazette? Eric Normand is usually more discriminating, but don't worry, this…
Back in August, I wrote two posts about an experimental framework for distributed functional programming called Girder . The idea, in summary, was to make distributed code look as much as possible like ordinary Clojure, as opposed to structuring it explicitly around message passing (as in the actor model) or data flows (as in map/reduce, storm, et al). As I say, it was an experiment, but it was…
In two previous posts, I went on about lenses in Clojure. Pinholes comprised a small library of higher order functions to formalize and simplify the viewing and manipulation of complex nested structures. Tinholes did essentially the same thing, but with macros instead. In both cases, there's recursion going on, as we burrow through layers of nesting, but macros had the advantage of doing it all…
In a previous post , I built up a framework for lens-like constructs in Clojure: essentially some fancified versions of assoc-in and get-in to allow for bidirectional transformations along the nesting path and some utilities to generate special-purpose getter/setter functions. The name, "pinhole," is supposed to suggest a more primitive, utilitarian mechanism for achieving focus. While still ruing…
Sseveral comments regarding the pinholes post , have forced me, against the deepest elements of my nature, to engage in thought. Since that might never happen again, I thought it meet to record the event. I'm going to say "I" a lot, because this is mostly my opinions. Bidirectional programming As pointed out by Christian Schuhegger in a comment on the original post, lenses were originally…
Lenses are a construct for getting, "setting" or "modifying" values within data structures, especially deeply nested data structures. The quotation marks have the usual meaning when they show up in funktionsprache: 1 not mutating anything per se , but instead producing an object, or reference thereto, that is identical except for the requested change. In Scala, the need for lenses is pretty…
TL;DR It's not too long, but, to summarize the summary, if you read Rich Hickey's 2014 blog post on transducers first, his 2012 post on reducers will be easier to understand. Brief Definitions Herewith, all in one place, are Clojuresque definitions of: reducible reducing function transducer reducer folder decomplected Longer elaborations of these definitions follow in the subsequent section.…
In part 1 and part2 of this almost unbearably exciting series, I outlined the concept of distributing purely functional programs and went through some implementation details of Girder . So far, however, I've only asserted that it works, so today I want to start explaining how I used Amazon Web Services to test pretty cheaply on gobs of machines. The art of AWS wrangling was somewhat new to me, and…
I have a confession to make. I don't find comedy funny. There are exceptions - Gilbert Gottfried and Louis CK can occasionally reduce me to putty - but, by and large, I appreciate comedy from a technical rather than aesthetic standpoint, combined with an admiration for and empathy with the drive that compels people to perfect their craft. I have similar feelings about magicians. Having spent a…
Update 2015-01-12 The algorithm as it exists in HEAD is somewhat different from the below, in ways that I'll describe (eventually) in an another post. In some ways, it's closer to Fork-Join, but with important differences to support reentrancy, share results of duplicate requests and adjust for the costs of distribution. Recap of recap In a previous post , I introduced a framework called Girder…
Ever realized you needed to modify some dependency (e.g. I needed to add weak cache support to core.cache ), go a head with forking on github, lein install your new version, modify your project.clj to pick it up, and then... somehow... you still seem to get the old version? That's usually because some other dependency is explicitly requesting a version of that library. If you do lein classpath ,…
Update 2015-01-12 The algorithm as it exists in HEAD is somewhat different from the below, in ways that I'll describe (eventually) in an another post. In some ways, it's closer to Fork-Join, but with important differences to support reentrancy, share results of duplicate requests and adjust for the costs of distribution. OSS and Commercial Grids Grid computing has always suffered the reputation of…
Overkill After reading my compelling post on a clojure Slack-bot, an astute reader 1 pointed out that using the fullblown apparatus of compojure, jetty, jvm, etc. for something this silly is really only justified when the entire purpose of the exercise is procrastination. Well, dear astute reader, that was the entire purpose, but you're right. Ok, here it is in perl, #!/usr/bin/env perl use…
Procrastination I really should be working on this obscure distributed RDF/FRP thing, but for various reasons my head isn't working properly write now. So I did this other stupid thing instead. Zippy Once upon a time, M-x yow in emacs would deliver a nice random quote from Zippy the Pinhead . Nowadays, you just get Yow! Legally-imposed CULTURE-reduction is CABBAGE-BRAINED! which has something to…
She was glad that the cozy house, and Pa and Ma and the firelight and the music, were now. They could not be forgotten, she thought, because now is now. It can never be a long time ago. Or, put another way, "It's constant, the moments, it's just - it's like it's always right now, you know?" So little Laura figures it all out at the age of 5, while Mason comes to the realization only after…
TL;DR There was this perplexing article in Ars Technica about replacing Fortran for scientific computation. The main performance advantage of Fortran is that its restrictive data structures facilitate vector optimization. There are many ways to match and even outperform Fortran, while using other languages. But it's OK to use Fortran if you want to. Ars Technica notwithstanding, Clojure, Haskell…
Time is confusing. Sometimes, it's so confusing that our thinking about it changes... over time. Which makes it triply confusing, or something. In this post, I want to talk about two aspects of temporal data, both common, useful and often misunderstood: Homogeneous vs Inhomogeneous Temporal vs Bitemporal Homogeneous Temporal Data The first thing to stress is that we're talking about homogeneous…
In honor of !!con , I was trying to think of programming-relevant uses of double exclamation marks. Other than denoting the end of a particularly funny programming joke (for example, a mangled homage to a famous paper ), it seemed the best place to look might be in the world of double factorials . As it turns out, that wasn't as fruitful as I'd hoped, but I learned something interesting on the way…
Marcus Ljungblad is going to give an amazing Scaladays talk about the process of learning Scala, and he put out a call to his fellow hackerschool ers for suggestions. I was glad to chip in, because I enjoy the opportunity to blurt out opinions without the obligation to structure them eloquently, and talking in front of people is scary, so it's awesome to have somebody else do it. Still, I kind of…
In my last post , I talked a bit about how FRP might look if state were maintained explicitly in persistent data structures, rather than in hidden mutable structures. The accompanying code was in Scala, but my first implementation was actually in Clojure. I was originally going to use the Clojure code in the post, but, having taken motivating example code from a Scala paper , it felt lazy to…
I totally agree with Paul Chiusano that the Reactive Manifesto not even wrong . In addition to the breezy non-falsifiability of its assertions, I have trouble with the name itself. Manifestos seldom work out well, in the sense of there not being a lot of corpses. (Plus which, "reactive" is acually a word, and a "reactive manifesto" doesn't sound like it would be very proactive, like.) BUT reactive…
Like any red-blooded American, I find regex golf fascinating. The idea, to paraphrase the comic strip, is to find a regular expression that matches all members of a group of related terms, but not any members of a different group. The hover text on the strip suggests a regex that matches all winning presidents. Peter Norvig went to town on this, first clarifying the problem and then building an…
No, not that wine . If you want to emulate Windows, just type :(){ :|:& };: in a bash session. This is going to be about the consumption of moderately priced fermented grape beverages in restaurants. Since my unrefined remarks are likely to upset a few people, I shall start by insulting them. There are three archetypes of wine experts: Comic-book Guy: Providing an outlet for obsessive accumulation…
What is Arrogance Recently, while reviewing some code, I found myself using the phrase, "breathtaking arrogance" and later wondering exactly what I meant. Well, I knew what I meant, but I did wonder what I might be implying. I feel reasonably confident that arrogance involves an offensive level of self-assuredness and claims of superiority. I am less sure about whether privately held (but possibly…
In another post , I prattled on at some length about the scala Set class. To understand its nuances, it was helpful to print out a graph of class and trait inheritance. Here's a contrived example that's simpler than Set : trait C1 {} trait C2 {} trait D extends C1 with C2 {} trait E1 extends D {} trait E2 extends D {} trait E3 extends C2 {} trait F extends E1 with E2 with E3 {} The hierarchy of F…
Around a year ago, there was a lively debate about the type invariance of the immutable Set in Scala. Dogpile argumentation on a subject far outside the popular interest is of course thrilling in itself, but the topic also provides a nice focal point for exploring and clarifying some important aspects of the Scala type system. We recall that Scala collections (and other higher kinded classes) can…
My pride in not being a "front end guy" is notoriously obnoxious and obviously compensatory. On the other hand, having crappy front ends for my projects might help disguise deeper flaws and thus actually be an advantage. Came a time, however, when I wanted to make myself a logo. Faced with the horrific prospect of doing actual art, I had no real choice but to use markup. Make no mistake, the…
The assumptions here are that you have a US land line, it has a phone number you've grown attached to, you or someone else in your household prefers to talk on something traditionally phone-like, you want to pay as little as possible for the privilege, that you have a high tolerance for - or ideally derive pleasure from - configuring gadgets, and that pains in the ass don't count as payment for…
Some time after my recent fiddles with IMDB , I read an interesting article about using a perceptron to classify words as parts of speech based on features that precede them in text. It's all done in python or some such sh*t, but whatever. Still very cool. Since I had all of this IMDB data accumulated in Mongo, I thought I would try to play with it, and the idea I had was to predict metacritic…
This is about a month old. Actually, scrap that. This kind of thing never gets old. To wit, a presentation on implementing and "using" Fractran , via clojure of course. The least silly aspects of the show are (1) that we can implement the language very concisely and functionaly, and (2) to interpret the results, there's a nifty sieve of Eratosthenes - the real one. The code is on github .
I am broadly sympathetic to view that scalable systems must be built with statically typed langages, for reasons outlined in this wonderful screed , and, until recently, that has made it difficult for me to recommend clojure for institutional use. With the introduction of core.typed , that has changed. The author has says that core.typed is now production-ready, and I agree. It's not perfect, but…
At some point, everyone wakes up in the middle of the night, in a cold sweat of panic that they don't truly understand how to derive the Y-combinator. Well maybe not everyone, but at least me. (Note that I'm talking about the higher order function , not the startup incubator.) I ended up reading through quite a few web pages, all of which presupposed a slightly different background, before I…
"Remember when you told me to tell you when you were acting rudely and insensitively?" First things first. It it is certainly a bad movie. That isn't in itself so rare, of course, but it's nice to think that foul intent is always accompanied by shoddy artistry (which requires momentarily forgetting Leni Riefenstahl, but never mind). It would also be comforting if hateful crap were technically…