Recent content in Home on teideal glic deisbhéalach
A Typography & Layout Demo This is a typography and layout test page for my Hugo + PaperMod setup. This file is not intended to be indexed or accessible to crawlers. It uses realistic content and exercises as many common Markdown features as I’m likely to use, so that I can see whether my font and CSS changes actually look good. I’ll be using Serpentine Andada for body copy, Schibsted Grotesk for…
Stub page for verifying internal typography links.
Over the past few months, the Sigma engineering team at Facebook has rolled out a major Haskell project: a rewrite of Sigma , an important weapon in our armory for fighting spam and malware. Sigma has a mission-critical job, and it needs to scale: its growing workload currently sees it handling tens of millions of requests per minute. The rewrite of Sigma in Haskell, using the Haxl library that…
Maintenance status: I no longer actively maintain this software. Expect details here to be outdated. criterion helps you write reliable microbenchmarks without becoming a benchmarking expert. It automates the noisy details, so you can focus on understanding and improving your Haskell code. Features Simple, declarative API that hides the busywork of configuring and running benchmarks.…
Almost five years after I initially released criterion, I’m delighted to announce a major release with a large number of appealing new features. As always, you can install the latest goodness using cabal install criterion , or fetch the source from github . Please let me know if you find criterion useful! New documentation I built both a home page and a thorough tutorial for criterion.…
(Summary: I’ve developed some algorithms for a statistical technique called the jackknife that run in O ( n ) time instead of O ( n ²).) In statistics, an estimation technique called “the jackknife” has been widely used for over half a century. It’s a mainstay for taking a quick look at the quality of an estimator of a sample. (An estimator is a summary function over a…
I’m pleased to introduce the third generation of my attoparsec parsing library . With a major change to its internals, it is both faster and more powerful than previous versions, while remaining backwards compatible. Comparing to C Let’s start with a speed comparison between the hand-written C code that powers Node.js’s HTTP parser and an idiomatic Haskell parser that uses…
I threw together a little code tonight to calculate the Katz centrality of packages on Hackage. This is a measure that states that a package is important if an important package depends on it. The definition is recursive, as is the matrix computation that converges towards a fixpoint to calculate it. Here are the top hundred Hackage packages as calculated by this method, along with their numeric…
Since the beginning of April, David Mazières and I have been back in the saddle teaching CS240H at Stanford again. If you’re tuning in recently, David and I both love systems programming, and we particularly get a kick out of doing it in Haskell. Let me state this more plainly: Haskell is an excellent systems programming language. Our aim with this class is to teach both enough advanced Haskell…
Maintenance status: I no longer actively maintain this software. Expect details here to be outdated. wreq is an HTTP client library that aims to be pleasant, pragmatic, and type-safe. It builds on top of modern HTTP primitives to give you a clean, high-level API for working with web services. Features A single, straightforward interface for making requests and handling responses. Built-in JSON,…
It’s been exciting to see the positive reception in the tech press and the open source community to Hack , the new programming language that my team at Facebook released last week. At the same time, one part of that coverage has made me wince: a couple of articles give the impression that I created Hack, but I didn’t. What I do is manage the Hack team, and since I’m incredibly…
It’s time someone finally wrote a proper review of Simon Marlow’s amazing book, Parallel and Concurrent Programming in Haskell . I am really not the right person to tackle this job objectively, because I have known Simon for 20 years and I currently happen to be his boss at Facebook. Nevertheless, I fly my flag of editorial bias proudly, and in any case a moment’s glance at…
I just released new versions of the Haskell text , attoparsec , and aeson libraries on Hackage, and there’s a surprising amount to look forward to in them. The summary for the impatient: some core operations in text and aeson are now much more efficient. With text , UTF-8 encoding is up to four times faster , while with aeson , encoding and decoding of JSON bytestrings are both up to twice as fast…
Yesterday, Michael Snoyman reported a surprising regression in version 1.0 of my Haskell text library: for some invalid inputs, the UTF-8 decoder was truncating the invalid data instead of throwing an exception. Thanks to Michael providing an easy repro, I quickly bisected the origin of the regression to a commit from September that added support for incremental decoding of UTF-8. That work was…
For a while now, I’ve had it in mind to improve the encoding performance of my Haskell JSON package, aeson . Over the weekend, I went from hazy notion to a proof of concept for what I think could be a reasonable approach. This post is a case of me “thinking out loud” about the initial design I came up with. I’m very interested in hearing if you have a cleaner idea.
Quoth Wikipedia: Big data[1][2] is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. The challenges include capture, curation, storage,[3] search, sharing, transfer, analysis,[4] and visualization. The trend to larger data sets is due to the additional information derivable from…
[Edit: a few hours after I wrote this post, I wrote some code to get rid of the inflation phenomenon it describes, and I’ll publish a corresponding update to criterion shortly. See below for details, and the bottom for a new chart that shows the effect of the fix.] A couple of days ago, Alexey Khudyakov did a little digging into the accuracy of criterion’s measurements . I thought his…
A few days ago, my Facebook colleague Andrei Alexandrescu posted a note entitled Three Optimization Tips for C++ , which reminded me that I had unfinished business with Haskell’s text package . I took his code, applied it to the text package, and this is the story of what happened. The text package provides a type named Builder for efficiently constructing Unicode strings from smaller…
I have spent quite some time over the last couple of months improving the Haskell hashable library, and all of my efforts eventually turned into a near-complete rewrite of the library. The 1.2 release of hashable is not backwards compatible, and for several good reasons. Read on for the details. Hash flooding The threat landscape faced by applications that use hash-based data structures has…
I’ve recently been talking with Johan Tibell about submitting his hashable package to become a part of the Haskell Platform. Once we get that submission accepted, we can fold Johan’s excellent hash-based data structures from his unordered-containers package into the standard containers package. For many applications, the hash array mapped tries in unordered-containers offer a decent…
The case of the mysterious explosion in space A few months ago, reports began to filter in of an unhappy problem with the Haskell text package: it was causing huge object files to be generated when a file contained lots of string literals . I didn’t notice the initial report (it was posted to a busy mailing list that I don’t try to keep up with), but Michael Snoyman was kind enough to…
While I was in New York for QCon last week, the temperatures started out quite mild, but soared back to their usual sweltering summertime levels by midweek. I thus found myself confined to my hotel room for a few hours one afternoon, feeling grateful for the air conditioning. As I waited for the sun to go down so that I might venture out in the heat without being cooked by both air and glare, I…
Back in 2005, Andy Wingo wrote a neat little statistical profiler named statprof that promptly disappeared into obscurity. It has since languished almost unknown, with a handful of people writing semi-private forks that themselves seem to be dead. Statistical profiling (also known as sampling profiling) is simple and sweet: the profiler periodically wakes up and samples the stack, then when all is…
I’m approaching my fourth anniversary of using github. I should hardly have to state that it’s a wonderful service, and especially so for being kept freely available to the open source community. At the same time, I’ve noticed over the past year or so that in many ways I feel less efficient using it now than I used to, even though the github team continues to roll out new…
After months of work, and a number of great contributions from other developers, I just released version 0.4 of aeson , the de facto standard Haskell JSON library. This is a major release, with a number of improvements. Enjoy! Ease of use The new decode function complements the longstanding encode function, and makes the API simpler. New examples make it easier to learn to use the package.…
(Cross-posted from the MailRank engineering blog .) You may have seen my exciting news about our upcoming move to Facebook . It’s been a total blast working on our product, and of course as we did so we released a number of open source libraries and tools. It only added to our pleasure to see so much of that code used outside of our own domain. I will continue to develop and maintain the…
I’m pleased to announce a major release of of the Haskell statistics library, version 0.10.0.0. I’d particularly like to thank Alexey Khudyakov for his wonderful work on this release. New features: Student-T, Fisher-Snedecor, F-distribution, and Cauchy-Lorentz distributions are added. Histogram computation is added, in Sample.Histogram . Forward and inverse discrete Fourier and cosine…
Last week, I flew to St Louis for the excellent Strange Loop conference, where I gave a 3-hour Haskell tutorial and a talk on how we use Haskell at my startup company, MailRank. Strange Loop is a pretty good approximation to my ideal conference, covering a narrow family of topics I’m interested in, mainly leading-edge matters in programming languages and distributed systems. The focus is not…
The other night, I had a random whim to spend a couple of minutes looking at the performance of UTF-8 decoding in the Haskell Unicode text package . Actually, rather than look at the actual performance, what I did was use Don Stewart’s excellent ghc-core tool to inspect the high-level “Core” code generated by the compiler. Core is the last layer at which Haskell code is still…
Here’s something I bet you never think about, and for good reason: how are floating-point numbers rendered as text strings? This is a surprisingly tough problem, but it’s been regarded as essentially solved since about 1990. Prior to Steele and White’s “ How to print floating-point numbers accurately ”, implementations of printf and similar rendering functions did…
Update : I just released attoparsec 0.9.1.0, which undoes all of the changes described below. The problem? While removing backtracking, I accidentally changed the semantics of the <|> operator in an unforeseen and unfortunate way. The bug I introduced was that a parser of the form (char 'a' *> char 'b') <|> char 'c' would now accept as valid the input "ac" , which is clearly highly undesirable.…
Looks like I’ve got a busy autumn ahead! Read on for two pieces of news that I’m very happy about. In September, I’ll be teaching a Haskell workshop at the Strange Loop Conference in St Louis. Here’s the abstract: Modern programming presents a daunting array of challenges: proliferating technologies, messy inputs, unreliable networks, huge volumes of data, how to verify…
It’s been a few weeks since I last wrote about the aeson library for working with JSON in Haskell, but this isn’t because I’ve been idle. In fact, just tonight I put out a new release. Where the previous releases focused on parsing performance, this one focuses on encoding performance. And the performance news is good: on real-world data, I’ve improved encoding performance…
Sometimes, when a software package meets a certain level of maturity (or the desire to hack on it fades sufficiently), it’s tempting to consider it “done”. Here’s a little tale of when done isn’t really done. About a week ago, I received a message from Finlay Thompson asking about my Haskell statistics package: he wanted to know how to generate pseudo-random variables…
Every functional programmer worth their salt seems to end up with at least a few stories to tell about programming in CPS, also known as continuation passing style . Here’s my latest one. As a user of it, you can’t tell that my attoparsec parsing library is implemented almost entirely using explicit continuations. Every combinator accepts two continuations: the failure continuation is…
I’ve spent some time over the past few weeks improving the performance of the attoparsec parsing library, and of the aeson JSON library. Since they’ve now reached a new plateau of performance and stability, I thought this would be a good time to release new versions. The major advance in the new version of aeson is a considerable speed improvement. [! Performance improvement ](…
I’m delighted to say that after a couple of years of a break from the startup world (which I’ve inhabited for most of the past decade), I’ve decided to throw my hat back into the ring. Together with Bethanye Blount, I’ve started a company named MailRank. We’re working on helping people to manage the all-too-common problem of email overload. You can read a little more…
Peter Seibel (the man behind Practical Common Lisp ) is launching a new publication, Code Quarterly , which looks intriguing to me. He’s hoping to run a coding challenge in each issue, and I know this is the sort of fun dear to many a coder’s heart. The first coding challenge has been up for a couple of weeks, and it involves implementing a parser for Markup, a lightweight markup…
One of the particularly nice things about working with a distributed revision control tool these days is that I can sidestep the choice of winning tool. Thanks to Scott Chacon and Augie Fackler’s excellent hg-git extension , I can use Mercurial and collaborate almost seamlessly with git users. This is exactly what I did when working with Johan on the new I/O manager subsystem in GHC 7, and…
Over the past couple of years, since David Roundy handed over control of the darcs revision control system to a talented team of developers, it has come a long way in stability and performance. I began using darcs essentially out of necessity, simply because it was the revision control system of choice for much of the Haskell community. I have always found my use of it to be somewhat clunky, and…
Ersin Er wrote a brief blog post about handling the Turkish language in Haskell . Because Turkish uses a character set that mostly looks familiar to Westerners, it is notorious for its ability to trip up the unwary programmer (see examples in PHP and PostgreSQL ). import Data.Text ( pack , unpack ) import Data.Text.ICU ( LocaleName ( Locale ), toLower ) main = do let trLocale = Locale "tr-TR" let…
I spent some time today trying to talk to a MySQL database server from a piece of middleware I’m writing in Haskell. You might think that talking to a database server would be easy, but it turned out to be quite a bother. Both of the major MySQL bindings, HDBC-mysql and HDBC-odbc , use the libmysqlclient C library behind the scenes. With GHC’s unthreaded runtime, which is still the…