Musings on technology, algorithms, software, and other topics from Dr. Josiah Carlson, Ph.D. - Author of Redis in Action, former maintainer of asyncore/asynchat and related libraries in Python. Consulting occasionally as companies need it.
I got my first email address somewhere in 1994, shortly after logging into my first BBS. This was just a year before most of the world got on the internet with Windows 95 and modems, but I continued using BBSes along with the general internet until at least '98. I wouldn't say I was hooked, but I would say that email, and networked communication has been very important to me for 31 years. Back in…
As teased in my last blog post, CRC64 in Redis was supposed to get faster, but it didn't. About a year after I posted my PR, Redis the company (formerly Garantia Data, then Redis Labs, whom I've worked a booth with at AWS, written articles for, gave several talks for, etc.) changed the license of Redis the software. I don't like the new license. Subsequently, Madelyn Olson asked me to port my…
On Friday last week, I submitted this PR to Redis: https://github.com/redis/redis/pull/11988 . In the PR, I make single-threaded CRC64 53-73% faster for a CPU that is 13 years old, and ~67% faster for a CPU that is 5 years old. I'll get into the details more as to how / why, but modern CPUs could see 2x+, and an update to the algorithm should allow for leveraging more CPU hardware as it scales.…
Back in 2008-2010, I was working as one of the software engineers tasked with developing features and functionality on YouTube for Google. I had joined as the Santa Monica team was growing, right before a summer-long hiring freeze, about 3 months after officially getting my doctorate in 2008. Among other things, I worked on auto-redirection based on demographics (age-gating alcohol-related Youtube…
Some people have heavy hearts, but I don't. I originally signed up for Twitter for development / API access. I wrote software that analyzed millions of tweets and twitter profiles every day for Adly and our celebrity tweet network. I'm about a decade removed from Adly, never much got into tweeting, and I spend most of my time on Twitter recently posting updates about my software, book, etc. Things…
After reading information about Amber Brown's talk regarding Python and its included "batteries" in the form of the standard library, there is a lot to agree with, and only a small amount to not agree with. I know I'm late. Here's my take... For experience consideration; I'm the reason that asyncore/asynchat/... (callback-defined async sockets in Python before asyncio) was included and updated in…
So... some updates on rom . The end of January, I posted a teaser picture about "Interleaved Indexes" in rom/Redis via tweet . If you haven't seen the picture, it's here: Interleaved index? I ended up building an interleaved index in Redis using a bit of Lua scripting and Python. No ZRANGEBYLEX, surprisingly. What is an interleaved index? It's what I was calling an indexing mode that has the same…
Over the last few months, I've been thinking about and implementing transactions for Lua scripting in Redis . Not everyone understands why I'm doing this, so let me explain with a bit of history. MySQL and Postgres In 1998-2003 if you wanted to start a serious database driven web site/service and didn't have money to pay Microsoft or Oracle for their databases, you picked either MySQL or Postgres.…
This article first appeared on November 3, 2014 over on Binpress at this link . I am reposting it here so my readers can find it easily. In Introduction to rate limiting with Redis [Part 1] , I described some motivations for rate limiting, as well as provided some Python and Lua code for offering basic and intermediate rate limiting functionality. If you haven’t already read it, you should,…
This article first appeared on October 9, 2014 over on Binpress at this link . I am reposting it here so my readers can find it easily. Over the years, I've written several different rate limiting methods using Redis for both commercial and personal projects. This two-part tutorial intends to cover two different but related methods of performing rate limiting in Redis using standard Redis commands…
This post is the first of a series of posts that I've been wanting to write for a long time, but I haven't been able to pick from among the collection of topics that I wanted to write about. After taking a quick vote on what people actually want me to write about, this one got the most votes, so you get to read about why I think that we (as a society) should take hurcluean efforts to get liquid…
Just so that I don't get caught out on this later in life, thinking that I might not get to say it later, I just wanted to tell you, Thank You. By reading these words, you have in some small way contributed to some part of me living on beyond my death. And all someone can really hope for is to be remembered in the future. I wish that everyone would record a bit of themselves every day, and that…
Like thousands of others last year, I saw the Soylent crowd funding last summer and the concept rang true to me. Sometimes you just need to eat. And if you are just going to eat to address your biological need to eat, it doesn't make sense to spend more time, money, and effort than absolutely necessary. Enter Soylent, which is intended to be a nutritionally-balanced meal replacement powder. How…
This morning I had the opportunity to listen to a bit of NPR, where a piece on Heartbleed was being discussed by Larry Mantle on Air Talk. You can read and download the piece: New report: The internet is too interconnected to fail . Medium story short: the segment and Larry have the apparent opinion that the way the collective technology industry handles these kinds of issues is "ineffective",…
If you are anything like me, you would prefer to deal with as few headaches when developing software as absolutely possible. One recent headache that decides to revisit me occasionally is Postgres permissions. For users/platforms where you need a single user (that isn't the "postgres" user), while offering password authentication with both local and remote access, keep reading to find out how you…
Python properties let programmers abstract away get, set, and delete methods using simple attribute access without exposing getValue() , setValue() , and delValue() methods to the user. Normally, properties can only be added to classes as either an instance-level property, or a class-level property (few people use class-level properties; I've only used them once, and I had to build a ClassProperty…
Recently, I received an email from a wayward Redis user asking about using Redis Sets and Sorted Sets to sort multiple columns of data, with as close to the same semantics as a traditional SQL-style "order by" clause. Well it is possible, with limitations, keep reading to find out how. What is Redis? For those people who don't quite know what Redis is already, the TLDR version is: an in-memory…
This post is off topic. I'm going to talk briefly about something that has been pissing me off for quite a while now, but recent articles , interviews , and a Reddit AMA about " Romeo Rose " (aka " Sleepless in Austin ") pushed me to the point of needing to say something. Please, just stop it. Look, I get it. Here is this physically unattractive, sexist, racist, douche-bag who thinks that he can…
This post will talk about the open source Redis object mapper that I've released called rom . I will talk about what it is, why I wrote it, and what I'm planning on doing with it. I've posted several articles about Redis in the past, and you can buy my book, Redis in Action , now (hard copy will be available on/around June 10, 2013 - about a week from now) - enter the code dotd0601au at checkout…
Book Deal: As a pre-article bonus, you can enter code dotd0511au during checkout to get 50% off of my book Redis in Action . If you want to keep seeing articles like this, please buy my book and give me positive reviews on Amazon ;) See me speak: LA Web Speed is hosting me for a talk at 7PM at Yahoo's offices in Santa Monica. Hit this page for meetup information:…
As we come into the home stretch of the release of Redis in Action , a lot of work has been going on behind the scenes. Easily 3-4 complete read-throughs over the last few months, a crew of editors poking and prodding, countless major and minor edits, and for those of you who have already downloaded version 11 of the MEAP (released last week), you will notice that it has taken a pass through the…
It may come as a surprise (or not) to those of you who see me posting on the Redis mailing list, or who've been involved in the writing of Redis in Action , or that work with me on a daily basis, or that I give technical advice to, or those that I would call friends, ... but I am tired. Actually no, I'm fucking exhausted. In the last year and four months, I've played the architect and lead…
Today is going to be a different sort of blog post. Normally I talk about technology, but helping to run a startup , finishing a book , and preparing for the imminent birth of my first child have all prevented me from posting for a while. But recently I've heard a lot of obvious bullshit about an issue that pushed me to the point of writing about it. A little back story before I get to the purpose…
Some 8 years ago today, two April fool's jokes appeared on python-list and its mirror newsgroup, comp.lang.python. At the time, I was too young or too much of an idiot to appreciate them. The first was a pronouncement of Guido's having decided that instead of the tab vs. space wars, that we would use the unicode character u'\x08' and our editors/IDEs could be told how wide to display the…
After having been featured on the High Scalability blog, there has been renewed interest in my 6 month old post about Improving Performance by 1000x . Occasionally I've received a few questions and/or comments about that post in other forums, but today I got a couple posts from two well-intentioned commentators that highlighted some very interesting misunderstandings about how modern computers and…