Musings about life, technology and some unrelated stuff
Introduction TinyBits, as presented in the previous blog, is promising to be a fast and extremely space efficient serializer. Multiple compression techniques contribute to that efficiency, but I reckon that the biggest contributor is the string deduplication feature. The string deduplication removes repeated strings (up to 256 strings per document, between 2 and 128 bytes [ ]
Introduction In my last blog, I went over multiple serialization solutions for Ruby including two JSON based flavors (Oj and the standard library JSON gem) alongside CBOR and MessagePack as binary alternatives. If you recall my verdict, MessagePack came out on top regarding encoded size and encoding performance, while the JSON gem was the best [ ]
Oj vs JSON vs MessagePack vs CBOR Introduction It had been a while since I had to look at serialization performance in Ruby. I generally would just use MessagePack for performance, and if I needed human readability I would opt for the Oj json gem. But due to my current project heavily relying on efficiently [ ]
Context In this paper, titled SQLite: Past, Present, and Future , the SQLite core team, along with researchers from the University of Wisconsin-Madison discussed the progress timeline of SQLite, how it became a world class OLTP database engine, but also how it started to struggle with some of the emerging analytical workloads. Compared to DuckDB, an [ ]
Introduction Recently, I have been working on a project that dealt with large sets of latency measurements. For these, I always needed to capture percentile values (e.g., p50, p95, p99, etc.). Since I was working with SQLite, I naturally used the stats extension to calculate those percentiles, which worked OK until it didn t! I realized [ ]
Introduction LiteCluster is a globally replicated SQL database with a leaderless architecture that delivers high performance operation while retaining ACID semantics and strong consistency. In this post I will walk through the features of LiteCluster, the design of such features and the tradeoffs that were taken during the design process. It is important to note [ ]
Introduction Versatile as it is, SQLite still suffers from one major drawback. Which is write concurrency. SQLite, using the Write-Ahead-Log (WAL) journaling mode, supports an unlimited number of readers and a single writer any given moment. Thus all write operations are effectively serialized. And while SQLite is very fast at performing writes, it gets stuck [ ]
Introduction Litestack, and its Litedb component in particular, provides powerful and flexible full text search capability via the Litesearch module. Any context that uses Litedb, whether directly using the driver or using ActiveRecord or Sequel adapters will have access to the Litesearch functionality. Litesearch is built on top of SQLite s FTS5, thus, it needs SQLite [ ]
Introduction In our latest article we saw how Litestack provides a high performance caching solution for Ruby/Rails SQLite based applications. Today we will look at the applications themselves. Not all Ruby/Rails applications are built equally and in this post we will have a look at how the environment that Litestack runs in can impact performance [ ]
Introduction If you are running a production Rails application using SQLite then you are probably using SQLite as a cache store as well. Compared to something like Redis or Memcached, SQLite offers a potentially much larger caching space and a significantly higher read performance, which is what matters most when we are dealing with caching. [ ]