It's tempting to think about software performance in vacuum, disconnected from any inputs and outputs, abstracted down to optimizing latency or throughput. This way the problem becomes much simpler to deal with, it's easier to convince others of its importance -- after all everyone wants to do more with less. It's also a quite dangerous approach, which has less and less place in the modern world.…
In these times of interdependency and extreme specialization it does not come as a surprise that open source projects depend on each other in many ways. Projects higher in the stack depend on features, provided by projects lower in the stack -- all while driving their development and sorting out issues specific for this particular project. But it's hardly breaking news that isolation between such…
PostgreSQL community conferences, as many other industry driven events, usually do not have proceedings or anything similar. All you get is a slide deck and, if you're lucky, a video recording. But it often feels not enough, as it was with one of my talks last year at PGConfDE, where the topic I was talking about included examples of mathematical modeling to predict how the database will behave.…
Everyone knows benchmarking is hard (and writing about benchmarking is double as hard), but have you ever asked 'why'? There could be at least few reasons, and they usually have something to do with the inherent duality: it's necessary to combine expertise from both the domain specific area and general analytics expertise; one have to take into account both known and unknown factors. In this…
My own personal white spot regarding BPF subsystem in Linux kernel was always programs performance and an overall introspection. Or to formulate it more specifically, I wasn't sure if there is any difference in how we reason about an abstract program performance versus a BPF program? Could we use the same technics and approaches? You may wonder why even bother when BPF programs are so small and…
What is a quantile? You face them all the time, sometimes even without knowing that. Computer vision, signal processing, risk management in finance, you name it. The most straightforward example for software engineers would be a monitoring tool that shows the measured value as well as its percentiles, which are special kind of quantiles. Yes, quantiles are everywhere, yet have you asked yourself…
Recently landed in PostgreSQL, jsonb subscripting support doesn't look as exciting as some other improvements around jsonb. But it's user visible changes are only tip of the iceberg. How many people were involved to make it, and what decisions choices were made? How long did it take, and what are the good/bad ideas to work on a patch?
What is your first association with the concept of B-tree? Mine is 'old and well researched, or in other words boring'. And indeed apparently it was first introduced in 1970! Not only that, already in 1979 they were ubiquitous. Does it mean there is nothing exciting left any more? It turns out that there are multitude of interesting ideas and techniques around B-Trees. They're all coming from…
Do not be afraid, this short blog post is about databases and does not contain any unreasonable metaphysical references. In fact, it's a result of a journey through couple of whitepapers and books with an unexpected intersection of two rather different fields. I will try to describe everything step by step, so that we can see if chess has anything in common with database query optimizer.
It's not a secret that databases are damn complicated systems. And they tend to run on top of even more complicated stacks of software. Nowadays you will not surprise anyone (or at least not that much) by running your database on a Kubernetes cluster or inside a virtual machine. It's probably still questionable whether it's good and appropriate, but this approach is something we have to face —…
As such, there’s really no “standard” benchmark that will inform you about the best technology to use for your application. Only your requirements, your data, and your infrastructure can tell you what you need to know. For already some time I can’t stop doing interesting/useful/weird (one at the time) benchmarks to reveal some details on how to apply document-oriented approach in the world of…
“How to start” is always a difficult question, and jsonb isn’t an exception. Here are few notes about converting different types of data into jsonb , that someone can find useful. Basically there are three possible cases of data conversion: Convert data from inside PostgreSQL Convert data from other database Convert plain data outside database From inside PostgreSQL First of all we shouldn’t…
I really don’t know how I lived before without pandoc. It’s an amazing tool, that saved me from a terrible pain of latex -> MS Word convertion. If you’re writing many scientific documents in latex format, and faced with the demand to convert it into MS Word, you can understand me. It’s like a tiresome point-and-click game, which can consume unbelievable amount of time. Pandoc can do it, and I’m…
As such, there’s really no “standard” benchmark that will inform you about the best technology to use for your application. Only your requirements, your data, and your infrastructure can tell you what you need to know. NoSql is everywhere and we can’t escape from it (although I can’t say we want to escape). Let’s leave the question about reasons outside this text, and just note one thing - this…
Finally, I decided to replace my old laptop, and my chose fell on the Lenovo u430p. As I understand now, it was not a good idea in case of Gentoo =) Actually, I was surprised, how many nerves you can lose only because of the adaptation of you hardware to your requirements. And here is the shortlist of what you shouldn’t forget, if you want to do the same more easily. EFI loader So you’ve…
This is a small notice about a very useful trick. I often have to deal with computations, because the CFD is the part of my activity. And one of the annoying problem in this kind of research is when after many hours of computations you got a Not A Number result, caused by a stupid mistake. It would be nice, if a computation was interrupted by the NaN. And there is the solution : #define…
I’m doing a small revolution in my environment from time to time. I think “hey, a cool stuff”, take a deep breath and doing something new. And this is story about my migration from the bash to the fish shell. Let’s see, what says the official tutorial: fish is a fully-equipped command line shell (like bash or zsh) that is smart and user-friendly. fish supports powerful features like syntax…
I’m at the beginning of my journey into the world of gnuchess. Its interface is a slightly tricky, than I’ve expected, but I was interested. So I’ve decided to make some notes about this. GNU Chess performs at the senior master/weak international master strength of 2500+ on the Elo rating system I don’t know, what does this mean really, but sounds cool =) It is often used in conjunction with a GUI…
A little bit about my new program-frankenstein. Now it is an endless Paginator for Django. It sounds crazy, isn’t? Standart Django Paginator uses the count() function for the verification of page number. It is converted to the SELECT COUNT(*) ... query, of course. But as I was explained (I really don’t know, maybe it’s just an exaggeration - you can post your opinion in the commentaries), this is…
Don’t be confused by the title of this post - I will tell you about my experience in the development of xmpp client xi . The first version of this client was written in Haskell in the shortest time (for me, of cource), and this fact provides the second emotional part of title =) First of all - xi was inspired by ii irc client. It explains the all of its features, design and main idea. In short -…
There are a some cases, when we prefer to use a PostgreSQL schemas for our purposes. The reasons for this can be different, but how it can be done? There are a lot of discussion about the implementation of PostgreSQL schemas in Django (for example one , two ). And I want to describe several caveats. First of all - you shouldn’t use the options key to choice a schema like this: DATABASES [ '…
Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface. It seems like there is no good IRC plugin for vim - I found none of them at least. But there is a brilliant geeky alternative - ii . Here is a quote from its site: ii is a minimalist FIFO and filesystem-based IRC client. It creates an irc…
Some time ago I was faced with the need to implement the sharding in Django 1.6 . It was an attempt to make step beyond the standart features of this framework and I felt the resistance of Django =) I’ll talk a bit about this challenge and its results. Let’s start with definitions. Wikipedia says that: A database shard is a horizontal partition in a database. Horizontal partitioning is a database…