""
In the previous blog post , we covered our story of migrating a giant database of almost 11 TB. Here comes the last part of the series—making Ruby on Rails applications work with RDS Proxy. I originally published this article on the Smily Engineering Blog . Why use RDS Proxy in the first place? Before migration, we were using PgBouncer . We hosted multiple databases (for multiple applications) per…
In the previous blog post , we covered our story of migrating to AWS RDS using AWS Database Migration Service (DMS) , a complex initiative with multiple issues we had to address. Nevertheless, almost all the migrations we did could have been generalized to the same strategy - in the end, we found a way to use DMS that works fine, even for mid-size databases. One database, though, required some…
In the previous blog post , an intro to our database migration series , we promised to tell the story of our challenges with AWS Database Migration Service , which turned out to be far from all sunshine and rainbows as it might initially seem after skimming through the documentation. When we started using it, it went significantly downhill compared to expectations, with all the errors and…
In recent months, one of our most significant initiatives was migrating from our self-managed PostgreSQL clusters to the managed service on Amazon - AWS RDS. Overall, we managed to migrate 5 database clusters of 54 applications (including staging ones), several of which had a size of close to 1 TB, and one giant cluster with a single database of a massive size - 11 TB, which required a lot of…
In the previous blog post , I promised we would introduce something special this time. So here we go - meet almighty Dionysus , who knows how to make the most of Kafka. I originally published this article on the Smily Engineering Blog . Change Data Capture Change Data Capture is a popular pattern for establishing communication between microservices - it allows to turn all inserts/updates/deletes…
In the last blog post , we covered some fundamental concepts about Kafka. This time, let’s discuss how we use Kafka in Smily, how we got where we are now, what the decision drivers were, and how the overall architecture has evolved over time. I originally published this article on the Smily Engineering Blog . A short story of Smily Architecture Like most of the technological startups, Smily (or…
In the last blog post , we covered a general overview of integration patterns for distributed architecture , and now it’s time to get into further details. Let’s start with perhaps the most exciting piece of tech we use in Smily - Apache Kafka . I originally published this article on the Smily Engineering Blog . What is Kafka? Generally speaking, Apache Kafka is an open-source distributed event…
Distributed architectures have been growing in popularity for quite a while for some good reasons. The rise of cloud services making the deployments simpler, as well as the ever-growing complexity of the applications, resulted in a shift away from monolithic architecture for many technical ecosystems. Microservices have emerged as an alternative solution offering greater modularity, scalability,…
We’ve finally launched Smily (formerly BookingSync) Engineering Blog which I’m going to primarily focus on instead of this blog here. If you enjoyed the content of my personal blog, make sure to check Smily blog as it’s going to be even better!
Recently, in BookingSync , we were performing a migration from Karafka 1.4 to 2.0, which we use for communication with Kafka . One of the great features available in version 1.4 was a custom partition assignment strategy for consumers . It was particularly useful for us as we’ve had some topics that had a way higher throughput than the other ones, so just a round-robin strategy with even…