This edition of the newsletter contains
Being hands-on is the best way for you to learn. Practice interesting programming challenges like building your own BitTorrent client, Redis, DNS server, and even SQLite from scratch on CodeCrafters.
Sign up, and become a better engineer.
We often focus on the wrong problem when asking for help. Instead of describing what we actually want to achieve, we end up asking about a side issue, i.e., instead of asking about X, we end up asking about Y. Let me explain...
Say you’re implementing image upload in your service, and while doing so, you get stuck converting a base64 string into an image. So, you ask for help decoding base64.
But the real problem isn’t decoding base64 - it’s handling image uploads. If you had asked about photo uploads, someone might have shown you a much simpler approach.
That’s the classic XY problem: you ask about your assumed solution (Y) instead of explaining the actual problem (X) you want to solve.
say what you are really trying to do
don’t assume your solution is the only way
explain the full problem, not just the code snippet
describe what you have already tried
You can and should always start with context. A bit of background helps others understand the bigger picture. Then, state your end goal clearly. What does success actually look like?
Be open to alternative solutions, even if they go beyond your original plan. Sometimes the best fix isn’t what you imagined at first. A fresh perspective can save you hours of unnecessary work.
Show what you’ve already researched or tested. It signals effort and prevents people from repeating what you already know. Plus, it helps others pick up right where you left off.
Here’s an example: Instead of “How do I decode base64 in Python?” Try “I’m building an image upload feature and need to store uploaded photos efficiently. What’s the best way to handle this?”
Before you ask, check yourself: are you describing Y, or explaining X?
Focus on the real problem, not your first guess at the solution, and that’s how you get better answers and solve the right issue.
I published a video - Database per Service Pattern in Microservices
Keeping microservices loosely coupled is our ultimate goal, and a high-level pattern that enables us to achieve it is - database-per-service pattern. I published a video where I cover
the database-per-service architecture pattern,
nuances around how it helps in modeling massive systems,
understand the advantages of adopting it, and
conclude by going through some drawbacks of it
I spent some time reading Tenzing: A SQL Implementation On The MapReduce Framework
In 2011, Google built a SQL Engine on top of MapReduce, which ran over 10,000 queries across 1.5 PB of data and powered Google Ads.
Some time back, I read a 2011 paper from Google titled Tenzing, which is a SQL engine built on top of MapReduce. This paper laid the foundation for all the big data technologies we see today, such as Spark SQL, Presto, and BigQuery, and included key features like joins, aggregations, and OLAP functions.
Tenzing leveraged persistent worker pools, shared in-memory state between nodes, and data streaming to bring end-to-end query latency under 10 seconds for most queries. They also implemented hash aggregation and joins by doing block-level shuffles and not sorting.
I would highly recommend you read this paper, as it will give you a glimpse into what it takes for a database to support SQL on a distributed database. You will be able to connect the dots when you use or dig deeper into the modern big data ecosystem.
You can download this and other papers I recommend from my papershelf.
I read a few engineering blogs almost every day, and here are the three articles I read and would recommend you read.
Thank you so much for reading this edition of the newsletter 🔮 If you found it interesting, you will also love my courses
I keep sharing no-fluff stuff across my socials, so if you resonate, do give me a follow on Twitter, LinkedIn, YouTube, and GitHub.
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.