Sharing my thoughts on life, data and AI engineering, while practising my writing skills.
You finally received a Calendly invitation for a call with the recruiter. You got past the recruiter call and received a home assignment. You promised you would never do them again, but here we are. The opportunity looks promising, the company seems solid, or it’s just the first application that got past the initial round after tens or hundreds of attempts. So, you decided to take the home…
It’s been more than a year that I have been introduced to AWS Step Functions and my team decided to use them for our internal data workflows. It all started aiming to support all our ETL processes to extract data from our various databases, external platforms and APIs and eventually load and transform our tables using dbt into our Snowflake data warehouse. However since we also started…
On-call is a huge issue in every tech company and a large discussion topic in tech communities and fora. Most people hate it, others burn out or switch positions, jobs and careers just to avoid being on-call.
One of my recent tasks was refining a script which exports our RDS databases’ snapshots into s3. The concept was straightforward; we would utilize the daily system snapshots of our RDS instances, and export them to one of our s3 buckets as parquet files using boto3 python library. The IAM roles and s3 buckets were setup following the AWS documentation, including a KMS key to encrypt our exports,…
It’s a superstitious day for some, it’s just another day for others including myself. In any case it’s my first post just to test that this works.