My blog, portfolio and technology related ramblings
If you’re on Windows, Git has almost certainly been silently messing with your line endings since the day you installed it. TLDR: The fix is this one command: git config --global core.autocrlf false It’s on by default. It’s a terrible default. Mainly due to historical reasons that no longer exist. What core.autocrlf Does In short, Windows has historically used CRLF ( \r\n ) as its line ending…
I’ve put together a reference site for engineering best practices, available at guidelines.ryanharrison.co.uk . The goal is a single place to find guidelines and conventions covering the full stack - from core principles through to deployment. Rather than being prescriptive about specific tools, most sections aim to explain the reasoning behind a recommendation so you can apply it to your own…
This site is built with Jekyll. That unfortunately often means wrestling with Ruby versions, gem dependencies, and environment configuration. Different machines require different setups, and what works on your machine might not work in CI or on your VPS. Docker solves this for many other areas by providing a standalone and reproducible build environment, so why not here as well? The challenge is…
A junior developer recently came to me for help with one of those classic issues which appears on a remote dev environment, but for whatever reason can’t (at least easily) be replicated locally. Their immediate thought was to add more log statements and redeploy the app to see what’s going on. Not unreasonable, but this is a dev environment, so I connected IntelliJ to one of the running containers…
So it turns out that Oracle Cloud (OCI) offers an extremely generous free tier, and not many people seem to know about it. Beyond the 30 day free trial, which gives you $300 worth of free credits to use (not bad at all), they also offer a pretty substantial amount of infrastructure in their Always Free tier as well. In terms of server (VPS) infrastructure, this includes (at the time of writing at…
Some quick high levels notes on Kafka vs MQ. This is a question that often gets asked by folks already who are familiar with traditional queues (IBM MQ/RabbitMQ etc) when they are introduced to the world of Kafka: Kafka High Level Uses Event input buffer for data analytics/ML Event driven microservices Bridge to cloud-native apps ActiveMQ Consumer gets pushed certain number of message by broker…
Most Kafka design and configuration choices are use case dependent and come with trade-offs, so it’s hard to define any best usage rules. However, below are some points of interest and general recommendations based on previous experience that you may want to consider or look more into: Topics Ensure all topics have a consistent naming scheme. If using a shared cluster, prefix with the system name.…
Environment Variables Set path to binaries export JAVA_HOME=/path/to/jdk export PATH=$PATH:/path/to/kafka/common/bin Add Kerberos / SASL config options (if required) export KAFKA_OPTS="-Djava.security.krb5.conf/etc/krb5.conf -Dzookeeper.sasl.client.username=myuser -Djava.security.auth.login.config=/var/tmp/jaas.conf" export KCONFIG=/var/tmp/kafka-client.properties Where properties contains:…
Part 1 - Logging In Part 2 - Securing Login Part 3 - Installing a Firewall Part 4 - Setup Nginx Web Server Part 5 - Install Git, Ruby and Jekyll Part 6 - HTTPS With Let’s Encrypt Part 7 - Email Forwarding with Postfix Part 8 - Sending Email Through Gmail Part 9 - Setup a Reverse Proxy with Nginx It’s likely that you will want to run containers of some kind on your server, for that we’ll be…
Part 1 - Installation and Instrumentation Part 2 - PromQL and Recording Rules Part 3 - Alerting Pushgateway The pull approach doesn’t work for ephemeral jobs which don’t run for long enough for Prometheus to scrape them. Pushgateway is a metrics cache for service-level batch jobs. Used to handle the exposition of metrics that have ben pushed from batch/cron jobs. If a Pushgateway instance…