Recent content on Joel Tok
The Tension Much of the tension in software engineering can be boiled down to two factors: Delivery and Engineering. We can visualize this with a quadrant chart: Delivery versus Engineering Quadrants The Axes Both axes represent a developer’s ability to execute on project requirements according to functional and non-functional requirements. Different people might define both of these…
A framework to detail a small organization's tech stack in markdown.
Introduction Nx is a tool that manages monorepos and standardises tooling within that monorepo. As part of these standards, Nx provides a set of tools that allow users to quickly generate a static vanilla HTML website, without any complicated tooling. Prerequisites To begin, make sure we have an Nx workspace setup. npx create-nx-workspace@latest <workspace-name> ✔ Which stack do you want to use? ·…
Introduction Component libraries are a tool that have been around for nearly a decade in the frontend world, yet it is still not a tool widely adopted in many organisations. Software engineers who work directly on the frontend have a better opportunity to understand – with nuance – the usefulness of component libraries in terms of longer-term productivity. This article is an attempt to convince…
Introduction to Our Problem I recently went to a grand uncle’s house. Evening hit as we were sitting on the patio, and a string of solar-powered lights lit up. It was a cool idea. Just hang up a bunch of solar lamps where there was sun, and voila! Free light at night. But it also got me curious: do these solar lamps actually pay for themselves in cost savings? Is the light that they produce…
Introduction This is a short guide to help us understand how we can deciper an existing Nx plugin, to figure out what it does. A basic understanding of Nx plugins will be useful to the reader. Understanding the Plugin’s Behavior Entrypoint The best place to start our investigation is with a few files at the root of the plugin. generators.json migrations.json executors.json These are the…
Introduction The basic idea behind an off-peak battery is to use cheaper power during an off-peak period to charge up a battery, then use that power during a peak period when such power would be more expensive. This article is meant to provide back-of-the-envelope calculations to help you decide if such a battery setup makes sense for you. Marginal Savings – A Litmus Test Given transfer and…
The Problem Over the years I’ve often needed to search for specific files by their name in Visual Studio Code, but never figured out how to do it. I had always assumed that a Command F would reveal the file to me, but nope, that always only searched within files. So I’ve had to always resort to using complicated bash commands from search engine results (searching a new method each time…
The way Nx is Marketed Nx is a build system with built-in tooling and advanced CI capabilities. It helps you maintain and scale monorepos, both locally and on CI. The above is a description of Nx taken directly from the nx.dev homepage. To me the keywords in this mission-statement-like paragraph are “build system” and “monorepo”. When I first read this, I came away with the…
Introduction When trying to figure out the Power Density of different batteries and fuel sources, the resources encountered both on the web and in scientific papers seemed confused. Definitions were presented in terms of power per unit volume, power per unit mass, some even unique measures like in terms of volume per unit time. Which is correct? I explore this question in this article.…
Problem Many component libraries, especially those built with vanilla Web Components, follow a specific way of dealing with boolean attributes, specified by the HTML Living Standard. Many Frameworks however do not apply the same standards and patterns in their approach, including but not limited to, frameworks that rely on JSX. We try to reconcile these two, by showing how we can use standards…
Problem Building a Lit Element component, and defining their properties for internal consumption is very well explained. However, in this article we are interested more with how to define the interface between our lit element and its consumer. In other words, we want to examine how to set lit element attributes and properties from outside of the component, and use these in the component. We also…
Introduction In this article we aim to develop a fundamental understanding of what Web Components are, by methodically and comprehensively building and modifying our own standalone Web Component through a variety of patterns. Let’s jump right in. The Component – A Terrapin Card To start off, let’s build our first Web Component and render it with an html file. I like pets, and I used to…
Introduction With growth in renewable energy, much of the conversation has shifted away from the economics of energy production to energy storage. This is because many sources of renewable energy like solar and wind are variable and intermittent, affected by factors like changes in weather or time of day. This reflects a lack of stability in power production, requiring the need for energy storage…
Problem Let’s say we want to implement a factory constructor in an abstract class. For example, given the abstract class Position, we want to enforce that all classes that implement it have two capabilities: Position.fromJson and toJson. Notice that the first of these is usually implemented as a factory constructor. We can show this in the example code below: abstract class Position {…
One way to think about electricity when using battery systems, whether off-grid or in other use cases.
Calculating the power capacity of a battery in Ah and kWh.
Python dependency management for reproducibility, using built-in libraries.
A simple docker debugging checklist
Problem I’ve been working with the deliciously productive React Hooks API recently, and have been using React Hook Form as my go to form abstraction. There was one issue I faced lately however that bothered the heck out of me: When the user hits enter on the keyboard, for some mysterious reason I was not able to get the input to blur. Even if there was no validation errors, and the data was…
Introduction I heard about this place while exploring Quarry Hills Park, and could not resist visiting just because of what it’s named. On the whole it was not a disappointing hike at all. Don’t expect to have much in terms of a lookout or vantage point. More of a peaceful, relaxing and serene kind of vibe. Weather was a little rainy just before I arrived, so the ground was muddy. But even…
Over the past few months, I have been working on and off on this side-project called Quackcheck. This project was built to help me deal with the opacity of articles on the web. You see, I figured out that because of the unique way in which search-engines operate, relevance is often prioritised over accuracy. What this means is that when doing research via Google Search, I often encounter articles…
This post is part of a 3 part series that uses a spreadsheet to answer the following question: can you retire? The 3 parts are as follows: Part 1: Can I retire? Part 2: Optimising your Expenses Part 3: Optimising your Income Introduction Before I start, this post is not about investment advice. There are a lot of resources out in the web for that, and investments is something I cannot claim to be…
This post is part of a 3 part series that uses a spreadsheet to answer the following question: can you retire? The 3 parts are as follows: Part 1: Can I retire? Part 2: Optimising your Expenses Part 3: Optimising your Income Introduction Welcome to Part 2 of the series Can I retire? In this part we will expand on the original post, and tackle with much more detail the expenses section of your…
This post is part of a 3 part series that uses a spreadsheet to answer the following question: can you retire? The 3 parts are as follows: Part 1: Can I retire? Part 2: Optimising your Expenses Part 3: Optimising your Income Introduction In this post I will walk you through how you can look at where you stand right now in your finances, and help you decide if (1) you will be able to retire right…
Given that we want to execute certain service tasks in Camunda directly, is there a noticeable difference in performance when using different implementations? What is Camunda? And what is a Service Task For those unfamiliar with the platform, Camunda is an open-sourced engine that allows you to design processes using visual diagrams. These visual diagrams are represented using a specification…
LocalStack is an open-sourced server application that lets developers emulate Amazon AWS Services on their local development environment. It can be used as part of one’s development environment, but also as a way to support the running of E2E tests, first in the local environment, then in the CI pipeline. LocalStack in the local environment E2E tests are usually run in the local environment first,…
Introduction In this post, we want to evaluate the memory footprint in Python 3 of data stored in various tabular formats. In particular, we want to compare DataFrames, to JSON-like data structures like List of Dictionaries, and Dictionaries of Lists. The above are 3 different ways to store table-like data. Table-like data is basically data represented by rows and columns. In this examination, we…
I became very intrigued with Metaflow because of its DAG (Directed Acyclic Graph) approach to building ML models, which seems rather intuitive when you think of a machine learning model as a series of processing steps. However, I did have one big concern when it came to Metaflow – production deployment. As it currently stands, production deployment with Metaflow seems highly coupled to AWS. This…
Dec 2022 Edit – Due to a significant amount of interest, readers can find a ready-made PyPI package here. Introduction Building efficient event buses requires strong support for parallel processing. This is because scalable event buses usually require multiple events be fired in parallel, and these events should not block each other’s execution during extended input-output (I/O) operations. Python…
Problem How do we run an asynchronous function in a synchronous script? Python’s await async syntax can be a real life-saver when it comes to running highly concurrent code in a performant manner. However, sometimes we just want to call an asynchronous function synchronously. Maybe we have a small application where performance is not a big issue, or we just want to reuse some async functions in an…
In this post we look at 4 different kinds of conversation engines that can be used for chatbots. The kind you choose to use can determine the power and flexibility of your final product — and ultimately how scalable it will be in handling diverse and myriad conversational flows. What is a conversation engine? To explain this, first we first have to understand a little about how chatbots work.…
Problem We want to be able to access different VueJS frontend applications from the same host, differentiated by a base-path for each app. For example, on an example host prototypes.com, we want to host 2 applications: alpha and beta. Both these applications should be accessible from separate base paths, prototypes.com/alpha and prototypes.com/beta, routed through the same reverse-proxy. Example…
There are many tutorials out there that teach you how to get started in Flutter for mobile. But resources on Flutter for desktop are still lacking. Today I will walk you through the steps of creating a Hello World! App on Flutter Desktop. Note: The following commands have been tested on Linux and MacOS only, since those are the OSes that I personally use. Setting up Flutter for Desktop First…
Asynchronous programming came to python awhile back, formally encapsulated in the asyncio module. This is a python library that allows developers to utilise an asynchronous pattern in writing code using the async-await pattern. While full-featured and mature, I personally encountered a confusing edge case when using create_task to spin off background tasks. Let’s start off with a working example:…
Introduction Working in Flutter to build my new mobile app (now defunct) ZapCam has been an amazing experience. For those who do not know, Flutter is an open-sourced cross-platform mobile app development framework, developed by Google and written in Dart. Developing in it has been surprisingly elegant. The type system of Dart is clean and expressive, and Flutter itself offers many development…
Problem While in the initial stages of a project, sometimes we have to choose between storing data with Pandas DataFrames or in native python lists of dictionaries. Both data structures look similar enough to perform the same tasks - we can even look at lists of dictionaries as simply a less complex Pandas DataFrame (each row in a DataFrame corresponds to each dictionary in the list). The question…
I have been using PyTorch extensively in some of my projects lately, and one of the things that has confused me was how to go about implementing a hidden layer of Rectified Linear Units (ReLU) using the nn.ReLU() syntax. I was already using the functional F.relu() syntax, and wanted to move away from this into a more OOP-approach. The following is a straightforward example on the way to convert an…
Before I started a corporate gig, I used to work freelancing gigs. When freelancing, your clients are your most important relationship. No clients = no revenue = starvation. So you do your best to keep them happy, understand their needs, and directly service their personal and corporate (read: business) goals. If you deliver value, you keep the relationship, and good things happen. It took me a…
When I first started working on a Facebook messenger chatbot back in early ’18, one of the key issues we faced was how inflexible it was for business to make changes to chatbot conversations. In fact, changes took anywhere from 2-3 days at best, to a month at worst — a pretty unbearably long feedback loop. Through a series of major changes to the underlying architecture, this average of 2 weeks…
Developer Productivity @ Scale There are multiple ways to scale When it comes to scaling, most developers assume there is only one way to scale: performance. How do I support the most number of users, or the highest amount of traffic, or the fastest processing times? The above is what I consider Performance @ Scale. But there is another way to scale: rate of development. Can my rate of development…
Tech Stack Using Tech Adoption Lanes, a personal framework inspired by the ThoughtWorks Tech Radar, modified to reflect the technology stack of a small organization or individual developer. Assess Trial Adopt Hold Languages & Frameworks Rust NextJSReact NativeNestJS JavaScriptTypeScriptPythonExpressJSVite + ReactJS Ruby + Ruby on RailsDart + FlutterAngularJSCreate React AppC/C++ Tools GraphQL…
about
contact
software engineering process
What am I up to now