A blog which majorly focuses on Data Science, Data Engineering and sometimes Kotlin / Scala.
Live Last read · last published · next check
One day I asked a chatbot how easy it would be to make my Balboa hot tub smart. Apparently: quite easy. A quick search turned up an existing GitHub repository ( shomanjk/esp32_balboa_spa ) that controls Balboa spas / hot tub over RS485 with an ESP32, and it supports my spa’s (Iva Idun) control board. Let’s take a step back: why do I want my spa to be smart? Only heat the water when my solar cells…
This spring my wife and I went ahead and bought a (used) EV, Skoda Enyaq iV80 2022. It’s not relevant for this blog but it’s a very nice car, and I do enjoy the EV “way”. With that said I found myself trying to optimize the charging and our electricity provider didn’t really integrate more than price optimization. I wished to include solar energy into the optimization engine. The problem I’m…
One day our baby monitor stopped producing sound. Scary with no audio from the baby’s room! At least the lights still flashed whenever there was sound or movement. Naturally I started to google new devices, hopefully cheap and good. After a good night of sleep I had the idea, what if I could repair it? I tried to figure out exactly what was wrong. The “parent” light acknowledged whenever the…
This post is not only be written by a real person but also hopefull short and to the point! The what Last year I blogged about img2lego , now renamed brickportraits to remove all connection to LEGO. In my tool I make something that’s not part of the LEGO franchise but can be built by any type of bricks. Today I got a website brickportraits.londogard.com which allows anyone to run the generation,…
To run your own container in AWS Sagemaker (Training Job) it’s required to have mamba . So how would you use pixi instead? It’s quite straight-forward, first add micromamba as part of your dependencies in pixi.toml . Then you update your docker to something like this: FROM --platform=linux/amd64 ghcr.io/pixi COPY pixi.toml pixi.lock ./ ENV PIP_DEFAULT_TIMEOUT=360 ENV UV_HTTP_TIMEOUT=360 RUN pixi…
I’ve been using pixi professionally in my team for a year now, and it has been a pleasure from the start. N.B. We’ve also been utilizing uv inside the pixi environment. Brief Intro uv : A faster & better pip. pixi : A faster and better pip+conda (bonus: beats mamba and works for other languages). Uses uv under-the-hood for pip installs. From the moment I (we) started using uv/pixi something…
ℹ️ This is a really small piece of “nothing”, but it might save you (and future me) some time! Recently I started playing around with scikit-image library which is really cool. I found that they had a decent resizing tool, but diving deeper it actually turned out to be really slow . I’d even go as far as to say that the anti-aliasing (AA) of scikit-image might be too agressive, but you can tune it…
When working with code, or anything really, you always apply trade-offs. One example is simplicity versus runtime efficiency , often talked about in CPU-cycles versus brain-cycles , where the latter usually wins. But sometimes the trade-off is hard, s5cmd is such a case - it’s a single new dependencies with massive gains. S5cmd is a very fast S3 and local filesystem execution tool . For those that…
Transformers.js is an ambitious project by HuggingFace to bring transformers to Web/JS and simplify inference on-device, running onnxruntime-web under-the-hood. I’ve written blogs and apps with onnxruntime-web 1 , and I must say - I’m a sucker for efficient on-device inference! Marimo WASM App: Before/After Prediction Test it live here , and yes it runs on your device in the browser! Tip Embedded…
Transformers.js is an ambitious project by HuggingFace to bring transformers to Web/JS and simplify inference on-device, running onnxruntime-web under-the-hood. I’ve written blogs and apps with onnxruntime-web 1 , and I must say - I’m a sucker for efficient on-device inference! Marimo WASM App: Before/After Prediction Test it live here , and yes it runs on your device in the browser! Tip Embedded…
There has been a lot of buss around DeepSeek (R1) and their Open Source mission, and lately they released their full stack to train State-of-the-Art LLM’s. One of the tools is a Distributed Data Processing framework named “smallpond” built on top of DuckDB & Ray . Mike made an excellent write-up on his blog. The summary? It’s a tool that you can’t even buy with millions $ , insanely valuable Open…
This post will be short, I recently built a WASM app that allows you to 1) convert between Parquet/CSV/JSON and 2) explore the data using Marimos built-in tooling. I shared an initial introduction to marimo in a blog earlier . Tip In short marimo is an up-and-coming Notebook that also have “App-mode” and can run using WASM. App:Converter There’s multiple libraries suppoted in Pyodide, Python WASM,…
ImportCollector It’s simple and requires 0 dependencies outside of the standard library. This script will recursively traverse the dependencies of a Class or python-script and find all relevant dependencies from your local project. It’s useful in multiple types of project, such as (remote) Machine Learning training jobs and serverless deployments, where you don’t want to include irrelevant files.…
MLFlow Models MLFlow is a popular tool to track your experiment to compare metrics, parameters and much more. It helps streamlining your job as a data scientist and machine learning engineers . Their MLFlow Models is a sub-project that helps making deployments smooth and integrates with their Model Registry that has versioned and tagged models which ties together with MLFlow Experiments. All in…
Marimo Marimo is the “new” kid on the block. Based on what Marimo tries to achieve you can’t help yourself but comparing it too other frameworks such as Gradio, Jupyter, Streamlit, Solara & Panel . Note A multitude of options The fact that there’s a plethora of options to build WASM apps/tools and “ literate programming” through notebook-style is nothing short of amazing We’re in for a great time!…
Gradio Client Most people in the AI-sphere (Deep Learning, LLM’s) are aware of the Gradio project (now under the huggingface umbrella). Tip Gradio in short Gradio is a simple Machine Learning App framework that provides easy components and reactivity. See for yourself: import gradio as gr def greet(name): return "Hello " + name + "!" demo = gr.Interface(fn = greet, inputs = "text" , outputs =…
Merry Christmas, Happy Holidays and a Good New Year! 🥳 Disclaimer: I’m not affiliated with LEGO and this is a personal project. Ever wished you could turn a photo into a LEGO masterpiece? This holiday season, while my son napped, I did! I’ve been passionately working on what might be my most entertaining project since I built my own Baby Monitor ( 1 , 2 ): a tool that transforms any image into a…
Managing dependencies and environments across multiple platforms can be a nightmare. That’s why I was thrilled to discover Pixi . I’ve previous talked about Pixi on LinkedIn/Twitter, but haven’t used it in any “serious” project until recently and so far it has worked exceptional! Imagine a tool that combines the speed and efficiency of uv with the robust package management of mamba . That’s Pixi…
This blog was supposed to be more in-depth but my enthusiasm was drastically cut and I felt like splitting it up into multiple smaller one, whereas daft one is already uploaded. I started writing a “recipe-book” for daft where I realized it wasn’t as smoothly integrated as a lot of other tools. I believe that the DataFrame format is both a winning and loosing concept, it’s very helpful but when…
I know I’ve been praising polars a lot lately, and I’m still in love. polars will be my continued go-to library for Data Analysis of Tabular data, and when building ETL (data pipelines) in 99% of the cases. However, when you work with Deep Learning and multi-modal data you need something to take the data from your Delta Lake, or wherever you store your data, and supply it to the model. That’s…
