Tech notes and tips
Maximizing LLM Usage on a 128GB M1 Ultra (Apple Silicon Mac Studio) Understanding Unified Memory and the 96GB “VRAM” Limit Apple’s M1 Ultra chip uses unified memory , meaning the CPU and GPU share the same 128 GB RAM pool. However, macOS does not allow the GPU to use all 128 GB for graphics/compute tasks by default. In practice, about 75% of the physical memory is the recommended maximum for GPU…
Buying GPU for Running Large Parameter Models (LLMs) on Your Local Machine Introduction and Motivation Running large language models locally comes with one major challenge: memory . I learned this firsthand using an NVIDIA RTX 4090 with 24 GB of VRAM. While the 4090’s raw performance is stellar, its limited VRAM quickly became a bottleneck when attempting to run models beyond roughly 30 billion…
Introduction Imagine transforming a simple toy into an intelligent machine that recognises and interacts with its environment. That's exactly what I set out to do with a Nerf turret. The journey began with a desire to create something fun and educational for my son and me, and it evolved into an exploration of open-source hardware, machine learning, and robotics. The Beginning: Assembling the…
Global variables antipattern Global variables make code harder to reason about, test, and debug. Instead, use local variables or pass variables as function arguments. x = 0 def increment (): global x x += 1 print ( x ) # Output: 0 increment () print ( x ) # Output: 1 Mutating arguments antipattern Modifying arguments passed to a function can lead to unintended side effects and make code harder to…
Follow the PEP 8 PEP 8 is the official style guide for Python code. It provides guidelines for writing readable and maintainable code. Following this guide can make your code more consistent and easier to read for other developers. Read changelog and new features when new version is released It's good to know what was changed in new python releases.For instance very useful f strings which were…
This is simple investigation on what happens with iptables on service creation with /and without targetport specified I've been recently told by someone that using target port in service creates mess in iptables so I thought it will be cool challenge to check if thats really the case. I have used simple iptables-save to get whole iptables rules at once and then added service with targetport, then…
What's etcd and what part does it play in kubernetes? ETCD is an open source distributed key-value store. In kubernetes it is a "single point of truth" as well as "single point of failure", it is the "definition of the cluster" as it holds it's configuration and status so it's best to have it replicated. There are of course alternatives like consul , zookeeper , doozerd , but I can't tell anything…
How I've build 4 nodes kubernetes i7 16GB cluster I always wanted to build my own k8s cluster, but couldn't see any reliable "budget" 3-4 nodes option to go for - yes there are a lot of attempts of building kubernetes which are: raspberry pi which are not x86 compatible so you wont run a lot of docker images there some tv hdmi sticks - better than raspberry but cpus are quite weak. So one day my…
Volumes Persistent storage with NFS In this example I have created nfs share "PersistentVolume" on my qnap NAS which IP is 192.168.1.11 Create persistentVolume.yml apiVersion : v1 kind : PersistentVolume metadata : name : pv0001 spec : capacity : storage : 100Gi accessModes : - ReadWriteMany mountOptions : - nfsvers=4.1 nfs : path : /PersistentVolume/pv0001 server : 192.168.1.11…
k8s_nfs_persistent_volume Create nfs persistent volume: What you need NFS Server I have used NFS already installed on my QNAP NAS (You need to enable NO_ROOT_SQUASH on permissions) K8s cluster Now having your NFS share here 192.168.1.11/Persistentvolume/ you can try if it works with mount sudo mount -t nfs 192.168.1.11:/PersistentVolume /mnt/PersistentVolume Later on you can secure access with…