programming: the action or process of writing computer programs. |
rants: speak or shout at length in a wild, [im]passioned way. 

How to install ollama with deepseek R1 (that will also install ROCm if you are using AMD GPU). curl -fsSL https://ollama.com/install.sh | sh sudo chown -R ollama /usr/share/ollama # just in case # additional if using AMD GPU curl -L https://ollama.com/download/ollama-linux-amd64-rocm.tgz -o ollama-linux-amd64-rocm.tgz sudo tar -C /usr -xzf ollama-linux-amd64-rocm.tgz # if you are using 6600XT and…
As much as I didn't like Google PubSub (because of it's inefficiency), especially compared to normal database that forced to use as queue (eg. Tarantool can do 200k events/s, Clickhouse can do 900k events/s, without automatic fan out). Here's how you can use it using Go. First you must create a docker compose file: services: testpubsub: image: gcr.io/google.com/cloudsdktool/cloud-sdk:latest #…
Last time we're dumping from Cassandra and restore it to Clickhouse for proper analytics. To do that we need cqlsh and cassandradump pip install -U cqlsh pip install cassandra-driver # list all tables tables=`cqlsh -k keyspace1 -e 'describe keyspace' | grep TABLE | cut -d '.' -f 2 | cut -d ' ' -f 1` # run copy to file mkdir backup echo $tables | while read x; do time cqlsh -k keyspace1 -e "COPY $x…
Normally for my projects I use docker-compose + normal bind persistence directory to avoid data loss for database (and keep it fast and simple to maintain), because it's simpler to upgrade version than normal installation (looking at that popular database that annoying to upgrade the major version that require a lot of copying) and the network/disk/cpu overhead also unnoticeable. Today we're gonna…
Today we're gonna dump Cassandra table and put it to Clickhouse. Cassandra is columnar database but use as OLTP since it have really good distributed capability (customizable replication factor, multi-cluster/region, clustered/partitioned by default -- so good for multitenant applications), but if we need to do analytics queries or some complex query, it became super sucks, even with ScyllaDB's…
Today we're going to create an UDF (User-defined Function) in Golang that can be run inside Clickhouse query, this function will parse uuid v1 and return timestamp of it since Clickhouse doesn't have this function for now . Inspired from the python version with TabSeparated delimiter (since it's easiest to parse), UDF in Clickhouse will read line by line (each row is each line, and each text…
Today we're gonna test ollama ( just like previous article ) with AMD GPU, to do this you'll need to run docker, for example using this docker compose file: version: "3.7" services: ollama: container_name: ollama image: ollama/ollama:0.1.22-rocm environment: HSA_OVERRIDE_GFX_VERSION: 10.3.0 # only if you are using 6600XT volumes: - /usr/share/ollama/.ollama:/root/.ollama # reuse existing model #-…
Today we're gonna try to use ollama to generate some code, it quite easy you just need to install it using curl like this, and install some model : curl https://ollama.ai/install.sh | sh # these models already predownloaded before first run time ollama run codellama 'show me inplace mergesort using golang' #time ollama run deepseek-coder 'show me inplace mergesort using golang' time ollama run…
So I have a past project from 2014 that still running until now (5 independent services: 3 written in Golang - 2 read-only service, 2 write-only service written in Ruby, 1 PostgreSQL, 1 Redis, ~45GB database / 2.7GB daily backup, 1.1GB cache, 8GB logs, <800MB compressed), and the server RAID already near death (since the server was alive without issue for 12 years), and on new server there's…
In this demo we're going to create mTLS using Go and Fiber . To create certificates that can be used for mutual authentication, what you need to have is just an OpenSSL program (or simplecert , or mkcert like in previous natsmtls1 example), create a CA ( certificate authority ), server certs, and client certs, something like this: # generate CA Root openssl req -newkey rsa:2048 -new -nodes -x509…
So today we're gonna benchmark between docker-volume (bind to docker-managed volume), bind/mount-fs (binding to host filesystem), and tmpfs. Which one can be the fastest? here's the docker compose: version: "3.7" services: web: image: ubuntu command: "sleep 3600" volumes: - ./temp1:/temp1 # mountfs - temp2:/temp2 # dockvol - temp3:/temp3 # tmpfs volumes: temp2: temp3: driver_opts: type: tmpfs…
As you might already know in the past article , I'm a huge fan of NATS , NATS is one of the fastest one at-most-once delivery non-persistent message broker. Unlike rabbitmq / lavinmq / kafka / redpanda , NATS is not a persistent queue, the persistent version is called NATS-Jetstream (but it's better to use rabbitmq/lavinmq/kafka/redpanda since they are more popular and have more integration than…
So today we're gonna learn a tool called chisel , from the same creator of overseer that I usually use to graceful restart production service. What chisel can do? It can forward traffic from private network to public network, for example if you have service/port that only accessible from your internal network, and you want it to be exposed/tunneled to server that has public IP/accessible from…
Usually I use extension in firefox or chrome, like UrbanVPN , but now I know that Cloudflare provides free VPN, I have problem where my ISP always block DNS queries, where my work mostly heavy on Web, DNS, Storage, any cloud related stuff. Normally I use DNSSec/DNSCrypt-proxy so I could bypass those restriction, but now I know that Cloudflare warp is available on Linux , all you need to do is just…
So today we're gonna use vault to make the configuration of an application to be in-memory, this would make debugging harder (since it's in memory, not on disk), but a bit more secure (if got hacked, have to read memory to know the credentials). The flow of doing this is something like this: 1. Set up Vault service in separate directory ( vault-server/Dockerfile ): FROM hashicorp/vault RUN apk add…
Autoscaling mostly useless, if the number of host/nodes/hypervisor is limited, eg. we only have N number of nodes, and we tried to autoscale the services inside of it, so by default we already waste a lot of unused resources (especially if the billing criteria is not like Jelastic , you use whatever you allocate not whatever you utilize). Autoscaling also quite useless if your problem is I/O-bound…
Today we're gonna benchmark nodejs/bin+uwebsocket with golang+nbio. The code is here , both taken from their own example. The benchmark plan is create 10k client/connection, send both text/binary string and receive back the from server and sleep for 1s,100ms,10ms or 1ms, the result is as expected: go 1.20.5 nbio 1.3.16 rps: 19157.16 avg/max latency = 1.66ms/319.88ms elapsed 10.2s 102 MB 0.6 core…
Dockerfile is quite simple, first we need to pick the base image for build phase (only if you want to build inside docker, if you already have CI/CD that build it outside, you just need to copy the executable binary directly), put command of build steps, choose runtime image for run stage (popular one like ubuntu/debian have bunch of debugging tools, alpine/busybox for stripped one), copy the…
So today we're gonna benchmark database that can store latitude-longitude (GPS coordinate), the benchmark spec is unbatched INSERT 100K records of (id, lat, long) tuple, search 200K times (or until deadline reached) 500 nearest point, id and the distance, move 100 points to another location 50 times, all benchmark done in 16 threads (so other 16 threads can be used by the database). The contender…
So we're gonna try to manipulate DNS records using go SDK (not REST API directly). I went through first 2 page of google search results, and companies that providing SDK for Go were: IBM networking-go-sdk - 161.26.0.10 and 161.26.0.11 - timedout resolving their own website AWS route53 - 169.254.169.253 - timedout resolving their own website DNSimple dnsimple-go - 162.159.27.4 and 199.247.155.53 -…
Lua is one of the most popular embeddable language (other than Javascript that already embedded in browser), there's a lot of products that embed lua as it's scripting language (probably because the language and the C-API is simple, VM size is small, and there's a product called LuaJIT that is currently the fastest JIT implementation for scripting language. Current version of Lua is 5.4, but…
Today we're going to benchmark CockroachDB one of database that I use this year to create embedded application. I use CockroachDB because I don't want to use SqLite or any other embedded database that lack of tooling or cannot be accessed by multiple program at the same time. With CockroachDB I only need to distribute my application binary, cockroachdb binary, and that's it, the offline backup…
Sometimes we want to convert from map to struct or struct to map (dictionary in other language), or even struct to struct. There's some library that can help us doing this, for example structs , mapstructure , copier , or smapping . We could also utilize serialization and deserialization libraries to do this. With caveats, that some serialization format (eg. JSON) doesn't allow integer larger than…
Today we're gonna learn about InfluxDB , a time series database that already been a standard for open source metric storage, a part of TICK stack ( telegraf -- a metric collector like datadog-agent/fluentd, influx -- the database, chronograf -- visualizer like grafana, kapacitor -- an alert manager like prometheus alert-manager also for ETL). To install influx use this steps from this link…
The result for Techempower framework benchmark round 21 is out , as usual the most important benchmark is the update and multi query benchmark: The top rankers are Rust, JS (cheating), Java, C++, C#, PHP, C, Scala, Kotlin, and Go. For multiple queries: Top rankers are Rust, Java, JS (cheating), Scala, Kotlin, C++, C#, and Go. These benchmark shows how efficient their database driver (which mostly…