Recent content on Charlie Chiang's blog
I like audio analyzers. It answers questions that my ears alone cannot answer quickly. Where is that resonance? Is the low end actually mono? How loud is this over the whole track? Is a limiter catching an occasional peak, or working all the time? What I do not like is an analyzer whose interface feels slower than the display it runs on. On my M1 Max, Excite Audio VISION 4X appeared to top out at…
I ordered an OVHcloud Kimsufi KS-5 dedicated server for 19.90 USD/month. It is not fast by modern standards, but it is a very usable small Proxmox box if you install it carefully. The goal for this machine was: Proxmox VE on mirrored ZFS root. Keep only part of each SSD for the root pool. Use the remaining SSD space as a fast disposable scratch pool. Put VMs and CTs behind NAT, because the server…
I sometimes need a tiny cloud instance that only forwards traffic. CPU and memory can be small, and the disk is mostly wasted space. On one Alibaba Cloud ecs.t6-c4m1.large instance, a 1 GB root disk on a long contract can push the cost down to about $0.40/month, but a normal Linux install leaves too little usable space. The trick is not complicated: Use Alpine Linux, because the base system is…
I recently tried to build QEMU on macOS and make the result portable enough that it could still run after uninstalling the Homebrew packages used during the build, or I can move the whole QEMU directory to another machine. I don’t want the Homebrew build because it installs too many dependencies on the system, and I want a more self-contained bundle, not a system-wide installation. The short…
背景 最近老家重建，家里需要重新布置无线网络。考虑到自建别墅面积比较大，单纯使用单个无线路由器覆盖不够，因此多个 AP 肯定是需要的。家里在装修的时候就每个房间吊顶中有留超六类线缆，这种情况对 AP 来说是非常友好的，因此决定使用企业级无线接入点（AP）来组建无线网络。由于华为系列的 AP 在国内使用量较大，闲鱼上非常好捡公司下架下来的二手设备，因此考虑华为系列的 AP。这些企业级 AP 并不需要担心二手问题，他们的质量是消费级产品无法比拟的，稳定性也是非常强。 目前是 2025 年 12 月，虽然在国内 Wi-Fi 7 （802.11be）的 6 GHz 频段还没批准无法使用（损失不少性能），但是上一下 Wi-Fi 7 追一下新也没什么问题。华为 Wi-Fi 7 AP 是 AirEngine xx7x 产品，我闲鱼花了 500 多，激情下单了一台 AirEngine 5773-21…
To install ZeroTier, one would typically allow port 9993/UDP on their firewall. However, in certain scenarios, you may need to run a ZeroTier moon on non-standard ports due to network restrictions (e.g. behind NAT) or conflicts with other services. This guide will walk you through the steps to set up a ZeroTier moon on non-standard ports. Setup Relays (Moon Nodes) on Non-Standard Ports Install…
Introduction Most schools and universities in China have a limited number of GPU servers (often only one or two, or even none). This makes the management of GPU development machines quite different than corporation which have virtually unlimited GPU servers. Let’s consider a normal lab environment with 12 students, each needing a GPU server for their projects, and we have one GPU server with…
启用 SSH 没SSH就别想着用Terminal了，赶紧去开了。 去绿联NAS的Web管理界面，控制面板 -> 终端机 -> SSH ，记得关闭自动关闭功能。 配置 root 用户 SSH 登录 用你的NAS用户名SSH到NAS上 ssh yourname@nas-ip 你需要先有一个SSH密钥对，如果你还没有，可以用 ssh-keygen -t rsa 生成一个。 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # 切root sudo su # 打开 root SSH 登录（仅允许密钥登录） # 注意，别去改 /etc/ssh/sshd_config ，因为绿联NAS的系统会保护这个文件，改了重启就失效了。 echo 'PermitRootLogin prohibit-password' >>…
If you want see the test results directly, please jump to the Test the unlocked performance section. What is P2P? From NVIDIA: GPUDirect Peer to Peer enables GPU-to-GPU copies as well as loads and stores directly over the memory fabric (PCIe, NVLink). GPUDirect Peer to Peer is supported natively by the CUDA Driver. As you can see, GPUDirect P2P allows for direct memory access between GPUs,…
Stripe Size When you create an hardware RAID array, you must encountered the term stripe size . For example, an 8-disk RAID 1/0 has a stripe width of 4, with a stripe element size of 64 KB has a stripe size of 256 KB (4 * 64 KB). A 5-disk RAID 5 (4+1) with a 64 KB stripe element size also has a stripe width of 256 KB drive (4 * 64 KB) . A stripe is the smallest chunk of data within a RAID array…
Background Consider a small lab, students need to use GPU for their projects. We have a NVIDIA GPU in our Proxmox VE server, and we want to share the GPU between multiple containers so that multiple students can use the GPU at the same time. Why not use a VM? Because a GPU can only be passed through to one VM at a time (only one student can use the GPU at a time). And resources are not flexible in…
Background One particular issue that I encountered when I tried to install some software using Homebrew on my Mac is that Homebrew deprecates support for older macOS versions just like Apple do. Once a macOS version is deprecated, Homebrew will not provide bottles (precompiled binaries) for that version. This means that you have to compile the software from source, which is extremely…
Background We want to test some sandbox application on our new cluster. My colleague reported that the performance is not as expected and wanted me to take a look. The sandbox application works like this: it starts a python container, runs some scripts, and exits. They executes scripts provided by the user, so the cluster is expected to create and destroy a lot of containers in a short period of…
Background The RDMA NIC names (InfiniBand HCAs in this case) of our nodes are inconsistent. We want to rename them to a consistent naming scheme. One node: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81…
So I decided to test NFS over RDMA somehow. Hostname ft2000 (Server) t3640 (Client) CPU Phytium FT-2000+/64 Intel Core i9-10900K RAM Quad Channel DDR4 128GB Dual Channel DDR4 64GB NIC Mellanox ConnextX-4 Lx 25GbE Mellanox ConnextX-4 Lx 25GbE OS UOS Server 20 1070a Debian 12 Two servers are connected using a 25GbE network. The goal is to set up NFS over RDMA and benchmark it to see the performance…
10G ConnectX-4 Lx NICs can be flashed to 25G with the right firmware. This is useful if you have a 10G NIC and want to upgrade to 25G without buying a new NIC. This article will guide you through the process of flashing a 10G Mellanox ConnectX-4 Lx NIC to 25G. Warning: Do not try this on MCX4121C. It will brick the NIC. Download firmware from Firmware for ConnectX®-4 Lx EN . Choose MCX4121A-ACUT .…
We have a 200Gbps InfiniBand network among about 100 nodes, connected using NVIDIA ConnectX-7 NICs (HDR200) to a NVIDIA MQM9700 switch. Each OSFP twin port (2xNDR, 2x400Gbps) on QM9700 is splitted into 4xNDR200 ports to connect to 4x NVIDIA ConnectX-7 NICs using 2xNDR to 4xNDR200 DAC/ACC (OSFP to 4x OSFP). We want to test the performance of the network between 2 nodes just to make sure it is…
Note Note that this blog currently acts as a reference for me. It is not a comprehensive guide on why and how to use Nix on macOS. I chose nix because I am tired of Homebrew’s slowness and NOT PROVIDING BOTTLES (BINARY) FOR OLD MACOS VERSIONS. Install I am not using the official installer because it cannot survive macOS updates. Instead, I am using the Determinate System’s Nix…
前言 由于最近接入了 10Gbps 网络，发现在 PT 做种时，传输速度并没有达到预期的速度，于是优化了一波。本文记录了一些针对高速做种的建议、注意点和技巧。 硬件条件 足够快的网络 做种本质上就是网络传输，快速的网络是做种的基础。 确保上传带宽 做种嘛，主要是给别人传输数据，吃的是你的上传带宽，跟下载几乎没有关系，这是最重要的。例如原本 50Mbps 的上传情况下，买一条 100Mbps 的宽带比以下所有的优化都重要。 但是在中国，一般来说家庭宽带的上行速度都是比较低的。例如，电信的 1000M 宽带，上行速度一般不会超过 100Mbps，通常在 50Mbps 左右。这样的话，你的上传速度就被限制在 6.25MB/s 左右，在这个速度下，其实最主要的瓶颈在于你的上传带宽上，不太需要下面说的硬件/软件优化🫠。你可能需要找不少资源来找到高上传带宽的宽带，例如某些地区的移动宽带会给 IPv6…
Recent Update : One month later, after we opened our issue #10264 , it turns out that the gVisor developers are also aware of tha lacking of documentation about the systemd-cgroup option. Finally: They have added the systemd-cgroup option to the documentation in 8c3abba , although the documentation website doesn’t seem to be updated as of now. runsc will throw a warning if it detects a…
Introduction Re-encoded BD anime series are my go-to when watching animes. They have great quality while being a fraction of the size of a Blu-ray disc. Let alone they come with all kinds of extra contents. Nowadays, re-encoded BD anime series usually have video tracks compressed using 10-bit x265 (a software implementation of the H.265/HEVC video compression standard), and audio tracks compressed…
Outline: Know ZFS filesystem and Docker’s zfs storage driver How Docker’s zfs driver works and Why it is slow Why we can’t directly use overlayfs on ZFS Why overlayfs don’t accept remote filesystems Why ZFS is identified as a remote filesystem How to actually solve the problem We will dive into the source code of moby , OpenZFS , and Linux kernel to find out. Note: I admit…