Recent content on Yisheng's Blog
AI agents live and die by their context windows. The system prompt, tool schemas, conversation history, retrieved documents — everything the model sees during a single request shapes what it does next. Context engineering is the art of deciding what goes in, where it goes, and when it gets removed. 
 Good context engineering makes agents more reliable, faster, and cheaper. This post explores…
In this post, we implement GPT-OSS-20B inference from scratch in PyTorch. Building GPT-OSS shares a lot of common ground with building the Qwen3 family — if you haven’t read my Qwen3 Inference from Scratch and Qwen3 MoE from Scratch posts, I recommend starting there. 
 GPT-OSS is also a sparse MoE transformer with RMSNorm + GQA + RoPE + SwiGLU experts — the shared components work the…
In this post, we implement Qwen3 MoE inference from scratch in PyTorch. It builds on the Qwen3 dense architecture — if you haven’t read my Qwen3 Inference from Scratch post, I recommend starting there, as it covers the full dense transformer pipeline (tokenizer, RMSNorm, RoPE, GQA, SwiGLU FFN, KV cache, generation). This post covers the key architectural advancement from dense to MoE:…
Better late than never — I’ve started a repo called llm-from-scratch , where I reimplement popular open-source LLMs piece by piece, with a focus on making every component’s structure clear and easy to follow. It’s meant to be a hands-on tutorial for beginners and a personal reference for myself. The first entry is a from-scratch implementation of Qwen3-0.6B in pure PyTorch — no…
Today I revisit the classic LRU cache problem in my playground repo. The standard answer comes to my mind immediately: HashMap + Doubly Linked List. Cool, I remember that. But then I ask myself, what about LFU, B+Tree, and other complex data structures? I can barely remember and have to look up the script I developed when I’m at school. 
 Yeah, the real problem is that I’m…
最近读到laike9m <工作三年我学到了什么> ，听完那期捕蛇者说博客，觉得自己的视角下有许多不同，想也来写一写自己的感想。我毕业之后就一直在一家硅谷小厂工作，到今年6月也满三年了。前两年的角色大致是DE/SDE，最近一年在转向AI/LLM。这篇想把主题扩展到工作之外，到生活，心态，团队，方方面面，也算一个阶段性的记录。 
 换方向是常有的事 
 换方向是常有的事，在小公司里更是。我刚进公司的时候是做维护data pipeline，花了6个月做了一次AWS EMR到databricks的迁移，稳定之后就几乎不需要时间维护了。随后便去接手了后端服务的性能优化，以这个契机接触了python / redis / mysql / elasticsearch后端技术栈，有段时间还去做了一些k8s…
Background 
 过去一年各大云服务厂商在Redis换license之后，力推使用Valkey代替Redis作为kv内存数据库的使用方案。Valkey作为Redis 7.2.4 fork，不仅兼容redis各个语言的client库，也解决了Redis目前存在的单线程性能瓶颈和内存占用问题，不禁好奇其性能有多大提升，周末找了个时间进行了性能benchmark。 
 已有youtube博主进行过benchmark，但由于方法不同结果仅供参考，可以与本文结果进行互相比较 
 
 
 

 关于Valkey相较于Redis的改进，主要有两个，一个是io-thread, 另一个是数据结构内存优化, 可以参见以下两篇blog post: 
 
 Unlock 1m rps - part1 
 Unlock 1m…
Background 
 Our team planned to upgrade one of our MySQL instance from 8.0.28 to 8.0.36 through AWS RDS blue/green deployment. The whole process includes 4 steps: 
 
 set up a new MySQL 8.0.36 instance (slave), recover data from snapshot 
 new MySQL instance (8.0.36) subscribe to old MySQL instance (8.0.28) 
 once there’s no replication lags, do switch-over. The downtime is…
Background 
 When I tried to run a big range query on mysql using aiomysql ‘s SSDictCursor , everything looks fine at first but after a period of time (15min to 20min), the query stopped with error.
The error message is: 
 Error Code 2013: Lost connection to MySQL server during query.
 Environment 
 Database
MySQL version:…
Background 
 One of the pipelines I maintained failed last week. The first try failed but the second try was successful. The pipeline’s structure is quite simple, it contains 3 steps, like most of the ETL pipelines: 
 
 Load data from database 
 Boardcast join a 2GiB dataset 
 Write to delta table 
 
 Error message is There is not enough memory to build the hash map .…
背景 
 摘抄自plantegg的blog 
 从2018年开始，我们有个业务陆续接到反馈 Nginx 线上集群经常出现不响应或者偶发性的“超慢”请求。这种卡顿每天都有少量出现。而只有多个集群中的一个出现，其他压力更大的集群皆未出现。 
 业务结构比较简单：LVS->Nginx->后端，如图 
 
 观察到的现象： 
 
 出问题前不久升级 Nginx 配置，打开了 reuseport 功能 
 在压力大的后端（upstream）服务环境不容易出现，后端压力轻对应的Nginx卡顿概率更高 
 关闭 reuseport 后 问题少了很多 
 失败的请求响应时间都是 0ms（Nginx日志不靠谱了） 
 从 Nginx 日志上看，所有失败的健康检查请求都是0ms 的499…
Background 
 As what is explained in my pervious post , B+ tree depth decides the worst case of MySQL’s query performance, only considering those use index. And with the help of SSD, that should be fine to have a B+ tree deeper than 4 layers. 
 So the next question is: how bad the performance is possible to be? Here I intentionally design a table using very long string as primary key and…
Rumor 
 There’s rumor around the internet that we should avoid having > 20M rows in a single MySQL table. Otherwise, the table’s performance will be downgraded, you will find SQL query much slower than usual when it’s above the soft limit. These judgements were made on HDD many years ago. I’m wondering if it’s still true for MySQL on SSD in 2023, and if true, why…
背景 
 这个问题一开始是在进行RDS实验的时候发现的。最初的情景是，多台机器同时对数据库进行select和insert操作时，会发现insert操作会造成select操作qps大幅下降，且insert操作结束之后select操作的qps仍不能回升。起初以为是RDS的问题，但是在复现问题、监控RDS之后发现RDS的压力其实很小。于是开始怀疑是网络的问题，在简化了场景和操作之后，发现能在过去做tcp实验的机器上复现，于是用这个更简单的场景进行问题复现和分析。 
 环境 
 linux kernal version: 6.1 
linux image: amazon/al2023-ami-2023.0.20230419.0-kernel-6.1-x86_64 
instance type: AWS t2.micro (1 vCPU, 1GiB RAM)…
send/receive buffer 
 Goal 
 Get a better understanding of how send/receive buffer size may affect the network rtt and throughput. Usually send/receive buffer size is managed by kernal and application should not hardcode these values. However, real world cases are more rare and complicated. Inapporiate buffer size may be the root case of some network issues. Rare but possible. 
 Method…
packet delay, loss, duplicate, corrupt, out-of-order and bandwidth limit 
 Goal 
 The goal of packet capture experiment here is to get a better understanding of TCP protocol. Besides, by reproducing the common network issues in the real world like packet delay, loss, out-of-order and brandwidth limit and saving a snapshot of packet capture result of circumstances above, we can identify and…
Observation 
 Spark Job runtime is not stable 
 
 As what can see from the screenshot, the pipeline (spark job) runtime is not quite stable. However, the pipeline deals with almost fixed amount data everyday, the runtime should not be so quite unstable like this. 
 Besides, the pipeline runtime is quite stable at around 15min before migration to databricks. 
 Analysis 
…
Observation 
 
 
 Spark cluster master node OOM 
 Master node shutdown due to OOM. Error log: 
 INFO Data stored in hdfs:///XXXX
INFO XXXXX updated
INFO Data has XXXXX records
INFO Data stored in hdfs:///XXXX
INFO XXXXX updated
#
# java.lang.OutOfMemoryError: Java heap space
# -XX:OnOutOfMemoryError='kill -9 %p'
# Executing /bin/sh -c 'kill -9…
Observation 
 
 
 Queue client performance went down 
 Everything worked well in the morning until we got monitoring alarm at 12:45 PM EST: <topic_name> Queue lag is too big . The issue happened in a service which is responsible for consuming message from message queue, processing message and writing it to database. The service also uses Redis to cache some objects which can be…
Observation 
 CronJob is taking more than 1h to complete 
 My colleagues told me that one of the cron job stuck in the middle after a random day. They received the warning: CronJob is taking more than 1h to complete. And the pod kept stucking there after a day, which is abnormal. However, another cron job which almost uses the same code works well. No database failure was reported during…
Observation 
 Signal file is widely used in Hadoop ecosystem. If you have experience with MapReduce, you’ll notice that by default MapReduce runtime writes an empty _SUCCESS file to mark successful completion of a job to the output folder. AWS DataPipeline and Databricks also support “file arrival” to trigger a downstream job. 
 Question 
 Is signal file a good architecture design?…

 

 My name is Yisheng Gong. 
 Software Engineer@ YipitData 
 Based in Santa Clara, California. 
 Recently working on Agent / RL Infra 
 I love open source and I’m a committer of sglang , miles and verl 
 And here’s some personal projects I built during my spare time: 
 
 playground - Learning by breaking things 
 llm-from-scratch - Build…