Wander around
New Hobbies I tried a lots of thing in recently two months. Piano An electronic piano is much cheaper, lighter, and takes up less space. However, it takes a lot of time to learn. I’ve been learning for two months, but I spend less time practicing recently than I did in the beginning. Right now, the hardest part for me is playing a different rhythm with each hand—it’s like drawing a circle with one…
This is the first week of my internship, I’m working on the onboarding process. There are two types of thing that make me surprise. One is the emotional quotient, most of the interns and the director of mine in the onboarding process have high EQ. They are good at communicating with others. They are confident, active and clear about the information that we want to express and the topic that they…
AI usage on academic research is one of the best reference for AI application. Terence Tao The earliest application of AI on academic research is from Terence Tao, who use and share the usage of cutting-edge AI products at early stages of AI development. I kept adobt at that time because AI is till a little bit dump sometimes, Academic research is the hardest and most sacred area. But the…
How mobile development will be next year There are many mobile development platforms for non-technical users nowadays, like Rork and Lovable. They are not perfect now but will become much better in a year, just like how ChatGPT changed in the year after it was published, and like how much vibe coding agents have progressed in the last year. These platforms can generate everything we need to submit…
ChatGPT was published in November 2022, it’s 3 years and 3 month till now. Look back, it’s hard to believe that it change so much for computer science and programming. How AI change programming When ChatGPT was published at frist time, we were surprised that it can answer unlimited questions, after that, we started to use it for every question while ridicule, we shared the hallucination with…
Previous Plan My first blog was summary of 2021, that is the start of the blog, it’s a little hard to believe that I write this blog for five years, I like this feeling. See back the plan of 2025, it’s just very simple. English: fluent Exercise New Products Read My english is much better than me in the start of 2025, one of the most important reasons is that I came to Canada and start to learn…
Why kindle is better I usualy read kindle on the bed before I fall asleep, I think the big advantage of reading kindle is that kindle leads less distractions. When I use computer and browser the website, it’s so eary to do other think when a notification jump, like text messages app or gmail. In contrast, When I read on kindle, I only focus on kindle and nothing else. In the past, I just readed…
Project The last month goes fast, because I almost spned all my last month into a new project. Right now, I finish the first relative muture version of the project. Of course, I still realize some feature that I intended to implement later, I just cannot help myself to optimize that part when I was thinking of that. I really enjoyed the feeling that I put all myself into a thing that I’m total…
English proficiency I though I can improve my english ability after staying in english speaking country for some time naturally, this is definitely wrong. English proficiency can be imporved only when I practice it. Changing naturally is just the cover of true life. Recently, I took two interviews, that’s the direct event that I realize I still have huge disadvantages on english proficiency. First…
In short, Target of fraud detection is to detect fradulent traffic and filtering or block them. What mekes this interesting and difficult are what we are detecting are human, it might be as hard as you can imagine to defense with huamn. Strategy Rules Most of the traffic are produced by pruely machine/device without any human involved, like Ddos. Rules usually can handle these kind of fraudulent…
Introduction PopTranslate is a translation chrome extension, main features: Pop up translated contents immediately after selecting contents without extra click Show english dictionary when selecting a word Show translated contenst in the right above corner user experience Sub window will show quickly after selecting, ensuring real-time experience for users User can click any other place but sub…
Today is the last day for me in netease, I don’t much as much feeling as that in baidu, on the one hand, I spend much more time in baidu, almose five days, one the other hand, that’s my first job, first time is always different. What makes my surprise is yesterday, the boss, all people in my small group have a bye-bye lunch, leader and boss are both friendly even I only stay here over four month.…
Copilot-typed GitHub Copilot, cursor, GPT-codex, winsurf, trae are representative products of Copilot-typed tools, their function is to help users complete code automatically when necessary. Of course, they provide interactive conversation for users with code as content, which is common feature. This is the direct usage intruction in coding field when LLM appears. CLI-typed Claude-code is the…
Gemini-cli Gemini-cli is command line tool supported by Gemini-2.5-pro model, it’s a similar product with Claude-code by Anthropic but free, what’s more, Google open source all code of Gemini-cli in github and receive 20k stars in only one day, 40k in three days, this’s scene that never happened for a long time. I use Gemini-cli to generate a project which can transfer model from huggingface into…
Prompt Prompt is the most direct way to influence response, tips for good prompt: Clear instruction about our demand Provide necessary context, role, tone, format guide LLM output reasoning process before final answer More instructions, less constraints Exampler can ensure the constructure is as same as example The purpose of prompt in post-training is building best reasoning architecture in…
How much do LLM memorize? key definition unintended memorization: memorize a specific dataset generalization (intended memorization): contains about the true data-generation process calculation method: by information entropy and mutual information double desent appear on the changing points from unintended memorization into generalization GPT-models store 3.6bits data per parameters value of…
ALL thing are certain in traditional computer technology, some programer say that there are beauty of certainty in traditional computer technology compared with current LLMs. For the influence or effects for this world, traditional computer technology can solve the certain problems and LMMs can slove all others, but LLMs sometimes cannot slove correctly. Compared with SFT, RL-base method like DPO,…
Main idea Key point it to understand the below pictures Iteration steps for each input, generator G outputs for each output, calculate logits_prob for each token in current, old, reference model calcualte objective value as loss update old model in each step update reference model in each epoch Objective function G is amount of outputs in each group for each input O_i is i-th output in current…
LLM Application The purpose of Appcalition is to try our best to meed the demand of users. Depth of technology is not very important. There are many unexplainable points in the field of LLM, as a result, there are some basic method we can use to improve the quality of product. Prompt is the first point engineers always use in the frist step, it’s easy to start but hard to advanced. In The first…
LLM Inference It’s a great milestone when I finish a mature and iterable project, even it’s still in the first stage. There’re plenty AI infra repositoreis and projects in github, but most of them are designed and developed for nvidia GPU, some of them are designed for AMD. LLM computation industry is a huge field and market, there’re also many domestic GPU brands in china. As a result, the work…
This is the first week to work as a LLM inference engineer, the real work content is what I want in the past. I don’t have much experience in this field, But it’s still acceptable. To do what I like is nice, I can put all my energy into my work. I hope freshness is not the only reason and I can maintain this status for longer time. Small company don’t have mature manage system, there’s also some…
This is the second time period that I need to seek a job position in my life till now. I don’t have much experience to take a interview in the past, I think I learn a lot this time. First lesson I learn Preparation is necessary for me to seek a better job than that in the past, not only for the project experience or basic knowledge, but also the attitude and future plan. I need to prepare for some…
Happy New Year! Feel more happiness this year, really enjoy.
Product AutoSwitch Translate Final make this extension, which can automatically switch target language when source language is same with target language. What surprised me was I only spen half day to program and build this extension, which inspired to do it immediately after have a new idea, this will be the best experience not only for result but also for my satisfaction. This can save lots of…
What I do keywods: Jan-May: work Travel: wuhan, huangshan IELTS Products: Chinese blog collection slack to discord IELTS speaking assistant PopTranslate Security clearance project AutoBuilder LLM acceleration Youtube xiaohongshu table tennis blog: 33 What I do are good: Try many my own products Try new-type products Try to maintain products which can last long time LLM translation, new idea What I…
Product Triton Try to debug the Liger-Kernel but found it works well except bfloat type. Learning news about LLM translation and try to realise the whole one in this kind of project, like cut cross entropy Glass My glass broke on 1.9, I’m really blind when I lost my glass. This is first experience I broke my glass and realise that how the backup of glass is important. This is what I do next: 1)…
Cross Entropy idea of fast cross entropy Based on previous knowledge on fast cross-entropy, realizing it by triton doesn’t spend too much. There are only 1e-7 difference between Pytorch and my Triton kernel, But the speed is significantly lower. Liger-Kernel Getting advise on gpumode discord group, I can read how it realized in other great Triton kernel repository. Liger-Kernel is the best option,…
Product Implementation of Triton I tried to realise better cross entropy loss by Triton, but failed, still trying to find the reason… PopTranslate Update to 3.2, replace self-hosted translation server with Google Gemini 2.0 experimental Thought I met three great table tennis coaches recently in table tennis room. They have different styles to teach people, one of them are strict but another of…
Product Triton-Puzzle This is a project tearch you how to use Triton, which is a new open-source alternative language for cuda. It’s always hard to understand and use it if you are not familiar with cuda-type language. This is definitly a great project for beginners, I thing every hard field should have this kind of toturial. Meanwhile it’s not possible for every filed to have, it’s luck for…
Product AutoBuilder Almost spend half of this week on AutoBuider, finish it and use it to build another project, ID_confuser, which programmed totally by AutoBuilder and spend only ten minutes. Summer internship submit 53 applications this week, still no interview. Reading The economist: recent issue and three issues(China part) in 2019 Harry potter Economy of China from fupeng Thought Table…
Product AutoBuilder I want to build AutoBuilder which can build simple product totally by AI model, like static websites. Compared copilot or cursor, AutoBuilder and see the result base on the current code after we build and run them. As a result, AI image model can provide feedback for current code, which can help text model to be better. First version is too ideal. After I show the static…
Product Visa Received a possible visa phone call, her tell me that my visa is not valid. All phenomenon show that it’s a fraud call except the offical-email, I cannot iamge that fraud group can steal the email-client of IRCC. This will interupt my rent plan, maybe I will still make a rent rencently, but really waster much of my time. Summer internship I realized that I should seek a summer…
Product xiaohongshu It’s hard to get regular flow, business transformation is also hard than expectation to obtain. Trust seems a big gap for normal user because my old account get some real transformations but new account cannot even it had more view and likes. Try another week. Reading AI agent for twitter, discord… get QQ notes run and debug in one platform video creation State of the software…
Product LLM acceleration Almost finish it xiaohongshu start the small business Reading lazy-leadership llm-server ShortMax: fill the space between TikTok and film. HARO shut down: typical influence to content industry from ChatGPT small company management tools focusing enough on the problems making plans for holidays M4 mini, most powerful CPU chip Amateur search engine build AI cluster with…
Summary Methods There are two main methods to acclerate LLM and another tricky methods low-rank: reduce dimension of matrix block: compute matrix with block trick: update model structure or change training process Reference xformers: collection of optimized transformers unsloth triton Liger-Kernel Flash-attention Megatron-LM from NVIDIA huggingface accelerate ColossalAI awesome LLM system…
Product LocalPictureCompress Spent whole one day to build LocalPictureCompress, really enjoy the monent when I publish it. Try AI code assistants continue: open source product, supports OpenAI’s, Anthropic’s and open-source models interpreter: works in terminal, suitable prompts are necessary if you want write code into file gpt-coplit: Only support server’s model, like ChatGPT, Claude computer…
Product New ideas computer use by local models polish anything by local models YouTube Upload five videos this week Reading Google build-AI challenge OpenAI ask me anything anthropic auto computer use talk with computer Thought There are more local models recently, that indicates local model has enough ability to build something, It also have more potentiality because pravicy is always a huge…
Product LLM acceleration read one paper Flash-attention: compute attention by blocks YouTube Upload five videos this week and start to try codeforces problems. Codeforces problems always contains more backgroupd information, which will practice more for my speaking ability, meanwhile, it has more users and can attract more viewers. PopTranslate two updates add speicial for chinese to reduce the…
Backgroud There are two common kinds of bound which limited the speed of training in deep learning. Memeory-bound: time spent on memeory-access is bottlenecked Computation-bound: time spent on computation is bottlenecked Introduction Inspiration: just tiling? Description: Split the Q, K, V into blocks and calculate output matrice O by block to avoid the store of softmax intermediate matrice which…
Collect, summary and adjust to get the following tutorial from multi-sources in Reference. How to learn Very quickly identify what the foundational knowledge is Build a personal curriculum to become an expert and avoid the trap of the expert beginner. Sprint hard the first 15-20 hours to impress initial memory, then decelerate to a more regular pace. Reading Research Papers Collect related…
Product LLM acceleration Matrix Multiplcation: Read more LoRA: start reading paper YouTube Upload four videos this week and receive 5 Subscribers Blog Update current blog to jekyll-theme-chirpy Updated commit Reading Searching content is the advanced ability compared reading from others flow is money, eliminating information gap can obtain money Product is the exchange of value Everyone should…
Introduction Inspiration: the change in weights during model adaptation have a low “intrinsic rank” Description: Change small matrices A and B when fine-tune, adding A * B to weight W, which significantly reduce the trainable number of parameters because r << d Novelty: Reducing Fine-tuen cost without additional latency and quality reduction or changing input consturction at the same time Benefits…
Background After read “Manual Autograd” in unsloth’s blog, I try to parse model and found more related point where we can optimize. torchview is a great similar tool to use. torchview what torchview can do I want to show what torchview can do after I try it. Model: torchview can parse model when inferencing and training, it support mlp, bert, Gemma, llama3.2. Node: the smallest node is tensor,…
Product Voice correction: failed, technology is not enough mature to meet this demand. PopTranslate: First available product. Cost of product with server is higher than expectation. unsloth: great start but just a start. Youtube: great start but just a start. Remote work: failed Security investage project: too much concerns that I didn’t realized in the past. Reflection can not concentrate enough…
Life I stay home for 14 days this time, it’s long. I spent the first seven days with my families and last seven days with friends. Everyone had different life, what are the same is that everyone have much pressure about life. Even though, there will be one or two friends who have much proactive atitude towards life, it’s so nice to talk with them. I want to be this kind of person. Product Visa…
Product process PopTranslate This is a multi stages story. 1) Lost my free $300 in Google Account In order to join the YouTube Premium family account, I need to change my location from Canada to America. So, I delete my previous payment profile, which is used for Google Translation API but I didn’t realize that at that time. I create a new payment where I set the address in America and linked this…
Product process PopTranslate Summary of promotion list of ways to publish promotion and is there response? ruanyifeng: Yes Product hunt: Yes Reddit: Yes HackerNews: No Twitter: No Discord: No pitchwall: No v2ex: No Users No much users Updating Got some feedbacks from users and update two versions, 1.1 -> 1.2 -> 2.0, details. It real make me happy when I found someone is using and like my product…
Theory of Fast Cross Entropy Loss As a start of LLM acceleration project, I learnt Fast Cross Entropy Loss in unsloth, it shows a better realize solution of cross entropy loss than default pytorch code. As default, pytorch will use log_softmax to realize CE loss, it’s common for all solution, but, the target has only one valid value in the training process of LLM and other element are 0, this is…
Product process PopTranslate Finally find the root reason why my chrome extension was rejected, I put the api_key in .gitignore file and it can’t upload to chrome extension store successfully when I upload the zip file. Actually I found the root reason thought the crx file generated by chrome extension. After I pointed out this reason, google extension team also responce to my email and tell me…
Product process PopTranslate I finally finish this project, it works on my computer and my gf computer, I like this project. Sad news is my review for this extension was rejected two times by google, I didn’t know the reason ans ask for further review. Other product attempts 1) Text-to-voice in-browser After know about transformer.js which aimed to bridge the gap between web and LLM, I tried this…