Technology reference and information archive.
Active recall == "Questions that force you to think" Active recall questions do not necessarily need an answer . Their objective, is rather: To prime you to read more carefully (when asked before you read some content) To solidify your understanding of something (when asked after you read some content) This is Pull-based Learning Push-based : Explain first, then show examples. Pull-based : Show…
Bad error messages will increase the load on support teams With good error messages, users can solve their own problems without resorting to support. Bad error messages mean that every problem becomes a ticket on internal support, costing money and time. In internal tools there is no need to hide the stacktrace In internet-facing tools you avoid showing the full error stacktrace, so as not to leak…
WIP Alert This is a work in progress. Current information is correct but more content may be added in the future. High-inflation scenarios add an additional challenge to training ML models because it accelerates model drift and it can easily break nonlinear relationships. Luckily there are several ways to mitigate this problem. Automatic retraining If you can, set up an automatic retraining…
WIP Alert This is a work in progress. Current information is correct but more content may be added in the future. See also Conda, Pip, Virtualenv and Pyenv: Commands Compared My current setup on MacOS is as follows: Manage virtualenvs with Conda Manage dependencies with uv but also use pip in some legacy projects Download, compile and sync dependencies with uv Create and Activate a Conda…
The tell-tale sign of great product management is that it seems to read your mind . Picture an app or digital product you use all the time; it mostly works well but there's this small thing that annoys you to no end: You don't get redirected after concluding an action; you need to do it manually every time. You need to provide the same piece of information multiple times. You need to manually…
Enable editor tab complete On MacOS : Cursor -> Settings -> VS Code Settings , enable Editor: Tab Completion : On Ubuntu : File -> Preferences -> VS Code Settings , enable Editor: Tab Completion : This one is NOT obvious at all! I mean, why wouldn't it be enabled by default? For some unknown reason, this is not always enabled by default.
Please note This post is mainly intended for my personal use . It is not peer-reviewed work and should not be taken as such. KTO: Model Alignment as Prospect Theoretic Optimization Source WHAT Authors develop a framework with HALOs (Human-Aware Losses), which add insights from Prospect Theory 1 to losses used to instruction-tune LLMs in DPO and similar strategies. They introduce one specific HALO…
Please note This post is mainly intended for my personal use . It is not peer-reviewed work and should not be taken as such. A General Theoretical Paradigm to Understand Learning from Human Preferences Source WHAT Authors propose a generalized framework called Ψ-PO to represent generic differentiable functions that can directly optimize preference data (similarly to DPO). DPO and RLHF are special…
Please note This post is mainly intended for my personal use . It is not peer-reviewed work and should not be taken as such. Fine-Tuning Language Models from Human Preferences Source WHAT RLHF is used to fine-tune pretrained LLMs for specific tasks (textual style transfer and text summarization). WHY Because RL/RLHF hadn't hitherto been used to fine-tune LLMs and authors thought it could lead to…
WIP Alert This is a work in progress. Current information is correct but more content may be added in the future. Feature vectors used during real-time ML scoring may break for some reasons: Normal model degradation (as time goes by) Operational problems (upstream feature sources break, services time-out, etc) Adversarial attacks There are many ways to make an ML model more robust in such a…
Truncate string column Need to use substring together with rpad , otherwise it doesn't work IME substring(rpad(MY_COLUMN,56,' '),1,55)
Please note This post is mainly intended for my personal use . It is not peer-reviewed work and should not be taken as such. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Source WHAT Chain-of-Thought (CoT) is a technique whereby the output begins with extra reasoning steps before giving the final answer. Chain-of-Thought prompting example. Source WHY Because even large…
Please note This post is mainly intended for my personal use . It is not peer-reviewed work and should not be taken as such. Learning to Forget: Continual Prediction with LSTM Source WHAT This paper introduces the "forget gate" to LSTM cells, that learn when they should "reset" the cell state. This was not present in the original 1997 paper by Hochreiter and Schmidhuber . WHY If LSTMs are applied…
See all posts related to testing here: Entries by Tag: testing Minimum Viable Post (MVP :grinning:) to send to your colleague who isn't sure whether tests are really needed. If you didn't test the code, you don't know if it works As simple as that. If it's not tested, you don't know if it works. You should assume it doesn't! "But this piece of logic is so simple, there's no need to test it." Yes,…
Please note This post is mainly intended for my personal use . It is not peer-reviewed work and should not be taken as such. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning Source WHAT A new strategy to induce reasoning capabilities (a la Chain-of-Thought, CoT) using pure Reinforcement Learning with Verifiable Rewards (RLVR), without Supervised Fine-Tuning (SFT).…
Format Unix timestamp as datetime string Use toDateTime(source_column, format) . All formats accepted by JodaTime are supported. Query: select created_at, toDateTime(created_at,'yyyy-MM-dd HH:mm:ssZZ') as created_at_str from my_table Sample results: created_at created_at_str 1744045102745 2025-04-07 16:58:22+00:00 1744045114395 2025-04-07 16:58:34+00:00 1744045166826 2025-04-07 16:59:26+00:00
Standardization 1 means following some sort of convention or rule when organizing code and architectural components within a codebase. What kinds of things should be standardized? Names (variables, components, files, services, libraries, repositories, etc) Folder structure (services, libraries, etc) Abstractions (similar abstractions should have similar names) Standardization reduces the cognitive…
Please note This post is mainly intended for my personal use . It is not peer-reviewed work and should not be taken as such. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models by Shao et al. Source WHAT A domain-specific LLM with 7B params is built to solve mathematical problems. A novel policy-gradient RL algorithm (Group Relative Policy Optimization or GRPO) is…
Please note This post is mainly intended for my personal use . It is not peer-reviewed work and should not be taken as such. Proximal Policy Optimization Algorithms Source WHAT Proximal Policy Optimization (PPO) is an enhancement to the Trust Region Policy Optimization (TRPO) Policy-Gradient algorithm for learning RL policies. WHY Because TRPO is complicated and it cannot be adequately used with…
WIP Alert This is a work in progress. Current information is correct but more content may be added in the future. Strategic: Having people rotate among teams TODO: Strategic: Explicit is better than Implicit TODO: Tactical: Prefer public forums over private ones Whenever possible, use public channels (e.g. on Slack) over private channels and DMs. Public channels increase accountability (other…
Please note This post is mainly intended for my personal use . It is not peer-reviewed work and should not be taken as such. Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study Source WHAT Authors run several experiments, including ablation tests, to compare the circumstances under which PPO is superior or inferior to DPO in RLHF. WHY To check why some papers report that DPO wins over…
Please note This post is mainly intended for my personal use . It is not peer-reviewed work and should not be taken as such. The Science of Detecting LLM-Generated Texts Source WHAT The authors survey the nascent field of machine-generated text detection, categorize the main types, explain the main approaches (mostly watermarking), and explain the need for fair evaluation. WHY There was no…
Please note This post is mainly intended for my personal use . It is not peer-reviewed work and should not be taken as such. Few-shot Fine-Tuning vs In-context Learning: a Fair Comparison and Evaluation Source WHAT Compare how LLMs generalize to out-of-domain problems when using Fine-tuning (FT) or In-context Learning (ICL) strategies. WHY Previous comparisons between FT and ICL did not take model…
Please note This post is mainly intended for my personal use . It is not peer-reviewed work and should not be taken as such. Multitask Prompted Training Enables Zero-Shot Task Generalization Source WHAT Investigate if and how fine-tuning a vanilla LM on text-to-text NLP tasks (like T5 summary ) helps it perform better on unseen tasks. Authors introduce the T0 (T-zero) model. WHY To see how the…
Please note This post is mainly intended for my personal use . It is not peer-reviewed work and should not be taken as such. Learning to summarize from human feedback Source WHAT RLHF 1 is applied to the task of generating abstractive summaries of an input text. WHY The authors wanted to extend the work by Ziegler et al 2019, using offline instead of online RL and better managing the labelers. HOW…
Please note This post is mainly intended for my personal use . It is not peer-reviewed work and should not be taken as such. Zephyr: Direct Distillation of LM Alignment Source WHAT Authors instruction-tune Mistral-7B vanilla by distillation : using DPO on open preference datasets and samples generated from previously aligned teacher models. WHY Because traditional distillation strategies are only…
WIP Alert This is a work in progress. Current information is correct but more content may be added in the future. List installed packages Use apt list --list installed , optionally using grep to limit the search $ apt list --installed | grep qua pngquant/focal,now 2.12.2-1 amd64 [installed] quarto/now 1.3.433 amd64 [installed,local] Install .deb package sudo dpkg -i my-file.deb Uninstall .deb…
Git version 2.x used Revert file from branch Use git checkout to retrieve a file from my-other-branch $ git checkout my-other-branch -- path/to/file Revert file from remote branch (using git log and checkout) Retrieve the last commit hash of the remote branch $ git log origin/main commit 123456abcde (origin/main, origin/HEAD, main) Author: John Doe <john-doe@example.com> Date: Mon Jul 3 12:44:25…
Please note This post is mainly intended for my personal use . It is not peer-reviewed work and should not be taken as such. Constitutional AI Source WHAT Constitutional AI (CAI) is a strategy to fine-tune LLMs that place a higher value on harmlessness 1 , without being overly evasive. CAI employs Reinforcement Learning with AI feedback (RLAIF), standing in contrast to RLHF, as used by InstructGPT…
Python 3x+, Pytest 7x+ used unless otherwise stated. Assert exception is raised Use with pytest.raises(ValueError): as a context manager: # inside my_test.py def test_raises_index_error (): # test will success if an IndexError is raised with pytest . raises ( IndexError ): arr = [ 1 , 2 , 3 ] arr [ 1 ] Assert exception with specific text Use pytest.raises(<class>, match=<regular_expression> ).…