Thoughts on Data Science, Machine Learning, and AI
Announcing: ๐ช๐ช-๐ฃ๐๐ โ ๐ช๐ฒ๐ถ๐ด๐ต๐๐ช๐ฎ๐๐ฐ๐ต๐ฒ๐ฟ ๐ฃ๐ฟ๐ผ๐ท๐ฒ๐ฐ๐๐ฒ๐ฑ ๐๐ฟ๐ฎ๐ฑ๐ถ๐ฒ๐ป๐ ๐๐ฒ๐๐ฐ๐ฒ๐ป๐ ๐ I just released WW-PGD, a small PyTorch add-on that wraps standard optimizers More
There is a deep connection between the open-source weightwatcher tool, which implements ideas from the theory of Heavy Tailed Self-Regularization More
Recently, Meta released LLama3.2 1B and 3B Instruct Fine Tuned LLM. To mixed reviews. On the one hand, it s ranking More
Are you Fine-Tuning an open-source LLMs ? Like Llama, Mistral, or Qwen? A That is, Instruct Fine Tuning. Whether you More
Double Descent (DD) is something that has surprised statisticians, computer scientists, and deep learning practitioners but it was known in the More
Recently, Microsoft Research published the LASER method: Layer-Selective Rank Reduction in this recent, very popular paper The Truth is in There: More
Recently, the Mistral models have taken the LLM world by storm. The Mistral Mixture of Experts (MOE) 8x7b model outperforms other More
Evaluating LLMs is hard. Especially when you don t have a lot of test data.In the last post, we saw how to More
if you are fine-tuning your own LLMs, you need a way to evaluate them. And while there are over a dozen More
WeightWatcher 0.7 has just been released, and it includes the new and improved advanced feature for analyzing Deep Neural Networks More