Personal site for Data Scientist and Machine Learning Engineer - Svilen Todorov
I wake up as usual by immediately going on my phone. I reply to some of my messages - yes dad, the solar installation you have at the villa could be redundant soon, but again, it can still benefit them (even if slightly) if you comply and let them integrate it into the new grid. A grid which operates on principles only slightly less opaque than divine providence, but is demonstrably more…
We often hear about the qualities and attributes that successful people possess or that you need to cultivate. Some believe that success is a result of innate talent, others argue that it’s the result of hard work and perseverance, and yet different people try to sell you their one weird trick for it. Realistically, most people either over-focus on one aspect, ignore others, or are simply…
A common question in the chess community is whether to play e4 or d4 as a beginner. Opinions are generally split between e4 being better for learning, no difference, and rarely that d4 is better. Typical reasoning being that e4 leads to sharper games and gives you more opportunities to learn. The other starting moves as white are rarely recommended for beginners altogether and rarely played at all…
I’m a wanderer now. My name is John Minter, my original world no longer exists, and I don’t grieve for it. What’s real and more importantly what’s going to be real is what matters. Did I kill Hitler? Of course. What self-respecting time-traveller wouldn’t? The very first time I told someone about my predicament they immediately asked why I hadn’t and it admittedly made me uncomfortable. So I did…
A topic I end up discussing with friends and thinking about regularly is how videos and games we used to like a decade or two ago looked completely fine back then and are currently unwatchable. I don’t mean this in the more common way of shifting tastes and values but in almost literally barely seeing what’s going on. Videos in 360p and even lower (how is 144p even still an option in youtube?)…
GANs have become the default image generation technique, and many are familiar with sites like artbreeder, thispersondoesnotexist, and its off-shoots such as thiswaifudoesnotexist. The generated samples are stunning and quite hard to distinguish from real images, and StyleGAN, in particular, has been used and finetuned by many with interesting results.
A lot of progress has been made in training large NLP models in recent years, and after finetuning GPT2-1.5b I’ve been thinking about context windows. Large Transformers are very good, but when you are only limited to 512 or more commonly now 1024 tokens, the model doesn’t know quite that much of what happened earlier at any point. Google’s Reformer paper introduces a more efficient way of doing…
Colab is one of my favourite machine learning tools and quite possibly the most useful online-based tool for democratizing Machine Learning currently in existence. It is used by many to share research, working models, ideas. It is a reproducible environment to present GitHub issues, a usable machine with a powerful GPU or TPU which many won’t have access to otherwise, and more. Colab is extremely…
Previously in this series - finetuning 117M, finetuning 345M
Google Search is ridiculously, utterly bad. Before your brain starts recalling successful searches you’ve had, and before this turns into only a rant let’s see some examples. This is a query for checking out what reddit thinks in regards to buying a phone, while specifying to only get results from the last month. Pretty simple I guess, yet none of the results achieved that goal, and none of them…