Recent content on .mischief.mayhem.soap.
If you ever looked at optimized code generated by Clang, you might have noticed it loves vectorizing. It is enabled by default at higher optimization levels/when not minimizing the size. Clang is a great compiler and it knows what it’s doing, but I was curious about two things: is it worth the increased binary size what functions are the most ‘affected’ The first question could…
(I had a more catchy post title, but it’d spoil a surprise). Last week we had an interesting debugging session, trying to find a source of a mysterious hitch that suddenly started plaguing one of our tools. Admittedly, the setup is maybe a bit exotic, but it has been working for fine. It’s a 3rd party Windows application that supports Lua plugins for custom commands. Some of our…
Last week I noticed some unexpected assembly code in a function calling std::unordered_map::find (yes, I know, friends don’t let friends use STL, but that is a topic for another discussion). I decided to investigate a bit and it turned out to be quite a journey. It was a Clang compiler, ARM-based platform, I will use x64 here as behavior is similar and it is easier to repro in the Compiler…
I have been comparing how locks (critical sections) behave on different platforms recently and noticed Windows seemed to be quite different than others. The behavior is consistent across two PC machines I tried (desktop and a laptop), test code is a bit too long to include it in the post, but it is linked here. The gist of it is really simple though - we have multiple threads (even 2 is enough)…
Recently we have updated some of our compilers (Clang) and started running into mysterious problems, mostly in the physics code. We were able to narrow it down to raycasts failing to hit… sometimes. What made it a bit more annoying - it was only happening on 1 platform I had no easy access too. It was ARM based, but another ARM platform was fine. It seemed like the problem was deep in the…
I had another interesting debugging session recently with quite an unexpected conclusion. It all started when we received a new crash report - a certain platform (and 1 platform only) was crashing in a fairly old and seemingly innocent fragment of code: someModule->GetElements(elements); std::sort(elements.begin(), elements.end(), [&origin](const Element* a, const Element* b) { return…
I’ve always felt conflicted about the subscription operator[] in standard containers. It makes sense for random access structures like vectors, but it gets a little bit problematic elsewhere. Consider this seemingly innocent piece of code (it’s not 1:1, but this is code you can easily find in many codebases): if(someMap[key].value < x) { someMap[key].value = x; } 2 lines of actual…
The other day I was looking at a crash dump for a friend. A discussion that followed made me realize it might be worth to write a short post explaining why sometimes 2 seemingly almost identical function calls behave very differently. Consider the following code snippet (simplified): 1struct Foo 2{ 3 __declspec(noinline) const int& GetX() const { return x; } 4 virtual const int& GetY() const {…
Today I’d like to share a simple multithreading trick you can use to minimize “bubbles” in your pipeline. “Simple” because it applies mostly to “oldschool” threading systems, ie. the ones with main thread concept that kicks jobs and eventually flushes them. Cool kids using proper task graphs where everything just flows beautifully should not need it.…
If you’ve been reading this blog for a bit, you might know that when I experiment with new programming languages, I like to write a simple pathtracer, to get a better “feel”. It’s very much based on smallpt (99-line pathtracer in C++), but I do not want to make it as short as possible. I am more interested in how easily I can get it to run and what language…
I’ve recently been asked by a friend for a little help with debugging a problem he was running into. Occasionally the program would freeze while trying to process a chunk of data and never moved on to the next one. Application is heavily threaded and processing is done by thread B, while thread A does its own job and periodically checks if work has been finished. If so, it sends it for…
Grafana is a very popular “analytics platform” or in more professional terms - a system to create pretty graphs. It’s very popular for monitoring system metrics, but really can be used for any timeseries data. It supports plethora of data sources and there is a decent chance you can use one of the off-the-shelf solutions to do 99% of the work for you (for example for some basic…
Microcorruption is my ongoing “distraction” – it’s an online CTF. I’m way late to the party and have been doing it on and off since… 2013. How does it work exactly? To use their own description: “tl;dr: Given a debugger and a device, find an input that unlocks it. Solve the level with that input. You’ve been given access to a device that controls a…
I’d be the first to admit I don’t have much experience designing public APIs. I typically work on code that’s fairly specific to the game we’re making and while some of it is expected to be reused, our potential user pool is very limited, we’re still talking just one team, so <40 people and definitely not thousands. I’m still successfully using some little…
Consider the following, seemingly innocent (and completely made up) fragment of code: typedef std::map<int, std::string> MyMap; void foo(const MyMap& m) { for(const std::pair<int, std::string>& i : m) { if(i.first) { printf('%s\n', i.second.c_str()); } } } Looks simple enough, right? Just iterate over all elements of the map and print the value for non-zero keys. We use range-based for construct,…
I probably mentioned this before, but SPSC (single producer, single consumer) queue is one of my favorite structures. If implemented correctly, it’s actually 100% lock-free and is also surprisingly versatile (not all problems require MPMC!). I typically use a slightly modified version of this or an unbounded version, based on code by Dmitry Vyukov. Both implementations are very simple and…
Circular buffers are one (if not the) of my favorite data structures for some quick&dirty debugging. A simple, not production ready version can be implemented in a few lines of code (not ashamed to admit, I usually just copy paste these and remove when I’m done) and they’re a great tool to “record” a history of generic events. Any time I run into a seemingly…
Yet another MSVC story. Visual Studio has a nice compile-time warning when trying to access a static array with invalid index - C4789. According to documentation it’s mostly meant for various ‘copy’ functions (memcpy/strcpy etc), but it seems to work on ‘simple’ accesses as well. Consider (here’s a Godbolt link): struct Tab { float tab[2]; }; void Foo(const…
I recently transitioned to Visual Studio 2017 and while it went relatively painless, the new and improved optimizer uncovered some subtle issues lurking in the code (to be fair, Clang/GCC has been behaving same way for a long time now). The code in question was actually quite ancient and originated from this Devmaster forum post (gone now, but found if using The Wayback Machine): Fast and accurate…
Ever since I was a kid I was always fascinated by the northern lights and hoped to see them in person one day. Poland is too far south/densely populated to spot them, though, so it wasn’t a very realistic dream. In 2009 I did move to Sweden, but still, this was Stockholm area, so my chances might have been bigger, but not by much (plus only been there for a year).
Machine learning and neural networks seem to be all the rage recently. I was sifting through my old drives the other day and found my master thesis. It’s actually vaguely related, subject was Neural Networks for Stock Market Forecasting. Complete thesis can be downloaded here, but comes with 2 major caveats: it’s 100% in Polish (so actual title is Sztuczne sieci neuronowe w…
The Old New Thing is one of my favorite blogs. It’s a collection of Windows development anecdotes, but every now and then Raymond will post a gnarly debugging/crash story. I’ve recently found some of my old notes related to a crash I was chasing in a third-party library, it reminded me a little bit of The Old New Thing and I decided to try something similar. The whole thing took place…
Few years ago I published an article about love and care for your cache. Every now and then I receive emails asking for clarification or some extra question. It seems like the basic rules are easy enought, but once you add multiple cores to the mix, it might get a little bit confusing. If we only have one core everything is fairly simple. There’s just 1 cache, which is essentially a variant…
A little bit late, as Halloween was yesterday, but I think std::get_temporary_buffer is scary enough to qualify. A co-worker called me today to show me an ‘interesting’ crash. It was deep in guts of the STL, more specifically in the std::_Merge method. Corresponding C++ line seemed innocent: *_Dest++ = _Move(*_First1++); Nothing very interesting, not our code and yet it was crashing…
I’ve just returned from the Digital Dragons 2017 conference. I’ve been presenting there last year and enjoyed the event so much I decided to repeat it. They changed a venue this year and I must say it was even better. Last year had a very nice ‘artsy’ feel to it, but ICE has a much better infrastructure. This year I’ve been talking about networking system. You can…
Last week I was investigating a crash originating somewhere in a code that looked like this (GPF in this C++ line): obj->GetFoo()->GetBar().Call(player->GetCat(), this, &MyType::SomeFunc, moreArgs We could discuss the number of indirections or the fact that if this code had been split into multiple lines it’d be obvious, but that’s not the main point here. I didn’t have this…
A few weeks ago we had a discussion a forum discussion. Question was - what tips did we have for junior developers (not only programmers, all specialties). My #1 tip was: always diff before checking in your changes! I still stand by it, but thought I’d elaborate a little bit and give a quick overview of how I handle my commits these days. Not a rocket science for sure, but the presentation…
I’ve been doing some minor refactoring recently and - once again - it got my thinking how seemingly tiny C++ changes can affect generated assembly code (and performance). It was a fairly simple scenario, had a collection of ~60 small items identified by an unique ID. ID was a 32-bit number, but realistically the range seemed to be limited to 0-5000 (although I couldn’t rely on it…
Back in the day I wrote about the ‘dynamic initializers’ problem. Basically, older versions of MSVC (up to 2012, not sure about 2013, seems better in 2015) had problems with static const floats that depended on other static const floats. Values were not calculated compile-time, there was actually a short function generated and it’d do it. The immediate problem is a code bloat (if…
I’ve been planning on converting my blog to some static site generator for a while now, but the (perceived) amount of work involved seemed scary, so I kept coming up with reasons not to do it. I really like the idea of static sites. Using gamedev jargon, this basically means we “precompute” as much as possible. Instead of retrieving posts from the database and building pages on…
If you’ve ever done any matchmaking coding (…or played chess) you’re most likely familiar with the Elo rating system (PSA: it’s Elo, not ELO, it’s not an acronym, it’s named after a person - Arpad Elo). It’s a rating system used by most chess federations (most notably, FIDE) and has been adapted by many video games. It’s trivial to implement and get…
A video from my DD2016 talk has been uploaded and can be viewed here:
A few days ago I came back from the Digital Dragons 2016 conference. I’ve been in talks with these guys in the past years, but timing was never quite right, this time I finally could make it. Luckily for me, it was the same year they invited John Romero, David Brevik and Chris Avellone. In December 1993 my classmate brought an old floppy disc to school. He claimed it contained the best game…
You have probably heard about an AI algorithm defeating a human professional in a game of Go. The algorithm itself has been developed by Deep Mind, an English company that’s now owned by Google. One of the founders is Demis Hassabis. If you were playing Bullfrog games in the late 90s, the name might ring a bell, he’s one of the designers of Theme Park. Back in the day I remember…
Recently, we’ve been collectively complaining on Twitter about going crazy in C++ (as we do every few weeks). This reminded me of my dark period around 2002 when I was really excited about templates and metaprogramming. I tried finding my old code, but turned out my previous website was completely gone. Fortunately (?), good folks from The Wayback Machine had some copies. I moved some of my…
I’m playing in a fantasy football league with some coworkers this year. I have always liked all kinds of sports, but it’s a little bit challenging to follow NFL from Europe. There’s not too many stations that actually show it (except for the Super Bowl) and even if they do, the time difference make it difficult to watch. I still followed major news and managed to catch a game…
I’ve been debugging a rare memory corruption bug recently and - as usual - it turned out to be an exercise in frustration. This time only part of that was because of the bug itself, the other was because methods I chose were not very helpful (in my defense, it’s been a while, so I was a little bit rusty). The bug itself was fairly boring - sometimes, when leaving a certain UI screen…
An entertaining head-scratcher from today. Application has suddenly started crashing at launch, seemingly inside steam_api.dll. It’d be easy to blame external code, but as mentioned - it’s a new thing, so most likely related to our changes. To make things more interesting, it’s only 32-bit build that crashes, 64-bit seems fine. High-level code looks like (simplified): 1struct…
I’ve been doing some micro-optimizations recently. One of the things I’ve been trying is eliminating virtual method calls in ‘leaf’ types. Consider the following snippet (simplified): 1struct Bar 2{ 3 virtual bool Func1() { return false; } 4}; 5struct Foo : public Bar 6{ 7 virtual bool Func1() 8 { 9 return true; 10 } 11 virtual void Func2() 12 { 13 if (Func1()) 14…
I’ve been planning to write a post about debugging multiplayer games (post-mortem) for a while now, but it keeps getting bigger and bigger, so instead of waiting until I can get enough time to finish it, I thought it’d be easier to share some quick’n’easy tricks first. I’d like to show a simple way of “instrumenting” a crash dump so that it gives us more…
A nice thing about Twitter is that single tweet can bring so many memories. It all started when Fiora posted this: (tweet now deleted). It reminded me of an old bug I encountered few years ago when working on a multi-platform (PC/X360/PS3) title. One day we started getting strange bug reports. Apparently, if you jumped down the roof at a very specific position, player would start to slide across…
(Please excuse the terrible pun, couldn’t help myself). As we all know, computer cache is a touchy beast, seemingly little modifications to the code can result in major performance changes. I’ve been playing with performance monitoring counters recently (using Agner Fog’s library I mentioned before). I was mostly interested in testing how cmpxchg instruction behaves under the…
Last year I briefly described my adventure with writing apathtracer in the Go language. This year, I decided to give Rust a try. It’s almost exact 1:1 port of my Go version, so I’ll spare you the details, without further ado - here’s a short list of observations and comparisons. As previously, please remember this is written from position of someone who didn’t know anything…
Had a very interesting debugging session recently. I’ve been investigating some of external crash reports, all I had was a crash dump related to a fairly innocent-looking piece of code. Here’s a minimal snippet demonstrating the problem: struct Tree { void* items[32]; }; #pragma intrinsic(_BitScanForward) __declspec(noinline) void* Grab(Tree* t, unsigned int n) { unsigned int j = 0;…
Recently, I had an inspiring discussion with fellow programmers, we were talking about interesting side projects/programs to quickly “try out” new programming language/job interview tasks. One that’s been mentioned was coding a Z-machine interpreter that’s capable of playing Zork I. The Z-machine is a virtual machine developed by Joel Berez and Marc Blank, used for numerous…
A short cautionary tale from today. I’ve been modifying some code and one of the changes I made was to use a type of Lol as a key in a map-like structure (key-pair container, uses < operator for comparisons). Structure itself looked like: 1struct Lol 2{ 3 byte tab[16]; 4 short cat; 5 bool foo; 6}; …and here’s the operator< 1bool Lol::operator<(const Lol& other) const 2{ 3…
There’s been some comments to my previous post wondering about C++ compilers and their capabilities. Normally, I’m all for compiler bashing, in this case I’d probably cut them some slack. It’s easy to optimize when you’re focused on a single piece of code, way more difficult when you have to handle plethora of cases. On top of that, uops handled differently on…
Few weeks ago I encountered a discussion on a Polish gamedev forum – participants were wondering whether it’s faster to access stack or heap memory. I didn’t pay much attention (there should be no significant difference) until someone had posted a test case. Out of curiosity, I ran it and to my surprise discovered, it was consistently faster for buffers allocated on the stack.…
There may come a time in game programmer’s life when he has to fix a bug in a library he doesn’t have the source code for. It doesn’t happen often, it might never happen, but it’s good to be prepared. If I remember correctly, I had to do it only two times, one was fairly recently. We were getting quite a few crash reports and were assured that fix in the third-party library…
Today’s article is brought to you by a friend of mine. He’s been doing some home experiments and noticed a particular piece of code was running faster in Debug than Release (using Visual C++ compiler). He mentioned this to me, I found this intriguing enough to ask him for this snippet and investigated it yesterday. Turned out it was a classical case of compiler trying to be too smart…