Hackers gonna hack
Last year I started working on stalld , a daemon that prevents thread starvation in real-time Linux systems. On systems with isolated CPUs running high-priority RT tasks, essential kernel threads can starve indefinitely. stalld monitors CPU run queues and temporarily boosts starving threads using SCHED_DEADLINE (or SCHED_FIFO as fallback). The main goal was switching from the old sched_debug…
Disclaimer: this is a mental note in the form of a blog post for future references. Most of it is a summary of a debug session to solve this issue . Kernel is one of hardest piece of software to debug. Although it has some limited support for a classical debug session , it is often impractical. Not surprisingly, over the years, kernel hackers developed a lot of tools to aid the art of debugging…
It is incredible how we can find room for improvement even in the eldest and battle-tested codebases out there. This post is about one of these cases. I was investigating a soft lockup bug report which only happened in an HP Proliant DL380 Gen9 . The trigger for the issue is when we try to load the scsi_debug with a few hundred virtual devices: $ modprobe scsi_debug virtual_gb = 1 add_host = 2…
The other day I was working on a bug where kernel-rt-debug reported some warnings in the middle of the rcutorture test followed by a kernel hang: rcu_torture_writer: Testing asynchronous GPs. rcu_torture_writer: Testing normal GPs. ------------[ cut here ]------------ DEBUG_LOCKS_WARN_ON(this_cpu_read(softirq_ctrl.cnt)) WARNING: CPU: 2 PID: 7996 at kernel/softirq.c:169…
As the last project of 2020, I and some other co-workers were assigned the task of creating a prototype for a graphical user interface preview Visual Studio extension for the NUI graphics toolkit. Notice the extension is for Visual Studio , not Visual Studio Code . The idea is the programmer create the user interface in a XAML file and the plugin displays the preview of the user interface. NUI…
C++ enums historically gave me some headaches due to the fact that compilers are free to choose the size of the type for whatever criteria they think. This makes particularly hard to write portable code among compilers from ABI point of view (Visual C++ vs C++ Builder, anyone?). Some days ago I was debugging a problem with dali-toolkit and the enum story scalated to another level. Let me give you…
Since I left Mozilla I came back to my old days as a C/C++ developer. One of the projects I am working on is porting the Tizen.NUI API to Windows. Tizen.NUI itself is written in C#, so it doesn’t require any major effort to run on Windows, but it is heavily based on DALi , and there where the job lies. DALi kind of works on Windows already, but I caught several bugs affecting 64 bits. You can…
For the past months, I have been working on adding Taskcluster support for packet.net cloud provider. The reason for that is to get faster Firefox for Android CI tests. Tests showed that jobs run up to 4x faster on bare metal machines than EC2. I set up 25 machines to run a small subset of the production tasks, and so far results are excellent. The problem is that those machines are up 24/7 and…
For the last year I have been using the Go programming language, working on the implementation of the taskcluster-worker project. If you want to learn more about taskcluster-worker , you can read this post . In the taskcluster-worker task payload, there is a field called command , that specifies the command the task must run. Through a configuration file, you can specify either the spawned command…
Some days ago, a friend emailed me desperated because a Java code was faster than his C++ implementation. These Java faster than C++ claims were very common when Java was the tech companies fashion language. I never liked Java because I always thought it was quite verbose (however, I never tried the new functional constructions in Java 8) and used to be very heavy. I mean, I could recognize that…
In this quarter, I worked on implementing the taskcluster-worker Mac OSX engine. Before talking about this specific implementation, let me explain what a worker is and how taskcluster-worker differs from docker-worker , the currently main worker in Taskcluster . The role of a Taskcluster worker When a user submits a task graph to Taskcluster, contrary to the common sense (at least if you are used…
One of my goals for 2016 Q1 was to write a monitoring dashboard for Taskcluster. It basically pings Taskcluster services to check if they are alive and also acts as a feed aggregator for services Taskcluster depends on. One problem with this approach is the same origin policy , in which web pages are only allowed to make requests to their own domain. For web servers which is safe to make these…
This post is about our plans for representing Taskcluster tasks inside the gecko tree. Jonas , Dustin and I had a discussion in Berlin about this, here I summarize what we have so far. We currently store tasks in an yaml file and they translate to json format using the mach command. The syntax we have now is not the most flexible one, it is hard to parameterize the task and very difficulty to…
In this post I am going to talk about my work for phone builds inside the Taskcluster infrastructure. Mozilla is slightly moving from Buildbot to Taskcluster. Here I am going to give a survivor guide on Firefox OS phone builds. Submitting tasks A task is nothing more than a json file containing the description of the job to execute. But you don’t need to handle the json directly, all tasks are…
So far I have been a heavy git user, but since I joined Mozilla I have given Mercurial a try. I must say that migrating from Subversion to git was far less painful than from git to Mercurial. I think there are a few reasons for that: When I started to learn git, I started it with PyUSB , a small personal project that was on its youth at that time. This means I needed just the most basic commands…
B2G has a built-in profiler that you can use to find hot spots in your app. The process is made easy by the profile.sh script, which helps you to generate a profile file for upload to the Cleopatra application. Earlier, you had to build your own Firefox OS image to use profile.sh tool, but that’s no longer the case. Thanks to Snappy symbolication server , you can grab a Mozilla nightly image and…