Description / Steps to reproduce the issue
Our usage of GitHub Actions Runners has hit 70 Full-Time Runners over the past 12 hours. Remember that ASF Infra Team has demanded that we keep this Under 25 Full-Time Runners, or we might lose access to GitHub Actions CI.
https://infra-reports.apache.org/#ghactions&hours=12&limit=15&group=name
I'm not sure whether this is due to high volume of PRs in NuttX Repo or NuttX Apps. Or possibly some runaway jobs? Let's continue to monitor this, hope it will drop within the next few days.
The URL above is accessible to NuttX PMC Members (possibly to NuttX Maintainers too). Other folks may use this simpler metric for monitoring: https://lupyuen.github.io/nuttx-metrics/github-fulltime-runners.png
I'm travelling overseas for a Marathon Race, my response will be slow. Thanks!
FYI: This is how we redesigned NuttX CI, 13 months ago: https://lupyuen.org/articles/ci3
Update: I have new videos that explain GitHub Runners and why we're using so many of them, also potential solutions...
- [Testers Needed] Building PRs in Our Own Repos #18568
- [Test] Workflow to testing PRs in their own repository on GitHub NuttX/manual-nuttx-ci#1
- [FEATURE] Retry for CI Builds #18562
- CI: Retry build upon failure #18576
- [FEATURE] NuttX Build Monitor #18659
- Apache NuttX RTOS: GitHub Runners and Mystery Spikes
- Apache NuttX RTOS: Too Many GitHub Runners
- Apache NuttX RTOS: Dumping the GitHub Jobs
- Apache NuttX RTOS: Most Expensive PR
- Apache NuttX RTOS: Cost of NuttX PRs vs NuttX Apps
- Apache NuttX RTOS: Analysing the Most Expensive PR
- Apache NuttX RTOS: Retrying a CI Build
- Apache NuttX RTOS: Building PRs in Our Own Repo
- Apache NuttX RTOS: Testing of Retry for CI Builds
- Apache NuttX RTOS: NTFC Download Failure + Second Most Expensive PR
- Apache NuttX RTOS: PR Monitoring + Third Most Expensive PR
- Apache NuttX RTOS: Building the PR Monitor + Most Expensive Apps PR
- Apache NuttX RTOS: Parsing the GitHub Build Logs
- Apache NuttX RTOS: Build Monitor
- Apache NuttX RTOS: More Testing of Retry for CI Builds
- Apache NuttX RTOS: CI Build Retry works OK yay!
GitHub Runners Usage up to 8 Mar 2026:
https://docs.google.com/spreadsheets/d/1JH7BCwTB-U6dwvSsy0tfpNXr534jVX-XYkOYi3FLM8g/edit?gid=0#gid=0
Based on the data from above, we have identified the following PRs as the Most Expensive PRs, in terms of GitHub Runner Usage. We need to study further why these PRs are consuming so many resources, and whether improving our NuttX Dev Process will fix this problem...
- sched/hrtimer: introduce scheduler support with hrtimer #17573
- sched/spinlock: Refactor critical section implementation with rspinlock foundation and SMP optimization #18135
- [EXPERIMENTAL]sched/hrtimer: hrtimer state machine improvement for SMP cases #17642
- !boards: Simplify NuttX initialization #18408
- sched: add a reference count to the TCB to prevent it from being deleted. #17468
- sched/sched: Part 2: Add high-resolution timer (hrtimer) and os tick support with hrtimer to NuttX #17489
- netutils/netinit: Support the NETINIT_MACADDR. nuttx-apps#3315
- NuttX GitHub Jobs Spreadsheet
- Most Expensive PRs
Which were generated by...
git clone https://github.com/lupyuen/nuttx-github-jobs cd nuttx-github-jobs ./dump-github-jobs.sh cargo run ## Import nuttx-github-jobs.tsv into Google Sheets for analysis: ## NuttX GitHub Jobs Spreadsheet: https://docs.google.com/spreadsheets/d/1HY7fIZzd_fs3QPyA0TX7vsYOjL86m1fNOf1Wls93luI/edit?gid=1187577522#gid=1187577522 ## Most Expensive PRs: https://docs.google.com/spreadsheets/d/1HY7fIZzd_fs3QPyA0TX7vsYOjL86m1fNOf1Wls93luI/edit?gid=70515654#gid=70515654
Over the Past 12 Months: NuttX Repo consumed 84% of GitHub Runners, while NuttX Apps Repo consumed 12%...
What about the past 4 months? The distribution is the same: 84% vs 12%. Interesting note: Within the past 4 months, we consumed Half of The Entire Year's GitHub Runners.
Conclusion: It's definitely worth the effort to do Dependency Analysis and speed up the CI Builds for NuttX Repo. I thought NuttX Apps might take more GitHub Runners (because every CI Build takes 3.5 hours), but I proved myself wrong today.
On which OS does this issue occur?
[OS: Linux]
What is the version of your OS?
Ubuntu on GitHub Actions
NuttX Version
master
Issue Architecture
[Arch: all]
Issue Area
[Area: Other]
Host information
No response
Verification
- I have verified before submitting the report.



