Read the articles...
-
"Continuous Integration Dashboard for Apache NuttX RTOS (Prometheus and Grafana)"
-
"Optimising the Continuous Integration for Apache NuttX RTOS (GitHub Actions)"
To ingest NuttX Build Logs into Prometheus Pushgateway: run.sh
## We prefer GitLab Snippets, since GitHub Gists will get blocked for overuse. ## For GitHub Gists: Any GitHub Token with read access will do ## export GITHUB_TOKEN=... . $HOME/github-token.sh ## Find all defconfig files cd $HOME ; git clone https://github.com/apache/nuttx find $HOME/nuttx -name defconfig >/tmp/defconfig.txt ## Ingest logs from lupyuen/nuttx-build-log GitLab Snippets. Remove special characters. ## gitlab-token.sh contains "export GITHUB_TOKEN=...", any GitLab Token with read access will do. set +x ; . $HOME/gitlab-token.sh ; set -x cargo run -- \ --user lupyuen \ --repo nuttx-build-log \ --defconfig /tmp/defconfig.txt \ | tr -d '\033\007' unset GITLAB_TOKEN ## Ingest logs from nuttxpr GitHub Gist. Remove special characters. cargo run -- \ --user nuttxpr \ --defconfig /tmp/defconfig.txt \ | tr -d '\033\007' ## Ingest logs from GitHub Actions ./github.sh ## Or: Start GitHub Actions Build, wait to complete then ingest logs ./build-github-and-ingest.sh ## Or: Sync NuttX Mirror Repo, start GitHub Actions Build, wait to complete then ingest logs ## https://github.com/lupyuen/nuttx-release/blob/main/sync-build-ingest.sh ../nuttx-release/sync-build-ingest.sh
(See the Ingest Log for GitHub Gists)
(See the Ingest Log for GitHub Actions)
Continuous Integration Dashboard
To install Grafana and Prometheus...
## Install Grafana brew install grafana brew services start grafana ## Browse to http://localhost:3000 ## Install Prometheus brew install prometheus brew services start prometheus ## Browse to http://localhost:9090 ## Install Prometheus Pushgateway brew install go git clone https://github.com/prometheus/pushgateway cd pushgateway go run main.go & ## Browse to http://localhost:9091
Update the Grafana and Prometheus Configuration...
Add the Grafana Dashboard and Panels...
Remember to check for suspicious activity!
tail -f /opt/homebrew/var/log/grafana/grafana.log \ | grep --line-buffered "logger=context " \ | grep --line-buffered -v "path=/api/frontend-metrics " \ | grep --line-buffered -v "path=/api/live/ws " \ | grep --line-buffered -v "path=/api/plugins/grafana-lokiexplore-app/settings " \ | grep --line-buffered -v "path=/api/user/auth-tokens/rotate " \ | grep --line-buffered -v "path=/favicon.ico " \ | cut -d ' ' -f 9-15
Highlight the HTTP Errors: iTerm > Profile > Advanced > Triggers...
- Regular Expression:
status=[4-9][^ ]+[ ], Action: Highlight Line, Background: Red - Regular Expression:
path=[^ ]+[ ], Action: Highlight Text, Background: Dark Blue
If we see too many HTTP 404 Errors for Dubious URLs (we're not a WordPress Server!): Turn on Cloudflare > Under Attack Mode. The errors should disappear.