Simulating SLURM scheduling on large clusters
Simulation using SLURM code itself.
B. Nikolic Software and Computing Blog
Software Engineering, Computing and Programming notes and articles from 20+ years at the cutting edge: C++, Python, PyTorch, Quantitative Finance, Cloud Scale-Out
Latest posts
A slice of BayeSys3 in SLURM cluster scheduler
Reviewing the source code of the SLURM sluster scheduler I happened on a file “skilling.c”. I thought it could not be! But it indeed refers to John Skilling and it is a portion of the BayeSys3 program (fully credited of course) to convert between integers and rectilinear positions using Hamiltonian space-filling curve. The code is in file slurm/contribs/skilling.c (github) and can be used to…
Is Git a database?
Some Friday afternoon thoughts on this perennial question which yet seems to have little good info found through Google :
Fully self contained builds of complex C/C++/Python packages with Spack
The Spack package manager has excellent in-built capabilities for building on system disconnected form the internet. As part of this, this includes the capability to collect all of the required source code to build any package. For example the collected source code for building Tensorflow (without MPI or CUDA) amounts to around 2.4GB after compression. In essence all that of the functionality is…
execution::par and parallel STL in C++
Had cause to use the STL parallel algorithms this week with excellent results. The algorithm was a small matrix inversion repeated for each in an image (to correct for effects of cross-polarisation).
Easy SLURM job sequencing
Mostly from: https://hpc.nih.gov/docs/job_dependencies.html .
Levenshtein distance calculation in Excel witl xlOil and python Levenshtein module
A very quick experiment to try a traditional data cleaning example with Excel and xlOil: calculate the Levenshtein distance between every pair of words. In this case I’m using the Python-Levenshtein module, and the xloil wrapping is trivial – each pair is calculated separately.
Postgresql is enough (for a room reservation system)
Of course PostgreSQL is a enough for a room reservation system.
Python forward compatibility & numpy
At the time of writing the most recent version of Python is 3.14.2. If you use this version then this innocuous looking sequence of commands will result in a seriously broken environment:
Notes on Marimo
Marimo is a reactive notebook system in Python. Here are some thoughts on it: