My clone repository
I was writing a data intensive code in Perl relying heavily on PDL for some statistical calculations (estimation of percentile points in some very BIG vectors, e.g. 100k to 1B elements), when I noticed that PDL was taking a very (and unusually long!) time to produce results compared to my experience in Python. This happened irrespective of whether one used the pct or oddpct functions in…
After a very long hiatus due to the triplet of work-vacation-work, we return to Part 3 of my AI assisted coding of a Perl interface to a foreign library. In the last couple of months, many things have happened on the AI front, including the release of additional models, and some much needed injection of reality into the hype, so my final part will have a different tone than the previous ones and…
Of makefiles and alienfiles
Introduction
In the second part of this we implement the solution that was outlined at the end of Part 1: utilize a Perl application that probes the operating system in real time for the RSS (Resident Set Size), i.e. the DRAM footprint of an application fire the application from within R as a separate process, provide it with the PID (Process ID) of the R session and put it in the background do the long,…
Another year, another opportunity for Perl to excel as a system’s language. Today I decided to take Perl for a (?)wild ride and use it to monitor peak physical DRAM use in a R script. Due to the multi-language nature of the post, there will be a lot of R code in the first part of the series; however, the code is self-explanatory, and should not be difficult to understand (the same applies to the…
VelociPerl is a closed source fork of Perl that claims performance gains of 45% over the stock (“your dad’s Perl” in their parlance) based on some public benchmarks. I will not go into how they achieved this performance boost, or why they released it as closed source, or even “but why the heck did you release it as closed source?”, as you can follow the Reddit discussion. However, even a modest…
Sometimes the unexpected happens and must be shared with the world … this one is such a case.
At this point one may wonder how numba, the Python compiler around numpy Python code, delivers a performance premium over numpy. To do so, let’s inspect timings individually for all trigonometric functions (and yes, the exponential and the logarithm are trigonometric functions if you recall your complex analysis lessons from high school!). But the test is not relevant only for those who want to do…
Having run this toy performance example, we will now digress somewhat and contrast the performance against a few Python implementations. First let’s set up the stage for the calculations, and provide commandline capabilities to the Python script. ```python import argparse import time import math import numpy as np import os from numba import njit from joblib import Parallel, delayed