Cautionary tales from a programmer
The European Commission has posted a "call for evidence" on open source for digital sovereignty. This seeks feedback from the public on how to reduce its dependency on software from non-EU companies through Free and Open Source Software (FOSS). This is my response, with proper formatting (the web form replies all seem to have gotten their spaces collapsed) and for future reference. The added value…
I feel a change is happening in how people produce and (want to) consume software, and I want to give my two cents on the matter. It has become more mainstream to see people critical of "Big Tech". Enshittification has become a familiar term even outside the geek community. Obnoxious "AI" features that nobody asked for get crammed into products. Software that spies on its users is awfully common.…
NOTE: This is another guest post by Felix Winkelmann , the founder and one of the current maintainers of CHICKEN Scheme. Introduction Hi! This post is about a new project of mine, called "CRUNCH", a compiler for a statically typed subset of the programming language Scheme , specifically, the R7RS (small) standard. The compiler runs on top of the CHICKEN Scheme system and produces portable C99 that…
NOTE: This is a guest post by Felix Winkelmann , the founder and one of the current maintainers of CHICKEN Scheme. Introduction This article is about the changes in the next major version of CHICKEN Scheme . The current version is 5.4.0. The next major version, 6.0.0, is mostly implemented. Currently we are adding final touches before preparing the next release. If you are already familiar with…
Recently, a user was asking on IRC why CHICKEN doesn't give proper line numbers for error call traces in the interpreter. This is indeed rather odd, because the compiler gives numbered traces just fine. After a brief discussion, we figured that this was probably because line numbers are stored in a "database" (hash table) which maps the source expression to a line number. Because you can keep…
Recently I joined bevuta IT , where I am now working on a big project written in Clojure . I'm very fortunate to be working in a Lisp for my day job! As I've mostly worked with Scheme and have used other Lisps here and there, I would like to share my perspective on the language. Overall design From a first view, it is pretty clear that Clojure has been designed from scratch by (mostly) one person…
As you may know, I co-maintain the uri-generic egg , together with Ivan Raikov. We had just been working on fixing a bug and porting it to CHICKEN 5 when I stumbled across the WHATWG URL specification , an evolution over RFC 3986 . I found it hard to believe they dropped the formal grammar from the RFC, so I checked the issue queue and found a closed ticket from 2015 . They replaced the BNF with a…
We're getting close to a CHICKEN 5 release, so let's take a look at the cool new stuff! Overhaul of built-in modules The biggest change you'll notice when you fire up CHICKEN and start to use it is that the modules that come shipped with core are completely different from CHICKEN 4. The functionality is mostly the same, but we moved things around (a lot!) to make things more logical. This is also…
Now that we have covered the most important algorithms, it's time to take a look at the internal data representation of extended numerals. This will be the final part in this series of posts. Ratnums and cplxnums Recall from my data representations article that fixnums are represented as immediate values, directly within a machine word. Flonums are represented in boxed form, by putting them in an…
In this instalment of the blog series, we'll take a look at how string->number and number->string are implemented for bignums. Performing base conversions Performing calculations is all nice and useful, but you eventually want to print the results back to the user. And of course the user needs a way to enter such large numbers into the system. So, converting between numbers and strings is an…
Now that you understand the basic bignum algorithms, let's look at various tricks to speed up these operations. Faster multiplication Like I mentioned in the previous part of this series, the primary school method for addition and subtraction is the fastest known, for bignums of any size. And you can't really get better than O(n) . However, primary school multiplication is O(n²) , and there are…
This is the second part documenting my journey to add full numeric tower support to CHICKEN core. In this post I explain some of the basic algorithms. You'll need to understand these before going on to the next part, which deals with fancier versions of these algorithms. Classical numerical algorithms Like I mentioned in my previous post, the Scheme48 numerical code used only the so-called…
Originally, CHICKEN only supported fixnums (small integers) and flonums (floating-point numbers). The upcoming CHICKEN 5 will support a full numeric tower , adding arbitrary-length integers, rational numbers and complex numbers. This is the first in a series of blog posts about my journey to make this a reality. We'll start with a bit of background information. Later parts will dive into the…
The recent trouble at GitHub, both cultural changes within the company and criticism from the community , reminded me how unstable the whole "free as in beer code hosting for the public good" idea really is. The good part is that it motivated me to finally look into setting up personal hosting for my own projects, because how hard can it be, really? History shows: code hosting is unreliable GitHub…
I just submitted a patch to add a statistical profiler to CHICKEN. In this post, I'll explain how it works. It's easier than you'd think! Instrumentation-based profiling There are two major ways to profile a program. The first way has been supported by CHICKEN as long as I can remember: You add instrumentation to each procedure. This counts how often the procedure is called, and how much time is…
Maybe you've noticed the little Flattr icons that I've added at the bottom of each post, and now you're wondering if, and why, you should support my blogging efforts with a donation. Let me state this first and foremost: I enjoy writing posts about subjects that interest me and I learn a lot from the research I put into my posts. So, even without donations I'd continue doing so. But, it's also a…
In my earlier post about the garbage collector, I lied a little bit about the data representation that CHICKEN uses. At the end of the post I briefly mentioned how CHICKEN really stores objects. If you want to fully understand the way CHICKEN works, it is important to have a good grasp on how it stores data internally. Basic idea CHICKEN attempts to store data in the most "native" way it can. Even…
When you're writing Scheme code in CHICKEN it's sometimes necessary to make a little excursion to C. For example, you're trying to call a C library, you're writing extremely performance-critical code, or you're working on something that's best expressed in C, such as code that requires a lot of bit-twiddling. This post contains a lot of code, including generated C code. If you get too tired to…
One of CHICKEN's coolest features has to be its unique approach to garbage collection. When someone asked about implementation details (hi, Arthur!), I knew this would make for an interesting blog post. This post is going to be long and technical, so hold on to your hats! Don't worry if you don't get through this in one sitting. Prerequisites There's a whole lot of stuff that we'll need to explain…
Today I'd like to talk about how CHICKEN Scheme handles distribution of language extensions (which we call "eggs"). There are some unique features of our setup that might be interesting to users of other languages as well, and I think the way backwards compatibility was kept is rather interesting. In the beginning First, a little bit of history, so you know where we're coming from. CHICKEN was…
Recently there was a small flame war on the Chicken-hackers mailing list. A user new to Scheme asked an innocuous question that drew some heated responses: Is there a good reason for this behavior? # perl -e 'print substr("ciao",0,10);' ciao # ruby -e 'puts "ciao"[0..10]' ciao # python -c 'print "ciao"[0:10];' ciao # csi -e '(print (substring "ciao" 0 10))' Error: (substring) out of range 0 10…
I've finally decided to get a proper domain name: http://www.more-magic.net . Please update your bookmarks and feed readers! I used to run this blog on a hostname from the good folks at DynDNS , which I registered in my college days. DynDNS had the benefit of being 100% free (great for poor college students!), but the disadvantage of having to run a tool called ddclient . This tool is intended to…
Last time I explained how sloppy representations can cause various vulnerabilities. While doing some research for that post I stumbled across NUL byte injection bugs in two projects. Because both have been fixed now, I feel like I can freely talk about them with a clear conscience . These projects are Chicken Scheme and the C implementation of Ruby . The difference in the way these systems deal…
The two biggest threats to the web are caused by the same underlying mistake. It is time this problem was fixed at its root. This article will attempt to provide the tools do so. Input sanitation, input filtering or output escaping? The Open Web Application Security Project (OWASP) does a great job at educating people and suggesting practical solutions to avoid common weaknesses. Unfortunately,…
Today we'll look at an old, experimental DSL of my own design. I've always referred to it as a failed experiment, but perhaps it's really a successful experiment, because it helped me figure out why this type of DSL doesn't work too well. Whatever the status, I'll use it as an example of what makes a bad DSL. The DSL in question is SSQL , a way of embedding SQL as S-expressions into Scheme code.…
Today I'd like to discuss Scheme Regular Expressions (SRE). Originally introduced in a library for the Scheme Shell , this DSL has recently been gaining some popularity due to the release of Irregex , a pure R5RS Scheme regex engine with SRE as its native syntax. Irregex has been integrated as the core regex system in Chicken Scheme and Jazz Scheme , and you can easily use it from any other Scheme…
After last time's example of SCSS , I'd like to take a look at SXML , another Lispy DSL I'm using in this blog. It's more successful and more widely-used than SCSS and even has an official specification ! The observation that XML is really an obnoxiously verbose Lisp without parens is common, but the details are (of course) hairier than that. Let's look at an XHTML example: < div > < span >Hello,…
Setting up this blog was a good excuse to try out SCSS , which I'd been meaning to do for quite a long time. Working with SCSS and exploring its limitations got me thinking about what makes a good Lispy DSL (domain specific language). This post is the first of a series. Today we'll look at SCSS; in future installments we'll explore other examples of Lispy DSLs. The idea behind SCSS isn't unique;…
Welcome to my renewed website. I've decided to stop procrastinating and instead of rolling my own blog software I decided to simply take Hyde and actually start blogging! The quest for better blogging software I've started writing blog software from scratch about three times by now. The first time I attempted making what one could call a "clone" of Wordpress or other similar blogging "platforms".…