Recent content on Lexy's blog
Last month I got to use ArkScript , a language I’ve been developing for nearly 7 years, for the Advent of Code. And this time, I got to the end, using only my language (and a few hints from programming.dev )! Adding attributes to functions’ arguments Advent of code challenges are often heavy on lists, with inputs sometimes being thousands of lines long. After benchmarking some of my…
Since my last update post , there was a total of 127 new commits, 402 files changed, 4782 insertions, and 2798 deletions. There was a lot of refactoring to make the project cleaner, a new Dict datatype, a ton of new tests that helped fix bugs, better position tracking for nodes in the parser, and a few more super instructions for optimization purposes! Dictionaries It has been a few years since I…
Since my last update post , there was a total of 941 files changed, 10'780 insertions and 4'105 deletions in 71 commits. I mainly focused on optimizing the runtime performances of the language. Better errors! I worked on improving error contexts, by adding an optional cause, which is very helpful when debugging macros extensions: 1 2 3 4 5 6 7 8 9 At (foo 1 2) @ 4:2 1 | (macro foo (a b c ...d) | ^…
Since the last update post , there was a total of 102 commits, for 947 files changed, 9395 insertions, 3149 deletions. I mainly focused on test coverage, adding around a hundred tests on type checking errors. Mordor… erm more documentation The documentation for the bytecode instructions is (finally!) automatically generated thanks to a small Python script I wrote . No more editing this…
Good error reporting is crucial in programming languages. Doing it at compile time was easy in ArkScript as we have all the context we need at hand, but since we compile code down to bytecode, which is then run by the virtual machine, we loose a lot of context: the source files, their path, their content, we don’t have that anymore! Our runtime errors could only show the VM internal state.…
The other night, I was talking about meta programming to other developers, and at one point someone asked how macros could be used to do meta programming. They were probably thinking about C type of macros , which are powerful but are just text processor tools. Using macros you could have the following code: 1 2 3 4 5 double Compute () { // doing something return result ; } and transform it into:…
In my previous article about Scala , I briefly mentionned we were publishing Swaggers inside JARs, that we could unzip to retrieve the Swagger definition and then run our code generation. It works, but using ZIPs would feel better and less confusing for users. I have decided to nerd snip myself and see how I could publish a ZIP using SBT, and then use said ZIP as a dependency for sbt-swaggerinator…
If you don’t know me yet, I have been working on ArkScript for nearly 6 years now. ArkScript is a scripting language in modern C++, running on a custom virtual machine (like Python or Lua), with the goal of having a syntax easy to learn and use, a C++ interface to embed it in programs, and decent performances (without trying to be as fast as Lua though, Mike Pall is a genius and did…
Tonight, I had a bit of a misadventure with WireGuard on MacOS. My Mac installed an update a few days ago, and as usual I didn’t think much about it. While lying on the sofa, I tried to connect to one of my servers to update a project, so I went to my VPN, Wireguard and searched for my tunnel. But it wasn’t there! Restoring your tunnels According to the internet, this is something that…
At work, we’ve been generating code from swaggers and publishing said generated code. Alas, this requires us to remember to generate the swagger(s), as well as fixing versions of libraries (which means you need to upgrade versions in the server, publish updated generated code, then update your client), see this examples from eikek/sbt-openapi-schema : 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16…
After having watch a video of Jason Turner about visibility=hidden , I wanted to try and apply this compiler switch to my project, ArkScript . This is useful for me as I build ArkScript in two phases: a shared library and an executable, and marking the symbols of the shared library as hidden unless specified otherwise allows the compiler to apply more optimizations. In this project, I also use…
It’s been about 46 days since the new year has started, but my brain hasn’t adjusted (yet) to us being in 2025 and not 2024. That’s why I’m only getting started now on establishing my goals for the new year! ArkScript, version 4.0.0 This year, I want to finish ArkScript v4 (or at least all the tasks I planned for this major version), and finally get some rest, working on…
Hello there! In the last 90ish days, there were 841 files changed, 9323 insertions, and 5070 deletions, in 106 commits. The primary goal was integrating an intermediate representation, but then… I got side tracked. Compiler ire…? The compiler didn’t get angry, but now outputs instructions in a new intermediate representation (IR), that is then compiled to bytecode for the…
In December 2022, a colleague gave me the idea to improve ArkScript import system. All it was doing was using a (import "folder/file.ark") syntax, and copy / pasting the content of the file (duplicate imports were removed). The idea was to be able to do something akin to Scala or Python imports: import a whole file, and have it in a namespace of it own ; eg (import std.List) import only a few…
A few weeks ago, I was feeling like making a new (split) keyboard, since I hadn’t assembled one for nearly a year now. It’s like Lego(tm), so of course I love that activity as much as coding, and since I’m on a keyboard 10 hours a day due to my job and hobbies, it’s quite fitting. This time, my crazy idea was to fit a split keyboard in the footprint of a floppy disk. 90mm x…
A few years back, I wanted to try my hand at making my own alarm clock, but using light to wake me up (basically, DIY domotic?). All I had available to me was an Arduino and a lighting garland (as well as a few other components: cables, resistors, diodes, potentiometers, LCD screen, relays). My goal Creating a somewhat accurate clock using an Arduino Being able to set the clock time Turning on and…
ArkScript is a scripting language, running on a VM. To accomplish this, we had (as of September 2024) a compiler generating bytecode for the virtual machine, receiving an AST from the parser (and a few other passes like name resolution, macro evaluation, name and scope resolution…). Exploring new optimizations The only thing we could optimize was the virtual machine and the memory layout of…
I think this is the longest title on my blog as of now. This problem is highly specific to my setup and homelab, but that could help some people out there. What I want to achieve: have CloudFlare serving as a proxy for my homelab, so that my IP address isn’t directly accessible rely on nginx-proxy-manager to create hosts on my homelab and serve them on the internet ban abusers and bots using…
A bit of context… Recently, more and more tech have been switching to USB C as its default connector to do pretty much everything, from charging, to transfering data, and even audio (thanks Apple!). While it’s great to have only a single cable for everything, I quickly discovered (as many other did) that not all cables were created equal. Some support charge up to 5W, others up to…
Hello, General Kenobii! In the last 90 days, 795 files have been modified, for 3105 insertions and 9135 deletions (dataset cleanup for fuzzers), in just 97 commits (I’m reworking the very last one, more on that soon). Did someone say… macros? Macros come in handy for improving syntax and creating DSLs like the one used for testing: 1 2 3 4 5 6 ( test:suite list { ( test:case 'append…
A common idiom in virtual machines or state machines is to read data from a list, execute some code depending on the value we read, advance in the list, rinse and repeat. That could be written as: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 std :: vector < uint8_t > bytecode = readBytecode (); std :: size_t pos = 0 ; while ( pos < bytecode . size ()) { uint8_t inst = bytecode [ pos ]; pos ++ ;…
In the previous article , we saw how to compile functions and handle their scopes. We also saw how to optimize a certain kind of function calls, the tail call ones in Understanding tail call optimization , now we are going to see yet another optimization, this time on conditions and expression evaluation. Basic way to deal with conditionals Following the previous article(s), we could implement and…
Python has received a lot of attention lately. The 3.13 release, planned for October this year, will begin the huge work of removing the GIL . A prerelease is already out for curious users who want to try a (nearly) GIL-less Python. All this hype made me dig in my own language, ArkScript , as I had a Global VM Lock, too, in the past (added in version 3.0.12, in 2020, removed in 3.1.3 in 2022), to…
For a while, when updating ArkScript website , I have needed to remember to update the copy on my VPS so that the changes would be reflected to the world. Since I’m quite lazy, I’d like to automate this! The solutions The first solution would be to use crontabs . Easy to setup, find a schedule, write a bash script that cd and git pull and you’re down. However, it has some…
Why? In a work related context, I had to create a hash algorithm working on a finite set of values ( [0, 0xFFFFFFFF] ) to output a non sequential serie from a sequential one (the output had to be rendered as a UUID . Basically, I wanted to avoid generating UUID looking like 00000000-0000-0000-0000-000000000001 , that will definitely appear to be sequential and easily abused to find other IDs. The…
Just tonight, I wanted to add support for caps word to my keyboard , but it needed an update to its qmk firmware. “No problem” I thought to myself, I already did this a lot in the past, what could go wrong? Well, everything. Dependencies, compilers, versions and brew I had QMK 1.1.1 when I first compiled my arkenswoop firmware, but in the mean time I ran brew install xyz several times.…
Hello again! It’s now been 581 days since I last posted here. In the meantime, that’s around 21880 lines added and 18238 deleted in various refactos, divided into 202 commits (the year 2023 was very empty, work resumed at the start of 2024)! A parser? Again? To continue from the end of the last message, the new parser has been permanently integrated into ArkScript, and the old one…
Everyone loves their tests, they can help with refactoring, code quality, adding new feature more easily… but you can also find the opposite points of view. Some may argue that yes, tests are great, but they can also hinder your ability to refactor your code. Or give you a false sense of security about your project, eg. you have thousands of tests but you mocked your database and never…
Good evening everyone! Since the last post, I’ve taken some time to work on the language’s error messages, to better help users when an error occurs, as well as more compile-time warnings: I’ve started bug hunting by integrating AFL++ , a handy fuzzer which has brought to my attention quite a few bits of code able of crashing the lexer, parser, macro processor, rarely the…
If you have ever worked on a large scale project, you know that finding and tracking bugs can be very tedious and lengthy. Did you know it could be automated? There are multiple ways to achieve this, starting with unit & integration tests run regularly to detect regressions, end-to-end tests to ensure a functionality is behaving as intended, and much more. However, writing those tests is also a…
As software developers, having our projects available easily to anyone is a goal, but it can be hard to achieve. Using package managers like apt , pacman or brew has become an industry standard (compared to wget + compile it yourself + install it), but publishing a project on it can be quite tedious. In this article, we will go through the basics of creating a tap for homebrew (available on both…
Hello there! The project is still alive, I was just running out of content to make a new post here. Between today and the last post, taking into account only the dev branch, a small refacto of the project has passed: 88 files modified, 3093 lines added, 1991 deleted I specifically said the dev branch, because a little new one has appeared, aimed specifically at the next big release, ArkScript 4.0!…
Lately, I’ve been working on optimizations for my language, ArkScript , and finally take some time to add tail-call optimization to my compiler. In this article, I’ll explain what is tail-call optimization , why it seems easy to implement, and how to do it. Definition Tail-call optimization is a method to optimize some recursive functions so that we don’t have to create a bunch…
Hello, it’s been a long time since I posted here, I’ve neglected communication too much to concentrate on code and school work. Progress Since the last post, the project has undergone a lot of refactoring: 210 files modified, 11,273 lines added, 8,065 lines deleted (not enough for my taste). Broadly speaking, the project has been reworked to make it easier to integrate into other…
This week, I had to design an API with protected routes, which needed the user to be logged in. There is even more to it, said API should be used by a website (where you have access to cookies). Technologies used : NodeJS & expressjs Making a simple server Let’s create a small server for this demonstration: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 const express = require ( 'express' ) const app =…
Until recently, when we wanted to create new releases for ArkScript , we had to build the language on all the system we support (currently Windows and Linux), build the modules (http, console, random, etc), test everything on each operating system, and then package the needed files and directory in ZIPs. We had to go to GitHub, create a new release, add the correct tag (and not mix it with the…
In the previous article , we discussed the global idea behind a bytecode interpreter, and explained the concept of a stack and how the instructions could be represented. Now we will dive a bit more in tables and call stack . Tables? As we saw in the previous article, we need a way to keep track of symbols’ identifiers, to refer to them using a fixed size number instead of a variable size…
Have you ever wondered how Java or Python work? What is a “virtual machine”? If those questions stir your interest, you’ve come to the right place! If everything goes according to the plan, this will be part of a serie of articles on understanding and making bytecode interpreters (we will even see a bit of how you could compile a language to a bytecode). Why should I care about…
In this article, I will try to highlight important things when developping your own games, according to me. The importance of a clean code I started programming game at a very young age, in C with the SDL , and I have to say that my code got messy very quickly. Functions with hundreds of lines, structures handling too many different things… which ultimately led to poor bug tracking, memory…
Disclaimer : this article is intended to be part of a Request For Comments on a language I am working on . Note : I will be using bytecode interpreter and virtual machine (sometimes written VM) interchangeably throughout this article. Introduction There are a lot of languages out there, and new ones are born everyday. We can often divide them into 4 categories: interpreted from a tree (often the…
Hello, it’s been a while, and I’m back with some good news! We’re still hard at work on the language, with UTF8 on the agenda (for strings and maybe identifiers), macros with variadic argument management coming soon, various optimizations, and we’re in the process of expanding the standard lib and modules. To test all this, I’ve created a mini blockchain in the…
Very often, I have to help friends and colleagues to build a C or C++ project, and my solution is always the same: “use CMake!”, but their documentation isn’t pretty good so we have to spend a few hours on stackoverflow to get it to work for their specific use cases. This blog post will try to provide basic CMake knowledge and how to produce a nice and working CMakeLists.txt .…
Hi there! So we’ve got a new interpreter (REPL) that has persistence between each line typed (previously each piece of executed code was forgotten as soon as new code was typed), coloring and auto-completion, all using the replxx lib (in C). Then, I’ve recently added dead code elimination (to the global scope only) during compilation, a home-made lexer with no regex and much faster…
Nowadays, we have code everywhere. We need to patch bugs, find how and why a bug was introduced to fix it, find when a functionality was added and by who, thus we need what is called “version control”. Version control is a bunch of things to solve the problem stated above: handling a bunch of small changes (= commit ) commits are signed to know who made them, thus who made the changes…
If you have some experience with Python, you must have already seen a TypeError: 'T' object is not callable . In this article we will try to demystify this kind of errors. What is a callable? A callable is anything that can be called, from a function, to a class constructor. For example, len is a callable because we can write len(object) . Creating a callable The easiest way is to write a function…
Two years ago, I wanted to make my own games from A to Z, starting from the library I would use to handle entities, worlds, graphisms… to the tools next to that such as a map editor. And I had this strange idea: what if I create my own scripting language? In fact, I was using Lua but a few points here and there didn’t suit me well (you have semi-classes with meta tables ; arrays start…
Hi there, fellow reader! Today, I want to talk about a small project I’ve been working on since April 2019, ArkScript . After the failure of Kafe, I wanted to make another language but this time Lisp inspired and interpreted. Then Ark was born (later renamed ArkScript, see this reddit post ). Then I tried actually compiling this language, and instead of starting to make the virtual machine…
In my previous post , I tried to introduce you to my journey: making a game (I forgot to mention that I want to make (more or less) the whole game myself, from the engine to the assets). I am now trying to render chunks of blocks and… that was harder than I thought ! I briefly mention the basic things needed when playing with 3D rendering in OpenGL (VAO, VBO, shaders) in my last post.…
Nota Bene : This serie of articles is mainly focused on my experience while making Voksel , a 3D game made in a Minecraft style, so I will only write about my opinion and how I solve the problems I met! A month ago, I got this strange idea after having taught myself on OpenGL: What if I made my own rendering engine? It could be funny, and I could learn a lot! That’s how I started making…