Gábor Melis' Blog
Version 0.5 of DRef , the definition reifier, is now available. It has moved to its own repository, completing its separation from PAX , where it was originally developed. This was a long time coming. Twelve years ago today, PAX was born. From the start, PAX used the concept of locatives to refer to definitions without first-class objects. For example, to generate documentation for the *MY-VAR*…
Classical literate programming A literate program intersperses narrative and code chunks. From this, source code to be fed to the compiler is generated by a process called tangling , and documentation by weaving . The specifics of tangling vary, but by allowing complete reordering and textual combination of chunks, it lets the human narrative drive the exposition at the cost of introducing an…
Thanks to Paul A. Patience , PAX now has PDF support . See pax-manual-v0.4.1.pdf and dref-manual-v0.4.1.pdf . The PDF is very similar to the HTML , even down to the locative types (e.g [function] ) being linked to the sources on GitHub, but cross-linking between PDFs doesn't work reliably on most viewers, so that's disabled. Also, for reading PDFs so heavy on internal linking to be enjoyable, one…
At ELS 2024 , I gave a talk on adaptive hashing , which focusses on making general purpose hash tables faster and more robust at the same time. Theory vs Practice Hash table theory most concerns itself with the asymptotic worst-case cost with a hash function chosen randomly from a family of hash functions. Although these results are very relevant in practice, those pesky constant factors, that the…
Version 0.4 of PAX , the documentation system, and DRef , the definition reifier, was released. There were large refactorings, bug fixes, minor features, cosmetics, documentation and performance improvements too numerous to list . Here is a summary of the new features and notable changes. DRef now supports DTYPE s , which allow filtering DEFINITIONS and DREF-APROPOS results according to the…
Over at argmin.net , Ben Recht is reflecting on Meehl's lectures on the metatheory of science, which is about how science progresses. The original lectures are fascinating but also long as well as long-winded, and I found Ben's blog series a much better read (especially since the originals are video recordings). Still, at the time of writing, with 13 blog posts covering less than half of the…
Catchy title, innit? I came up with it while trying to name the development style PAX enables. The original idea was something vaguely self-explanatory in a straight out of a marketing department kind of way, with tendrils right into your unconscious. Documentation-driven development sounded just the thing, but it's already taken. Luckily, I came to realize that neither documentation nor any other…
Try , my test anti-framework, has just got light Emacs integration. Consider the following test: ( deftest test-foo ( ) ( is ( equal "xxx" 5 ) ) ( is ( equal 7 7 ) ) ( with-failure-expected ( t ) ( is ( same-set-p ' ( 1 ) ' ( 2 ) ) ) ) ) The test can be run from Lisp with (test-foo) (interactive debugging) or (try 'test-foo) (non-interactive), but now there is a third option: run it from Emacs and…
DEFSECTION needs to refer to definitions that do not create a first-class object (e.g. stuff like (*DOCUMENT-LINK-TO-HYPERSPEC* VARIABLE) ), and since its original release in 2014, a substantial part of PAX dealt with locatives and references, which reify definitions. This release finally factors that code out into a library called DRef , allowing PAX to focus on documentation. Being very young,…
PAX got a live documentation browser to make documentation generation a more interactive experience. A great thing about Lisp development is changing a single function and quickly seeing how it behaves without the delay of a full recompile. Previously, editing a docstring required regenerating the full documentation to see how the changes turned out. The live documentation browser does away with…
I put the sources of the Two-Tailed Averaging paper on github . Of course, the sources are also available on arxiv , but this may give better visibility to the LaTeX grid typesetting code in there. Also, note how much cleaner the paper looks with the XCharter font compared to Times New Roman . No wonder Matthew Butterick pushes Charter . By the way, see what he has to say about grids and baseline…
In short, comparing fonts at the same font size is almost never the right thing to do. Compare them at the same x-height or, better yet, at the same space usage. In full, recently I wanted to choose a font that looks good on screen at various resolutions and also in print. This is fairly subjective, of course, so there is a lot of noise out there. Going by the LaTeX Font Catalogue , Google Fonts…
This is a complement to the Two-Tailed Averaging paper , approached from the direction of what I think is a fairly common technique: averaging checkpoints. We want to speed up training and improve generalization. One way to do that is by averaging weights from optimization, and that's a big win (e.g. 1 , 2 , 3 ). For example, while training a language model for the down-stream task of…
Do or do not. There is now Try. I forgot to announce Try , my Common Lisp test framework, on this blog. Try does equally well in interactive and non-interactive mode by minimizing the function-test impedance mismatch. It provides a single, extensible check macro . All other checks are built on top of it. It is highly customizable: what to debug interactively, what to print, what to describe in…
PAX v0.1 is released. At this point, I consider it fairly complete. Here is the changelog for the last year or so. New Features To reduce deployment size, made the MGL-PAX system autoload navigation, documentation generation, and transcription code . Symbols in the CL package are linked to the hyperspec like this: PRINT , which renders as PRINT . Hyperspec sections and issues can be linked to with…
Ever wished for machine-readable logs and TRACE s, maybe for writing tests or something more fancy? The Journal library takes a simple idea: user-defined execution traces and implements logging , tracing , a testing "framework" with mock support, and an Event Sourcing style database on top. Perhaps the highlight from the linked tutorials is how easy persistence is. Let's write a simple game: (…
After more than five years of silence, I may be resurrecting my old blog . I already got as far as rewriting it using MGL-PAX , which is a curious choice because PAX is a documentation generator for Common Lisp. The blog "engine" is rather bare-bones but works admirably, especially considering that the implementation is only 72 lines of code, most of which deals with post categories and overview…
2020-05-03 – Things have changed the during last 5 years. This is a non-issue in Tensorflow and possibly in other frameworks, as well. I believe there is one design decision in MGL-MAT that has far reaching consequences: to make a single matrix object capable of storing multiple representations of the same data and let operations decide which representation to use based on what's the most…
Bigger because documentation for named-readtables and micmac has been added. Badder because clicking on a name will produce a permalink such as this: *DOCUMENT-MARK-UP-SIGNATURES* . Clicking on locative types such as [variable] on the page that has just been linked to will take you to the file and line on github where *DOCUMENT-MARK-UP-SIGNATURES* is defined.
A promise of PAX has always been that it will be easy to generate documentation for different libraries without requiring extensive markup and relying on stable URLs. For example, without PAX, if a docstring in the MGL library referenced the matrix class MGL-MAT:MAT from the MGL-MAT library, it would need to include ugly HTML links in the markdown: "Returns a…
I've been cleaning up and documenting MGL for quite some time now, and while it's nowhere near done, a good portion of the code has been overhauled in the process. There are new additions such as the Adam optimizer and Recurrent Neural Nets. My efforts were mainly only the backprop stuff and I think the definition of feed-forward: ( build-fnn ( :class 'digit-fnn ) ( input ( ->input :size…
I'm getting so used to the M-. plus documentation generation hack that's PAX , that I use it for all new code, which highlighted an issue of with code examples. The problem is that, the ideally runnable, examples had to live in docstrings. Small code examples presented as verifiable Transcripts within docstrings were great, but developing anything beyond a couple of forms of code in docstrings or…
I've just committed a major feature to MGL-PAX: the ability to include code examples in docstrings. Printed output and return values are marked up with .. and => , respectively. ( values ( princ :hello ) ( list 1 2 ) ) .. HELLO => :HELLO => ( 1 2 ) The extras are: parsing back and updating a transcript, auto-checking of up-to-dateness at documentation generation time, readable return values can be…
Due to the bash security hole that keeps giving , I had to disable gitweb at http://quotenil.com/git/ and move all non-obsolete code over to github. This affects: Six the Hex AI , the Planet Wars bot , MiCMaC , FSVD , Lassie , and cl-libsvm .
The Higgs Boson contest on Kaggle has ended. Sticking to my word at ELS 2014 , I released some code that came about during these long four months. MGL-GPR is no longer a Genetic Programming only library because it got another Evolutionary Algorithm implementation: Differential Evolution . My original plan for this contest was to breed input features that the physicists in their insistence on…
Actually, I'll only link to the post-mortem I wrote in the forum. There is a also a model description included in the git repo . A stand-alone distribution with all library dependencies and an x86-64 linux precompiled binary is also available . This has been the Kaggle competition that attracted the most contestants so it feels really good to come out on top although there was an element of luck…
In addition to the cl-libsvm asdf system, there is now another asdf system in the cl-libsvm library: cl-liblinear that, predictably enough, is a wrapper for liblinear . The API is similar to that of cl-libsvm.
After almost two years without a single competition, last September I decided to enter the Stackoverflow contest on Kaggle . It was a straightforward text classification problem with extremely unbalanced classes. Just as Bocsimackó did the last time around, his lazier sidekick (on the right) brought success . I would have loved to be lazy and still win, but the leaderboard was too close for…
This is a short rant on what I consider to be a good puzzle and why the trick 36Cube pulls is rather dirty. WARNING , spoilers ahead. Got this crafty beast for Christmas, and since then I lost many evenings to juggling permutations of towers over the inverted city skyline. The game can be solved two ways: by being cleverer than Euler, by being streetwise. We are going to concentrate on option 2…
My ISP replaced a Thomson modem with a Cisco EPC3925 modem-router to fix the speed issue I was having. The good news is that the connection operates near its advertised bandwidth, the bad news is that tcp connections started to hang. It didn't take long to find out that this particular router drops "unused" tcp connections after five minutes. The fix recommended in the linked topic (namely sysctl…
I've moved to an OfflineIMAP + Gnus setup that's outlined at various places . Gnus can be configured to use ~/.authinfo as a netrc style of file to read passwords from and can easily use encrypted authinfo files as well. Offlineimap, on the other hand, offers no such support, and passwords to the local and remote imap accounts are normally stored in clear text in .offlineimaprc . For the local…
It hasn't even been a year yet since I first promised that alpha–beta snippet, and it is already added to Micmac in all its 35 line glory . The good thing about not rushing it out the door is that it saw a bit more use. For a tutorialish tic-tac-toe example see test/test-game-theory.lisp . The logging code in the example produces output , which is suitable for cut and pasting into an org-mode…
While I seem to be unable to make my mind up on a good interface to alpha–beta with a few bells and whistles, I added a Nash equilibrium finder to Micmac , which is becoming less statistics oriented. This was one of the many things in Planet Wars that never really made it. Let's consider the Matching pennies game. The row player wins iff the two pennies show the same side. The payoff matrix is: |…
I can't believe I won . I can't believe I won decisively at all. The lead in the last month or so was an indicator of having good chances, but there was a huge shuffling of ranks in the last week and some last minute casualties. Code Note that the git repository is available at [https://github.com/melisgl/planet-wars(https://github.com/melisgl/planet-wars). Denial I had promised myself not to…
First, is it possible to get something as simple as RESOLVE-BATTLE wrong? Apparently, yes. That's what one gets for trying to port Python code that's pretty foreign in the sense of being far from the way I'd write it. More importantly, I found out the hard way that sbcl 1.0.11, that's still on the official servers, has a number of bugs in its timer implementation making WITH-TIMEOUT unreliable.…
Released v0.6 ( git , latest tarball ). The way the server compiles lisp submissions was fixed, and this revealed a problem where MyBot.lisp redirected *STANDARD-OUTPUT* to *ERROR-OUTPUT* causing the server to think compilation failed.
The Google AI Challenge is back with a new game that's supposed to be much harder than Tron was this spring. The branching factor of the game tree is enormous, which only means that straight minimax is out of question this time around. Whether some cleverness can bring the game within reach of conventional algorithms remains to be seen. Anyway, I'm adding yet another starter package ( latest…
As promised, my UCT implementation is released, albeit somewhat belatedly. It's in Micmac v0.0.1, see test/test-uct.lisp for an example. Now I only owe you alpha–beta.
For what has been a fun ride, the official results are now available . In the end, 11th out of 700 is not too bad and it's the highest ranking non-C++ entry by some margin. I entered the contest a bit late with a rather specific approach in mind: UCT , an algorithm from the Monte Carlo tree search family. It has been rather successful in Go (and in Hex too, taking the crown from Six ). So with UCT…
Tron is a fun little game of boxing out the opponent and avoiding crashing into a wall first. The rules are simple, so the barrier to entry into this contest is low. Thanks to aeruiqe , who made the Common Lisp starter pack, it took as little as a few hours to get a very bare-bones algorithm going. It's doing surprisingly well: it is number 23 on the leaderboard at the moment with 43 wins, 2…
Debian Squeeze finally got Xorg 7.5 instead of the old and dusty 7.4. The upgrade was as smooth as ever: DPI is off , keyboard repeat for the Caps Lock key does not survive suspend/resume and the trackpoint stopped working . Synaptics click by tapping went away before the upgrade so that doesn't count.
From a failed experiment today, I salvaged Micmac , a statistical library wannabe, which for now only has Metropolis-Hastings MCMC and Metropolis Coupled MCMC implemented. The code doesn't weigh much, but I think it gets the API right. In other news MGL v0.0.6 was released.
Let me interrupt the flow of the MGL introduction series with a short report on what I learnt playing with Deep Boltzmann Machines . First, lots of thanks to Ruslan Salakhutdinov, then at University of Toronto now at MIT , for making the Matlab source code for the MNIST digit classification problem available. The linked paper claims a record of 99.05% in classification accuracy on the permutation…
UPDATE – This post is out of date with regards to current MGL. Please refer to the documentation instead. In Introduction to MGL (part 2), we went through a trivial example of a backprop network. I said before that the main focus is on Boltzmann Machines so let's kill the suspense here and now by cutting straight to the heart of the matter. Cottrell's Science article provides a clear and easy to…
UPDATE – This post is out of date with regards to current MGL. Please refer to the documentation instead. After Introduction to MGL (part 1), today we are going to walk through a small example and touch on the main concepts related to learning within this library. At the top of the food chain is the generic function TRAIN : ( defgeneric train ( sampler trainer learner ) ( :documentation "Train…
This is going to be the start of an introduction series on the MGL Common Lisp machine learning library. MGL focuses mainly on Boltzmann Machines (BMs). In fact, the few seemingly unrelated things it currently offers (gradient descent, conjugate gradient, backprop) are directly needed to implement the learning and fine tuning methods for different kinds of BMs. But before venturing too far into…
I'm cleaning up the accumulated junk and found this guide that was written eons ago. This build is focused on survival. No save/loading, killap's final patch , hard combat and game difficulty. As it is not only ironman but a munchkin too, it must be a Sniper since HtH is a bit underpowered. See this for good insights into ironman survival. The two most important pieces of advice it has is: Sneak…
Debian Lenny was released back in February. My conservativeness only lasts about half a year, so I decided to upgrade to Squeeze aka Debian testing. The upgrade itself went rather smoothly with a few notable exceptions. With KDE 4.3, I should have waited more. Notes: Who thought it a grand idea that in the default theme (Oxygen) the color of the panel and window title bar cannot be customized?…
Consider a class with a trivial initialization dependency between slots A and B : ( defclass super ( ) ( ( a :initarg :a :reader a ) ( b :initform 0 :initarg :b :reader b ) ) ) ( defmethod initialize-instance :after ( ( super super ) &key &allow-other-keys ) ( setf ( slot-value super 'a ) ( 1+ ( slot-value super 'b ) ) ) ) ( a ( make-instance 'super ) ) => 1 ( a ( make-instance 'super :b 1 ) ) =>…
A quick note to library implementors: the effects of DECLAIM are permitted to persist after the containing file is compiled, and it is unkind to mutate your user's settings. Personally, I find DECLAIM too blunt and prefer to add declarations within functions, even going as far as introducing LOCALLY subforms just to have a place on which to hang declarations. But if you are really set on using…