This is Part II of porting Andrej Karpathy's microgpt to the data-parallel language Futhark . Part I covered the forward pass. Now: training and inference. This part of the port is more involved than Part I, and doesn't map as cleanly 1-to-1, so I won't blog every line of code like I did in Part I, just excerpts. The full code (so far) is on GitHub . Examples of things left out of this post…
I have been wanting to find a project to try out the data-parallel language Futhark . They have a very good blog that I've been following for years, but I've never actually written anything in it. Andrej Karpathy's microgpt , a self-contained implementation of a GPT-2-like neural network in 200 lines of Python, finally provided the excuse. I like microgpt, but it does not scale at all . Obviously…
I saw an exhibition by Shohei Fujimoto yesterday that I liked, at Artechouse Houston , titled Intangible Forms . I was not familiar with his work, and came away thinking it was pretty interesting! I believe this is a quasi-permanent (?) exhibition that opened in June 2024 with the opening of Artechouse Houston, and shows alongside whatever rotating exhibit they have. Some of the pieces were…
This demo repeatedly generates and autoplays levels for the dungeon crawler toy domain at the end of the the previous post , which was in turn adapted from Chapter 8 of our PCG textbook . It uses clingo running in the browser via clingo-wasm to generate the levels. The level generation logic ensures that the level is solvable and obeys stylistic constraints (see the previous post for details).…
This is an updated, interactive web version of all but one of the answer-set programming (ASP) level-generation examples in Chapter 8 of our Procedural Content Generation textbook : Mark J. Nelson and Adam M. Smith (2016). ASP with applications to mazes and levels . In: Noor Shaker, Julian Togelius, and Mark J. Nelson, eds., Procedural Content Generation in Games: A textbook and an overview of…
A message I sent to my programming languages class for the start of the fall semester, on why I like teaching it: Welcome to Design & Organization of Programming Languages! This is one of the classes I most enjoy teaching, because it covers many parts of what makes computer science interesting to me. There are human-centered angles to designing programming languages: how to make them easy to use,…
In class earlier today, I demoed prompting LLM base models. Base models only autocomplete prompts using a large neural network model of language, rather than answering queries like a chatbot, so one needs to use various strategies to get responses out of them. One strategy is to start a definition, and have the model complete it. Prompt: A good definition of artificial intelligence is Five…
I received an intriguing notification in the Microsoft Outlook 365 web interface: This message can't be saved because it no longer exists. It can only be discarded. Make sure you copy the contents of the message before you discard if you want to use them later. I have some ontological questions. * * * This message can't be saved because it no longer exists. Sensible enough. Only messages that…
Photo: Luis Bartolomé Marcos, CC BY-SA Novelist and essayist Walter Kirn doesn't like ChatGPT . He especially doesn't like ChatGPT screenshots: Of all the posts I ignore on social media... the ones I ignore most thoroughly these days, almost with a vengeance, are the results of prompts to AI chatbots. The tell-tale fonts and formats of these posts allow me to spot them instantly... It's…
In mid-2021, I was preparing the final version of a paper for publication in an IEEE proceedings volume . Part of the IEEE's process is that you run your final publication-ready PDF through IEEE PDF eXpress , which runs various checks – ensuring the PDF has fonts embedded, isn't a scanned document, isn't password-protected, etc. My PDF kept being rejected, but instead of an error message in the…
Prompt: "a torrent of email" An AI advance I did not expect to happen as quickly as it did: open domain text-to-image synthesis sort of works now. You type in a phrase describing an image you want, and get one generated for you that, a lot of the time, resembles your request. Like many people, I'm using Stable Diffusion , an open source model about 4 gigabytes in size that was released publicly a…
Operator precedence is a traditional feature of mathematical notation where, absent explicit parentheses, some operators bind tighter than others ("have higher precedence"). For example, a + b × c would be interpreted as a + (b × c) , not as (a + b) × c , because multiplication has higher precedence than addition. This is a bit annoying to implement in programming languages. As a result, some…
The job of a professor is traditionally split into three categories: research, teaching, and service. For example, I'm evaluated for promotion in those three categories. What exactly goes in each is sometimes fuzzy, but roughly: Research: Running experiments, analyzing data, writing code, reading papers, writing papers, submitting grant proposals, managing grants, supervising research students,…
Note: A version of this blog post is available as an interactive Jupyter Notebook hosted on Google Colab. The Wikidata relation doctoral advisor (P184) links researchers to the advisor or advisors who supervised their Ph.D. Note that coverage is currently a bit spotty. But, it's at least a superset of the data in the Mathematics Genealogy Project (because MGP data was imported into Wikidata), plus…
This is a bit late since some of the deadlines have already passed, but nonetheless, here are the deadlines I know of for 2021 editions of the conferences I track for my lists of institutions and researchers active in technical games research. Plus two journal special issues. Deadlines still in the future (as of April 26): Deadline Venue Paper type 2021 May 28 CoG Short/demo/other papers 2021 May…
I had the following question: Given a number in $base^{exponent}$ format, like $3^{500}$ or $7^{300}$, is there a way to convert it to scientific notation without explicitly constructing the (possibly very large) number as an intermediate step? We will assume here that both $base$ and $exponent$ are positive. For example, $7^{300}$ should be converted to $3.384\ldots \cdot 10^{253}$, also written…
This post is for American University students who are interested in me serving as faculty supervisor for an internship-for-credit. Like many universities, AU allows students in some cases to receive academic credits for undertaking internships with an external organization (company or nonprofit). This can be in addition to being paid for the internship. To receive credits, there must be an AU…
A large urban–rural divide is a current feature of American politics. Republicans rarely win cities, and Democrats rarely win rural areas. This goes beyond regional concepts like "the heartland" and "coastal elites", and is replicated within just about every state. There are many ways to slice and dice that divide. This post takes one of them. Republicans can rarely win in a major city. But how…
A list of four-year universities in the United States who announced before August that their fall 2020 undergraduate classes will be taught all or almost all online, sorted by date of announcement. There's variation within the plans in the list. Some are exclusively online, while others plan to have a limited number of in-person courses, e.g. science labs. Also, some universities plan to have the…
During grad school in the late 2000s, I used to maintain a list of newsgames , i.e. games that commented in some kind of timely and usually editorial way on current events. I remembered it existed while looking through an archive of my old website , and decided to reproduce it here in case it helps anyone looking for information on newsgames of that period. You can also browse the original list on…
I'm recruiting a funded Masters student to study how AI bots play games. The goal is to systematically understand the kinds of difficulty posed by games to AI algorithms, as well as the robustness of any conclusions. Some example experiments include: looking at how performance scales with parameters such as CPU time and problem size; how sensitive results are to rule variations, choice of…
Enter a search term in one of the following six languages, and we'll try to find an image for it using Wikidata, via two search methods. Search terms can be people, places, events, objects, plants, etc. E.g. try "granite" or "Obama" in English, "ordinateur" in French, "δάσος" in Greek, etc. Full explanation and code below. Search for: Direct Wikidata search: English Greek Chinese French German…
A recent article on UK private bank C. Hoare & Co. made the rounds, noting that it has been run for more than 300 years by the Hoare family. Somehow this found its way into tech discussion boards, which led some people to wonder whether pioneering British computer scientist C.A.R. "Tony" Hoare is from the same Hoare family. I investigated briefly, and as it turns out the answer appears to be: yes,…
The UK-based Google subsidiary, DeepMind , which brought big news to the AI & games world in 2016 with its bot AlphaGo defeating a world-class Go player, says they've done it again with AlphaStar, a bot that beat a high-level human player in the real-time strategy (RTS) game StarCraft II, previously considered to be a top-tier challenge. I'm currently teaching an AI & Games special-topics course…
A few days ago I moved to Washington, DC, to start a new job as Assistant Professor in the Computer Science department of American University . The CS department here is small but growing , and I'm looking forward to being a part of that. Thanks especially to profs. Mike Treanor and Nathalie Japkowicz for their part in hiring me and helping me to get oriented here. For the past 2½ years I was…
The name Trump is transliterated into Greek as Τραμπ. This is a fairly straightforward transliteration, but the reverse direction is surprisingly ambiguous. I found one news article where Google Translate translates Τραμπ back into English three different ways in the same article, none of them correct: US President Donald Trub President Trabble President Trabid What's going on here? There are two…
If you try to translate 'ribbit' (the sound a frog makes in English), Google Translate will usually just leave it untranslated. But in a few cases it makes amusing mistakes: English → Greek κουνέλι ('rabbit') English → Spanish costilla ('rib') English → Chinese 收购Ribbit ('acquisition of Ribbit') In the Greek and Spanish cases, it makes two different spurious autocorrections on the English side.…
McGill University's txtLAB blog posted an article earlier this week plotting the proportion of academic papers in the field of literary studies that have a colon in the paper title, from 1950 to 2010. Colon-titles are a thing academics sometimes like to do, with patterns such as "Main Idea: Additional Explanatory Text" and "Joke Title: Real Topic Explained After the Colon". In the txtLAB analysis,…
When preparing conference proceedings in computer science, a common administrative task for the proceedings chair, after all the accepted papers have been collected, is to stamp some kind of copyright or venue information onto the first page of each paper. (Sometimes authors are asked to do this directly, but it's also pretty common for the proceedings chair to add this information later.) I've…
What would game jams be like if they lasted 1–2 hours instead of 1–2 days? And to take it further, if jammers tried out a dozen game designs in that time rather than focusing on prototyping one? A bit like a brainstorming session with paper-prototyping materials, but still producing playable digital games as output, like a game jam. We've been hosting rapid game jams to experiment with one way of…
The U.S. National Science Foundation announced its 2018 Graduate Research Fellowship Program recipients today. A full list of the 2,000 recipients is available from NSF FastLane . Alas, as these things often go, the data is messy, e.g. the same university's name appears with multiple spellings ("University of California, Berkeley" versus "University of California-Berkeley", etc.). I've cleaned it…
My preference, since the late '90s, has been to write webpages as plain HTML in a text editor. This isn't because I'm a giant HTML fan or expert. I don't do anything fancy with it, just text with some links and images, which is part of why I prefer this approach. HTML is an acceptable markup language for hypertext documents. For a long time, this worked fine. And today it still works, but only…
During the years I lived in Denmark, I often used Google Translate for news articles. Since Danish has relatively simple grammar that is not too dissimilar from that of English, and also tends towards short sentences without a lot of complex clauses, machine translation does pretty well. But sometimes it makes mistakes. Since Google Translate is based purely on statistical machine learning applied…
I've stayed in my share of accommodations with questionable internet setups and wifi failure modes—mainly B&Bs and AirBnBs equipped with consumer-level access points that don't really work. Of those, I thought I'd share two, along with a workaround for one of them, though admittedly a pretty janky workaround. Failure number one Here is one failure, the regularity of which is oddly satisfying, yet…
Boulder Dash , a VGDL version of the 1984 computer game The General Video Game AI Competition (GVG-AI) is a recurring videogame-playing competition intended to stimulate progress in general videogame AI (as distinguished from AI written for a specific game) by testing submitted AI agents against previously unseen videogames. Games used in the competition are written in the Video Game Description…
The Communications of the ACM is, since 1958, the flagship magazine/journal of the Association for Computing Machinery , a professional society for computer scientists and computing professionals. In its current format, which dates to a revamp initiated by incoming editor Moshe Vardi in 2008, it carries a mixture of news items, interviews, op-eds, columns (1–3 pages ea.), articles on current…
In a traditional Unix mail setup, users receive mail on a remote server, which they log into through a terminal to read. This would once commonly have been a server run by their university or employer. The mail is delivered to their home directory (or possibly a spool file), after which it can be read or otherwise manipulated as the user would like. More on that part in a minute. This setup has…
Tapper I have had an occasional interest in the 1983 videogame Tapper . But I did not know, until recently discovering it by accident, that its designers had patented the game [pdf] . Leaving aside the oddity of patenting a game design, the patent document itself is remarkable. For those who haven't played the game, a gameplay summary I wrote some years ago: In Tapper , the player is a bartender…
Marco Scirea , a PhD student who Julian Togelius and I work with, built a generative-music system that is parameterized by two control axes. Our goal here is to understand what effect varying those axes has on listeners' perception of the music. The axes did not come from nowhere, of course, so we have some guesses about what they do. They were designed with the intent to control perceived mood of…