Daemon

Retrospective

This is a front-end for a language model, written in 2021. That sentence means something very different now, and the code is a decent record of the difference.

GPT-2’s weights were public, so generating text meant running the model yourself. It could be deployed to Google Cloud Run using minimaxir¹. Daemon is the frontend.

There is no backend

There’s a Dockerfile, which sounds impressive until you read it:

FROM httpd:alpine
EXPOSE 80
COPY ./build /usr/local/apache2/htdocs/

Apache serving a build folder. The model lives somewhere else entirely, and the app finds out where from a text input in the sidebar:

host: data !== null ? data.host : 'http://localhost:3000',

A settings field where you paste the URL of your own model server, defaulting to localhost. That’s the part that dates hardest. Using a language model meant having one running.

The knobs were the interface

The sidebar is four controls: length, temperature, top_k, and a prefix textarea.

I didn’t write that for a general audience because there wasn’t one. GPT-2 shipped without a chat wrapper or any instruction tuning. You gave it the opening of some text and it continued, so the UI exposed the sampling parameters directly. Those were the only things there were to adjust.

The default prefix is The grasshopper lies heavy., the book inside the book in The Man in the High Castle.

Sending the request

The whole thing:

fetch(`${state.host}`, {
method: 'POST',
body: JSON.stringify({
  length: state.longText,
  prefix: state.prefix,
  temperature: state.temperature,
  top_k: state.top_k,
}),
})
.then((response: any) => response.json())
.then((data) => { /* ... */ });

Nothing here catches a failure, tracks that a request is in flight, or disables the button while it waits. A cold Cloud Run container booting GPT-2 took several seconds, and for all of them the interface sat there looking exactly as it had before. I’d hit send, see nothing happen, and hit it again.

Model output straight into the DOM

let cleanData = `<p>${data.text.replace(
/(?:\r\n|\r|\n)/g,
'<br>'
)}</p>`;

That string gets rendered through dangerouslySetInnerHTML. cleanData is a generous name for a variable that cleans nothing: it swaps newlines for <br> and wraps the rest in a paragraph. Whatever the model returned went into the page as markup. The model was mine and the server was mine, so nothing came of it, but that’s luck rather than a decision.