Creative writing help from your LLM

Anyone who has tried creative writing using a Large Language Model knows they all have their quirks and their prefered language and phrasing which they spit out over and over.

Often they’re more trouble than they’re worth, “I could have written this with a pencil faster.” You end up running endless regenerations, rewriting and copy pasting and after a while you'll ask yourself, there must be a better way to get what I want from this huge word beast?

Really you want to use an LLM for what they’re best at; the task of an editor, and co- writer. Fixing sentences, rewrites, grammar. Bits and pieces. They’re great at the micro (sentences), dreadful at the macro (story).

I wanted to be able to use an LLM’s statistical knowledge of language to rework sentences and paragraphs, the standard chat interface is a terrible way to interact for this type of task. You need something better, something more specific.

I imagined a way where you could interogate a generated sentence, see how it was (statistically) generated and allow a user to inspect the words which were not chosen, and to try out regenerate using an alternative starting point, all the time keeping the originals to compare against. Holding onto the previous generations seems key here, for comparisons and invetigations.

Through July I was lucky enough to work with Daniel Hibbin from Exeter University as part of their A2i scheme and we created a prototype which is available at github:
https://dhibbin.github.io/LexiSelect/

The current prototype has an unfortunatly high technical barrier to entry. You need to use the llama.cpp project for inference as it’s API is able to provide the token probabilities and if you’re running your LLM on a different machine or remotely you need to host the LexiSelect page on the same machine or from your local file system to avoid any cross site (CORS) content issues. I’d love to iron out all these issues and hook it up to OpenAI’s API which is also capable of supplying the probability data.

Here’s a brief overview of using it.

The source code for the prototype (built as a Vue/Vuetify project) is available on github
https://github.com/dhibbin/LexiSelect/

I’d love to get some feedback from the community to see if they find it useful, ways they use it and suggestions for moving it forwards to a more fully functional project.