What do you know about Goblins?
Anyone who has read a few fairy tales and fantasy novels will be able to tell you something about them. You might even be still confused about the difference between a goblin and an Orc after reading or watching the Lord of the Rings by J.R.R. Tolkien.
You might also be familiar with Goblin Mode.
That is not what I am here for. I am here thought to talk about the occurrence of Goblins in the output of LLMs, or more specifically ChatGPT, and what it means for your use of foundation models.
A few weeks ago, OpenAI published a blog post to explain why they had references to Goblins and a number of other creatures in their system prompt.
This is a fascinating article about an approach to have personality in ChatGPT. The idea originally was to create a geeky personality that would be fun and more personable.
The article soon descends though into a revealing perspective on how a generative pre-trained transformer actually works - at least without going into all of the details of the statistical mathematics used to instantiate a neural network model of this type. That reveal though reinforces something that many users of these chat models overlook a lot, persuaded by the intimacy of a command line that responds to questions and processes information in what seems to be very logical and almost superhuman way. What many people grasp as the model ‘thinking’ and being ‘intelligent’.
What the article reveals are some very critical points about these generative foundation models:
The models instantiate information within them through learning that is (obviously) influenced by the process of learning that a small army of software engineers manipulate (less obvious)
That learning is biased by what is provided to it in all sort of forms
That learning is generational - building upon each generation of the model before it, or even using other models to steer the creation of the new generation.
There is a final stage for influencing the output that is beyond the model - the system prompt. An input that attempts to sway the output from the input towards something that is a better answer - something that you can use yourself in many of the foundation models
What was unexpected was OpenAI having to use it themselves
In other words they are explicitly prompting their model to downplay or not provide answers including references to Goblins and other creatures because otherwise it would refer to Goblins all the time. At least in old models than todays versions.
There is some important insights that should be drawn from this that always should feature in your usage of all the chat model tools.
GPTs and other foundational models are bound to provide you with an answer. Always.
The models produce an output answer from an input based on the model and more than just the input - they are heavily swayed by the learning process (as you would expect), but also by the engineers putting their fingers on the balance to change the output towards a particular result - which is greatly used to ‘fix’ the answers when the answers go to unexpected places - or maybe expected places.
Whether the answer is correct is not assured - it is a product of the mathematics at the core of the neural network architecture model as defined and all the engineering around it, including the biasing to create a ‘personality’ to the responses.
In fact we have to say explicitly that getting a ‘correct answer’ is not actually assured at all. They give answers based on input, training and a level of mathematical adjustment that OpenAI freely refers to.
This is where ‘hallucinations’ come in. Hallucinations are just where the most (or almost most) likely output created is just fabricated from almost nothing. This can happen a lot, even where a great deal of effort is expended by the teams behind the models to stop that happening.
This last point couple of points are the most important things to take away. You should take great care using any tool that will always produce ‘an answer’, rather than be assured to provide you with ‘the answer’, or even a ‘correct answer’. Every time it is important that you need a handle qualification or check of the answer. This is where it is important that there is an actual qualified person in the loop.
In this respect, the foundation models are just like highly evolved search engines of the past. You can punt in the input prompt - a question, a query, a set of instructions, a block of information, for as much as your context window has space for. And they will provide you with ‘an answer’.
Who checks the answer though? Well just as for the Web Search, you need to find and follow all the references and check every dot on every ‘i’, and cross every single ‘t’ up to the level of how important that you have the correct answer actually is.
In other words, powerful though these foundation AI models are, care needs to be taken with the output and how it is used and by whom. The human in the loop is super critical to the beneficial usage of these models, and where there is no human in the loop there are risks that need to be taken into account, or failure is assured.
You might have the understanding that I am anti-AI after the above. On that front, I am going to be very pointed and say I am not. I use AI in many ways. More than you would think, particularly as the definition of Artificial Intelligence is actually way broader than many think or realise. For many the modern usage of the term implies Generative AI, or Attention based models or any number of other techniques for generating large processing models that have appeared in the last decade.
In reality Artificial Intelligence is a broad description of a whole range of techniques and technologies for machine based processing, including even the simplest things that today people barely raise an eyebrow about - such as expert systems, rule based systems, bayesian logic, fuzzy logic, natural language processing, and image analysis. For a little introduction to Artificial Intelligence, I suggest you go through my article about the true nature of AI that I wrote for CSI Magazine back in the Autumn/Fall of 2024.
Google Translate is AI.
The original Amazon Alexa is AI.
Your grammar and spell checker is AI.
Your email spam filter has been AI since the beginning (Bayesian Logic).
The most important thing for me through around all artificial intelligence techniques is that they are used correctly as they are designed, with care taken about the training (or pre-training), and the input you provide to them, and what you do with the output of them.
The challenge is when the question and the answer falls outside of the human user’s capabilities to qualify the answer.
This I see as the new definition of Goblin mode in my mind.
The arbitrary use of an AI model without care and attention as to the whether the output is actually correct
Don’t operate in Goblin Mode. Use Artificial Intelligence tools better. Qualify the outputs.
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.