Transformers, deep learning
(Not NLP-specific work.) Things I want to learn: Transformer architecture TensorFlow and alternatives conditional random fields
David Corney
A blog about natural language processing, machine learning, fact checking. My opinions about GPT3, generative AI. My work at Full Fact, part of the International Fact Checking Network.
Latest posts
NLP, language models, linguistics
Things I know about: language models BERT Things I want to know (more) about: BERT alternatives - mT5, GPT3
Digital Media and (modern) journalism
https://www.journalismaifestival.com/ Seminar: most journalists are NOT technical, often have slow take up of new tech. So they’re cautious of AI tools. A few pioneers may embrace them, but things spread slowly within newsrooms. Should therefore involve editorial in decisions about AI tech early on. AI = NLP for analysis; synthetic text; finding patterns in data faster than people can; article…
Plans for notes about ideas
So new plan (from December 2020): I want to raise my public profile as an expert in using NLP/AI for digital news analysis, especially around trust, verification, misinformation etc. I want to identify a series of areas and become an expert in them: read key papers, make notes on them (hereish), share findings with colleagues. And I want to raise my profile through regular blogging, engaged…
Data Feminism, bias detection
Notes from the book “Data Feminism” and related sources I’m slowly reading Data Feminism, by Catherin D’Ignazio and Lauren Klein. It’s good - lots of it is eye-opening to me, as it shows data science from a perspective other than my own. Matrix of domination (p.15): Four domains of domination structural Plans and policies hegemonic cultural disciplinary enforcement interpersonal invidival…
What happens when "LLM grooming" fills an "information void"?
I want to discuss two concepts around mis-/disinformation and what happens if they mix. LLM Grooming This is when a coordinated network of websites and social media accounts is used to push a particular narrative at a massive scale. Unlike typical disinformation campaigns, the target is not (directly) people, but rather search engine web crawlers and the scraper bots that collect training data for…
Does using LLMs as fact checkers make things better or worse?
TL;DR: Worse. I’ve been developing AI tools to support (human) fact checkers for several years now. The tools that Full Fact produces are now used every day by fact checkers around the world to help monitor the media and find claims that might be worth checking, and to find repeats of known falsehoods. This helps them to hold powerful people to account and to push back against misinformation. But…
What cognitive science teaches us about AI and vice versa
I recently came across a new paper 1 making a bold claim: that machine learning will never lead to actual artificial intelligence. Not just that AGI is hard, but rather that it is provably impossible to achieve through ML. It’s a bold claim because it flies in the face of so many big announcements in recent years from OpenAI, Google and many others, who claim that as machine learning is used to…
Factuality, LLMs and fact checking
A few months after ChatGPT was launched, I joined a group of academics and practitioners with expertise in the intersection of disinformation, fact checking and AI. We were all concerned that making these new generative AI tools widely available would lead to a proliferation of false information online that would be hard to distinguish from reliable information. We spent some time discussing our…
What makes a claim worth checking?
As a species, we tell stories and share information all the time. But when we’re given bad information, we make bad decisions. So when a statement made in public is wrong it may be worth considering for fact checking, especially if it may cause harm. My fact-checking colleagues at Full Fact spend time every day deciding which claims to focus on. We fact check around 800 claims per year , which is…