excursus in truth theory

A MOST MINIMAL THEORY OF MEANING FOR MACHINE-CONSCIOUSNESS

Do LLMs handle meaning in the sense defined by Logic and Theory of Meaning? If not, can they recognise truth? What is the assonance between a spatial positioning system like GPS and words embeddings? Do we, as system developers, minimise or increase semantics in the logic we program?

Incipit

This exploratory work has been inspired by the reading of Bernard Williams’s “Descartes” for the methodological part and by Michael Dummett (1997, minute 18:24) on the role of logic and semantics: “the logician is concerned with the theory of inference, in characterising which inferences are valid he needs to rely on a semantic theory … there are competing semantic theories. The logician is not concerned to judge between them but to work out the consequences for the theory of inference … of the choice of a particular semantic theory. Which semantic theory is the correct one is not the concern of the logician but … of the theory of meaning”

Let’s start accessibly: is Information theory (IT) a theory of inference? No (because semantics is explicitly left out of IT), then which theories of meaning can be the coherent basis of a theory of inference derived from IT? (see the last section for a tentative answer).

Let’s fail upward from a philosophical trick; another thought experiment to describe the interrelations between a theory of inference and a theory of meaning (on our usual pathways of intuition pumps). Starting from a concept of truth based on IT, e.g. keeping out a theory of meaning as much as possible (hypothetically an approximation of what a non-embodied non-spatial or “mathematical” intelligence could achieve, see a previous post that describes this hypothesis): truth is that recognition of fact[s] that facilitates the development of devices (tech or dispositifs) based on a particular class (locality) of ordering functions (in-formation) which effects mitigates the entropy of the system in which that recognition happens.

The only injection of (strictly human) meaning in this tentative pure-inferential definition is the possibility of the device to be a (humanly social) dispositif; applying some simple analogy for the sake of fixing this easy point, we found here a first example of a choice of a semantic theory. As a broad example to familiarise the naming “ordering functions”: a reasoner with a background in Foucault’s philosophy would maybe find basis for the ordering functions in the foucaultian definition of (social, juridical, …) dispositif; a reasoner with an econometric background would maybe base its ordering functions on Arrow’s social choices and preferences. I call all of those “devices”, deployers of ordering functions. This is all nice good debating but the questions of methods are:

1. (logical) which “mix” of ordering functions? to build an inference;
2. (semantical) with which objective functions? (totally different kinds of functions from the ordering ones; in addition how these functions are justified in terms of human condition, this part only if we want to get really pop-philosophical)

I will try to answer only the first question here as it is the one in the subject matter of finding an inference theory that is workable with IT (and can be used to underpin or used as corollary for the machine-consciousness hypothesis, or hopefully maybe to establish an ordering between inference and semantics for the sake of the question of method or for the realm of metalogic). [Why! you could ask. Because the “mathematical” intelligence in its condition of alien intelligence is much more interesting and explainable than the most familiar kind to us.]

Derivation and basics

Skip this if you are familiar with these concepts. Note: equations are broken by rendering, see original for proper rendering.

A minimal theory of meaning

The ones mentioned in the Synthesis above are all nice options but let’s try a practical experiment to remove some theoretical overhead. What if we can encode meaning in spatial relationships? Like relative positions or any spatial reference of objects. Is it useful to describe what happens in contemporary Large Language Models (LLMs) where semantics is reduced to distances among tokens on a vector space?

As in the best tradition of my blogposts, and in the spirit of first principles thinking for lean design and engineering, I try here to develop the aforementioned to add the minimal possible layer of semantics to IT and adapt it in accordance with the reasoning adopted in my hypothesis on machine consciousness.

To develop a theory of inference grounded in spatial frames of reference as the most minimal semantic layer, I propose **Spatial Information Semantics (SIS)**, an hypothesis for which meaning derives exclusively from empirically verifiable spatial relationships (i.e. vector distances, neurons relative positions, particles positions). This aligns with Dummett’s verificationist semantics but restricts semantic content to positional data and geometric transformations (less overhead). So to implement a minimal verificationism, trying to identify the minimal conditions for which IT can support meaning (“Meaning arises from conditions under which a statement is verified.” quotes [1] in Derivation and basics above; yes but what are the minimal conditions from which meaning arises? Semantics meets complexity).

Note: equations are broken by rendering, see original for proper rendering.

This simple model makes clearer also to non-specialists some obscure points of current AI trends and groundbreaking events. By analogy, we can see here how similar this is to what happens in vector similarity; in the case of LLMs we just substitute words embeddings to GPS coordinates. We can step forward by trying to describe LLMs hallucinations in this terms: GPS works on a portion of space that is strictly limited by the specification of the GPS triangulation and its equipment while natural language is a completely open space where new constructs and meaning are built. Can this simplified spatial semantics be enough to allow recognition of valid statements? This is another core question. Another is: is the space of natural language too undefined to allow deterministic objective functions? A class of objectives functions/processes (so inference processes) that separates “generative content that makes sense according the ‘rules’ of the portion of space humans are accustomed to” from hallucinations is the subject of the current AI gold rush where systems in competition try to find the right boundaries for this undefined space through the pipes lining of software components.

Verification

We can think about how this approach if further developed may tackle some important barriers for understanding AI systems now and in the future by putting down some cornerstones:

• Syntax-Semantics Integration: Encode data with metadata specifying its verification conditions (e.g., cryptographic hashes as proof-of-work).
• Inferential Layers: Use deductive rules (e.g., logical constraints in error correction[3]) to derive higher-order meanings.
• Contextual Utility: Assign meaning based on operational goals (e.g., medical data interpreted differently in diagnosis vs. research).

Beyond SIS, QSIS

As anticipated let’s extend SIS to adapt it to the basics principles of a quantum system (non-locality, entanglement, superposition) for the sake of getting rid of possible ballast to our layers.

To extend **Spatial Information Semantics (SIS)** to quantum systems, we redefine spatial positions as quantum states, incorporating superposition, entanglement, and non-locality. Below is the framework and code showing quantum SIS (QSIS) using Qiskit. Note: see original for correct rendering of equations.

This framework tries to bridge logic-meaning with quantum information theory, enabling spatial reasoning in quantum algorithms. It is a thought experiment to boost understanding of how logic and meaning have interacted in 20th century philosophical thinking and how these concepts can be used to describe underlying principles of contemporary AI processes as described in my previous post.

Quotations:
[1] An operational distinction between quantum entanglement and ... https://arxiv.org/html/2405.15692v1
[2] Quantum query algorithms https://learning.quantum.ibm.com/course/fundamentals-of-quantum-algorithms/quantum-query-algorithms
[3] Is there a Qiskit function to encode a list of ints into a quantum state ... https://quantumcomputing.stackexchange.com/questions/29366/is-there-a-qiskit-function-to-encode-a-list-of-ints-into-a-quantum-state-by-usin
[4] Hello world - IBM Quantum Documentation https://docs.quantum.ibm.com/guides/hello-world
[5] Quantum Objects as Meaningful Symbols - Shabda Journal https://journal.shabda.co/2023/07/04/quantum-objects-meaningful-symbols/
[6] Quantum semantics of text perception | Scientific Reports - Nature https://www.nature.com/articles/s41598-021-83490-9
[7] [1206.0630] Three-dimensionality of space and the quantum bit - arXiv https://arxiv.org/abs/1206.0630
[8] Circuit library | IBM Quantum Documentation https://docs.quantum.ibm.com/guides/circuit-library
[9] Coding with Qiskit 1.x | Programming on Quantum Computers https://www.youtube.com/watch?v=93-zLTppFZw
[10] [PDF] The Potential of a Thick Present through Undefined Causality and ... https://pdfs.semanticscholar.org/7427/ad1b141b8e2d1de65364b0f2b8299708c8c4.pdf
[11] [PDF] Everything is Entangled in Quantum Mechanics - CONICET https://ri.conicet.gov.ar/bitstream/handle/11336/254895/CONICET_Digital_Nro.f7bb433f-fe00-4b48-a2cb-64b352a1a7b4_B.pdf?sequence=2&isAllowed=y
[12] [PDF] Quantum Entanglement vs Non-Locality - PhilArchive https://philarchive.org/archive/CAPQEV
[13] [PDF] A Quantum-Theoretic Approach to Distributional Semantics https://aclanthology.org/N13-1105.pdf
[14] Quantum causal inference with extremely light touch - Nature https://www.nature.com/articles/s41534-024-00956-0
[15] [PDF] Quantum Information Theory and The Foundations ... - PhilSci-Archive https://philsci-archive.pitt.edu/2344/1/qinfandfoundsqm.pdf
[16] End-to-end examples - IBM Quantum Documentation https://docs.quantum.ibm.com/migration-guides/qiskit-runtime-examples
[17] Introduction to Coding Quantum Algorithms: A Tutorial Series Using ... https://arxiv.org/abs/1903.04359
[18] Encoding patterns for quantum algorithms - IET Journals - Wiley https://ietresearch.onlinelibrary.wiley.com/doi/10.1049/qtc2.12032
[19] Training a Quantum Model on a Real Dataset - GitHub Pages https://qiskit-community.github.io/qiskit-machine-learning/tutorials/02a_training_a_quantum_model_on_a_real_dataset.html
[20] Programming on Quantum Computers — Coding with Qiskit S2E6 https://www.youtube.com/watch?v=OKbcJCUx6xA

— — — — — — — — — —

Bibliography

[8] [PDF] Dummett on the Relation between Logics and Metalogics https://media.philosophy.ox.ac.uk/assets/pdf_file/0015/35340/Dummett.pdf
[9] Information theory: A foundation for complexity science - PNAS https://www.pnas.org/doi/10.1073/pnas.2119089119
[10] The Correspondence Theory of Truth https://plato.stanford.edu/entries/truth-correspondence/
[11] Philosophical Aspects of Quantum Information Theory - arXiv https://arxiv.org/abs/quant-ph/0611187
[12] The Information Theoretic Perspective on Medical Diagnostic ... https://pmc.ncbi.nlm.nih.gov/articles/PMC6993929/
[13] [PDF] What Theories of Truth Should be Like (but Cannot be) https://www.impan.pl/~kz/truthseminar/Leitgeb_PhilCompass07.pdf
[14] [PDF] Quantum Information Theory https://qi.rub.de/files/qi-book-draft.pdf
[15] Truth - Stanford Encyclopedia of Philosophy https://plato.stanford.edu/entries/truth/
[16] Semantics of logic - Wikipedia https://en.wikipedia.org/wiki/Semantics_of_logic
[17] Language, Thought, and Logic: Essays in Honour of Michael Dummett https://academic.oup.com/book/53284
[18] Lecture 1: Introduction to Information Theory - YouTube https://www.youtube.com/watch?v=BCiZc0n6COY
[19] [PDF] Introduction to Logical Entropy and Its Relationship to Shannon ... https://philsci-archive.pitt.edu/19983/1/IntroductionToLogicalEntropy-revised.pdf
[20] Correspondence Theory of Truth - The Information Philosopher https://www.informationphilosopher.com/knowledge/correspondence.html
[21] Three Different Theories of Truth - Mr. Hoye’s TOK Website http://mrhoyestokwebsite.com/Knower/Useful%20Information/Three%20Different%20Theories%20of%20Truth.htm
[22] Entropy (information theory) - Wikipedia https://en.wikipedia.org/wiki/Entropy_(information_theory)
[23] Towards a Semantic Information Theory (Introducing Quantum ... https://arxiv.org/abs/2201.05478
[24] [PDF] Lectures Notes on Quantum Information Theory https://qi.rub.de/courses/qit21/qit22.pdf
[25] Principle of locality - Wikipedia https://en.wikipedia.org/wiki/Principle_of_locality
[26] Michael Dummett - Wikipedia https://en.wikipedia.org/wiki/Michael_Dummett
[27] Michael Dummett, What Is a Theory of Meaning? - PhilPapers https://philpapers.org/rec/DUMWIA-4
[28] [PDF] MICHAEL DUMMETT’S LOGICAL BASIS OF METAPHYSICS https://www.collectionscanada.ca/obj/s4/f2/dsk2/ftp02/NQ27659.pdf
[29] [PDF] Dummett on the Origins of Analytical Philosophy https://wab.uib.no/agora-alws/article/download/2706/3135
[30] Theories of Meaning - jstor https://www.jstor.org/stable/40343862
[31] Proof-Theoretic Semantics - Stanford Encyclopedia of Philosophy https://plato.stanford.edu/entries/proof-theoretic-semantics/
[32] Information theory | Definition, History, Examples, & Facts | Britannica https://www.britannica.com/science/information-theory
[33] Shannon Theory - an overview | ScienceDirect Topics https://www.sciencedirect.com/topics/computer-science/shannon-theory
[34] [PDF] Shannon information basics https://bernstein-network.de/wp-content/uploads/2021/02/17_Lecture-17-Shannon-information.pdf
[35] Correspondence theory of truth - Wikipedia https://en.wikipedia.org/wiki/Correspondence_theory_of_truth
[36] Three Different Theories of Truth - Mr. Hoye’s TOK Website https://www.mrhoyestokwebsite.com/Knower/Useful%20Information/Three%20Different%20Theories%20of%20Truth.htm
[37] The Five Main Theories Of Truth | UKEssays.com https://www.ukessays.com/essays/philosophy/the-five-main-theories-of-truth-philosophy-essay.php
[38] FUNDAMENTAL NATURAL CONCEPTS OF INFORMATION THEORY https://www.jstor.org/stable/42581366
[39] Meaning in Music and Information Theory - jstor https://www.jstor.org/stable/427154
[40] [PDF] 1. Introduction: information theory after Shannon - Dialnet https://dialnet.unirioja.es/descarga/articulo/5259075.pdf
[41] Information Theory–based Compositional Distributional Semantics https://direct.mit.edu/coli/article/48/4/907/112556/Information-Theory-based-Compositional
[42] Algorithmic Information Theory - an overview | ScienceDirect Topics https://www.sciencedirect.com/topics/computer-science/algorithmic-information-theory
[43] Quantum Information Theory - an overview | ScienceDirect Topics https://www.sciencedirect.com/topics/physics-and-astronomy/quantum-information-theory
[44] [PDF] The Theory of Quantum Information https://cs.uwaterloo.ca/~watrous/TQI/TQI.pdf
[45] [PDF] Quantum Information Chapter 10. Quantum Shannon Theory http://theory.caltech.edu/~preskill/ph219/chap10_6A.pdf
[46] [quant-ph/0011036] Quantum information theory - arXiv https://arxiv.org/abs/quant-ph/0011036
[47] Locality and entanglement of indistinguishable particles - Nature https://www.nature.com/articles/s41598-021-94991-y

YT: S5yvVm5wlE4