1 of 30

Tensor Networks and Optimisation of �Structure-Aware Language Models

Theodor Iosif

University College London

2 of 30

Motivation

Mathematical models

of language

(Quantum) natural

language processing

New approach to

structure-aware model

Multimodal tasks

Efficient extraction

of meaning from text

Python package

lambeq

1,2,3,4,9

3 of 30

Background

4 of 30

Language models

String: “John gave Mary a flower”

Word sequence model

BOW model

DisCoCat model

3

5 of 30

Pipeline (I)

6 of 30

Categories 101

For the particle physicists

out there …

5

7 of 30

Grammar as a pregroup

6

Category:

8 of 30

Grammar as a pregroup

Example:

Diagram:

9 of 30

Meaning as a CCC

Example:

Extension:

10 of 30

Combining grammar and meaning

(DisCoCat)

11 of 30

Rewriting rules

Modifying

word structure:

Modifying cups:

12 of 30

Pipeline (II)

✔️

13 of 30

Tensor Networks 101

Tensor:

Tensor network:

Tensor network layouts:

7,8

14 of 30

Parametrisation

Quantum Ansatz

Classical Ansatz

15 of 30

Classical Ansätze

String diagram:

MPSAnsatz:

16 of 30

Classical Ansätze

String diagram:

SpiderAnsatz:

17 of 30

Quantum Ansätze

IQPAnsatz:

…and many others.

18 of 30

Pipeline (III)

✔️

19 of 30

New Ansatz

String diagram:

NewAnsatz:

(~meaning, diagonal)

(~syntax)

20 of 30

New Ansatz (simplified)

TensorAnsatz

NewAnsatz

21 of 30

Comparison

Noun size:

Sentence size:

MPS bond size:

Noun �vocabulary size:

Verb �vocabulary size:

Number of �“connectors”:

22 of 30

Back to string diagram

String

diagram:

NewAnsatz:

functional

“words”

23 of 30

Conclusion

DisCoCat ⬄ (syntax + semantics);

New Ansatz:

improves structure;
expected to be efficient;
particularly useful for large vocabularies;
map to string diagram 🡪 and beyond.

24 of 30

Outlook

Implementation on lambeq;
More general diagonal tensor;
String diagram 🡪 quantum circuits:

discussed in report;

Application to other word types:

Also discussed in report!

Parametrisations seen as functors

… and many other future directions.

25 of 30

Thank you

26 of 30

References:

[1] E. Grefenstette and M. Sadrzadeh, “Experimental Support for a Categorical Compositional Distributional Model of Meaning”, arXiv:1106.4058 [cs], 10.48550/arXiv.1106.4058 (2011).

[2] R. Lorenz, A. Pearson, et al., “QNLP in Practice: Running Compositional Models of Meaning on a Quantum Computer”, Journal of Artificial Intelligence Research 76, arXiv:2102.12846 [cs], 1305 (2023).

[3] D. Kartsaklis, I. Fan, et al., “lambeq: An Efficient High-Level Python Library for Quantum NLP”, arXiv:2110.04236 [cs], 10.48550/arXiv.2110.04236 (2021).

[4] H. Hawashin and M. Sadrzadeh, Multimodal structure-aware quantum data processing (2025).

[5] G. Vercleyen, “The Mathematical Structure of Tensor Networks”, MA thesis (Ghent University, 2018).

[6] B. Coecke, M. Sadrzadeh, et al., “Mathematical Foundations for a Compositional Distributional Model of Meaning”, arXiv:1003.4394 [cs], 10.48550/arXiv.1003.4394 (2010).

[7] H.-M. Rieser, F. K¨oster, et al., “Tensor networks for quantum machine learning”, en, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 479, 20230218 (2023).

[8] A. Cichocki, “Tensor networks for big data analytics and large-scale optimization problems”, arXiv:1407.3124, 10.48550/arXiv.1407.3124 (2014).

[9] Quantinuum, lambeq logo, https://docs.quantinuum.com/lambeq/index.html, Accessed: 2025-03-24, n.d.

27 of 30

Extra: Other sentences

Using

prepositions:

Relative pronouns:

28 of 30

Extra: Simplifications revisited

29 of 30

Extra: Simplifications revisited

30 of 30

Extra: Ansatz on prepositions