1 of 75

The Future of

Knowledge Graphs

in a World of

Large Language Models

Denny Vrandečić

New York, NY - May 11, 2023

2 of 75

“When in doubt, you can’t be wrong”

  • “This isn’t what it looks like”, by Pseudonymous Bosch

Disclaimer 1

3 of 75

CC-BY-SA 3.0, Pnautilus

4 of 75

CC-BY-SA 3.0, Pnautilus

5 of 75

6 of 75

7 of 75

8 of 75

Nothing in this talk is generated

*unless explicitly marked, or in a screenshot from an LLM

Disclaimer 2

9 of 75

This is a focused talk

Disclaimer 3

10 of 75

11 of 75

CC-BY-SA 4.0, Andrew Lih

12 of 75

CC-BY-SA 4.0, BrunelloN

13 of 75

14 of 75

15 of 75

16 of 75

17 of 75

18 of 75

19 of 75

~5 seconds

20 of 75

~5 seconds

0.56 seconds

21 of 75

~5 seconds

0.56 seconds

0.26 seconds

22 of 75

~5 seconds

0.56 seconds

0.26 seconds

23 of 75

~5 seconds

0.56 seconds

0.26 seconds

24 of 75

~5 seconds

0.56 seconds

0.26 seconds

25 of 75

Large generative model

  • 6 tokens input
  • 60 tokens output (2 tokens)
  • 96 layers
  • 175 billion parameters

26 of 75

Large generative model

  • 6 tokens input
  • 60 tokens output (2 tokens)
  • 96 layers
  • 175 billion parameters

Knowledge graph lookup

  • Find item out of 100m
  • Find key out of 10k
  • Logarithmic operations

27 of 75

28 of 75

29 of 75

30 of 75

“ChatGPT-like Google would be 10x more expensive per query”

  • John Hennessy, Chairman of Alphabet

31 of 75

32 of 75

33 of 75

34 of 75

35 of 75

36 of 75

37 of 75

38 of 75

39 of 75

40 of 75

41 of 75

42 of 75

You can machine learn Obama’s birthplace every time you need it, but it costs a lot and you’re never sure it is correct

  • Jamie Taylor, Google Knowledge Graph

43 of 75

44 of 75

45 of 75

46 of 75

47 of 75

48 of 75

49 of 75

32 x 412 = � 15,129

50 of 75

Don’t get bedazzled by LLM’s capabilities, but use it where efficient

51 of 75

52 of 75

53 of 75

CC-BY-SA 4.0, EdoardoRamalli

54 of 75

CC-BY-SA 4.0, EdoardoRamalli

55 of 75

56 of 75

Stable diffusion

Public Domain, All images generated by Stable Diffusion

57 of 75

890,000,000

175,000,000,000

58 of 75

890,000,000

175,000,000,000

7,000,000,000

59 of 75

In a world of infinite content, knowledge becomes valuable

60 of 75

Overfit for truth

61 of 75

CC-BY-SA 4.0, EdoardoRamalli

62 of 75

63 of 75

64 of 75

author

it’s complicated

65 of 75

SuperlativeSubject: Jupiter� Quality: large� Class: Planet� Locality: Solar System

“$Subject is the $Superlative( $quality) $Class in $Locality.”�

Jupiter is the largest planet in the Solar system.

66 of 75

LLMs are awesome, but…

  • Hallucinations
  • Expensive to train & run
  • Difficult to fix & update
  • Hard to audit & explain
  • Inconsistent answers
  • Low resource languages
  • Coverage gap on long tail

67 of 75

The future of Knowledge Graphs is brighter than ever

Thanks to a world with Language Models

68 of 75

Contact me

denny@wikimedia.org

69 of 75

That’s 9:34 am in New York time

70 of 75

Should LLMs store knowledge end to end?

71 of 75

72 of 75

A.I. TURNS THIS DATASET INTO A TEXT ON WHICH I CAN TRAIN THE LLM ON

A.I. EXTRACTS TRIPLES OUT OF THIS TEXT, SO I CAN QUERY IT EFFICIENTLY

73 of 75

74 of 75

75 of 75