1 of 40

LLMs, AI Search & Answer Optimization

THE FUTURE OF SEARCH

2 of 40

Ethan

CEO @ Graphite

Greg

Chief AI Officer @ Graphite

Who We Are

3 of 40

Questions

General Chatter / Networking

4 of 40

Before

  • Go to search engine
  • Search
  • Click links
  • Visit websites
  • Find answer

Now

  • Go to AI answer engine
  • Ask a question
  • Read answer
  • Click links to learn more

AI Is Transforming How We Get Information

5 of 40

New Optimization Problems for AI Search

Citation Optimization

How do you get your page to be cited in the answer?

Answer Optimization

How do you get your product / company to appear in the answer?

6 of 40

Technologies for AI Search

LLM

LLM + RAG

Search

7 of 40

The best project management tool is ______

How LLMs Work: Next Word Prediction

8 of 40

The best project management tool is Trello

How LLMs Work: Next Word Prediction

9 of 40

​

The best project management tool is JIRA (36%)

The best project management tool is Notion (27%)

The best project management tool is Trello (20%)

The best project management tool is Atlassian (9%)

The best project management tool is Monday (8%)

LLMs Generate by Repeatedly Sampling the Next Word

10 of 40

​

Best for Kanban-Style boards, Trello is a project …

teams can use Trello to build a complete project workflow …

One of the best project management tools is Trello.

If you are using Agile, Trello is a good option …

Trello: Best for simple project management.

​

Next Word Predictions Are Based On Co-Occurrences

11 of 40

“Answer Optimization”

Is The New SEO

12 of 40

Answer Optimization - Co-Occurrence

13 of 40

Answer Optimization - Co-Occurrence

14 of 40

AI Answer Optimization - UGC

15 of 40

AI Answer Optimization Varies by Surface

“What’s the best project management software?”

16 of 40

AI Answer Optimization Varies by Question Form

"What is the best project management software?"

1

2

3

4

"Which project management tool is considered the best?"

"Can you suggest the best project management software for teams?"

"What is the top-rated project management software?"

"Which project management software offers the best features and usability?"

5

17 of 40

  • Track by Surface: ask questions across multiple surfaces
  • Track by Variations: ask the same question in many different ways
  • Track by Run: re-ask the same question many times, average
  • Rather than single position, result is a distribution or “share of”

SERP Tracking > Tracking Share Of AI Answers

0.38

0.28

0.67

0.41

0.37

0.19

0.21

0.35

0.14

Trello

Notion

Jira

ChatGPT

CoPilot

Perplexity

What is the best project management tool?

18 of 40

LLMs + Retrieval Augmented Generation (RAG)

19 of 40

LLM Answers Are As Good As Available Data

False information

Little information

Lots of information

20 of 40

AI Answers Frequently Start with a Search

LLM

LLM + RAG

Search

21 of 40

Using LLMs with RAG is Becoming Standard

Perplexity

CoPilot

ChatGPT

Google

22 of 40

LLM

Question

Prompt

LLM

Answer

23 of 40

LLM + RAG

Question

Data

Search Engine

Prompt

LLM

Answer

LLM acts as a reasoning engine and summarizer

Retrieved pages

Additional information is provided by the search engine

24 of 40

Provides extra data to LLM to help with limited / out-of-date information

Provides opportunity to filter out low-quality and bad information

LLM becomes less creative

Facilitates citing sources (retrieved pages)

RAG Improves AI Answers

25 of 40

AI search with LLM+RAG does not replace traditional search, it builds on it

​

​

Optimizing for AI Search is an Evolution of SEO

26 of 40

Answer Optimization for LLM+RAG - Perplexity

“How to get a day pass at West Hollywood Edition?”

Resortpass ranks in both SEO and LLM Answer

Optimizing for search engines (SEO) can help optimize for RAG + LLM answer

27 of 40

“Good Content” Answers Every Question

“How can I get a day pass at West Hollywood Edition?”

1

2

3

4

"When does the West Hollywood Edition pool open?”

“How much is a day pass at the West Hollywood Edition?”

“Can I reserve a cabana at the West Hollywood Edition?”

“What’s a fun day vacation idea this weekend?”

5

28 of 40

Answer Optimization - Owned vs. Earned

“What Is The Best Credit Card?”

Optimize by getting featured by popular citations

(e.g. US News, Forbes)

Listicle Optimization

29 of 40

AI Overview Citation Optimization

30 of 40

AI Search is backed by a traditional search engine

Optimizing for SEO means optimizing for citations

Citation Optimization Starts With SEO

31 of 40

Track how often my content is cited across LLM Answers, and position of citation

  • Track by Surface: ask questions across multiple surfaces
  • Track by Variations: ask the same question in many different ways
  • Track by Run: re-ask the same question 100 times

Citations are sometimes end, sometimes inline, sometimes below the fold; they likely have different CTRs

Citation Tracking - Owned

What is the best business credit card?

0.38

0.28

0.67

0.41

0.37

0.19

0.21

0.35

0.14

Forbes

Nerd Wallet

US News

ChatGPT

CoPilot

Perplexity

32 of 40

Brands Need To Earn Citations

Yes

Yes

Yes

Yes

No

Yes

No

No

Yes

Ramp

Brex

Mercury

Forbes

US News

Nerd Wallet

What is the best business credit card?

33 of 40

  • Expertise
  • Authority
  • Trustworthiness
  • Personal Experience

People Have Higher Expectations For

Answers Than Search

34 of 40

AI Answers - Organic vs. Ads

Best Bicycles

Wedding Photographers NYC

Executive Coaching

35 of 40

AI search builds on traditional search

AI Answer Optimization:

  • appear on many pages with language similar to the target questions
  • publish high-quality, comprehensive, trustworthy content about your product

​

Key Takeaways

01

02

03

AI Citation Optimization:

  • publish high-quality, comprehensive, trustworthy content
  • avoid publishing fully AI-generated content

36 of 40

Thank You!

Slides: bit.ly/ai-optimization

Graphite.io

37 of 40

AI Content

38 of 40

Dilemma - If AI Content Wins

If AI generated content performs well in search and is summarized by AI, then you have an infinite loop of derivatives of derivatives which results in convergence of a single point of view for everything.

39 of 40

AI Content Is Not Winning…Today

40 of 40

AI Content Is Not Winning…Today