1 of 11

�

DATA MINING GROUP

SIEBEL SCHOOL OF COMPUTING AND DATA SCIENCE

UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN

AUG 10, 2026

1

Structure Shapes

the Future of DataxLLM Systems:

Retrieval, Structuring, and Reasoning

1

2 of 11

Instructors

  • Wonbin Kweon
  • Postdoc @ UIUC
  • Jiawei Han
  • Michael Aiken Chair Professor @ UIUC
  • Jash Parekh
  • M.S. Student @ UIUC
  • Cynthia Fan
  • Ph.D. Candidate @ UIUC

2

3 of 11

Outline

  • 0: An Introduction to LLM, RAG, and RAS
  • I: Retrieving: Enhancing Power of Retrieval with Structures
  • II: Reasoning: Enhancing LLM Reasoning & Generation with Structures
  • III: Agents: Enhancing LLM Agents with Tools and Search

3

4 of 11

Outline

  • 0: An Introduction to LLM, RAG, and RAS
  • I: Retrieving: Enhancing Power of Retrieval with Structures
  • II: Reasoning: Enhancing LLM Reasoning & Generation with Structures
  • III: Agents: Enhancing LLM Agents with Tools and Search

4

5 of 11

  • Artificial Intelligence: “The science and engineering of making intelligent machines” ― John McCarthy
    • Tasks include perception, learning, reasoning, problem-solving, decision-making
  • Machine Learning: Algorithms and models that learn from data
    • Learning approaches include supervised, unsupervised, semi-supervised, and reinforcement learning
  • Deep Learning: Utilize deep artificial neural networks and learn representations of data through multiple layers
    • Effective for tasks such as image recognition, natural language processing, etc.
  • Generative AI: Focus on creating models to generate new data
    • Examples include Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), Large Language Models (LLMs)

From Machine Learning to LLM: An AI Landscape

5

6 of 11

  • Comparison among Fine-Tuning, Prompt Engineering and RAG (Retrieval Augmented Generation)
    • Prompt Engineering: require low model modification & external knowledge, focusing on harnessing the capabilities of LLMs themselves
    • Fine-tuning: Involve further training the model
    • Naive RAG: Low demand for model modifications
    • Modular RAG: More integrated with fine-tuning techniques
    • ? Retrieval and Structuring ?

Figures adapted from Y. Gao et al, RAG Survey. arXiv:2312.10997

O. Ovadia, et al (2023), “Fine-tuning or retrieval? comparing knowledge injection in LLMs,” arXiv:2312.05934

[Ovadia, et al 23]: RAG consistently outperforms unsupervised fine-tuning (FT). LLMs struggle to learn new factual information through unsupervised FT. In some cases, combining RAG and FT may lead to optimal performance.

Retrieval and Structuring to Unleashing the power of LLM

Empowering LLMs: Prompting, Fine-Tuning vs. RAG

6

7 of 11

From RAG to RAS: A Retrieving-Structuring-Reasoning Framework

7

8 of 11

Outline

  • 0: An Introduction to LLM, RAG, and RAS
  • (9:00-9:50) Retrieving: Enhancing Power of Retrieval with Structures
  • (10:00-10:50) Reasoning: Enhancing LLM Reasoning with Structures
  • (11:00-11:50) Agents: Enhancing LLM Agents with Tools and Search

8

9 of 11

Un-used slides

9

10 of 11

  • Language models are impressive for many NLP tasks!
    • Automatically learned/trained from massive world-scale data
    • Human-like performance for some NLP tasks and beyond
      • Story generation, QA, summarization, code generation, …
  • But LLM still hallucinates (An example from ChatGPT)
    • Q: “List the title, venue and authors of a highly cited paper on heterogeneous information network”
    • A: “Heterogeneous Information Network Analysis and Mining: A Comprehensive Survey”, by Jiawei Han, Micheline Kamber, and Jian Pei, KDD 2011 (cited over 4,300 times as of March 2023)
  • There is no easy way for ChatGPT alone to do it right!
  • Why does ChatGPT hallucinates?―Because it does not really understand the text!
  • Solution?―We need structured knowledge on texts for LLM to understand the texts!
  • How to get such structured knowledge?―Text mining or mining structures from text!

LLM May Hallucinate, but RAG + Structuring will Help

10

11 of 11

  • RAG (Retrieval Augmented Generation)
    • Role: Incorporating external data and knowledge to LLM
    • Challenges
      • Data quality: Retrieving theme-relevant data without annotation/supervision?
      • Structure: How to incorporate structures and structured knowledge into LLM?
  • RAS (Retrieval and Structuring): Our proposed approach
    • Retrieving by corpus-based analysis: Taxonomy, topics & text classification
    • Structuring by entity/relation recognition, typing and knowledge graph construction
        • Ontology-guided, fine-grained entity-recognition and typing
        • Ontology-guided relation extraction and KG construction
    • Theme-focused and LLM-guided exploration

RAG vs. Retrieval and Structuring

11