1 of 26

When the Chain Breaks: �Interactive Diagnosis of LLM Chain-of-Thought Reasoning Errors��

Shiwei Chen¹, Niruthikka Sritharan¹, Xiaolin Wen¹, Chenxi Zhang¹, Xingbo Wang², Yong Wang¹

1 – Nanyang Technological University, Singapore

2 – Bosch Research North America, Sunnyvale, California, USA

2 of 26

The Problem

2

Background

Error Detection Pipeline

Visual Interface

Evaluation

87 steps. one question.

find the “one” wrong step?

A Deepseek sample without searching ability

3 of 26

The Problem

3

Verbosity:

Easily hundreds of steps
Complex logic

Background

Error Detection Pipeline

Visual Interface

Evaluation

Unreliability:

Contain subtle mistakes
Answer overconfidently

✕ Hard to build a coherent mental model

✕ Cause misplaced trust

✓ Explore many possible paths

✓ Explicitly state reasoning steps

4 of 26

Research Gap

Hidden by default

Click and show summary

4

Background

Error Detection Pipeline

Visual Interface

Evaluation

It shows only high level description and loses context.

It risks overconfidence and over-reliance.

5 of 26

Formative Study

5

Background

Error Detection Pipeline

Visual Interface

Evaluation

Findings

F1

Users prioritize different steps.

F2

Long CoTs cause cognitive overload.

F3

Nonlinear logic kills traceability.

F4

Limited verification resources.

derive

R1

Automatically detect potential errors.

R2

Provide an overview of the overall reasoning.

R3

Support traceable navigation across steps.

R4

Visualize how they propagate.

R5

Enable cause analysis of individual errors.

Design requirements

9 participants with intense LLM usage

Each was given an long CoT sample from the Deltabench;

Process

What is their practice?

What is the main difficulty?

6 of 26

System Overview

6

Background

Error Detection Pipeline

Visual Interface

Evaluation

ReasonDiag = automatic detection + interactive diagnosis

Error-Detection Pipeline

R1

1

Premise Tree Generation

2

Error Detection

Fact Check

suggestions

human verification

Interactive Interface

R2–R5

Overview — step distribution + error propagation

Section View — hierarchy + premises + error causes

Original CoT — full text, linked on demand

Logic Check

7 of 26

Error Detection Pipeline

7

Background

Error Detection Pipeline

Visual Interface

Evaluation

Step 1. Premise Tree Generation

Step 2. Error Detection

8 of 26

Step 1. Premise Tree Generation

8

Background

Error Detection Pipeline

Visual Interface

Evaluation

9 of 26

Step 1. Premise Tree Generation

9

Background

Error Detection Pipeline

Visual Interface

Evaluation

ID

S1

S2

S3

S4

STEP

The question asks for the time span between the launch of the Hubble Space Telescope and 2025.

The Hubble Space Telescope was launched in 1992, when NASA sent it into orbit.

To determine the number of years that have passed, we subtract the launch year from the target year.

So we compute 2025 – 1992.

……

10 of 26

Step 1. Premise Tree Generation

10

Background

Error Detection Pipeline

Visual Interface

Evaluation

ID

S1

S2

S3

S4

STEP

The question asks for the time span between the launch of the Hubble Space Telescope and 2025.

The Hubble Space Telescope was launched in 1992, when NASA sent it into orbit.

To determine the number of years that have passed, we subtract the launch year from the target year.

So we compute 2025 – 1992.

……

TYPE

VERIFIABILITY

11 of 26

Step 1. Premise Tree Generation

11

Background

Error Detection Pipeline

Visual Interface

Evaluation

12 of 26

Step 2. Error Detection

12

Background

Error Detection Pipeline

Visual Interface

Evaluation

13 of 26

Step 2.1 Factual Error

13

Background

Error Detection Pipeline

Visual Interface

Evaluation

14 of 26

Step 2.2 Logical Error

14

Background

Error Detection Pipeline

Visual Interface

Evaluation

15 of 26

Step 2.3 Propagated Error

15

Background

Error Detection Pipeline

Visual Interface

Evaluation

16 of 26

ReasonDiag UI

16

Background

Error Detection Pipeline

Visual Interface

Evaluation

A DeltaBench math problem

17 of 26

ReasonDiag UI

17

Background

Error Detection Pipeline

Visual Interface

Evaluation

Section View

Overview

18 of 26

Overview

18

Background

Error Detection Pipeline

Visual Interface

Evaluation

error source --- propagated error

Error

Uncertainty

Link

Node

“uncertained” step --- uncertain step

an erroneous step

an propagated error step

uncertain step

19 of 26

Retroactive reasoning patterns

19

Background

Error Detection Pipeline

Visual Interface

Evaluation

20 of 26

Section View

20

Background

Error Detection Pipeline

Visual Interface

Evaluation

21 of 26

Technical Evaluation

21

Background

Error Detection Pipeline

Visual Interface

Evaluation

Dataset:

13 CoT samples from Deltabench

2,030 sentence-level annotations

User Interviews

16 LLM users with diverse background (aged 18-31)

Effective,

Well-designed,

Good usability

Highest Recall

Covering the most errors

22 of 26

Takeaway

22

CoT reasoning traces generated by LLMs can be problematic

ReasonDiag is an attempt to use visualization help general users easily identify CoT reasoning errors .

NTU VIDA Lab

Case Demo: csw0109.github.io/reasondiag-demo/

Dataset: huggingface.co/datasets/CoTDiagnosis/DeltaBench_CoT_Diagnosis/

Thanks for listening !

csw0109.github.io

🌟 Looking for PhD position

23 of 26

Backup Pages

24 of 26

Appendix: Precision of the diagnose pipeline

44.48% of false positives come from error propagation.

When a few early core steps are misclassified, many downstream steps inherit the flag.

In practice the interface mitigates this: a burst of flagged steps traces back to a small set of core steps that can check quickly.

User prefer AI system with higher recall than precision [1]
Future: agreement across multiple verifiers (our pipeline + prompt-based) to cut false positives while preserving recall.

24

many flagged steps

[1] Kocielnik R, Amershi S, Bennett P N. Will you accept an imperfect AI? exploring designs for adjusting end-user expectations of AI systems[C]//Proceedings of the 2019 CHI conference on human factors in computing systems. 2019: 1-14.

1 core error

25 of 26

Appendix: Limitations and future works

25

Visual clutter at scale

Dense cross-links obscure structure on 100+ step traces.

→ adaptive granularity: aggregate simple ops into composite nodes.

The semantic gap

Locating a wrong step is relatively easy; explaining WHY a logic error is wrong is not.

→ AI-assisted explanations + interactive debugging (edit / fix steps).

Precision

Many false positives come from propagation of a few core errors.

→ agreement across multiple verification methods, keeping high recall.

Generalizability

Validated on verifiable domains (math, logic, retrieval).

→ softer encodings for creative / legal text: semantic drift, plausibility; agentic pipline.

26 of 26

Appendix: Need of Diagnose CoT

Reasoning is relevant to the result:

High correlation with the reasoning process result with the final output.

CoT is naturally interpretable:

Easier for user to interpret CoT than internal components

“Judge a person by the action not the thoughts”

26