1 of 9

COCOA: Confidence- and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in LLMs

Anant Khandelwal, Manish Gupta, Puneet Agrawal

Microsoft India

2 of 9

Why is it important to care about conflicts between LLM’s parametric memory and external context?

  •  

Anant Khandelwal, Manish Gupta, Puneet Agrawal. CoCoA: Confidence- and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in LLMs. EMNLP 2025.

2

3 of 9

How do various conflict resolution methods work?

  •  

Anant Khandelwal, Manish Gupta, Puneet Agrawal. CoCoA: Confidence- and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in LLMs. EMNLP 2025.

  •  

3

4 of 9

How does CoCoA work?

  •  

Anant Khandelwal, Manish Gupta, Puneet Agrawal. CoCoA: Confidence- and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in LLMs. EMNLP 2025.

  •  

4

5 of 9

How does CoCoA perform?

Anant Khandelwal, Manish Gupta, Puneet Agrawal. CoCoA: Confidence- and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in LLMs. EMNLP 2025.

  • Long form QA: CLAPNQ, ExpertQA, HAGRID, ELI5-WebGPT, QuoteSum
    • COCOA shows best relevance (ROUGE-L) and factuality (Faithfulness)

​

5

Exact match on QA (gold contexts provided by each dataset)

Summarization (Llama3-70B)

6 of 9

How sensitive is CoCoA to high conflict vs low conflict cases?

Anant Khandelwal, Manish Gupta, Puneet Agrawal. CoCoA: Confidence- and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in LLMs. EMNLP 2025.

6

High (NQ-SWAP) vs. Low (NQ-SYNTH) Conflict

 

7 of 9

How sensitive is CoCoA to high conflict vs low conflict cases?

Anant Khandelwal, Manish Gupta, Puneet Agrawal. CoCoA: Confidence- and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in LLMs. EMNLP 2025.

7

8 of 9

Qualitative examples showcasing COCOA’s improved factual grounding and conflict resolution

Anant Khandelwal, Manish Gupta, Puneet Agrawal. CoCoA: Confidence- and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in LLMs. EMNLP 2025.

8

9 of 9

Summary

  •  

Anant Khandelwal, Manish Gupta, Puneet Agrawal. CoCoA: Confidence- and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in LLMs. EMNLP 2025.

​

9