1 of 1

Timeline Summarization Task

Datasets and Tasks

Method1: Sequential Temporal Chunking

Method2: KG+LLMs

Timeline Summarization in the Era of LLMs

Daivik Sojitra1, Raghav Jain1

Sriparna Saha1, Adam Jatowt2, Manish Gupta3

1IIT Patna, India 2University of Innsbruck, Austria 3Microsoft, India

  • TimeLine17: Timelines from popular news agencies such as CNN, BBC, NBC News, etc.
  • Crisis: News articles on the Egypt Revolution, Syria War, Yemen Crisis, and Libya War. Timelines from 24 news agencies
  • Entities: Timelines from CNN Fast Facts.

Models and Metrics

Method3: Time Ranker

    • Sort articles by time; grouped into chunks; summarize using LLM.
    • Median date from articles in chunk is assigned to chunk summary.
    • FLAN-T5 is best for Crises and Entities; GPT-4 performs well for TimeLine17.
    • Better than baselines only for Crisis Concat F1.
    • Use rebel-large model for relation extraction from raw text to create a temporal KG.
    • Use temporal relation extraction techniques to extract temporal info from articles.
    • Temporal KG: Head, Type, Tail, and Date.
    • GPT-4 is best for everything except Crisis Concat-F1 where FLAN-T5 is better.
    • Worse Concat F1 compared to Sequential Temporal Chunking but better Align F1, Agree F1 and Date F1.
    • Use HeidelTime to extract sentences with temporal expressions from article text (e.g., “19 November”, or “last Friday”).
    • Use date ranker approach to select top 150 dates with highest numbers of temporal mentions.
    • FLAN-T5 is best except for Date-F1 where Long-LLaMA is the best.
    • Better than other 2 LLM methods.
    • Compared to baselines, Agree-F1 and Align-F1 are competitive but not better.

CNN’s early reporting on the coronavirus

December 31, 2019 - Cases of pneumonia detected in Wuhan, China, are first reported to WHO. During this reported period, the virus is unknown. The cases occur between December 12 and December 29, according to Wuhan Municipal Health.

January 1, 2020 - Chinese health authorities close the Huanan Seafood Wholesale Market after it is discovered that wild animals sold there may be the source of the virus.

January 5, 2020 - China announces that the unknown pneumonia cases in Wuhan are not SARS or MERS. In a statement, the Wuhan Municipal Health Commission says a retrospective probe into the outbreak has been initiated.

January 7, 2020 - Chinese authorities confirm that they have identified the virus as a novel coronavirus, initially named 2019-nCoV by WHO.

January 11, 2020 - The Wuhan Municipal Health Commission announces the first death caused by the coronavirus. A 61-year-old man, exposed to the virus at the seafood market, died on January 9 after respiratory failure caused by severe pneumonia.

January 17, 2020 - Chinese health officials confirm that a second person has died in China. The United States responds to the outbreak by implementing screenings for symptoms at airports in San Francisco, New York and Los Angeles.

  • Summarize multiple documents using a timeline: significant events and their corresponding dates.
  • Manual timelines: Expensive and cumbersome
  • Current Methods: Event graph approaches; Graph neural network-based approaches
    • How about LLMs?

Sequential Temporal Chunking

KG+LLMs

Time Ranker