Textual Entailment Annotation
For each pair of sentences, assume that the first sentence (S1) is true, describes a real scenario, or expresses an opinion. Using your best judgment, indicate how likely it is that the second sentence (S2) is also true, describes the same scenario, or expresses the same opinion. Several examples are given below

Example 1: In the below example, the pair is extremely likely (rating of 5) because we can be very confident that if a person is ‘‘on a beach’’ than that person is ‘‘outside’’. 
S1: A woman is on a beach with her feet in the water. 
S2: The woman is outside. 

Example 2: In the below example, the pair is not at all likely likely (rating of 1) because we can be very confident that if a person is ‘‘on a beach’’ then that person is NOT ‘‘in her living room’’. 
S1: A woman is on a beach with her feet in the water. 
S2: The woman is in her living room.

Example 3: In the below example, the pair is in the middle (rating of 3) because knowing that woman is on the beach does not give us any information about the color of her hair and so we cannot reasonably make a judgment about whether or not her hair is brown. 
S1: A woman is on a beach with her feet in the water. 
S2: The woman has brown hair.

Examples and the idea from this annotation task come from Pavlick and Kwiatkowski (2019), Inherent Disagreements in Human Textual Inferences
Sign in to Google to save your progress. Learn more
Sentence Pair 1
S1: Someone confessed that a particular thing happened.
S2: That thing happened
Not likely at all
Extremely likely
Clear selection
Sentence Pair 2
S1: A guy on the street watching someone.
S2: The person is male.
Not likely at all
Extremely likely
Clear selection
Sentence Pair 3
S1: Paula swatted the fly.
S2: The swatting happened in a forceful manner.
Not likely at all
Extremely likely
Clear selection
Submit
Clear form
This form was created inside of Middlebury. Report Abuse