Published using Google Docs
ISC procedures and policies
Updated automatically every 5 minutes

ISC procedures and policies

Author: Jakub “Kuba” Łącki with input from ISC

Last updated: April 2022

Status: Approved by the ISC

Scope

Procedures

Task selection process

Updating test data and rejudging

Time extensions

Reverse-engineering the test data

Scope

This document outlines some of the procedures and policies that the ISC has been following in the last few years. The goal of this document is to:

The policies described in this document are not binding, and the ISC may decide to deviate from them in cases that are deemed exceptional by the ISC.

Procedures

Task selection process

  1. The submitted tasks are reviewed by the HSC, which identifies a longlist of 20-30 tasks.
  1. In most cases only tasks that are clearly unsuitable for the IOI are removed at this step (clearly out of syllabus, extremely standard/textbook problems, task proposals without a description of the intended solution, etc)
  1. These tasks are presented to the ISC, and graded according to the following criteria:
  1. Difficulty. The measure of difficulty is the expected number of perfect scores
  1. Solution novelty
  1. Task novelty
  1. Subtaskability
  1. Is the task accessible?
  1. Once the tasks have been graded, the final step is to choose a set of 6 tasks for the two contest days plus (typically 3) backup tasks. The main constraints are:
  1. More novel tasks are generally preferred.
  2. Each contest day should consist of (sub)tasks of varying difficulty.
  3. The topics covered by the tasks should be diverse.
  4. Each contest task should have a suitable replacement. Note that a backup task can serve as a replacement for multiple contest tasks.
  5. The total length of the task statements should be reasonable.
  6. The total length of the solutions of all tasks in a day should be reasonable, i.e., there should be at most one task with a lengthy model solution.

Updating test data and rejudging

The SC may occasionally update the test data and/or rejudge some submissions. This may lead to some scores changing. In particular, nondeterministic solutions may get a different number of points (either lower or higher) after every rejudge.

Rejudging submissions is often accompanied by changing the test data and/or graders. The following rules are typically followed:

Time extensions

A contestant may lose time during the contest due to unforeseen issues. This is always an unfortunate event, which introduces some amount of unfairness into the contest. A time extension can compensate for the time lost, but is also not an entirely fair solution, given that:

We believe that a contestant can be given a time extension, when ALL of the following conditions are met:

The above rules apply to the cases when the disruption is resolved at least 10 minutes before the end of the contest. The remaining cases are handled on a case-by-case basis.

Reverse-engineering the test data

IOI 2021 rules describe the following cheating behavior:

contestants must not reverse engineer the test data in order to solve the problems in highly test-data-dependent manners. One example of such behavior is using the feedback system to extract the test data and then applying this knowledge to build solutions adapted to the specific test cases in the grading system. This behavior would be considered cheating only if a contestant submits a solution that would solve significantly fewer test cases correctly if the test data were replaced by an equivalent set of test cases (e.g., one generated with a different random seed).

Note that “and” highlighted in the above paragraph is key in the formulation of the rule. In particular, extracting some information from the test data (e.g. using it for debugging) is acceptable, as long as the extracted knowledge is not used to solve the problem for the particular test cases provided, and not in full generality.

We acknowledge that this rule is (deliberately) not very specific. In order to better explain ISC stand, we provide examples that the ISC does/does not consider cheating.

Examples of allowed behaviors:

Examples of cheating/highly suspicious behavior: