Skip to content

Adaptive stopping

AdaptiveStopping dataclass #

AdaptiveStopping(
    max_responses: int | None = None,
    min_confidence: float | None = None,
    ambiguity_threshold: float | None = None,
)

Stops collecting responses on a compare datapoint once the outcome is clear.

Responses are collected until it is clear which option wins, until the two options are so close that the datapoint is called ambiguous, or until max_responses is reached. Clear pairs finish after a few responses; close pairs use more.

It is the default for compare job definitions. Pass it as responses_per_datapoint to change a parameter, or assign it to RapidataJobDefinition.responses_per_datapoint to switch an existing compare definition. Other job types raise ValueError.

Leave a parameter as None to use its default. A saved job definition always carries the resolved values.

Parameters:

Name Type Description Default
max_responses int

The most responses collected per datapoint. Default: 100. Cost and progress count this many responses per datapoint.

None
min_confidence float

How certain the result must be before a winner is called, between 0 and 1 (exclusive). Default: 0.9.

None
ambiguity_threshold float

How close the two options must be before the datapoint is called ambiguous, as a normalized Gini impurity between 0 and 1 (exclusive). Higher is stricter: it needs a closer tie, so it costs more responses. Default: 0.64, which matches a 20%-80% split.

None

Attributes:

Name Type Description
max_responses int | None

The response cap, or None for the default.

min_confidence float | None

The confidence level, or None for the default.

ambiguity_threshold float | None

The ambiguity threshold, or None for the default.