Skip to content

Classify early stopping referee

ClassifyEarlyStoppingReferee #

ClassifyEarlyStoppingReferee(
    threshold: float = 0.999, max_vote_count: int = 100
)

Bases: Referee

A referee that stops the task when confidence in the winning category exceeds a threshold.

This referee implements an early stopping mechanism for classification tasks. It terminates the task when the confidence in the leading category surpasses a specified threshold or when the maximum number of votes is reached.

The threshold behaves logarithmically, meaning small increments (e.g., from 0.99 to 0.999) can significantly impact the stopping criteria.

Attributes:

Name Type Description
threshold float

The confidence threshold for early stopping.
max_vote_count int

The maximum number of votes allowed before stopping.

Parameters:

Name Type Description Default
threshold float

The confidence threshold for early stopping. Defaults to 0.999.

 0.999
max_vote_count int

The maximum number of votes allowed before stopping. Defaults to 100.

 100
Source code in src/rapidata/rapidata_client/referee/classify_early_stopping_referee.py 
23
24
25
26
27
28
29
30
31
32
33
def __init__(self, threshold: float = 0.999, max_vote_count: int = 100):
    """Initialize the ClassifyEarlyStoppingReferee.

    Args:
        threshold (float, optional): The confidence threshold for early stopping.
            Defaults to 0.999.
        max_vote_count (int, optional): The maximum number of votes allowed
            before stopping. Defaults to 100.
    """
    self.threshold = threshold
    self.max_vote_count = max_vote_count