Thresholds Module

gaico.thresholds.apply_thresholds

apply_thresholds(results, thresholds=None)

Apply thresholds to scores for single pair or batch of generated and reference texts. Type ThresholdedResults is a dictionary where keys are metric names and values are either scores or dictionaries with scores. Specifically, it is of type Dict[str, float | Any].

Parameters:

Name	Type	Description	Default
results	ThresholdedResults | List[ThresholdedResults]	Either a single dictionary of scores or a list of score dictionaries Single: {"BLEU": 0.6, "JSD": 0.1} Batch: [{"BLEU": 0.6, "JSD": 0.1}, {"BLEU": 0.4, "JSD": 0.2}]	required
thresholds	Optional[Dict[str, float]]	Dictionary of metric names to threshold values. Defaults to get_default_thresholds() if not provided.	None

Returns:

Type	Description
ThresholdedResults | List[ThresholdedResults]	For single input, returns a dictionary. For batch input, returns a list. Single: {"BLEU": {"score": 0.6, "threshold_applied": 0.5, "passed_threshold": True}, ...} Batch: [{"BLEU": {"score": 0.6, ...}, ...}, {"BLEU": {"score": 0.4, ...}, ...}]

gaico.thresholds.get_default_thresholds

get_default_thresholds()

Returns the default thresholds for each metric. This is useful for testing and can be overridden by user-defined thresholds.

Returns:

Type	Description
Dict[str, float]	A dictionary of default thresholds for each metric. e.g., {"BLEU": 0.5, "JSD": 0.5}

gaico.thresholds.calculate_pass_fail_percent

calculate_pass_fail_percent(results, thresholds=None)

Calculate pass/fail percentages for each metric across results.

Parameters:

Name	Type	Description	Default
results	Dict[str, List[float]]	Dictionary where keys are metric names and values are lists of scores	required
thresholds	Optional[Dict[str, float]]	Dictionary of thresholds for each metric	None

Returns:

Type	Description
Dict[str, Dict[str, float | int]]	Dictionary with metric names as keys and pass/fail statistics as values

gaico.thresholds.DEFAULT_THRESHOLD module-attribute

DEFAULT_THRESHOLD = {'BLEU': 0.5, 'ROUGE': 0.5, 'JSD': 0.5, 'BERTScore': 0.5, 'Jaccard': 0.5, 'Cosine': 0.5, 'Levenshtein': 0.5, 'SequenceMatcher': 0.5, 'ActionSequenceDiff': 0.5, 'TimeSeriesElementDiff': 0.5, 'TimeSeriesDTW': 0.5, 'SSIM': 0.5, 'PSNR': 0.5, 'AudioSNR': 0.5, 'SpectrogramDistance': 0.5}