Semantic Similarity Metrics

GAICo provides a semantic similarity metric to evaluate the semantic similarity between text inputs.

This metric is useful for tasks such as semantic text matching, paraphrase detection, and understanding the meaning of text in a more nuanced way.

gaico.metrics.semantic_similarity_metrics.BERTScore

Bases: TextualMetric

This class provides methods to calculate BERTScore for individual sentence pairs and for batches of sentences. It uses the BERTScore library to calculate precision, recall, and F1 scores.

__init__

__init__(model_type='bert-base-uncased', output_val=None, num_layers=8, batch_size=64, additional_params=None)

Initialize the BERTScore metric.

Parameters:

Name	Type	Description	Default
model_type	str	The BERT model to use, defaults to "bert-base-uncased"	'bert-base-uncased'
output_val	Optional[List[str]]	The output value to return, defaults to None Should be one of "precision", "recall", or "f1" to return a single score of type float Wrap in a list to return multiple scores of type dict Default returns a dictionary of all scores. Equivalent to passing ["precision", "recall", "f1"]	None
num_layers	int	Number of layers to use from BERT, defaults to 8	8
batch_size	int	Batch size for processing, defaults to 64	64
additional_params	Optional[Dict[str, Any]]	Additional parameters to pass to the BERTScorer class from the bert_score library, defaults to None Default only passes the model_type, num_layers, and batch_size	None