wehub-resource-sync
2.2 KiB
2.2 KiB
PaddleNLP Metrics API
Currently, PaddleNLP provides the following model evaluation metrics:
| Metric | Description | API |
|---|---|---|
| Perplexity | Perplexity, commonly used to evaluate the quality of language models, and applicable to tasks such as machine translation and text generation. | paddlenlp.metrics.Perplexity |
| BLEU (BiLingual Evaluation Understudy) | A commonly used evaluation metric for machine translation. | paddlenlp.metrics.BLEU |
| Rouge (Recall-Oriented Understudy for Gisting Evaluation) | A metric for evaluating automatic summarization and machine translation. | paddlenlp.metrics.RougeL, paddlenlp.metrics.RougeN |
| AccuracyAndF1 | Accuracy and F1-score, applicable to MRPC and QQP tasks in GLUE. | paddlenlp.metrics.AccuracyAndF1 |
| PearsonAndSpearman | Pearson correlation coefficient and Spearman's rank correlation coefficient, applicable to the STS-B task in GLUE. | paddlenlp.metrics.PearsonAndSpearman |
| Mcc (Matthews correlation coefficient) | Matthews correlation coefficient, used to measure the classification performance of binary classification tasks. Applicable to the CoLA task in GLUE. | paddlenlp.metrics.Mcc |
| ChunkEvaluator | Computes precision, recall, and F1-score for chunk detection. Commonly used in sequence labeling tasks such as Named Entity Recognition (NER). | paddlenlp.metrics.ChunkEvaluator |
| Squad Evaluation | Evaluation metrics for SQuAD and DuReader-robust. | paddlenlp.metrics.compute_predictions, paddlenlp.metrics.squad_evaluate |
| Distinct | A diversity metric commonly used to measure the form diversity of sentences generated by text generation models. | paddlenlp.metrics.Distinct |