Jesús Giménez

Asiya has been designed to assist both system and metric developers by offering a rich repository of metrics and meta-metrics.

We compiled a rich set of specialized similarity metrics operating at different linguistic levels (lexical, syntactic and semantic). We also studied how the scores conferred by different metrics may be integrated into a single measure of quality, without having to adjust their relative importance.

Asiya is an evolution, extension, refactoring, and a replacement for its predecessor, IQMT.


Technical Requirements (optional):
  • Individual metrics (BLEU, NIST, GTM, ROUGE, METEOR, ...)
  • Linguistic processors:
    - C&C Tools
    - SwiRL
    - BIOS
    - Charniak Parser
    - MINIPAR dependency parser
    - SVMTool
Asiya allows for evaluation of translation quality aspects at different linguistic levels. Metric scores can also be combined into a single measure of quality


  • Jesús Giménez and Lluís Màrquez. Asiya: An Open Toolkit for Automatic Machine Translation (Meta-)Evaluation. The Prague Bulletin of Mathematical Linguistics. Number 94, pages 77-86, 2010.


Scroll to Top