Evaluating Automated Theorem Provers Using Adimen-SUMO

9 pages•Published: March 1, 2017

Javier Álvez, Paqui Lucio and German Rigau

Abstract

We report on the results of evaluating the performance automated theorem provers using \ADIMENSUMO{}. The evaluation follows the adaptation of the methodology based on competency questions \cite{GrF95} to the framework of first-order logic, which is presented in \cite{ALR15}, and is applied to \ADIMENSUMO{} \cite{ALR12}. The set of competency questions used for this evaluation has been semi-automatically generated from a small set of semantic patterns and the mapping of \WORDNET{} to \SUMO{}, also introduced in \cite{ALR15}. Our experimental results demonstrate that improved versions of the proposed set of competency questions could be really valuable for the development of automated theorem provers.

Keyphrases: benchmark, ontologies, system evaluation

In: Laura Kovacs and Andrei Voronkov (editors). Vampire 2016. Proceedings of the 3rd Vampire Workshop, vol 44, pages 74-82.

Links:	https://easychair.org/publications/paper/4LFw
	https://doi.org/10.29007/hplh

BibTeX entry

@inproceedings{Vampire2016:Evaluating_Automated_Theorem_Provers,
  author    = {Javier Álvez and Paqui Lucio and German Rigau},
  title     = {Evaluating Automated Theorem Provers Using Adimen-SUMO},
  booktitle = {Vampire 2016. Proceedings of the 3rd Vampire Workshop},
  editor    = {Laura Kovacs and Andrei Voronkov},
  series    = {EPiC Series in Computing},
  volume    = {44},
  publisher = {EasyChair},
  bibsource = {EasyChair, https://easychair.org},
  issn      = {2398-7340},
  url       = {/publications/paper/4LFw},
  doi       = {10.29007/hplh},
  pages     = {74-82},
  year      = {2017}}

Download PDF Open PDF in browser