An Empirical Methodological Study of Evaluation Methods Applied to Educational Timetabling Visualizations

EasyChair Preprint 4425

Abstract

The conception, and usage, of methods designed to evaluate information visualizations is a challenge that goes along with the development of these visualizations. In the scientific literature there is a myriad of proposals for such methods. However, none of them was able to pacify the field or establish itself as a "de facto" standard, due to difficulties like: (a) the complexity of its usage; (b) high financial and time costs; and (c) the need of a large number of raters to guarantee the reliability of the results. One way to circumvent such adversities is the usage of Heuristic-Based Evaluations thanks to its simplicity, low cost and speed of application and the quality of reached results. This article intends to conduct an empirical methodological study about the use of three heuristic-based methods (Zuk bet al., Forsell & Johansson and Wall et al.) for evaluation of visualizations in the context of Educational Timetabling Problems. Five different visualizations, extracted from literature, were evaluated using the original methods and versions modifiedby the authors (where an importance factor were assigned to each statement being evaluated, as well as the level of confidence of the rater in his/her appointment) in order to improve their efficiency when measuring the quality of visualizations. The experimental results demonstrated that for the two first heuristics, only the modification on the importance of the statements proved to be (statistically) relevant. For the third one, both factors did not induce different results.

Keyphrases: Educational Timetabling Problems, Heuristic Evaluation, Information Visualization Evaluation, NP-hard, Problem Domain, Timetabling Problem, combinatorial problems, empirical study, evaluation method, heuristic based evaluation, heuristic based evaluation method, heuristic method, heuristic-based method, information visualization, interactive visualization, rater confidence, user interface, visualization evaluation

@booklet{EasyChair:4425,
  author    = {Wanderley de Souza Alencar and Walid Abdala Rfaei Jradi and Hugo Alexandre Dantas Do Nascimento and Juliana Paula Félix and Fabrizzio Alphonsus Alves de Melo Nunes Soares},
  title     = {An Empirical Methodological Study of Evaluation Methods Applied to Educational Timetabling Visualizations},
  howpublished = {EasyChair Preprint 4425},
  year      = {EasyChair, 2020}}