Evaluating Learnability of software with Users: A Quantitative and Automated Approach to Evaluation

EasyChair Preprint 1827

Abstract

Usability is an important feature of software quality Where the user interaction with the software is concerned. There are various features to evaluate software usability. One of these features is the ability to learn the use of software with users. In this study, a quantitative and automated method for evaluating software learning capability by users was presented. Then, this method was used to evaluate the learning capability of a mobile software. In this evaluation, the criteria for extra activities, missed activities, and the number of errors, based on the history of user interactions with the software, were reviewed over three consecutive weeks. To conduct this study, an empirical experiment was designed and 29 users were asked to use certain functionalities of software for three consecutive weeks.Software interactions with users are automatically recorded. Then, by using the process mining method, the amount of deviation from the optimal way of interacting with the software is evaluated and the rate of improvement is analyzed over several weeks.

Evaluation results show that in the software in question, given the amount of benchmarks achieved within three weeks, learning how to perform some tasks in the second week is such that their re-execution in the third week had no effect on learning. It can also be seen that learning how to perform some other tasks has increased over the course of three weeks.

Keyphrases: Process Mining, Quality, Usability, conformance checking technique, quantitative and automated evaluation, software

@booklet{EasyChair:1827,
  author    = {Mahla Sharani and Mohammad Ali Soleymani and Shiva Vafadar},
  title     = {Evaluating Learnability of software with Users: A Quantitative and Automated Approach to Evaluation},
  howpublished = {EasyChair Preprint 1827},
  year      = {EasyChair, 2019}}