Abstract

Context: There is an increasing need to perform controlled experiments in software engineering.

Objective: This systematic review (SR) shows the current state of the art on the tools and infrastructures that provide automated support for controlled experiments in software engineering.

Method: We performed manual searches in journals and conferences proceedings for papers describing supporting tools and environments to conduct controlled experiments. Results: We found and reviewed 25 primary studies according to inclusion and exclusion criteria, resulting in 15 relevant studies.

Conclusion: There are few supporting environments for conducting controlled experiments, despite of the increasing demand for this kind of study in software engineering. We also highlight many limitations of these tools, which configures great potential for future research.

Bibtex

@inproceedings{FreireCNMKAS13,
  author    = {Mar{\'{\i}}lia Aranha Freire and
               Daniel Alencar da Costa and
               Edmilson Campos Neto and
               Tain{\'{a}} Medeiros and
               Uir{\'{a}} Kulesza and
               Eduardo Aranha and
               S{\'{e}}rgio Soares},
  title     = {Automated Support for Controlled Experiments in Software Engineering:
               {A} Systematic Review },
  booktitle = {The 25th International Conference on Software Engineering and Knowledge
               Engineering, Boston, MA, USA, June 27-29, 2013.},
  pages     = {504--509},
  year      = {2013},
}