Fraco's Evaluation Dataset

Summary

In pursuance of one of the contributions stated in the thesis titled "Detecting Fragile Comments", we make the data used in evaluation of our approach available here, publicly.

This archive contains the data we used for evaluation of our tool Fraco, a tool to detect fragile comments. We evaluated our tool using 3 different datasets - Development set, Test set A and Test set B. The data presented in this archive is related to the Chapter 7 of the thesis.

Each dataset in this archive is presented using two sets of files - the first set contains the manually annotated data used to evaluate Fraco's performance and the second set contains the data used to compare Fraco's performance with Eccore (Eclipse's refactoring tool's in-comment text replacement feature).

The data in Test Set B is annotated by multiple annotators and therefore, each data file in the dataset contains annotations done by both the annotators.

Development Set

Test Set A

Test Set B