FaultExpSuits is a set of collections which includes bug reports and scripts preparing resources for Fault Localization experiment. This documents explains this repository, what we include and how to use this data.
This is the top-level directory of FaultExpSuits. The directory structure is as follows:
-
techniques: This folder includes source codes and executable files of previous techniques such as BugLocator, AmaLgam, Locus and more. We revised the source codes to output result with same format.
- releases: The executable files and environment settings for each techniques get together here.
-
analysis: The execution result of previous techniques which is refind for scripts in forlder "scripts > analysis".
-
scripts: Python scripts to prepare resources for Fault Localization, to execute previous techniques and to organize the results.
-
packing.sh: Shell script to pack resource data per each subject.
-
unpacking.sh: Shell script to unpack resource data per each subject.
We used the 5 old subjects and the 46 new subjects in the below table. The subjects classified into 6 groups to manage them (The Previous group is old subjects). Each of the archive contains bug reports, bug report repositories that we made, cloned git repository and metadata of them that we made.
We have got all the result of the paper. This scirpts has 5 folders.
* repository: Scripts to prepare the resources to execute each technique.
* results: Scripts to collect the execution results of each technique and export to Excel.
* analysis: Scripts to analysis for the result of each technique and features extracted from resources.
We applied Mann-Whitney U test, Pearson correlation and so on.
* commons: Scripts to managing subjects and common functions.
* utils: Personal libraries for experiments.
All the experiments are executed in Ubuntu 16.04 LTS.
- Clone the repository by using the following command. (We cloned into the "Suits" directory.)
$ sudo apt-get update
$ sudo apt-get install git
$ git clone https://github.com/irblsensitivity/irblsensitivity.git Suits
- We used java 8 and python 2.7
- If you have java and python in your computer, please skip this section.
// install java
$ sudo apt-get install python-software-properties
$ sudo add-apt-repository ppa:webupd8team/java
$ sudo apt-get update
$ sudo apt-get install oracle-java8-installer
// install python
$ sudo add-apt-repository ppa:fkrull/deadsnakes
$ sudo apt-get update
$ sudo apt-get install python2.7 python
$ sudo apt-get install python-pip
// install python dependencies
$ pip install numpy scipy matplotlib pytz GitPython bs4 xlswriter nltk
- Download all subjects from the Subjects table and save them in the cloned repository path
- In our case, we save it to the Suits/_archives directory.
- Each subject must be stored in the group directory to which it belongs.
- Finally, unpacking all archives by using the unpacking.sh script.
$ cd Suits
Suits$ mkdir _archives
Suits$ cd _archives
Suits/_archives$ mkdir Apache
Suits/_archives$ cd Apache
Suits/_archives/Apache$ wget -O CAMEL.tar "https://drive.google.com/uc?export=download&id=0B78iVP5pcTfKdEZZZnJrWmZxWjg"
....work recursively....
Suits$ mkdir data
Suits$ ./unpacking.sh _archives data
- In the file scripts/commons/Subject.py, there are variables that stores a resource PATH information as a string.
- The variables are Subjects.root, Subjects.root_result, and Subjects.root_feature.
- You should change the variables according to cloned path of this repository.
- We used multiple versions of source code for the experiment.
- The script, launcher_GitInflator.py clones a git repositories and inflates it into the multiple versions which are used in the experiment.
- Since the provided archives have only a git repository, you need to inflate also.
- The version information that needs to inflate exists in the Python script and provided archives.
- The information for the inflation are in the provided scripts and archives. See a file versions.txt in any subject's data directory.
Suits$ cd scripts
Suits/scripts$ python launcher_GitInflator.py
- We need to build a repository for the bug reports with pre-crawled bug reports.
- We are already providing the result of this works in provided subject's archives.
Suits/scripts$ python launcher_repoMaker.py
Suits/scripts$ python launcher_DupRepo.py
- The script of Counting.py makes a count information for bug and source code.
Suits/scripts$ python Counting.py
-
To get the result of each technique, you can use scripts/launcher_Tool.py.
-
Preparing step
- You need to set the PATHs and JavaOptions in the launcher_Tool.py file.
- Open the file, launcher_Tool.py and check the following variables
- ProgramPATH: Set the directory path which contains the release files of the IRBL techniques. (ex. u'~/Suits/techniques/releases/')
- OutputPATH: Set the result path to save output of each technique (ex. u'~/Suits/expresults/')
- JavaOptions: Set the java command options. (ex. '-Xms512m -Xmx4000m')
- JavaOptions_Locus: Set the java options for Locus. Because Locus need a large memory, we separated the option. (ex. '-Xms512m -Xmx4000m')
-
The script executes 6 techniques for all subjects.
-
The script basically works for the multiple versions of bug repository and each of the related source codes.
-
Options
- -w : [necessary] With this option, users can set the ID for each experiment, and each ID is also used as a directory name to store the execution results of each Technique. Additionally, if the name starts with "Old", this script works for the previous data, otherwise works for the new data.
- -g : A specific group. With this option, the script works for the subjects in the specified group.
- -p : A specific subject. To use this option, you should specify the group name.
- -t : A specific technique. With this option, the script makes results of specified technique.
- -v : A specific version. With this option, the script works for the specified version of source code.
- -s: Single version mode, With this option, the script works for the only latest source code.
- -m: With this option, the bug repositories created by combining the text of duplicate bug report pairs are used instead of the normal one.
-
Examples
Suits/scripts$ python launcher_Tool.py -w NewData
Suits/scripts$ python launcher_Tool.py -w NewDataSingle -s
Suits/scripts$ python launcher_Tool.py -w NewData_Locus -t Locus
Suits/scripts$ python launcher_Tool.py -w NewData_CAMLE -g Apache -p CAMEL
- To execute BLUiR and AmaLgam, you need to install indri.
- Since there are compile problems, we chose indri-5.6 version.
- In the installing process, please memorize the path in the first line in the "make install" log.
(In my case, /usr/local/bin. This is the installed path of indri) - And then, Change Settings.txt file.
- Commands to install indri
// Install g++ and make for indri
$ sudo add-apt-repository ppa:ubuntu-toolchain-r/test
$ sudo apt-get update
$ sudo apt-get install g++
$ sudo apt-get install make
$ sudo apt-get install --reinstall zlibc zlib1g zlib1g-dev
// download and install indri (If you faced an error in the compiling, please try with another version.)
$ wget https://downloads.sourceforge.net/project/lemur/lemur/indri-5.6/indri-5.6.tar.gz
$ tar -xzf indri-5.6.tar.gz
$ cd indri-5.6
$ ./configure
$ make
$ make install
/usr/bin/install -c -m 755 -d /usr/local/bin
/usr/bin/install -c -m 755 -d /usr/local/include
/usr/bin/install -c -m 755 -d /usr/local/include/indri
...
...
/usr/bin/install -c -m 644 Makefile.app /usr/local/share/indri
// changeSettings.txt file
$ cd ~/irblsensitivity/techniques/releases // We assume you cloned our repository to
$ vi Settings.txt
indripath=/usr/local/bin/ <-- edit this value as a the first log of "make install"
We changed previous techniques on Eclipse. But we didn't include eclipse environment files (.metadata folder, .project and .classpath file) in each previous techniques folders.
So, If you want to load these techniques on Eclipse, please follow next sequence.
-
Open Eclipse
-
Make a 'techniques' folder into workplace of Eclipse. Then .metadata folder will be created in 'techniques' folder.
-
On the 'Package Explorer' panel, Open context menu by clicking right mouse button.
-
Select 'Import', Then a pop-up windows will be placed.
-
Except BLUiR project, choose 'General > Projects from Folder or Archive' item and click 'Next' button.
-
Designate project folder in 'techniques' and click 'Finish' button.
-
Then, the project will be loaded and be shown in the Package Explorer.
-
BLUiR is made as Maven project. So, You should import with 'Maven > Existing Maven Project'. And then, just choose project folder. You don't need to change any other options.
-
Especially BLIA project, need to add library JUnit.
- OutputPATH: Set the result path to save output of each technique (ex. u'~/IRBL/expresults/')
- JavaOptions: Set the java command options. (ex. '-Xms512m -Xmx4000m')
- JavaOptions_Locus: Set the java options for Locus. Because Locus need a large memory, we separated the option. (ex. '-Xms512m -Xmx4000m')
-
The script executes 6 techniques for all subjects.
-
The script basically works for the multiple versions of bug repository and each of the related source codes.
-
Options
- -w : [necessary] With this option, users can set the ID for each experiment, and each ID is also used as a directory name to store the execution results of each Technique. Additionally, if the name starts with "Old", this script works for the previous data, otherwise works for the new data.
- -g : A specific group. With this option, the script works for the subjects in the specified group.
- -p : A specific subject. To use this option, you should specify the group name.
- -t : A specific technique. With this option, the script makes results of specified technique.
- -v : A specific version. With this option, the script works for the specified version of source code.
- -s: Single version mode, With this option, the script works for the only latest source code.
- -m: With this option, the bug repositories created by combining the text of duplicate bug report pairs are used instead of the normal one.
-
Examples
IRBL/scripts$ python launcher_Tool.py -w NewData
IRBL/scripts$ python launcher_Tool.py -w NewDataSingle -s
IRBL/scripts$ python launcher_Tool.py -w NewData_Locus -t Locus
IRBL/scripts$ python launcher_Tool.py -w NewData_CAMLE -g Apache -p CAMEL
We changed previous techniques on Eclipse. But we didn't include eclipse environment files (.metadata folder, .project and .classpath file) in each previous techniques folders.
So, If you want to load these techniques on Eclipse, please follow next sequence.
- Open Eclipse
- Make a 'techniques' folder into workplace of Eclipse. Then .metadata folder will be created in 'techniques' folder.
- On the 'Package Explorer' panel, Open context menu by clicking right mouse button.
- Select 'Import', Then a pop-up windows will be placed.
- Except BLUiR project, choose 'General > Projects from Folder or Archive' item and click 'Next' button.
- Designate project folder in 'techniques' and click 'Finish' button.
- Then, the project will be loaded and be shown in the Package Explorer.
- BLUiR is made as Maven project. So, You should import with 'Maven > Existing Maven Project'. And then, just choose project folder. You don't need to change any other options.
- Especially BLIA project, need to add library JUnit.