How to compile and test peephole patterns

Introduction

After checking out the latest public_html/ in the SVN repository, you will find the class benchmarks in benchmarks/2004/joos, benchmarks/2007/joos and benchmarks/2008/joos. The group-x folders contain benchmarks from each group, we will use them to benchmark your joos peephole optimizer.

  • benchmark_list_joos is a plain text file listing thees benchmarks. This file is used by the scripts. Feel free to add your own benchmarks.
  • compile_and_run is a script that tries to (1) compile benchmarks using javac, joosa- and joosa+; (2) compiled code in different folders; then (3) run them one after another; and (4) compare the output files to check if the benchmarks behave exactly the same. In addition, this script decompiles the bytecode to jasmin to count the total bytes. This is what you will be judged on.
  • compile_and_run_opt is pretty similar to compile_and_run except the "-O" flag is passed to the three compilers so that optimization is turned on. Output files are checked in the end, to see if the benchmarks' behaviour does not change after your peephole optimization.
  • totalbytes_all is a script that will report the total size of the compiled files
  • cleanup is a script that will remove all the generated files.

Environment Settings

Before running those scripts, make sure you have set up your environment variables. In particular, note that the joosa- wrapper expects to find a binary called joos on your PATH that implements your peephole optimizations. Also, please note: if you are not using bash as your shell you might need to replace instances of ${OSTYPE} and $OSTYPE in the scripts with a hard path to the binaries you want in $JOOSDIR/bin.

Scripts Usage

  • compile_and_run / compile_and_run_opt:

    You should pass the benchmark list as the only parameter when running these scripts:

    ./compile_and_run benchmark_list_joos

    ./compile_and_run_opt benchmark_list_joos

  • totalbytes_all:

    You can run it by simply typing

    ./totalbytes_all

  • cleanup:
  • You should pass the benchmark list as the only parameter when running:

    ./cleanup benchmark_list_joos

  • totalbytes:

    Yet another script to help you calculate the size of your code, it is located in public_html/joos/bin, it can report the total size of the .sj files in the specified directory:

    totalbytes <dir_name> where <dir_name> is one of aminus-opt, aplus-opt, javac-opt

Finally, you might want to read the README.2007 and README.2004 files if you want more insight as to how the scripts work and how you can use them to focus on specific problems.

Note: Unfortunately we were unable to integrate group 4's benchmark from this year because it depends on some non-standard API. Also not that the benchmark of group 1 throws a "joos.lib.JoosException" when being executed. This is normal.

Maintained by Eric Bodden. [HOME]