Benchmarking

Your task is to measure time performance of three different implementations of matrix multiplication and compare them:

1. Standard multiplication
2. Multiplication with the second matrix transposed before the multiplication
3. Multiplication of the matrices in 1D representation

You probably need to do the following steps to achieve the desired speedup:

1. Download the program from git repository: git clone https://gitlab.fel.cvut.cz/cuchymar/benchmarking.git
2. Open the source code in IDE
3. Run the program (cz.esw.benchmark.Main class)
4. Do the measurements on each of the implementations of the matrix multiplication
   1. Determine a warm-up period for each implementation
   2. Measure a time performance for each implementation and compute the average performance and 95% confidence interval of the measurements
   3. Compute comparison ratios of the implementations and 95% confidence intervals of the ratios
5. Upload a report with the results to the upload system

You can make any changes in the Main class to perform the measurements but you are not allowed to modify the MatrixUtils class.

The report should include the following parts:

• Machine specification - CPU, memory, OS, Java version (java -version), etc.
• Used JVM parameters (if applied)
• Warm-up period:
  • Brief description of how the warm-up period was determined
  • The results (with graphs) for each implementation
• Time performance:
  • Brief description of the measurement procedure
  • Results including graphs with error bars
• Comparison:
  • Brief description of how the comparison was done
  • Resulting ratios with confidence interval
• Conclusions:
  • Summary of the results and a discussion

Report templates: benchmark_template.doc, LaTeX template

• The slides from the second lecture
• https://kar.kent.ac.uk/33611/ (mainly chapters 9.3 and 10.1)

Guides on how to create error bars:

• MATLAB
• gnuplot
• Excel