diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..36ba3cd --- /dev/null +++ b/.gitignore @@ -0,0 +1,2 @@ +# Editor backup files +*~ diff --git a/README.md b/README.md index 64db27e..219ff8f 100644 --- a/README.md +++ b/README.md @@ -72,6 +72,12 @@ tool chains as follows: - GDB 14.2 - Glibc 2.39 +## Adding SPEC CPU 2017 + +[SPEC CPU 2017](https://www.spec.org/cpu2017/) must be obtained independently +(it is not free). It should be placed manually in the `specpu2017` directory +at the top level (i.e. as a peer to this directory). + ## Building QEMU QEMU can be built using the `--build-all.sh` script. @@ -96,3 +102,127 @@ The `--build-clang` option will also build a Clang/LLVM tool chain. The Various other options can be used to fine-tune the build. You can use the `--help` option to see all these. + +## SPEC CPU 2017 benchmarks under QEMU + +### Design of the scripts to run SPEC CPU 2017 under QEMU + +SPEC CPU 2017 really assumes it is running native. It is not perfectly set up +for running on a remote target. We could run QEMU in system mode, but this +would necessitate running all the commands to build the QEMU benchmarks under +QEMU, which would be slow. + +So we choose to build the benchmarks on the host machine using the RISC-V +cross-compiler, and then run them under QEMU in application mode. In order to +do this we use the standard standard SPEC CPU 2017 `runcpu` command to build +the benchmarks, with the SPEC CPU `submit` configuration option inserting QEMU +commands for execution. + +We then use `runcpu` to perform a _dummy_ run of the benchmarks. With an awk +script, we can then extract the commands to run the benchmarks and check their +results afterwards. We then run all these scripts in parallel, waiting until +they have all completed. We use the QEMU `libinsn` plugin to count the number +of instructions executed by each run. We record statistics of how many +benchmarks built correctly and then ran correctly. + +Postprocessing scripts (see below) are then used to extract the results. + +### Running the SEPC CPU 2017 benchmarks under QEMU + +The script `runspec-qemu.sh` runs the benchmarks. The most important options +are as follows. + +- `--lto` or `--no-lto`. Indicates whether the benchmarks should be built + using LTO or now. Default `--no-lto` +- `--vector` or `--no-vector`. Indicates whether the benchmarks should be + built for the RISC-V Vector (RVV) extension. Default `no-vector`. +- `--benchmarks `. Indicates the set of benchmarks to use. This can be + a space separated list of benchmarks, but for convenience the following + lists are defined: + - `dummy` - just the four `specrand` benchmarks; + - `quick` - `602.gcc_s`, `623.xalancbmk_s` and `998.specrand_is`; + - `intrate` - the SPEC CPU 2017 integer rate benchmarks; + - `fprate` - the SPEC CPU 2017 floating point rate benchmarks; + - `intspeed` - the SPEC CPU 2017 integer speed benchmarks; + - `fpspeed` - the SPEC CPU 2017 floating point speed benchmarks; + - `rate` - all the SPEC CPU 2017 rate benchmarks; + - `speed` - all the SPEC CPU 2017 speed benchmarks; and + - `all` - all the benchmarks. +- `--size test|train|ref`. The size of datasets to use. Full runs should use + the `ref` datasets, but depending on the size of your server can take 2-3 days + to complete. Most benchmarking for this project uses the `test` datasets. +- `--help`. Print details of all options to the script. + +There are many options to tune SPEC CPU 2017. However, since the purpose of +this project is to improve QEMU, not tune SPEC CPU 2017, we do not generally +use them. + +The script will produce messages as it progresses. At the end it will report +on how many benchmarks built correctly and how many ran correctly. Finally it +will print the name of the full log file. This file will be used later by the +scripts to report metrics. + +### Choice of metrics + +SPEC CPU 2017 is designed to work with timings, not instruction counts. To +facilitate the standard scripts, we convert instruction counts to a nominal +time, by treating QEMU as a machine which can execute 109 +instructions per second. + +But the point of the project is to know how fast QEMU is running. We time +each benchmark run (some benchmarks have more than one run, using different +datasets). This is our first QEMU metric. + +More usefully we divide this time by the number of instructions executed. +This gives us an average execution time per instruction. The goal of this +project is to reduce this time, and bring the average time when running with +vector enabled closed to that without vector enabled. + +### Scripts to extract results + +To get the SPEC CPU 2017 scores, we use the `calc-spec-qemu.sh` script. +``` +./calc-spec-qemu.sh --speclog +``` +where `` is the log file reported at the end of the `runspec-qemu.sh` +run. The output is a table with a line for each benchmark showing the official +baseline time (in seconds), the number of QEMU instructions executed, and the +SPEC Ratio, computed on the basis of 109 instruction being executed +per second. There are a number of options to control the format of the output. + +- `--md` - produce output as a MarkDown table +- `--csv` - produce output as a CSV file + +The default is to produce plain text output. + +To get the timing data we use the `dump-qemu-times.sh` script. +``` +./dump-qemu-times.sh --speclog +``` + +This will provide a table of real, user and system times for each benchmark. +As with `calc-spec-qemu.sh` scripts, the `--md` and `--csv` options control +output format. In addition, the `--verbose` option will print additional +tables with a break down of timings for each invididual benchmark run. + +### Post processing + +At present, post processing is up to the user, typically using a spreadsheet +(CSV output is useful). When working out QEMU times, we use the sum of user +and system time. Real time is of less use, since it is too affected by +external factors. + +### Sanity checks + +When comparing different versions of QEMU, the results from +`calc-spec-qemu.sh` should be the same, or at least very similar. There can be +small variations due to timing differences when interacting with the operating +system, random number generation and the like. + +### Known limitations + +The script generation from the dummy SPEC CPU 2017 run is not yet perfect. +Some of the scripts used are not to be run on the target platform, but on the +host. Thus some benchmarks may fail their checks, when in fact they have +executed correctly. Further work is needed to fix this. + diff --git a/calc-spec-qemu.sh b/calc-spec-qemu.sh new file mode 100755 index 0000000..24ffccd --- /dev/null +++ b/calc-spec-qemu.sh @@ -0,0 +1,109 @@ +#!/bin/bash + +# Script to compute SPEC CPU 2017 scores obtained by QEMU + +# Copyright (C) 2023, 2024 Embecosm Limited +# Contributor Jeremy Bennett + +# SPDX-License-Identifier: GPL-3.0-or-later + +# This script searches for generated instruction count (*.icount) files in the +# SPEC directory. This is converted to a time, on the basis of QEMU executing +# 10^9 instructions per second. Usage: +# +# calc-spec-qemu.sh [--csv|--md] [--specdir ] + +set -u + +# Print out the help message +dohelp () { + echo "Usage: calc-spec-qemu.sh [--csv|--md]" + echo " [--speclog ]" + echo " [--help|-h]" +} + +# Set default values +topdir="$(dirname $(cd $(dirname $0) && echo ${PWD}))" +tooldir="${topdir}/rise-rvv-tcg-qemu-tooling" +installdir="${topdir}/install" +tmpfile="$(mktemp -p /tmp calc-spec-qemu-XXXXXXXX)" +pformat="" +speclog= +specdir= + +# Parse command line options +set +u +until + opt="$1" + case "${opt}" + in + --csv|--md) + pformat="$1" + ;; + --speclog) + shift + if [ ! -e "$1" ] + then + echo "ERROR: non-existent SPEC log: exiting" + exit 1 + else + speclog="$1" + specdir="$(sed -n -e 's/^specdir:[[:space:]]\+\(.*\)$/\1/p' < $1)" + if [ ! -d ${specdir} ] + then + "ERROR: non-existent SPEC directory \"${specdir}\"" + exit 1 + fi + fi + ;; + --help|-h) + dohelp + exit 0 + ;; + ?*) + echo "Unknown argument '$1'" + dohelp + exit 1 + ;; + esac + [ "x${opt}" = "x" ] +do + shift +done +set -u + +if [ "x${specdir}" = "x" ] +then + dohelp + exit 1 +fi + +# Work out which base data set to use +datasize=$(grep '^size: ' ${speclog} | head -1 | sed -e 's/size:[[:space:]]\+//') + +case ${datasize} +in + test) + basedata="${tooldir}/specbasedata-test.txt" + ;; + ref) + basedata="${tooldir}/specbasedata-ref.txt" + ;; + *) + echo "Unsupported size: ${size}" + exit 1 + ;; +esac + +# Total the workloads for each benchmark +rm -f ${tmpfile} +touch ${tmpfile} +for icf in $(find ${specdir} -name '*.icount' -print) +do + bm=$(basename ${icf} | sed -e 's/-.*//') + ic="$(sed -n -e 's/total insns: //p' < ${icf})" + printf "%-15s %15d\n" ${bm} ${ic} >> ${tmpfile} + done + +awk -f ${tooldir}/collate-times.awk ${pformat} ${basedata} < ${tmpfile} +rm ${tmpfile} diff --git a/collate-qemu-times.awk b/collate-qemu-times.awk new file mode 100644 index 0000000..56c50e0 --- /dev/null +++ b/collate-qemu-times.awk @@ -0,0 +1,184 @@ +# Awk script to collate QEMU run times + +# Copyright (C) 2023 Embecosm Limited +# Contributor Jeremy Bennett + +# SPDX-License-Identifier: GPL-3.0-or-later + +# Invoke as awk -f collate-times.awk < + +# Dump help message and exit +# \param[in] rc Return code on exit +function dohelp(rc) { + printf "Usage: awk -f collate-qemu-times.awk --txt|--csv|--md --quiet|--verbose < \n" + return rc +} + +# Print out table header in standad format +function phdr(title, pformat) { + printf "\n%s\n", title + for (i = 0; i < length(title); i++) + printf "=" + printf "\n\n" + + switch (pformat) { + case "--csv": + printf "\"Benchmark\",\"Real\",\"User\",\"Sys\"\n" + break + case "--md": + printf "| %-25s | %12s | %12s | %12s |\n", "Benchmark", \ + "Real", "User", "Sys" + printf "| %-25s | %12s | %12s | %12s |\n", \ + ":------------------------", "-----------:", "-----------:", \ + "-----------:" + break + case "--txt": + printf "%-25s %12s %12s %12s\n", "Benchmark", "Real", "User", "Sys" + printf "%-25s %12s %12s %12s\n", "---------", "----", "----", "---" + break + default: + break; + } +} + +BEGIN { + # Get the baseline data + if (ARGC != 3) + dohelp(1) + else { + switch (ARGV[1]) { + case "--txt": + case "--csv": + case "--md": + pformat = ARGV[1] + break + default: + dohelp(1) + } + switch (ARGV[2]) { + case "--verbose": + case "--quiet": + verbosity = ARGV[2] + break + default: + dohelp(1) + } + + ARGC = 1 + } + # Flag to show when we should be capturing data + capture = 0 +} + +# Start capturing results +/^Appending benchmark run logs/ { + capture = 1 +} + +# Stop capturing results +/^Checking results/ { + capture = 0 +} + +# Get name for new set of data +capture && /^Run log for/ { + bmrun = $4 + split (bmrun, t, "-") + if ( bm != t[1]) { + bm = t[1] + bmtotreal[bm] = 0 + bmtotuser[bm] = 0 + bmtotsys[bm] = 0 + bmmaxreal[bm] = 0 + bmmaxuser[bm] = 0 + bmmaxsys[bm] = 0 + } +} + +# Capture real times +capture && /^(real|sys|user)/ { + patsplit ($2, t, "([[:digit:]]+)|m|.|s") + tot = t[1] *60 + t[3] + t[5] / 1000 + switch ($1) { + case "real": + bmreal[bmrun] = tot + bmtotreal[bm] += tot + if (tot > bmmaxreal[bm]) + bmmaxreal[bm] = tot + break + case "user": + bmuser[bmrun] = tot + bmtotuser[bm] += tot + if (tot > bmmaxuser[bm]) + bmmaxuser[bm] = tot + break + case "sys": + bmsys[bmrun] = tot + bmtotsys[bm] += tot + if (tot > bmmaxsys[bm]) + bmmaxsys[bm] = tot + break + } +} + +END { + phdr("Total timings per benchmark", pformat) + PROCINFO["sorted_in"] = "@ind_str_asc" + for (bm in bmtotreal) { + switch (pformat) { + case "--csv": + printf "\"%s\",\"%.3f\",\"%.3f\",\"%.3f\"\n", bm, \ + bmtotreal[bm], bmtotuser[bm], bmtotsys[bm] + break + case "--md": + printf "| %-25s | %12.3f | %12.3f | %12.3f |\n", bm, \ + bmtotreal[bm], bmtotuser[bm], bmtotsys[bm] + break + case "--txt": + printf "%-25s %12.3f %12.3f %12.3f\n", bm, bmtotreal[bm], \ + bmtotuser[bm], bmtotsys[bm] + break + default: + } + } + if (verbosity == "--verbose") { + phdr("Timings per run", pformat) + PROCINFO["sorted_in"] = "@ind_str_asc" + for (bm in bmreal) { + switch (pformat) { + case "--csv": + printf "\"%s\",\"%.3f\",\"%.3f\",\"%.3f\"\n", bm, \ + bmreal[bm], bmuser[bm], bmsys[bm] + break + case "--md": + printf "| %-25s | %12.3f | %12.3f | %12.3f |\n", bm, bmreal[bm], \ + bmuser[bm], bmsys[bm] + break + case "--txt": + printf "%-25s %12.3f %12.3f %12.3f\n", bm, bmreal[bm], \ + bmuser[bm], bmsys[bm] + break + default: + } + } + phdr("Longest run by benchmark", pformat) + PROCINFO["sorted_in"] = "@ind_str_asc" + for (bm in bmmaxreal) { + switch (pformat) { + case "--csv": + printf "\"%s\",\"%.3f\",\"%.3f\",\"%.3f\"\n", bm, \ + bmmaxreal[bm], bmmaxuser[bm], bmmaxsys[bm] + break + case "--md": + printf "| %-25s | %12.3f | %12.3f | %12.3f |\n", bm, \ + bmmaxreal[bm], bmmaxuser[bm], bmmaxsys[bm] + break + case "--txt": + printf "%-25s %12.3f %12.3f %12.3f\n", bm, bmmaxreal[bm], \ + bmmaxuser[bm], bmmaxsys[bm] + break + default: + } + } + } +} diff --git a/collate-times.awk b/collate-times.awk new file mode 100644 index 0000000..72eb68c --- /dev/null +++ b/collate-times.awk @@ -0,0 +1,143 @@ +# Awk script to collate benchmark results + +# Copyright (C) 2023, 2024 Embecosm Limited +# Contributor Jeremy Bennett + +# SPDX-License-Identifier: GPL-3.0-or-later + +# Invoke as awk -f collate-times.awk < + +BEGIN { + # Get the baseline data + if ((ARGC < 2) || (ARGC > 3)) { + printf "Usage: awk -f collate-times.awk [--csv|--md] < \n" + exit 1 + } else { + if (ARGC == 2) { + pformat = "default" + basefile = ARGV[1] + ARGC = 1 + } + else if (ARGV[1] ~ "--csv") { + pformat = "csv" + basefile = ARGV[2] + ARGC = 1 + } + else if (ARGV[1] ~ "--md") { + pformat = "md" + basefile = ARGV[2] + ARGC = 1 + } + else { + printf "Usage: awk -f collate-times.awk [--csv] < \n" + exit 1 + } + } + while ((getline < basefile) > 0) + baset[$1] = $2 +} + +/[[:digit:]]{3}./ { + bmres[$1] += $2 +} + +END { + switch (pformat) { + case "csv": + printf "\"Benchmark\",\"Base (s)\",\"QEMU insns\",\"Ratio\"\n" + break + case "md": + printf "| %-15s | %9s | %15s | %7s |\n", " Benchmark", \ + "Base (s)", "QEMU insns", "Ratio" + printf "| %-15s | %9s | %15s | %7s |\n", ":--------------", \ + "-------:", "---------:", "----:" + break + case "default": + printf "%-15s %9s %15s %7s\n", "Benchmark", "Base (s)", "QEMU insns", \ + "Ratio" + printf "%-15s %9s %15s %7s\n", "---------", "--------", "----------", \ + "-----" + break + default: + break; + } + specprod = 1 + numbm = 0 + PROCINFO["sorted_in"] = "@ind_str_asc" + for (bm in bmres) + if (bm in baset) { + if (bmres[bm]) { + ratio = baset[bm] * 1000000000 / bmres[bm] + specprod *= ratio + numbm++ + } + else + ratio = 0 + + switch (pformat) { + case "csv": + if (ratio) + printf "\"%s\",\"%d\",\"%d\",\"%.3f\"\n", bm, baset[bm], \ + bmres[bm], ratio + else + printf "\"%s\",\"%d\",\"%d\",\"%s\"\n", bm, baset[bm], \ + bmres[bm], "-" + break + case "md": + if (ratio) + printf "| %-15s | %9d | %15d | %7.3f |\n", bm, baset[bm], \ + bmres[bm], ratio + else + printf "| %-15s | %9d | %15d | %7s |\n", bm, baset[bm], \ + bmres[bm], "-" + break + case "default": + if (ratio) + printf "%-15s %9d %15d %7.3f\n", bm, baset[bm], bmres[bm], \ + ratio + else + printf "%-15s %9d %15d %7s\n", bm, baset[bm], bmres[bm], \ + "-" + break + default: + } + } + else if (bm !~ "specrand") { + if (misslist) + misslist = misslist ", " bm + else + misslist = bm + } + + if (numbm) { + specratio = exp (log (specprod) / numbm) + switch (pformat) { + case "csv": + printf "\"SPEC ratio\",\"\",\"\",\"%.3f\"\n", specratio + break + case "md": + printf "| %-15s | %9s | %15s | %7s |\n", "", "", "", "" + printf "| %-15s | %9s | %15s | %7.3f |\n", "SPEC ratio", "", "", \ + specratio + break + case "default": + printf "\nSPEC ratio: %7.3f\n", specratio + break + default: + break + } + if (misslist) { + switch (pformat) { + case "csv": + printf "\"Unknown benchmarks\",\"%s\"\n", misslist + break + case "md": + case "default": + printf "\nUnknown benchmarks: %s\n", misslist + break + default: + break + } + } + } +} diff --git a/dump-qemu-times.sh b/dump-qemu-times.sh new file mode 100755 index 0000000..6aaf086 --- /dev/null +++ b/dump-qemu-times.sh @@ -0,0 +1,73 @@ +#!/bin/bash + +# Script to dump SPEC CPU 2017 QEMU execution times + +# Copyright (C) 2023 Embecosm Limited +# Contributor Jeremy Bennett + +# SPDX-License-Identifier: GPL-3.0-or-later + +# This script searches a SPEC CPU 2017 log script for records of the execution +# times of QEMU for each benchmark workload run. + +set -u + +# Print out the help message +dohelp () { + cat < + [--verbose | --quiet] + [--help|-h] +EOF +} + +# Set default values +topdir="$(dirname $(cd $(dirname $0) && echo ${PWD}))" +tooldir="${topdir}/rise-rvv-tcg-qemu-tooling" +installdir="${topdir}/install" +basedata="${tooldir}/spec-basedata.txt" +specdir="${installdir}/spec" +pformat="--txt" +speclog="" +verbose="--quiet" + +# Parse command line options +set +u +until + opt="$1" + case "${opt}" + in + --csv|--md) + pformat="$1" + ;; + --speclog) + shift + if [ ! -e $1 ] + then + echo "ERROR: non-existent SPEC log file: exiting" + exit 1 + else + speclog="$1" + fi + ;; + --verbose|--quiet) + verbose="$1" + ;; + --help|-h) + dohelp + exit 0 + ;; + ?*) + echo "Unknown argument '$1'" + dohelp + exit 1 + ;; + esac + [ "x${opt}" = "x" ] +do + shift +done +set -u + +awk -f ${tooldir}/collate-qemu-times.awk ${pformat} ${verbose} < ${speclog} diff --git a/runspec-breakout.awk b/runspec-breakout.awk new file mode 100644 index 0000000..a0f7f35 --- /dev/null +++ b/runspec-breakout.awk @@ -0,0 +1,61 @@ +# Awk script to extract commands to run SPEC CPU 2017 benchmarks + +# Copyright (C) 2023 Embecosm Limited +# Contributor Jeremy Bennett + +# SPDX-License-Identifier: GPL-3.0-or-later + +# Invoke as awk -f runspec-breakout.awk < + +BEGIN { + # State machine variables + invoking = 0 + verifying = 0 + + # Get the template for the scripts + if (ARGC != 2) { + printf "Usage: awk -f runspec-breakout.awk < \n" + exit 1 + } else { + bmtemplate = ARGV[1] + ARGC = 1 + } +} + +/Benchmark invocation/ { + invoking = 1 + verifying = 0 + invnum = 0 +} + +/Benchmark verification/ { + invoking = 0 + verifying = 1 + invnum = 0 +} + +invoking && /# Starting run for copy/ { + scriptfile = bmtemplate "-run-" invnum ".sh" + invnum++ +} + +verifying && /# Starting run for copy/ { + scriptfile = bmtemplate "-check-" invnum ".sh" + invnum++ +} + +(invoking || verifying) && /^cd/ { + print $0 > scriptfile +} + +invoking && /^qemu-riscv64/ { + print $0 > scriptfile +} + +verifying && /^[^[:space:]]+\/specperl/ { + print $0 > scriptfile +} + +verifying && /The log for this run is in/ { + print $8 +} diff --git a/runspec-qemu.sh b/runspec-qemu.sh new file mode 100755 index 0000000..4b6ab5f --- /dev/null +++ b/runspec-qemu.sh @@ -0,0 +1,712 @@ +#!/bin/bash + +# Script to run SPEC CPU 2017 on QEMU + +# Copyright (C) 2009, 2013, 2014, 2015, 2016, 2017, 2022, 2023, 2024 Embecosm Limited +# Contributor Jeremy Bennett + +# This file is part of the Embecosm GNU toolchain build system for RISC-V. + +# SPDX-License-Identifier: GPL-3.0-or-later + +# Usage +# +# runspec-qemu.sh +# + +set -u + +# Useful functions + +# Duplicate a string +# - $1: String to duplicate +# - $2: How often to duplicate +dup () { + printf "$1%.0s" $(seq 1 $2) +} + +# Print out a heading to log only +# - $1: Heading +loghdr () { + echo "" >> ${logfile} + echo "$1" >> ${logfile} + echo "$(dup = ${#1})" >> ${logfile} +} + +# Print out a heading to log and to screen +# - $1: Heading +# - $2: Optional elipsis for screen only +hdr () { + if [[ $# -eq 2 ]] + then + echo "$1$2" + else + echo "$1" + fi + echo "" >> ${logfile} + echo "$1" >> ${logfile} + echo "$(dup = ${#1})" >> ${logfile} +} + +# Print a message to log only +# - $1: Heading +logmess () { + echo "$1" >> ${logfile} +} + +# Print a message to log and to screen +# - $1: Heading +# - $2: Optional elipsis for screen only +mess () { + if [[ $# -eq 2 ]] + then + echo "$1$2" + else + echo "$1" + fi + echo "$1" >> ${logfile} +} + +# Print a time-stamped message to log only +# - $1: Heading +logtmess () { + ts="$(date +\"%Y-%m-%d-%H:%M:%S\"):" + echo "${ts} $1" >> ${logfile} +} + +# Print a time-stamped message to log and to screen +# - $1: Heading +# - $2: Optional elipsis for screen only +tmess () { + ts="$(date +\"%Y-%m-%d-%H:%M:%S\"):" + if [[ $# -eq 2 ]] + then + echo "${ts} $1$2" + else + echo "${ts} $1" + fi + echo "${ts} $1" >> ${logfile} +} + +# Print out the help message +dohelp () { + cat <] + [--toolbindir ] + [--topdir ] + [--installdir ] + [--specsrcdir ] + [--specdir ] + [--builddir ] + [--logdir ] + [--cc gcc | clang] + [--cxx g++ | clang++] + [--fc gfortran | flang] + [--benchmarks ] + [--size test|train|ref] + [--tune base|peak|all] + [--config ] + [--march] + [--mabi] + [--lto] + [--no-lto] + [--vector] + [--no-vector] + [--spec-flags ] + [--qemu64-flags ] + [--keeptmp] + [--clean] + [--build-only] + [--static] + [--help|-h] + +The benchmark list may include the shorthands quick, intrate, fprate, +intspeed, fpspeed, rate, speed, all. Otherwise it is a space separated +list of benchmark names. + +--march defaults to "rv64gc" +--mabi defaults to "lp64d", +--spec-flags the base list of flags to use for all SPEC runs, which defaults + to "-Ofast". The LTO and vector flags are appended to this. +--cc/-cxx/-fc compilers default to gcc, g++ and gfortran +--[no-]lto remove/add "-flto=auto" to the SPEC flags, defaults to --no-lto. +--[no-]vector shorthand for "--march rv64gcv -mabi lp64d". Defaults to --no-vector. +--qemu64-flags the list of supplementary QEMU CPU flags, defaults to + "zicsr=true,v=true,vext_spec=v1.0,zfh=true,zvfh=true" + +For other vectorization behavior, the --spec-flags will need to be set explicitly. for example to set a fixed maximum VLEN, use + + ".. --param=riscv-autovec-preference=fixed-vlmax" + +If wished, the value of LMUL may be set to "m2", "m4" or "m8" using + + --param=riscv-autovec-lmul=" +EOF +} + +# Set up directories +rundate="$(date +%Y-%m-%d-%H-%M-%S)" +topdir="$(dirname $(cd $(dirname $0) && echo ${PWD}))" +tooldir="${topdir}/rise-rvv-tcg-qemu-tooling" +toolbindir="${topdir}/install" +installdir="${topdir}/install" +specsrcdir="${topdir}/speccpu2017" +builddir="${topdir}/build" +qemubuilddir="${topdir}/build/qemu" +logdir="${topdir}/logs" +specdir="" + +if [[ -e "${qemubuilddir}/tests/plugin/libinsn.so" ]] +then + qemuplugindir="${qemubuilddir}/tests/plugin" +elif [[ -e "${qemubuilddir}/tests/tcg/plugins/libinsn.so" ]] +then + qemuplugindir="${qemubuilddir}/tests/tcg/plugins" +else + echo "Cannot find QEMU plugin directory: terminating" + exit 1 +fi + +# Common SPEC shorthand +spec_dummy="996.specrand_fs 997.specrand_fr 998.specrand_is 999.specrand_ir" +spec_quick="602.gcc_s 623.xalancbmk_s 998.specrand_is" +spec_intrate="500.perlbench_r 502.gcc_r 505.mcf_r 520.omnetpp_r 523.xalancbmk_r 525.x264_r 531.deepsjeng_r 541.leela_r 548.exchange2_r 557.xz_r 999.specrand_ir" +spec_fprate="503.bwaves_r 507.cactuBSSN_r 508.namd_r 510.parest_r 511.povray_r 519.lbm_r 521.wrf_r 526.blender_r 527.cam4_r 538.imagick_r 544.nab_r 549.fotonik3d_r 554.roms_r 997.specrand_fr" +spec_intspeed="600.perlbench_s 602.gcc_s 605.mcf_s 620.omnetpp_s 623.xalancbmk_s 625.x264_s 631.deepsjeng_s 641.leela_s 648.exchange2_s 657.xz_s 998.specrand_is" +spec_fpspeed="603.bwaves_s 607.cactuBSSN_s 619.lbm_s 621.wrf_s 627.cam4_s 628.pop2_s 638.imagick_s 644.nab_s 649.fotonik3d_s 654.roms_s 996.specrand_fs" +spec_rate="${spec_intrate} ${spec_fprate}" +spec_speed="${spec_intspeed} ${spec_fpspeed}" +spec_all="${spec_rate} ${spec_speed}" + +# Set up default script parameters +benchmarks="${spec_intspeed}" +config="linux-riscv64-qemu" +size="ref" +tune="base" +arch="rv64gc" +abi="lp64d" +spec_flags="-Ofast" +lto_flags="" +vector_flags="" +static_flags="" +cc_compiler="gcc" +cxx_compiler="g++" +fc_compiler="gfortran" + +# May need to change this for other supported extensions +qemu64_flags="zicsr=true,v=true,vext_spec=v1.0,zfh=true,zvfh=true" +keeptmp="" +doclean="no" +dorun="yes" +logfile="spec-qemu-${rundate}" + +# Parse command line options +set +u +until + opt="$1" + case "${opt}" + in + --specsrcdir) + shift + specsrcdir="$1" + ;; + --specdir) + shift + specdir="$1" + ;; + --tooldir) + shift + tooldir="$1" + ;; + --toolbindir) + shift + toolbindir="$1" + ;; + --installdir) + shift + installdir="$1" + ;; + --builddir) + shift + builddir="$1" + ;; + --logdir) + shift + logdir="$1" + ;; + --benchmarks) + shift + case "$1" + in + dummy) + benchmarks="${spec_dummy}" + ;; + quick) + benchmarks="${spec_quick}" + ;; + intrate) + benchmarks="${spec_intrate}" + ;; + fprate) + benchmarks="${spec_fprate}" + ;; + intspeed) + benchmarks="${spec_intspeed}" + ;; + fpspeed) + benchmarks="${spec_fpspeed}" + ;; + rate) + benchmarks="${spec_rate}" + ;; + speed) + benchmarks="${spec_speed}" + ;; + all) + benchmarks="${spec_all}" + ;; + *) + benchmarks="$1" + ;; + esac + ;; + --size) + shift + case "$1" + in + test|train|ref) + size=$1 + ;; + *) + echo "Unknown size: \"$1\" - ignored." + ;; + esac + ;; + --tune) + shift + case "$1" + in + base|peak|all) + tune=$1 + ;; + *) + echo "Unknown tune: \"$1\" - ignored." + ;; + esac + ;; + --config) + shift + config=$1 + ;; + --march) + shift + arch="$1" + ;; + --mabi) + shift + abi="$1" + ;; + --vector) + arch="rv64gcv" + abi="lp64d" + vector_flags="" + ;; + --no-vector) + arch="rv64gc" + abi="lp64d" + vector_flags="" + ;; + --lto) + lto_flags="-flto=auto" + ;; + --no-lto) + lto_flags="" + ;; + --spec-flags) + shift + spec_flags="$1" + ;; + --qemu64-flags) + shift + qemu64_flags="$1" + ;; + --keeptmp) + keeptmp="-keeptmp" + ;; + --logfile) + shift + logfile="$1" + ;; + --clean) + doclean="yes" + ;; + --build-only) + dorun="no" + ;; + --static) + static_flags="-static -Wl,-Ttext-segment,0x10000" + ;; + --cc) + shift + case "${1}" + in + gcc|clang) + cc_compiler="${1}" + ;; + *) + echo "Unknown C compiler \"$1\"" + dohelp + exit 1 + ;; + esac + ;; + --cxx) + shift + case "${1}" + in + g++|clang++) + cxx_compiler="${1}" + ;; + *) + echo "Unknown C++ compiler \"$1\"" + dohelp + exit 1 + ;; + esac + ;; + --fc) + shift + case "${1}" + in + gfortran|flang) + fc_compiler="${1}" + ;; + *) + echo "Unknown C compiler \"$1\"" + dohelp + exit 1 + ;; + esac + ;; + + --help|-h) + dohelp + exit 0 + ;; + ?*) + echo "Unknown argument '$1'" + dohelp + exit 1 + ;; + esac + [ "x${opt}" = "x" ] +do + shift +done +set -u + +# Compose the complete spec flags +spec_flags="${spec_flags} ${lto_flags} ${vector_flags}" + +# Make exisiting directories absolute, creating those that may not exist first +mkdir -p "${builddir}" +mkdir -p "${logdir}" +topdir="$(cd ${topdir} && echo ${PWD})" +tooldir="$(cd ${tooldir} && echo ${PWD})" +toolbindir="$(cd ${toolbindir} && echo ${PWD})" +installdir="$(cd ${installdir} && echo ${PWD})" +specsrcdir="$(cd ${specsrcdir} && echo ${PWD})" +builddir="$(cd ${builddir} && echo ${PWD})" +logdir="$(cd ${logdir} && echo ${PWD})" + +# Set up derived files and directories. Note that the script directory is +# always cleaned! +logfile="${logdir}/${logfile}.log" +tmpdir=$(mktemp -d -p /tmp spec-qemu-XXXXXX) + +if [[ "x${specdir}" == "x" ]] +then + mkdir -p "${installdir}" + specdir="${installdir}/spec-${rundate}" +fi + +# Timestamp start (has to be after logfile declared) +tmess "Run started, logging to ${logfile}" + +# Log all the parameters +echo "Parameters:" >> ${logfile} +echo "===========" >> ${logfile} +echo "topdir: ${topdir}" >> ${logfile} +echo "tooldir: ${tooldir}" >> ${logfile} +echo "toolbindir: ${toolbindir}" >> ${logfile} +echo "installdir: ${installdir}" >> ${logfile} +echo "specdir: ${specdir}" >> ${logfile} +echo "specsrcdir: ${specsrcdir}" >> ${logfile} +echo "builddir: ${builddir}" >> ${logfile} +echo "logdir: ${logdir}" >> ${logfile} +echo "logfile: ${logfile}" >> ${logfile} +echo "tmpdir: ${tmpdir}" >> ${logfile} +echo "benchmarks: ${benchmarks}" >> ${logfile} +echo "size: ${size}" >> ${logfile} +echo "tune: ${tune}" >> ${logfile} +echo "config: ${config}" >> ${logfile} +echo "arch ${arch}" >> ${logfile} +echo "abi ${abi}" >> ${logfile} +echo "spec_flags: ${spec_flags}" >> ${logfile} +echo "qemu64_flags: ${qemu64_flags}" >> ${logfile} +echo "static_flags: ${static_flags}" >> ${logfile} +echo "cc_compiler: ${cc_compiler}" >> ${logfile} +echo "cxx_compiler: ${cxx_compiler}" >> ${logfile} +echo "fc_compiler: ${fc_compiler}" >> ${logfile} +echo "keeptmp: ${keeptmp}" >> ${logfile} +echo "doclean: ${doclean}" >> ${logfile} +echo "dorun: ${dorun}" >> ${logfile} + +# Install SPEC CPU 2017 if desired +if [[ -d ${specdir} ]] +then + mess "SPEC CPU 2017 already installed in ${specdir}" +else + hdr "Installing SPEC CPU 2017" "..." + cd ${specsrcdir} + if ./install.sh -f -d ${specdir} >> ${logfile} 2>&1 + then + mess "SPEC CPU 2017 installed in ${specdir}" + else + mess "ERROR: failed to install SPEC CPU 2017" + mess "See ${logfile}" + exit 1 + fi +fi + +# Make the SPEC directory abolute (can only do after installation) +specdir="$(cd ${specdir} && echo ${PWD})" + +# Set up the script directory and ensure it is clean (only possible after +# installation). +scriptdir="${specdir}/scripts" +rm -rf ${scriptdir} +mkdir -p ${scriptdir} + +# Set up SPEC CPU environment +cat ${tooldir}/spec-configs/${config}.cfg \ + ${tooldir}/spec-configs/peak-flags-qemu.cfg \ + > ${specdir}/config/${config}.cfg +cd ${specdir} +source shrc +export PATH=${toolbindir}/bin:$PATH +export QEMU_LD_PREFIX=${toolbindir}/sysroot + +# Clean the installation +cleanlog= +if [[ "${doclean}" == "yes" ]] +then + hdr "Cleaning" "..." + + runcpu --config ${config} --define gcc_dir=${installdir} \ + --define qemu_plugin_dir=${qemuplugindir} \ + --define build_ncpus=$(nproc) \ + --define model="-march=${arch} -mabi=${abi}" \ + --define spec_flags="${spec_flags}" \ + --define qemu64_flags="${qemu64_flags}" \ + --define static_flags="${static_flags}" \ + --define cc_compiler="${cc_compiler}" \ + --define cxx_compiler="${cxx_compiler}" \ + --define fc_compiler="${fc_compiler}" \ + --action scrub ${benchmarks} \ + >> ${tmpdir}/cleaning-log 2>&1 + cat ${tmpdir}/cleaning-log >> ${logfile} + cleanlog="$(sed -n -e 's/^The log for this run is in \(.*\)$/\1/p' \ + ${tmpdir}/cleaning-log)" + rm -f ${tmpdir}/cleaning-log +fi + +# Build all the benchmarks and set up execution +hdr "Building" "..." +buildlog= +bd="${tmpdir}/building-log" +runcpu --config ${config} --define gcc_dir=${installdir} \ + --define qemu_plugin_dir=${qemuplugindir} \ + --define build_ncpus=$(nproc) --define use_submit \ + --define model="-march=${arch} -mabi=${abi}" \ + --define spec_flags="${spec_flags}" \ + --define qemu64_flags="${qemu64_flags}" \ + --define static_flags="${static_flags}" \ + --define cc_compiler="${cc_compiler}" \ + --define cxx_compiler="${cxx_compiler}" \ + --define fc_compiler="${fc_compiler}" \ + --tune=${tune} --size=${size} ${keeptmp} \ + --loose --action setup \ + ${benchmarks} >> ${bd} 2>&1 +cat ${bd} >> ${logfile} +buildlog="$(sed -n -e 's/^The log for this run is in \(.*\)$/\1/p' ${bd})" +goodbuilds="$(sed -n -e 's/^Build successes for.*: //p' < ${bd} | \ + sed -e 's/([^)]\+),\?//g' | sed -e 's/ *None *//')" +badbuilds="$(sed -n -e 's/^Build errors for.*: //p' < ${bd} | \ + sed -e 's/([^)]\+),\?//g' | sed -e 's/ *None *//')" + +numgoodbuilds=0 +numbadbuilds=0 + +if [[ "x${goodbuilds}" != "x" ]] +then + for b in ${goodbuilds} + do + numgoodbuilds=$(( numgoodbuilds + 1 )) + done +fi +if [[ "x${badbuilds}" != "x" ]] +then + for b in ${badbuilds} + do + numbadbuilds=$(( numbadbuilds + 1 )) + done +fi + +rm -f ${bd} + +numbad=0 +numgood=0 + +# Only do running if requested +if [[ "x${dorun}" == "xyes" ]] +then + # Construct scripts to execute each benchmark run (which will include ones + # which failed to build). + hdr "Creating scripts" "..." + declare -A runlog + for bm in ${benchmarks} + do + mess "Creating script to run $bm" + bmlog="${tmpdir}/$bm.log" + runcpu --config ${config} --define gcc_dir=${installdir} \ + --define qemu_plugin_dir=${qemuplugindir} \ + --define build_ncpus=$(nproc) --define use_submit \ + --define model="-march=${arch} -mabi=${abi}" \ + --define spec_flags="${spec_flags}" \ + --define qemu64_flags="${qemu64_flags}" \ + --define static_flags="${static_flags}" \ + --define cc_compiler="${cc_compiler}" \ + --define cxx_compiler="${cxx_compiler}" \ + --define fc_compiler="${fc_compiler}" \ + --tune=${tune} --size=${size} ${keeptmp} \ + --loose --fake --action run ${bm} > ${bmlog} 2>&1 + runlog["${bm}"]="$(awk -f ${tooldir}/runspec-breakout.awk \ + ${scriptdir}/${bm} < ${bmlog})" + cat ${bmlog} >> ${logfile} + done + + # Now do all the runs together. + hdr "Launching run scripts" "..." + numprocs=0 + pidlist= + loglist= + for script in ${scriptdir}/*-run-*.sh + do + chmod ugo+x ${script} + bmlog="${tmpdir}/$(basename ${script} | sed -e 's/sh$/log/')" + if [[ "x${loglist}" == "x" ]] + then + loglist="${bmlog}" + else + loglist="${bmlog} ${loglist}" + fi + (time ${script}) > ${bmlog} 2>&1 & pid=$! + if [[ "x${pidlist}" == "x" ]] + then + pidlist="${pid}" + else + pidlist="${pid},${pidlist}" + fi + numprocs=$(( numprocs + 1 )) + done + + tmess "Launched ${numprocs} run scripts" + + # Wait for them all to complete + tcheck=0 + while [[ ${numprocs} -ne 0 ]] + do + sleep 10 + numprocs=$(ps --no-headers -q ${pidlist} | wc -l) + if [[ ${numprocs} -eq 0 ]] + then + tmess "All run scripts completed" + else + # Show we are alive every 30 x 10 = 300s + tcheck=$(( tcheck + 1 )) + if [[ ${tcheck} -eq 30 ]] + then + tmess "${numprocs} run scripts still running" "..." + tcheck=0 + fi + fi + done + + # Append all the benchmark logs + hdr "Appending benchmark run logs" "..." + for bmlog in ${loglist} + do + loghdr "Run log for $(basename ${bmlog} | sed -e 's/.log$//')" + cat ${bmlog} >> ${logfile} + done + + # Finally check for correctness. We do this in benchmark order for clarity + hdr "Checking results" "..." + numgood=0 + numbad=0 + for bm in ${goodbuilds} + do + bmfails=0 + for script in ${scriptdir}/${bm}-check-*.sh + do + chmod ugo+x ${script} + if ! ${script} >> ${logfile} 2>&1 + then + logmess "${script} failed check" + bmfails=$(( bmfails + 1 )) + fi + done + if [[ ${bmfails} -eq 0 ]] + then + numgood=$(( numgood + 1 )) + else + numbad=$(( numbad + 1 )) + mess "${bm} failed ${bmfails} checks" + fi + done +fi + +mess "${numgoodbuilds} benchmarks built correctly, ${numbadbuilds} failed" +if [[ "x${dorun}" == "xyes" ]] +then + mess "${numgood} benchmark runs passed, ${numbad} failed" +fi + +if [[ "${doclean}" == "yes" ]] +then + logmess "SPEC CPU cleaning log in ${cleanlog}" +fi +logmess "SPEC CPU build log in ${buildlog}" +if [[ "x${dorun}" == "xyes" ]] +then + for run in ${!runlog[@]} + do + logmess "SPEC CPU run log for ${run} in ${runlog[${run}]}" + done +fi + +tmess "Run completed, log in ${logfile}" + +# Clean up +rm -rf ${tmpdir} + +# Return code is always successful if we get this far, otherwise Jenkins gives +# up. In a perfect world, we would use the following: +# +# if [[ ${numbad} -ne 0 || ${numbadbuilds} -ne 0 ]] +# then +# exit 1 +# fi + +exit 0 diff --git a/spec-configs/linux-riscv64-qemu.cfg b/spec-configs/linux-riscv64-qemu.cfg new file mode 100644 index 0000000..98b101e --- /dev/null +++ b/spec-configs/linux-riscv64-qemu.cfg @@ -0,0 +1,268 @@ +#------------------------------------------------------------------------------ +# SPEC CPU(R) 2017 config for gcc/g++/gfortran on 64-bit RISC-V QEMU User Mode +# emulation +#------------------------------------------------------------------------------ +# +# Contributor: Jeremy Bennett +# +# Based on Example-gcc-linux-x86.cfg +# +# The author tested this config file with: +# Compiler version(s): GCC 12.1 +# Operating system(s): QEMU for RISCV-64 version 8.0.50 +# Hardware: Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz +# +# Note: This is not intended for use on GCC versions older than GCC 10. +# +# For SPEC help: https://www.spec.org/cpu2017/Docs/techsupport.html +#------------------------------------------------------------------------------ + + +#--------- Default user defined symbols ------------------------------------ +# Symbols can be overridden on the command line with +# +# runcpu --define = ... \ + +%ifndef %{gcc_dir} +% error Please define the GCC installation directory - see instructions in config file +%endif + +# We also need to know the location of the QEMU plugin directory to access the +# instruction counting builtin. There is no point in hard coding this, so +# warn if it's mssing. It should be set from the command line: +# +# runcpu --define qemu_plugin_dir= ... \ +%ifndef %{qemu_plugin_dir} +% error Please define QEMU plugin directory - see instructions in config file +%endif + + SPECLANG = %{gcc_dir}/bin/ + CC = $(SPECLANG)riscv64-unknown-linux-gnu-%{cc_compiler} %{static_flags} -std=gnu99 %{model} + CXX = $(SPECLANG)riscv64-unknown-linux-gnu-%{cxx_compiler} %{static_flags} -std=c++03 %{model} + FC = $(SPECLANG)riscv64-unknown-linux-gnu-%{fc_compiler} %{static_flags} %{model} + + # How to say "Show me your version, please" + CC_VERSION_OPTION = --version + CXX_VERSION_OPTION = --version + FC_VERSION_OPTION = --version + +default: + sw_base_ptrsize = 64-bit + sw_peak_ptrsize = 64-bit + +#--------- Portability ---------------------------------------------------- +default: # data model applies to all benchmarks + EXTRA_PORTABILITY = -DSPEC_LP64 + +# Benchmark-specific portability (ordered by last 2 digits of bmark number) + +500.perlbench_r,600.perlbench_s: #lang='C' + PORTABILITY = -DSPEC_LINUX_AARCH64 + +521.wrf_r,621.wrf_s: #lang='F,C' + CPORTABILITY = -DSPEC_CASE_FLAG + FPORTABILITY = -fconvert=big-endian + +523.xalancbmk_r,623.xalancbmk_s: #lang='CXX' + PORTABILITY = -DSPEC_LINUX + +526.blender_r: #lang='CXX,C' + PORTABILITY = -funsigned-char -DSPEC_LINUX + +527.cam4_r,627.cam4_s: #lang='F,C' + PORTABILITY = -DSPEC_CASE_FLAG -Wno-implicit-int + +628.pop2_s: #lang='F,C' + CPORTABILITY = -DSPEC_CASE_FLAG + FPORTABILITY = -fconvert=big-endian + +#---------------------------------------------------------------------- +# GCC workarounds that do not count as PORTABILITY +#---------------------------------------------------------------------- +# The workarounds in this section would not qualify under the SPEC CPU +# PORTABILITY rule. +# - In peak, they can be set as needed for individual benchmarks. +# - In base, individual settings are not allowed; set for whole suite. +# See: +# https://www.spec.org/cpu2017/Docs/runrules.html#portability +# https://www.spec.org/cpu2017/Docs/runrules.html#BaseFlags +# +# Integer workarounds - peak +# + +500.perlbench_r,600.perlbench_s: + EXTRA_CFLAGS = -fno-finite-math-only -fno-unsafe-math-optimizations +500.perlbench_r,600.perlbench_s=peak: # https://www.spec.org/cpu2017/Docs/benchmarks/500.perlbench_r.html + EXTRA_CFLAGS = -fno-strict-aliasing -fno-unsafe-math-optimizations -fno-finite-math-only +502.gcc_r,602.gcc_s=peak: # https://www.spec.org/cpu2017/Docs/benchmarks/502.gcc_r.html + EXTRA_CFLAGS = -fno-strict-aliasing -fgnu89-inline +505.mcf_r,605.mcf_s=peak: # https://www.spec.org/cpu2017/Docs/benchmarks/505.mcf_r.html + EXTRA_CFLAGS = -fno-strict-aliasing +525.x264_r,625.x264_s=peak: # https://www.spec.org/cpu2017/Docs/benchmarks/525.x264_r.html + EXTRA_CFLAGS = -fcommon +# +# Integer workarounds - base - combine the above - https://www.spec.org/cpu2017/Docs/runrules.html#BaseFlags +# +intrate,intspeed=base: + EXTRA_CFLAGS = -fno-strict-aliasing -fgnu89-inline -fcommon +# +# FP workarounds - base - combine the above - https://www.spec.org/cpu2017/Docs/runrules.html#BaseFlags +# +fprate,fpspeed=base: + EXTRA_CFLAGS = -fno-strict-aliasing + EXTRA_FFLAGS = -fallow-argument-mismatch -fmax-stack-var-size=65536 + +# +# Floating Point workarounds - peak +# + 511.povray_r=peak: # https://www.spec.org/cpu2017/Docs/benchmarks/511.povray_r.html + EXTRA_CFLAGS = -fno-strict-aliasing + 521.wrf_r,621.wrf_s=peak: # https://www.spec.org/cpu2017/Docs/benchmarks/521.wrf_r.html + EXTRA_FFLAGS = -fallow-argument-mismatch # GCC 10 or later only + 527.cam4_r,627.cam4_s=peak: # https://www.spec.org/cpu2017/Docs/benchmarks/527.cam4_r.html + EXTRA_CFLAGS = -fno-strict-aliasing + EXTRA_FFLAGS = -fallow-argument-mismatch # GCC 10 or later only + # See also topic "628.pop2_s basepeak" below + 628.pop2_s=peak: # https://www.spec.org/cpu2017/Docs/benchmarks/628.pop2_s.html + EXTRA_FFLAGS = -fallow-argument-mismatch # GCC 10 or later only + +#-------- Tuning Flags common to Base and Peak -------------------------------- +# +# Speed (OpenMP and Autopar allowed) +# +intspeed,fpspeed: + EXTRA_OPTIMIZE = -fopenmp -DSPEC_OPENMP +fpspeed: + # + # 627.cam4 needs a big stack; the preENV will apply it to all + # benchmarks in the set, as required by the rules. + # + preENV_OMP_STACKSIZE = 120M + +#-------- Base Tuning Flags ---------------------------------------------- +# EDIT if needed -- If you run into errors, you may need to adjust the +# optimization - for example you may need to remove +# the -march=native. See topic "Older GCC" above. +# +default=base: # flags for all base + OPTIMIZE = -g %{spec_flags} + +#-------- Peak Tuning Flags ---------------------------------------------- +default=peak: + OPTIMIZE = -g %{spec_flags} + PASS1_FLAGS = -fprofile-generate + PASS2_FLAGS = -fprofile-use + +# 628.pop2_s basepeak: Depending on the interplay of several optimizations, +# 628.pop2_s might not validate with peak tuning. Use the base +# version instead. See: +# https:// www.spec.org/cpu2017/Docs/benchmarks/628.pop2_s.html +628.pop2_s=peak: + basepeak = yes diff --git a/spec-configs/peak-flags-qemu.cfg b/spec-configs/peak-flags-qemu.cfg new file mode 100644 index 0000000..6f41608 --- /dev/null +++ b/spec-configs/peak-flags-qemu.cfg @@ -0,0 +1,5 @@ + +##################################################################### +# Peak Optimization QEMU +##################################################################### + diff --git a/specbasedata-ref.txt b/specbasedata-ref.txt new file mode 100644 index 0000000..8e312f5 --- /dev/null +++ b/specbasedata-ref.txt @@ -0,0 +1,43 @@ +600.perlbench_s 1774 +602.gcc_s 3981 +605.mcf_s 4721 +620.omnetpp_s 1630 +623.xalancbmk_s 1417 +625.x264_s 1764 +631.deepsjeng_s 1432 +641.leela_s 1706 +648.exchange2_s 2939 +657.xz_s 6182 +603.bwaves_s 8997 +607.cactuBSSN_s 6670 +619.lbm_s 5238 +621.wrf_s 3225 +627.cam4_s 8863 +628.pop2_s 1873 +638.imagick_s 4425 +644.nab_s 7472 +649.fotonik3d_s 9115 +654.roms_s 5745 +500.perlbench_r 1591 +502.gcc_r 1415 +505.mcf_r 1615 +520.omnetpp_r 1312 +523.xalancbmk_r 1055 +525.x264_r 1751 +531.deepsjeng_r 1145 +541.leela_r 1655 +548.exchange2_r 2619 +557.xz_r 1079 +503.bwaves_r 0027 +507.cactuBSSN_r 1266 +508.namd_r 949 +510.parest_r 2616 +511.povray_r 2335 +519.lbm_r 1053 +521.wrf_r 2239 +526.blender_r 1523 +527.cam4_r 1748 +538.imagick_r 2486 +544.nab_r 1683 +549.fotonik3d_r 3897 +554.roms_r 1589 diff --git a/specbasedata-test.txt b/specbasedata-test.txt new file mode 100644 index 0000000..0ae8272 --- /dev/null +++ b/specbasedata-test.txt @@ -0,0 +1,43 @@ +600.perlbench_s 74 +602.gcc_s 2 +605.mcf_s 40 +620.omnetpp_s 16 +623.xalancbmk_s 2 +625.x264_s 186 +631.deepsjeng_s 41 +641.leela_s 17 +648.exchange2_s 74 +657.xz_s 30 +603.bwaves_s 117 +607.cactuBSSN_s 16 +619.lbm_s 61 +621.wrf_s 50 +627.cam4_s 260 +628.pop2_s 15 +638.imagick_s 2 +644.nab_s 9 +649.fotonik3d_s 10 +654.roms_s 17 +500.perlbench_r 74 +502.gcc_r 2 +505.mcf_r 40 +520.omnetpp_r 16 +523.xalancbmk_r 2 +525.x264_r 186 +531.deepsjeng_r 41 +541.leela_r 17 +548.exchange2_r 74 +557.xz_r 30 +503.bwaves_r 117 +507.cactuBSSN_r 16 +508.namd_r 16 +510.parest_r 24 +511.povray_r 2 +519.lbm_r 61 +521.wrf_r 50 +526.blender_r 2 +527.cam4_r 260 +538.imagick_r 2 +544.nab_r 9 +549.fotonik3d_r 10 +554.roms_r 17