We don't want the result in the repo as it's a sizeable binary file that doesn't compress well.
Storing the output file in the repository becomes infeasible for
large inputs. There are two possible remedies:
- Generate the result files during make boot (Phab:D5426). We discovered some drawbacks (like missing dependency files to build compress during boot) to this approach which make it infeasible.
- Shrink the output files, for example by hashing the string that we would normally output and compare that instead.
This patch implements the second alternative. This somewhat distorts
the runtime profile, so we might want to consider doing hashing within the
benchmark runner in the future.