Over 2,600 SPEC benchmarks have been invalidated for playing dirty in the CPU 2017 benchmark. These Intel processors leveraged the oneAPI DPC++ compiler to improve the performance of a specific test (523.xalancbmk_r / 623.xalancbmk_s). This optimization accelerates the workload by using preparatory data on the SPEC code. Luckily, this invalidation is mostly limited to the 4th Gen Sapphire Rapids-SP processors across specific workloads.
OneAPI is an free and open-source framework for developing high-performance applications for data center workloads. Its primary goal is to simplify low-level code by offering a universal programming language, code base, and workflows for all architectures. It is based on C++, more specifically DPC++, an SYCL derivative that supports unified shared memory, group algorithms, and sub-groups.
OneAPI
The above-explained CPU optimization can improve the SPECint speed by up to 9%, meaning a lot of Intel chips higher up in the rankings will be expelled. According to Phoronix, Intel oneAPI 2022.0 through 2023.0 are affected by this compiler optimization, legalizing the latest release.
SPEC has ruled that the compiler used for this result was performing a compilation that specifically improves the performance of the 523.xalancbmk_r / 623.xalancbmk_s benchmarks using a priori knowledge of the SPEC code and dataset to perform a transformation that has narrow applicability.
In order to encourage optimizations that have wide applicability (see rule 1.4 https://www.spec.org/cpu2017/Docs/runrules.html#rule_1.4), SPEC will no longer publish results using this optimization.
This result is left in the SPEC results database for historical reference.
Although oneAPI and its DPC++ compiler are open-source, they contain optimizations targeting Intel architectures rather than the wider ecosystem. SPEC only allows code optimizations with multi-vendor support that don’t favor one architecture over another (at least on paper).
