Over 2,600 Intel CPU SPEC Benchmarks Invalidated for “Cheating”

The concerned optimization can improve performance by up to 9% on Intel CPUs

Over 2,600 SPEC benchmarks have been invalidated for playing dirty in the CPU 2017 benchmark. These Intel processors leveraged the oneAPI DPC++ compiler to improve the performance of a specific test (523.xalancbmk_r / 623.xalancbmk_s). This optimization accelerates the workload by using preparatory data on the SPEC code. Luckily, this invalidation is mostly limited to the 4th Gen Sapphire Rapids-SP processors across specific workloads.

OneAPI is an free and open-source framework for developing high-performance applications for data center workloads. Its primary goal is to simplify low-level code by offering a universal programming language, code base, and workflows for all architectures. It is based on C++, more specifically DPC++, an SYCL derivative that supports unified shared memory, group algorithms, and sub-groups.


The above-explained CPU optimization can improve the SPECint speed by up to 9%, meaning a lot of Intel chips higher up in the rankings will be expelled. According to Phoronix, Intel oneAPI 2022.0 through 2023.0 are affected by this compiler optimization, legalizing the latest release.

SPEC has ruled that the compiler used for this result was performing a compilation that specifically improves the performance of the 523.xalancbmk_r / 623.xalancbmk_s benchmarks using a priori knowledge of the SPEC code and dataset to perform a transformation that has narrow applicability.

In order to encourage optimizations that have wide applicability (see rule 1.4 https://www.spec.org/cpu2017/Docs/runrules.html#rule_1.4), SPEC will no longer publish results using this optimization.

This result is left in the SPEC results database for historical reference.

Although oneAPI and its DPC++ compiler are open-source, they contain optimizations targeting Intel architectures rather than the wider ecosystem. SPEC only allows code optimizations with multi-vendor support that don’t favor one architecture over another (at least on paper).

Areej Syed

Processors, PC gaming, and the past. I have written about computer hardware for over seven years with over 5000 published articles. I started during engineering college and haven't stopped since. On the side, I play RPGs like Baldur's Gate, Dragon Age, Mass Effect, Divinity, and Fallout. Contact: areejs12@hardwaretimes.com.
Back to top button