Sugon ParaStor F9000 Demoted From IO500 Production List Due to Reproducibility Criteria Failures

High-Performance Computing (HPC) benchmarking standards have once again sparked intense industry debate following a major administrative decision by the IO500 Committee. The committee officially removed storage subsystems powered by Sugon’s ParaStor F9000 all-flash storage architecture from its elite Production IO500 list. This demotion follows rigorous reviews prompted by independent observations regarding the reproducibility requirements mandated for top-tier production tiering. Consequently, Intel’s Aurora supercomputer storage system at Argonne National Laboratory has been reinstated to the number-one position on the prestigious Production ranking.

The IO500 benchmark serves as the premier storage counterpart to the renowned TOP500 supercomputing list. While the TOP500 categorizes systems primarily by raw computational floating-point operations per second, the IO500 evaluates High-Performance Computing storage subsystems based on overall input/output (I/O) bandwidth and metadata performance. As modern scientific workloads, artificial intelligence training models, and complex simulations generate petabytes of data, the ability to rapidly read, write, and manage files becomes a critical bottleneck. The IO500 list is thus divided into distinct categories—most notably the Production list and the Research list—to separate systems deployed for steady, real-world operational environments from experimental or proprietary testbeds.

The Timeline and Controversy of the ISC 2026 Submissions

The controversy began taking shape during the International Supercomputing Conference (ISC 2026), where SCNet introduced impressive benchmarking submissions utilizing Sugon’s ParaStor storage software and ParaStor file system implemented on F9000 all-flash hardware.

The larger of the two submissions, designated as SCNet AICS-A, executed the IO500 benchmark leveraging an immense footprint of 500 client nodes encompassing 64,000 client processors. The results submitted to the committee were staggering: AICS-A achieved an overall IO500 score of 79,110.05. Furthermore, the system demonstrated a sequential throughput bandwidth of 26,888.39 GiB/s alongside a metadata performance metric of 232,754.76 kIOPS (thousand input/output operations per second).

A secondary, smaller submission labeled AICS-B represented a 10-client configuration running across 2,560 client processors. This system recorded an IO500 score of 7,839.30, backed by 2,551.40 GiB/s of bandwidth and 24,086.69 kIOPS in metadata performance.

At the time of submission, these figures drastically overshadowed the previous benchmarks established by Argonne National Laboratory’s Aurora supercomputer. Aurora utilizes a custom, highly complex storage subsystem integrating Intel Optane Persistent Memory modules, conventional solid-state drives (SSDs), and the open-source Distributed Asynchronous Object Storage (DAOS) file system. A comparable Production benchmark for Aurora had previously yielded a score of 32,165.90, featuring 10,066.09 GiB/s of bandwidth and 102,785.41 kIOPS.

The raw metrics submitted by SCNet meant that the AICS-A configuration outpaced Aurora’s overall score by approximately 2.46 times. In the 10-client category, the AICS-B submission was roughly 2.72 times faster than its Aurora counterpart. Initially accepted by the committee, these numbers immediately propelled the SCNet submissions to the top of both the main Production and 10-Client Production rankings, temporarily displacing the American-built Aurora architecture.

The Core Issue: Reproducibility and Transparency Standards

The sudden shift in standings did not go unnoticed within the global HPC community. Noted high-performance computing expert Glenn K. Lockwood was among the first to publicly raise questions regarding the architectural transparency of the Sugon submission, sparking deeper scrutiny from the IO500 Committee.

Following comprehensive evaluations, the IO500 Committee issued a formal statement explaining the administrative reversal. "After further review, the Sugon ISC26 submission has been transferred from the Production List to the Research List, as it did not satisfy the criteria for the highest level of Reproducibility due to the lack of widely available architectural details and limited general availability of the file system," the committee announced. "Accordingly, the previous #1 position on the Production and Production 10-Client lists (Argonne’s DAOS system) has been restored."

To qualify for the Production IO500 list, systems must adhere to strict reproducibility guidelines. Much like the elite tiers of the TOP500 list, which frequently feature bespoke, one-off supercomputing installations, the IO500 accommodates customized storage subsystems built on specialized hardware and parallel file systems. However, the Production category enforces a rigorous standard: the underlying architecture must be thoroughly documented, and the software ecosystem—particularly the parallel file system—must exhibit general availability. This ensures that independent researchers can understand, verify, and potentially reproduce the architectural methodologies on varying scales.

For example, while replicating a massive 230-petabyte storage architecture akin to Intel’s Aurora within a standard corporate data center is financially and practically prohibitive, the underlying software remains accessible. DAOS is open source, comprehensively documented, and publicly downloadable. Its hardware dependencies, implementation strategies, and architectural blueprints are open for independent audit and experimentation.

Proprietary Architectures and the Research IO500 List

In contrast, proprietary file systems such as Sugon’s ParaStor present significant transparency barriers. The IO500 Committee noted that the F9000 submission lacked the requisite public architectural specifics, and the restricted availability of the software prevented the independent verification required to maintain a standing on the Production list.

This policy is applied consistently across similarly structured proprietary systems, particularly those originating from domestic technological developments in China. Proprietary frameworks like Huawei’s OceanFS and SuperFS architectures are similarly categorized under the Research IO500 list rather than Production.

The Research IO500 list routinely showcases extreme technological capabilities that operate outside standard commercial availability frameworks. For instance, the Research list is spearheaded by Pengcheng Laboratory’s CloudBrain system, which pairs Huawei OceanStor A800 storage with the OceanFS file system. That particular setup achieved a phenomenal IO500 score of 603,334.56, registering 8,291.11 GiB/s in sequential throughput alongside an astronomical 43,903,983.64 KIOPS in random performance.

An administrative curiosity noted by industry observers involves the disparity in labeling within the submission records. While historical entries such as Pengcheng Laboratory’s CloudBrain-II are explicitly designated as "proprietary" within the reproducibility tracking columns of the Research list, initial documentation for the SCNet-A submission carried a "fully reproducible" badge despite the underlying software’s proprietary nature—an inconsistency that likely accelerated the committee’s audit and subsequent reassignment.

Broader Implications for Global HPC Benchmarking

The reclassification of the Sugon ParaStor F9000 systems underscores a foundational tension within the high-performance computing community: the balancing act between raw, unbridled performance metrics and open scientific integrity.

As artificial intelligence workloads and massive data sets drive unprecedented demand for high-throughput storage architectures, vendors worldwide are pushing the boundaries of all-flash performance. Systems capable of delivering tens of thousands of gigabytes per second and hundreds of thousands of metadata operations per second represent monumental engineering achievements. However, benchmarking organizations face increasing pressure to ensure that leaderboard rankings reflect technologies that are not merely theoretical or closed-room marvels, but demonstrable components of a transparent technological ecosystem.

By tightening enforcement of its reproducibility criteria, the IO500 Committee aims to preserve the utility of its lists as reliable guides for system architects, procurement officers, and researchers worldwide. While the ParaStor F9000 retains its status on the Research list as one of the fastest storage devices ever built, its removal from the Production ranking sends a clear signal: verifiable transparency is just as vital to modern high-performance computing as raw input/output velocity.