--- description: SPEC released a new version of its storage solution benchmark, designed to help IT managers with purchasing, configuration & optimization decisions title: Inside the new SPEC storage solution benchmark image: https://www.storagereview.com/wp-content/uploads/2020/12/StorageReview-SPECstorage2020_topology.jpg --- [![Storage Review]()](https://storagereview.com/) --- [Facebook](https://www.facebook.com/@storagereview) [X (Twitter)](https://twitter.twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS Feed](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview) [Facebook](https://www.facebook.com/@storagereview) [X (Twitter)](https://twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS Feed](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview) [![StorageReview.com]()](https://www.storagereview.com) ≡ Menu - [Home](https://www.storagereview.com/) - [Storage Reviews](https://www.storagereview.com/review) - [Consumer Reviews](https://www.storagereview.com/consumer) - [Enterprise Reviews](https://www.storagereview.com/enterprise) - [Ubiquiti Reviews](https://www.storagereview.com/best/ubiquiti-reviews) - [SR Merch](https://store.storagereview.com) - [Leaderboards](https://www.storagereview.com/best) - [Best Storage Arrays](https://www.storagereview.com/best/storage-arrays) - [Best Enterprise SSDs](https://www.storagereview.com/best/enterprise-ssds) - [Best Servers](https://www.storagereview.com/best/servers) - [Best Desktops for Local AI](https://www.storagereview.com/best/desktops-local-ai) - [Best Laptops for Local AI](https://www.storagereview.com/best/laptops-local-ai) - [Best Local LLM Tools](https://www.storagereview.com/best/local-llm-tools) - [Agentic AI Hardware](https://www.storagereview.com/best/agentic-ai-hardware) - [Best SSDs & Hard Drives](https://www.storagereview.com/best_drives) - [Best Portable SSDs](https://www.storagereview.com/best/portable-ssds) - [Best Business Laptops](https://www.storagereview.com/best/business-laptops) - [Best Mobile Workstations](https://www.storagereview.com/best/mobile-workstations) - [Best Desktop Workstations](https://www.storagereview.com/best/desktop-workstations) - [Best Laptop Battery Life](https://www.storagereview.com/best/laptop-battery-life) - [Storage Reference Guide](https://www.storagereview.com/storage-reference-guide) - [About SR](https://www.storagereview.com/about-storagereview) - [StorageReview.com Sweepstakes Rules and Regulations](https://www.storagereview.com/storagereview-com-sweepstakes-rules-and-regulations) Search [Home](https://www.storagereview.com/) » [News](https://www.storagereview.com/news) » Inside the new SPEC storage solution benchmark # Inside the new SPEC storage solution benchmark by Guest Author on December 15, 2020 [Enterprise](https://www.storagereview.com/enterprise)  ◇  [Enterprise Storage](https://www.storagereview.com/enterprise/enterprise-storage) *by Don Capps, SPEC Storage subcommittee chair* The Standard Performance Evaluation Corp. (SPEC) has released a major new version of its storage solution benchmark, designed to help IT managers make better purchasing, configuration and optimization decisions. The new SPECstorage Solution 2020 benchmark is based on actual applications and real-world scenarios. It includes new workloads for artificial intelligence (AI) and genomics, expanded custom workload capabilities, massively better scaling, and a statistical visualization mechanism for displaying benchmark results. There are also dozens of upgrades that make the benchmark easier to use and more efficient in delivering results. ## **Real-world workloads** Workloads within the SPECstorage Solution 2020 benchmark represent applications used within five different markets. Each workload has a set of pass-fail criteria that must be met in order to validate the benchmark results. Examples of criteria include: - Whether the storage solution under test maintained the requested sustained average op\_rate. - Whether the solution maintained an average latency below a given threshold. - Whether the solution maintained an equal balance of operations across the subcomponents of the workload. - Whether the solution maintained a maximum latency below a specific threshold for any subcomponent in the workload. The new AI workload represents AI Tensorflow image processing environments, with traces collected from systems running COCO, Resnet50 and CityScape datasets. Metrics are based on the number of simultaneous *jobs*— defined as a set of subcomponents that encompass one complete workflow — that can be sustained under the defined pass/fail criteria. The new genomics workload comes from traces of I/O operations used by commercial and research facilities to perform genetic analysis. The data has been sanitized so that it doesn’t contain any of the original genome data. Metrics are based on the number of simultaneous jobs that can be maintained under the workload’s pass-fail criteria. Three workloads are carried over from the SPEC SFS 2014 SP2 benchmark, released in December 2017: - The electronic design automation (EDA) workload represents the typical behavior of a mixture of EDA applications, including front- and back-end processing. Metrics are based on the number of jobs that can be completed within a defined set of pass/fail criteria. - The software build environment workload comprises metadata-intensive tests derived from analysis and system traces of real-world applications. Metrics are based on the number of simultaneous builds that can be completed under the pass-fail criteria. - The video data acquisition workload (VDA) simulates applications that store data acquired from a time-sensitive source, such as a surveillance camera. Metrics are based on the number of video streams that can be captured under the pass-fail criteria. ## **Customized workloads** The new benchmark enables users to customize existing workloads or create new ones to gain insight into storage performance issues. Customized workloads can accommodate a combination of Unix and Windows load generators for applications that use this combination in their workflows. An example of this combination would be a Windows system that collects sensor data and stores it using SMB while a Unix-based compute farm is analyzing the data using NFS, Lustre, GPFS, or some other file system technology. ![SPEC storage benchmark]() A sophisticated synchronization mechanism within the SPECstorage Solution 2020 benchmark keeps all of the geographically distributed load-generating processes in sync at a sub-millisecond resolution. ## **Massive scaling with fewer constraints** New internal IPC mechanisms in SPECstorage Solution 2020 significantly reduce the number of TCP ports and DNS lookups, improving scalability, reliability, and start-up and run-time performance. Scaling has been increased from 60,000 load-generating processes in previous versions to more than 4 million processes that can be distributed throughout the world. A sophisticated synchronization mechanism keeps all of the geographically distributed load-generating processes in sync at a sub-millisecond resolution. The new scaling efficiency is especially significant for testing cloud-based storage solutions. This is an important development, since among cloud providers, TCP ports are a scarce resource, with most virtual machines able to accommodate only a few thousand at best. The new benchmark never requires more than 1,000 TCP ports. This enables users to scale up to 4 million processes without running into cloud constraints. ![SPEC storage benchmark visual 2]()*The new statistical visualization mechanism within SPECstorage Solution 2020 allows users to extract runtime counters and connect them to a database for visualization.* ## **Visualization for greater insights** The new statistical visualization mechanism within SPECstorage Solution 2020 allows users to extract runtime counters and connect them to a database such as Graphite, Carbon or Grafana for visualization. This provides greater insight into the behavior and operation details of the system under test. ## **Available now for downloading** The SPECstorage Solution 2020 benchmark is available for [immediate download](http://www.spec.org/order.html) on the SPEC website for $2,000, with discounts for qualified research and academic institutions. Results from the new benchmark are not comparable to those from past SPEC storage benchmarks. SPEC corporate members active in the development of the benchmark include Dell, IBM, Intel, iX Systems, NetApp, Pure Storage, and WekaIO. Supporting individual contributors includeUdayan Bapat, Sorin Faibish, and Brian Pawlowski. \### *Don Capps has more than 30 years of experience in performance engineering for storage systems. He is a performance engineer for Netapp; a founder of Iozone.org, a non-profit organization that produces and distributes free software for measuring the performance of computer storage systems; and chair of the Standard Performance Evaluation Corp. (SPEC) Storage subcommittee.* **Engage with StorageReview** [Newsletter](https://www.storagereview.com/storage_newsletter) |  [YouTube](https://www.youtube.com/user/StorageReview "Opens in a new window") | Podcast  [iTunes](https://podcasts.apple.com/gb/podcast/storagereview-com-storage-reviews/id1060681115 "Opens in a new window")/ [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0 "Opens in a new window") |  [Instagram](https://www.instagram.com/storagereview/ "Opens in a new window") |  [Twitter](https://twitter.com/storagereview "Opens in a new window") |  [TikTok](https://www.tiktok.com/@storagereview? "Opens in a new window") |  [RSS Feed](https://www.storagereview.com/rss.xml) ![]() ### Guest Author Previous post: [News Bits: Cisco, Atempo, Arcserve, XenData, AWS, Firebolt, Komprise, HPE, Trilio, & More](https://www.storagereview.com/news/news-bits-cisco-atempo-arcserve-xendata-aws-firebolt-komprise-hpe-trilio-more) Next post: [NetApp StorageGRID SG100 Released](https://www.storagereview.com/news/netapp-storagegrid-sg100-released) Trusted Vendors Products and solutions from our affiliate partners: - [ ![Ubiquiti Logo]() Ubiquiti](https://store.ui.com/us/en?a_aid=StorageReview "Opens in a new window") - [ ![Newegg Logo]() Newegg](https://click.linksynergy.com/fs-bin/click?id=g5terNMDhz0&offerid=1207190.18&subid=0&type=4 "Opens in a new window") - [ ![Amazon Logo]() Amazon](https://amzn.to/4fLJUEW "Opens in a new window") Newsletter Subscribe to the StorageReview newsletter to stay up to date on the latest news and reviews. We promise no spam! 1 Leave this field empty if you’re human: ## Advertisement Content Categories [Facebook](https://www.facebook.com/@storagereview) [X](https://twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview) Copyright © 1998-2025 Flying Pig Ventures, LLC Cincinnati, Ohio. All rights reserved. Manage your privacy To provide the best experiences, we and our partners use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us and our partners to process personal data such as browsing behavior or unique IDs on this site and show (non-) personalized ads. Not consenting or withdrawing consent, may adversely affect certain features and functions. Click below to consent to the above or make granular choices. Your choices will be applied to this site only. You can change your settings at any time, including withdrawing your consent, by using the toggles on the Cookie Policy, or by clicking on the manage consent button at the bottom of the screen. Functional Functional Always active The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network. Preferences Preferences The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user. Statistics Statistics The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you. Marketing Marketing The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes. Statistics Marketing Features Always active Always active - Manage options - Manage services - Manage {vendor\_count} vendors - [Read more about these purposes](https://cookiedatabase.org/tcf/purposes/) Accept Deny Manage options Save preferences Manage options - {title} - {title} - {title} Manage your privacy To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions. Functional Functional Always active The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network. Preferences Preferences The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user. Statistics Statistics The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you. Marketing Marketing The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes. Statistics Marketing Features Always active Always active - Manage options - Manage services - Manage {vendor\_count} vendors - [Read more about these purposes](https://cookiedatabase.org/tcf/purposes/) Accept Deny Manage options Save preferences Manage options - {title} - {title} - {title} Manage consent Manage consent ```json {"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.storagereview.com\/news\/inside-the-new-spec-storage-solution-benchmark","url":"https:\/\/www.storagereview.com\/news\/inside-the-new-spec-storage-solution-benchmark","name":"Inside the new SPEC storage solution benchmark - StorageReview.com","isPartOf":{"@id":"https:\/\/www.storagereview.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.storagereview.com\/news\/inside-the-new-spec-storage-solution-benchmark#primaryimage"},"image":{"@id":"https:\/\/www.storagereview.com\/news\/inside-the-new-spec-storage-solution-benchmark#primaryimage"},"thumbnailUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2020\/12\/StorageReview-SPECstorage2020_topology.jpg","datePublished":"2020-12-15T10:00:05+00:00","description":"SPEC released a new version of its storage solution benchmark, designed to help IT managers with purchasing, configuration & optimization decisions","breadcrumb":{"@id":"https:\/\/www.storagereview.com\/news\/inside-the-new-spec-storage-solution-benchmark#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.storagereview.com\/news\/inside-the-new-spec-storage-solution-benchmark"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.storagereview.com\/news\/inside-the-new-spec-storage-solution-benchmark#primaryimage","url":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2020\/12\/StorageReview-SPECstorage2020_topology.jpg","contentUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2020\/12\/StorageReview-SPECstorage2020_topology.jpg","width":897,"height":497},{"@type":"BreadcrumbList","@id":"https:\/\/www.storagereview.com\/news\/inside-the-new-spec-storage-solution-benchmark#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.storagereview.com\/"},{"@type":"ListItem","position":2,"name":"News","item":"https:\/\/www.storagereview.com\/news"},{"@type":"ListItem","position":3,"name":"Inside the new SPEC storage solution benchmark"}]},{"@type":"WebSite","@id":"https:\/\/www.storagereview.com\/#website","url":"https:\/\/www.storagereview.com\/","name":"StorageReview.com","description":"StorageReview.com is a leading provider of news and reviews throughout the entire IT stack - from the datacenter to the edge, and all points in between.","publisher":{"@id":"https:\/\/www.storagereview.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.storagereview.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.storagereview.com\/#organization","name":"StorageReview.com","url":"https:\/\/www.storagereview.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.storagereview.com\/#\/schema\/logo\/image\/","url":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2020\/02\/Storage-Reviews-2-2.png","contentUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2020\/02\/Storage-Reviews-2-2.png","width":344,"height":61,"caption":"StorageReview.com"},"image":{"@id":"https:\/\/www.storagereview.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/storagereview","http:\/\/youtube.com\/user\/storagereview"]}]} ```