We return now to our analysis of the latest SPECsfs® 2008* benchmark results[1]. There have been six new NFS and no new CIFS/SMB submissions since our September report.  The new NFS submissions include four Huawei OceanStor 9000 Storage Systems with 10, 20, 40 and 100 nodes and two Hitachi Unified Storage (HUS) Model 4100 systems with four and two nodes. The new submissions changed a number of our top ten charts, which are reviewed below. All the other SPECsfs2008 charts are viewable in previous dispatches available on our website.[2]

Latest NFS results

Image may be NSFW.
Clik here to view.Figure 1 SPECsfs2008 NFSv3 Scatter plot: NFS throughput operations/second by Flash storage size

We are still trying to get a better handle on SSD/Flash advantages and have shown charts similar to Figure 1 before. We have used Flash memory size or SSD/PAM card counts as the X-axis but believe at the moment that using Flash memory size shows a more interesting story. The two groups of systems shown above include Hybrid SSD and Disk storage systems (red square) and SSD only storage systems (blue diamonds).

Figure 1 presents an interesting paradox. The increase in the amount of NFS throughput operations/second (NFS thruput ops/sec) per byte of Flash memory is higher for hybrid systems than for all-flash systems.  The regression equations are shown on the chart and indicate that for hybrid systems one can obtain, on average, ~45 NFS thruput ops/sec per byte of flash storage. The linear regression equation for all-flash systems reveals that on average, one can generate ~1 NFS thruput ops/sec per byte of flash storage.

I expected to see better performance per byte of flash memory from all-flash systems rather than from hybrid SSD and disk systems. Figure 1 clearly disproves this hypothesis. However, a couple of caveats are in order.

• We classify any system that has as much or more SSD capacity as SPECsfs2008 exported capacity as an SSD only or all-Flash system. All other systems that have any amount of SSD or Flash memory as hybrid systems.
• There are only a few (7) all-flash submissions for SPECsfs2008.  Given the paucity of results, the linear regression may not be a very good predictor of future results. Also the regression coefficient is relatively small at ~0.1 which could also be due to the lack of submissions. 

Next we turn to NFS throughput operations per second in Figure 2.

Image may be NSFW.
Clik here to view.Figure 2 SPECsfs2008 Top 10 NFSv3 throughput operations per second

Higher is better in throughput results. We had to change the scale here to accommodate the new Huawei OceanStor 9000 100-node storage system which came in at number one with a skosh over five million NFS thruput ops/sec handily beating out their previous number one. Their new 40-node OceanStor system came in at number three with just over two million NFS thruput ops/sec and their 20-node system came in at number nine with just over one million NFS thruput ops/sec. It used to be a million NFS thruput ops/sec was a pretty impressive feat, but all of our top ten systems exceeded this level of performance.

All of the new Huawei OceanStor 9000 systems had a good chunk of SSD flash (from 10TB to over 44TB of flash) with a sizeable amount of disk storage (from ~150 TB to over 1300 TB). It’s unclear if the OceanStor 9000 is doing auto-tiering, SSD caching or something else, but whatever they are doing, it seems to be working for NFSv3 thruput ops/sec.

Finally, in Figure 3 we turn to NFSv3 Overall Response Time (ORT) results.

Image may be NSFW.
Clik here to view.Figure 3 SPECsfs2008 Top 10 NFS Overall Response Time results

In Figure 3, lower is better. The new dual-node, Hitachi Unified Storage (HUS) Model 4100 storage system came in as the new number one and has beaten the previous 0.7 msec. record ORT with on ORT of 0.59 msec. The HUS Model 4100 is paired with HUS VM backend storage and provided a better response time than the four-node, HUS 4100 storage system which came in at number five with a 0.89 msec. ORT.

In contrast to SPC-1 block storage results, it seems that some hybrid and disk-only systems are still able to supply top ten ORT results (at least for the time being).  In Figure 3 there are four all-flash systems (two HUS Model 4100s, and the two EMC storage systems), one disk only system (Huawei-Symantec OceanStor N8500) and the rest are all hybrid storage systems.

Significance

There are still no new CIFS/SMB results. I keep hearing rumors of SPECsfs (2014?) moving to SMB2 or even SMB3 protocol levels. But I have no official confirmation on this.

The six new NFS results for this quarter all did well on most of our charts. Huawei’s new OceanStor 9000 multi-node clusters dominated the NFS thruput ops/sec. And Hitachi’s HUS 4100 did very well in response time.

The flash memory size scatter plot is a bit of a paradox but it could be entirely due to the lack of results. So as more all-flash submissions come in, we will update this chart to see if the results are more inline with our hypothesis. At the moment, the little data we have shows that the hypothesis is wrong.

We provide more detailed file system performance and NAS system feature analysis plus our SPECsfs2008 ChampionsCharts™ for NFS and CIFS in our recently updated (September, 2014) NAS Buying Guide available from our website. Also Top 20 versions of some of these charts are displayed in our recently updated (October, 2014) SAN-NAS Buying guide also purchasable from our website.

We are always interested in improving our performance analyses for SPECsfs2008 as well as other performance benchmark results. If you have any ideas on how to improve this analysis or ways to supply better insight into file or storage system performance, please drop us a line (contact information is at bottom left).

[This performance dispatch was originally sent out to our newsletter subscribers in December of 2013.  If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) and we will send our current issue along with download instructions for this and other reports. Dispatches are posted to our website at least a quarter or more after they are sent to our subscribers, so if you are interested in current results please consider signing up for our newsletter.]  

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community.