闪存的时代 - 重新定义企业级存储的经济学 王文惠 高级存储系统顾问

存储行业的 重要趋势

闪存发展迅猛：企业级存储市场预测（IDC） I/O密集型解决方案正在以57%的年均复合增长率增长，同时固态硬盘的价格却在下降 热/活动 数据 冷/静态 数据 非结构化数据的爆炸式增长推动了成本/容量优化型存储的增长，其年均复合增长率高达19% I/O密集型（闪存） 云计算和大数据、移动互联网、物联网对数据容量和数据性能的需求更高! <Continued setup for Compellent which can span both, I/O-intensive and cost optimized storage segments> Per IDC, we are seeing growth at the top and at the bottom of the enterprise storage market: At the high-performance storage market for I/O-intensive storage solutions for Tier 1 business critical workloads that require ultra fast response and minimal latencies. Flash is a storage technology that can effectively help address these performance sensitive requirements At the cost/capacity storage market for solutions that can handle unstructured or cold data in a cost efficient way. One way to cover both ends of the storage market is with tiered storage which uses different types of disk and storage media to: Move the hot data to I/O intensive flash tiers; Move cold data to cost-optimized disk tiers. 性能优化 成本优化 IT管理者正在寻找可以兼顾活动数据和静态数据，且已针对性能和成本进行优化的存储方案 IDC,，全球企业级存储系统预测(2013-2017)，2013年5月；文档编号241033

闪存发展迅猛：中国市场潜力巨大 全球市场：全闪存阵列(All Flash Array，简称AFA) 2013年Q4, 中国全闪存阵列仅占全球的2%

闪存发展迅猛：各部件的快速发展 闪存价格的下降速度比硬盘更快；预期二者价格 曲线将于2015-2017年出现交叉 新的存储形式：PCIe卡、专用设备、共享存储 15K硬盘 固态硬盘 美元/GB 7.2K硬盘 多核处理器既提高了处理器利用率，也提高了单 台服务器上用户对更多I/O的需求 虚拟化正在提高具有“混合器效果”的随机I/O的 需求 Flash storage, available today from many vendors in form factors such as PCIe solid state drives (SSDs) in application servers, flash appliances and all-flash and hybrid-flash storage arrays, can deliver impressive results by eliminating rotational and seek latencies that are common in traditional hard disk drive (HDD) storage systems. In the past, flash was very expensive and mostly used in application servers as cache memory or reserved to improve performance of the highest priority workloads on the arrays. Current market trends are helping the increase of flash adoption to support most demanding enterprise applications $/GB prices falling faster than rotating disk The $/GB flash prices are falling faster than the HDDs, with the analysts predicting the crossover point of SSD and 15K HDD prices to occur between 2015-2017. Prices of flash offerings are decreasing, and the adoption is accelerating. Once flash reaches the right price point, enterprises will be able to deploy it in larger capacities, more like HDDs, with multiple flash drive types used to populate multiple tiers of storage in an array. Many new form factors PCIe SSD cards for application servers; flash-only appliances; SAN arrays that are all-flash or hybrid-flash. Proliferation of multi-core processors driving higher demand I/O per server Multi-core processors and virtualized server and desktop architectures drive up processor utilization and hence also the demand for I/O per server. In the virtualized datacenter, it is now commonly I/O performance that limits server consolidation ratios, not CPU or memory. Widespread use of virtualization with its “mixer effect” Virtualization has the effect of multiplexing multiple logical workloads across a single physical I/O path. The greater the consolidation achieved through virtualization, the more randomized the physical I/O requests become. And random I/O is the Achilles heel of the rotational disk drive, because seek and rotational latency dominate transfer times 20 to 1. Per analysts: the rapid expansion of the all-flash storage arrays market to $1 billion in three to four years validates that there is pent-up demand for storage performance and accelerating applications at the right price point. As the underlying flash media decreases in price, so will the storage arrays prices, making the extreme performance SSD accessible to more and more end users. Benefits of flash: Random I/O performance Flash memory excels at random I/O performance, offering greater than 10x gains in I/Os per second (IOPS). A solid state storage solution based on flash has no seek time, no rotational latency, and is equally fast on random workloads as on sequential ones. Flash can accelerate virtual server and desktop deployments while affording higher consolidation and greater efficiency. Predictable low latency With flash memory, any block of data can be fetched in nearly constant time. This means that applications can be designed to expect sub-millisecond latency no matter what the I/O stream (random or sequential) or data distribution. Dramatic power and space savings Power and space are often capped for a specific data center footprint. When you reach capacity, it often means an expensive build-out, migration, or a whole new data center. Solid state storage uses 10x less power and space than rotational hard drives, allowing you to substantially expand capacity in place.

2014-04-30, SanDisk发布世界上第一款4TB企业级固态硬盘 (Optimus Max SAS SSD)；计划2016年，发布16TB SAS SSD

闪存市场让人眼花缭乱 特定应用程序加速 无特定应用程序加速 性能容量兼顾 服务器内部 一个设备 磁盘阵列 Flash storage, available today from many vendors in form factors such as PCIe solid state drives (SSDs) in application servers, flash appliances and all-flash and hybrid-flash storage arrays, can deliver impressive results by eliminating rotational and seek latencies that are common in traditional hard disk drive (HDD) storage systems. In the past, flash was very expensive and mostly used in application servers as cache memory or reserved to improve performance of the highest priority workloads on the arrays. Current market trends are helping the increase of flash adoption to support most demanding enterprise applications $/GB prices falling faster than rotating disk The $/GB flash prices are falling faster than the HDDs, with the analysts predicting the crossover point of SSD and 15K HDD prices to occur between 2015-2017. Prices of flash offerings are decreasing, and the adoption is accelerating. Once flash reaches the right price point, enterprises will be able to deploy it in larger capacities, more like HDDs, with multiple flash drive types used to populate multiple tiers of storage in an array. Many new form factors PCIe SSD cards for application servers; flash-only appliances; SAN arrays that are all-flash or hybrid-flash. Proliferation of multi-core processors driving higher demand I/O per server Multi-core processors and virtualized server and desktop architectures drive up processor utilization and hence also the demand for I/O per server. In the virtualized datacenter, it is now commonly I/O performance that limits server consolidation ratios, not CPU or memory. Widespread use of virtualization with its “mixer effect” Virtualization has the effect of multiplexing multiple logical workloads across a single physical I/O path. The greater the consolidation achieved through virtualization, the more randomized the physical I/O requests become. And random I/O is the Achilles heel of the rotational disk drive, because seek and rotational latency dominate transfer times 20 to 1. Per analysts: the rapid expansion of the all-flash storage arrays market to $1 billion in three to four years validates that there is pent-up demand for storage performance and accelerating applications at the right price point. As the underlying flash media decreases in price, so will the storage arrays prices, making the extreme performance SSD accessible to more and more end users. Benefits of flash: Random I/O performance Flash memory excels at random I/O performance, offering greater than 10x gains in I/Os per second (IOPS). A solid state storage solution based on flash has no seek time, no rotational latency, and is equally fast on random workloads as on sequential ones. Flash can accelerate virtual server and desktop deployments while affording higher consolidation and greater efficiency. Predictable low latency With flash memory, any block of data can be fetched in nearly constant time. This means that applications can be designed to expect sub-millisecond latency no matter what the I/O stream (random or sequential) or data distribution. Dramatic power and space savings Power and space are often capped for a specific data center footprint. When you reach capacity, it often means an expensive build-out, migration, or a whole new data center. Solid state storage uses 10x less power and space than rotational hard drives, allowing you to substantially expand capacity in place.

行业分析公司Storage Strategies NOW： 谁是优选的固态硬盘提供商？ 31%的存储专业人士选择戴尔闪存/固态硬盘 戴尔是第一大固态硬盘提供商，居于其后的是EMC(25%)、惠普(23%)、IBM(22%) 1/3的存储专业人士正在使用或计划部署闪存 1/2的存储专业人士将把闪存用于实施服务器虚拟化 1/3的存储专业人士将把闪存用于桌面虚拟化、分析和/或联机事务处理 38%的存储专业人士将使用现有的存储 28%的存储专业人士将购买新的存储 Storage Strategies NOW report “Enterprise Use Cases for Solid State Storage/Flash Memory” (Sep 13), more than 550 IT pros surveyed 31% of storage pros surveyed said Dell flash provider of choice—EMC 25%, HP 23%, IBM 22% 32% currently use or plan to deploy flash (estimated to be $5 billion market) 29% were SMBs, 44% were SMEs, 27% were LEs (across 47 verticals) 48% to use for server virtualization, 30% for desktop virtualization, 29% for analytics, 28% for OLTP 38% to leverage existing arrays, 28% to buy new arrays, 17% to add PCIe server adapters to servers, 13% to use appliances, 11% to use caching software New NAND innovations driving down cost/GB and footprint (e.g., cell size, SSD types—MLC, eMLC) Various SSD configurations maximize performance and reduce operating costs (server drive bay, server cache, storage array/appliance, and SSD/HDD hybrid deployments), depending on workload Dell offers all these options, differentiated by advanced auto-tiering across MLC/SLC/HDD Keys to enterprise SSD deployment are flash management and scalability SSD performance/longevity increased by disk virtualization, data use analysis algorithms, sequential writing, wear leveling, error detection and correction, automated tiering, deduplication, and compression

来自DELL的闪存建议

闪存市场在发生这改变 Flash 五层递进法 以前 (5年多以前) 现在 15K SAS硬盘用于存储高性能层 很少的SSD硬盘解决数据热点问题 SSD硬盘价格昂贵，但容量很小 解决方案在设计、部署、维护方面都过于复杂 固态盘开始替代部分15K和10K的SAS盘，用于提供高速访问 成本更加经济的固态盘使得更多地部署闪存盘成为一种可能 现在有可能用15K的SAS硬盘价格购买固态盘 解决方案易于设计、部署和维护 Flash 五层递进法

戴尔流动数据技术 一直领先！ 手工操作 自动 2004年开始 7K 10K 15K MLC SLC 7K 10K 15K MLC SLC 快道技术 7K 10K 15K MLC SLC 主机端 压缩 快道技术 7K 10K 15K MLC SLC 主机端 压缩 快道技术 7K 10K 15K MLC SLC 主机端 戴尔流动数据技术 一直领先！ 快道技术 7K 10K 15K MLC SLC 快道技术 7K 10K 15K SLC 7K 10K 15K 快道技术 7K 10K 15K 10K 7K 戴尔存储系统 成功 发明了 流动数据 2013年Fluid Cache 横空出世 手工操作 自动 2004年开始 传统存储

并非所有的闪存驱动器都相同 企业级 SAS 闪存盘 双SAS口，为性能和高可用提供了可能 内建使用损耗监控wear monitoring Over-provisioned为 读写寿命及持续性能提供保障 非易失性写缓存 全端到端的数据保护措施，错误检测及校正

数据分层-闪存优化 理想的多层SSD分层架构 只读数据被自动的放置到读IO SSDs (MLC) 写数据被保留在写IO密集型的SSDs (SLC) 这由CML 自己的系统自动的进行分配和管理 冷数据被移动到更低的普通SAS硬盘上(15K, 10K, 7.2K) $$$ RAID 10 $$ RAID 5 $ RAID 6 1.6TB Read intensive Enterprise SAS SSD Hard Drives 400GB Write-intensive Enterprise SAS SSD Tier 1 Tier 2 Tier 3 “Very large pool of cost-effective, enterprise grade SAS SSDs provides microsecond latency, and extreme IOPS scaling that grows with the customers demands”

Fluid Cache概述 什么是 Fluid Cache Dell Fluid Cache for SAN是一款用来提高性能的解决方案。Fluid Cache软件可聚合多台PowerEdge服务器中的高速PCIe固态盘，为戴尔SC存储提供一个共享的分布式高速缓存池。 它有哪些 优点？ Dell Fluid Cache for SAN可加快应用程序的响应速度，为OLTP、VDI和IaaS等应用程序提供稳定、无 与伦比的I/O性能。 它的工作 原理是什么？ Fluid Cache为通过戴尔存储SC系列部署的卷添加了一个高速回写缓存。这样即可根据需要更 快地访问数据，又能提供戴尔存储SC系列体系结构的所有优势。Fluid Cache与戴尔存储SC系 列提供的功能（包括数据调度和数据即时回放）完全集成在一起，并通过企业存储管理 器进行管理。 Fluid Cache for SAN is a solution that accelerates the performance of data hosted in a Compellent storage center.It is made up of software that is used to aggregate high-speed PCIe SSD’s located within multiple PowerEdge servers into a shared, distributed Cache pool. Fluid Cache for SAN can speed up a number of different applications and works especially well on applications that generate small, random workloads, such as OLTP databases, VDI, and IaaS (Infrastructure as a service,) to name a few. Fluid Cache works by adding a high-speed write-back cache in front of volumes that are deployed from a Compellent Storage Center.This allows faster access to your data when it is necessary while offering all of benefits of a the Compellent architecture.Fluid Cache is fully integrated with the features that Compellent offers, including data progression and data instant replays, and is managed through Enterprise Manager.Fluid Cache utilizes a high-speed, private network to connect the servers that form the cache pool. Now that we’ve covered the overview of Fluid Cache, let’s talk about some of the terms that describe the components of Fluid Cache.

Fluid Cache for SAN的体系结构图 3 在所有高速缓存群集服务器上安装Dell Fluid Cache软件 2 安装Dell Express Flash PCIe固态盘以创建高速缓存池 此服务器中 没有PCIe固 态盘 高速缓存池 4 设置低延迟的专用RoCE网络 网络 (10/40 GbE) 存储网络 （任何协议） Dell Fluid Cache for SAN是一款用来提高性能的解决方案。Fluid Cache软件可聚合多台PowerEdge服务器中的高速PCIe固态盘，为Compellent Storage Center提供一个共享的分布式高速缓存池。 Dell Fluid Cache for SAN可加快应用程序的响应速度，为OLTP、VDI和IaaS等应用程序提供稳定、无与伦比的I/O性能。 Fluid Cache为通过Compellent Storage Center部署的卷添加了一个高速回写缓存。这样即可根据需要更快地访问数据，又能提供Compellent体系结构的所有优势。Fluid Cache与Compellent提供的功能（包括数据调度和数据即时回放）完全集成在一起，并通过企业存储管理器进行管理。 服务器：支持Dell Express Flash PCIe固态盘的Dell PowerEdge服务器： Dell PowerEdge M620 Dell PowerEdge M820 Dell PowerEdge R620 Dell PowerEdge R720 Dell PowerEdge R820 Dell PowerEdge T620 操作系统 ESXi 5.5和vCenter Server 5.5（x86 64位） RH Linux 6.4 RAM和硬盘空间 150 GB可用磁盘空间 至少32 GB RAM，推荐64 GB（VMware VSA配置为使用16 GB RAM。） SAN软件 Dell Compellent Enterprise Manager 2014 R2 Dell Compellent Storage Center 6.5.1 SAN硬件 Dell Compellent SC8000控制器 Dell Compellent SC200或SC220扩展盘柜 将与运行SC 6.5.1的SC 40协同工作 可选：Fluid Cache Client服务器（这些服务器没有固态盘，但属于群集成员） 所有Dell PowerEdge服务器；如果采用其他厂商的服务器，则这些服务器必须安装了受支持的操作系统，可安装受支持的RoCE NIC 每个SC8000是否可以配备1个以上的高速缓冲池？ 可以，通过同一SC8000可配置由多个高速缓存构成的群集。 但是，同一个卷不能映射到多个高速缓存群集。 每个PVT高速缓存网络只能有一个高速缓存群集 许可要求？ 基于服务器提供许可，而不论服务器上有多少高速缓存或CPU核心 永久软件许可证。每三个服务器节点需要一个群集许可证，每增加一个服务器节点需要一个扩展许可证。 必须在POS和可选的APOS进行年度软件维护 PSfW中包含支持服务 定价？ YTF 按照Mo Khan的观点，Fluid Cache属于数据中心解决方案 This diagram will provide an architectural overview of the components of Fluid Cache for SAN as well as an overview of the steps needed to configure it.Those configuration steps are as follows: Connect all servers that will participate in the Fluid Cache network up to a Compellent SAN.Any combination of iSCSI, FC, or FCoE for the SAN fabric will work. Determine how many servers will join the cache cluster as providers of cache to the pool.On the servers that will become cache providers install at least 1 Express Flash PCIe SSD. Install the Fluid Cache software on all servers that will participate in the Fluid Cache cluster Configure the Fluid Cache network by installing a supported Mellanox card into each node in the Fluid Cache cluster and connecting them all to a set of switches that supports the RoCE protocol. 1 将服务器连接至Dell Compellent SAN存储 Dell Compellent SC 8000阵列

2013-12-13，迈克尔.戴尔本人亲自演示Fluid Cache惊人能力! 516万的IOPS，延时不到6毫秒 Fluid Cache for SAN was then turned on, response time dropped from nearly a second to six milliseconds, with over 12,000 transactions per second and supporting over 14,000 simultaneous users. 2013-12-13， #Dell World 2013# Dell FluidCache for SAN，500万IOPS，30%延时缩减，二分之一的成本，迈克尔·戴尔在大会现场进行了测试。516万IOPS，不到6毫秒的延时。 2013-12-13，迈克尔.戴尔本人亲自演示Fluid Cache惊人能力! 存储性能问题获得彻底根治

非结构化数据的扩展建议

灵活扩展的典范：DELL FluidFS支持Scale Up和Scale Out 客户端网络 全面一体化的NAS命名空间 支持纵向扩展 当只是容量不够时，只需支付扩盘的成本，无需支 付新NAS控制器的成本 横向扩展式架构 - 对于数据和管理，只有一个集 群 用更低的采购成本获得线性扩展性能的能力 无中断的增加引擎和硬盘来适应不断增长的块数 据和文件系统数据 将最繁忙的数据保存在高性能存储上，将不活动 数据保存在高容量、低成本的存储空间上 FluidFS NAS 动力节点 引擎0 引擎1 FluidFS NAS 动力节点 引擎2 引擎3 FC网 或 以太网 存储网络 戴尔存储SC系列 控制器 0 控制器 1 Tier I 盘 (SSD, 15K) Tier II and III disks (10K, 7K) 戴尔存储SC系列 控制器 2 控制器 3 Tier I 盘 (SSD, 15K) Tier II and III disks (10K, 7K) Tier II and III 盘 Tier II and III 盘 (10K, 7K)

FluidFS (可扩展文件系统)的竞争优势 数字宇宙中的结构化和非结构化数据的选择：DELL FluidFS FluidFS (可扩展文件系统)的竞争优势 支持SAN(FC, iSCSI), FCoE, NAS(SMB, NFS)多协议 支持Scale Out(横向扩展)和Scale Up(纵向扩展) 支持存储虚拟化、自动分级、快照、容灾 支持去重压缩、防病毒 适用于大多数客户 基于业界标准 操作简单

权威第三方 评测报告 – 配置和拓补 硬件配置： 24块SLC SSD+120块MLC SSD 4个FluidFS NAS设备

SPECsfs 测试结果: 4个FS8600节点, 2 Storage Centers 性能结果 494,244 SPECsfs OPS 1.85 ms 整体响应时间 配置 4 X FS8600 设备 24 SLC 和120 MLC SSD

FS8600 OPS 性能随着节点数 线性增加 Configuration list price File OPS $3.53 $3.34 494,244 Configuration list price File OPS 254,412 131,684 $3.53 $3.34 $3.34 2-node FS8600 1 Storage Center 36 drives 4-node FS8600 1 Storage Center 72 drives 8-node FS8600 2 Storage Centers 144 drives

FS8600 带宽随着节点数 线性增加 Still testing GB/s $3.53 2-node FS8600 1 Storage Center 94 15K drives 4-node FS8600 2 Storage Centers 188 15K drives 8-node FS8600 2 Storage Centers 376 drives

一个很棒的性能分析工具 DPACK

我们相信， 重新定义企业级存储的经济学 能帮助企业更好地发展壮大 我们竭尽全力为客户提供 更优质的技术解决方案

谢 谢