At FMS 2024, Kioxia had a proof-of-concept demonstration of their proposed a new RAID offload methodology for enterprise SSDs. The impetus for this is quite clear: as SSDs get faster in each generation, RAID arrays have a major problem of maintaining (and scaling up) performance. Even in cases where the RAID operations are handled by a dedicated RAID card, a simple write request in, say, a RAID 5 array would involve two reads and two writes to different drives. In cases where there is no hardware acceleration, the data from the reads needs to travel all the way back to the CPU and main memory for further processing before the writes can be done.

Kioxia has proposed the use of the PCIe direct memory access feature along with the SSD controller's controller memory buffer (CMB) to avoid the movement of data up to the CPU and back. The required parity computation is done by an accelerator block resident within the SSD controller.

In Kioxia's PoC implementation, the DMA engine can access the entire host address space (including the peer SSD's BAR-mapped CMB), allowing it to receive and transfer data as required from neighboring SSDs on the bus. Kioxia noted that their offload PoC saw close to 50% reduction in CPU utilization and upwards of 90% reduction in system DRAM utilization compared to software RAID done on the CPU. The proposed offload scheme can also handle scrubbing operations without taking up the host CPU cycles for the parity computation task.

Kioxia has already taken steps to contribute these features to the NVM Express working group. If accepted, the proposed offload scheme will be part of a standard that could become widely available across multiple SSD vendors.

Western Digital's BiCS8 218-layer 3D NAND is being put to good use in a wide range of client and enterprise platforms, including WD's upcoming Gen 5 client SSDs and 128 TB-class datacenter SSD. On the external storage front, the company demonstrated four different products: for card-based media, 4 TB microSDUC and 8 TB SDUC cards with UHS-I speeds, and on the portable SSD front we had two 16 TB drives. One will be a SanDisk Desk Drive with external power, and the other in the SanDisk Extreme Pro housing with a lanyard opening in the case.

All of these are using BiCS8 QLC NAND, though I did hear booth talk (as I was taking leave) that they were not supposed to divulge the use of QLC in these products. The 4 TB microSDUC and 8 TB SDUC cards are rated for UHS-I speeds. They are being marketed under the SanDisk Ultra branding.

The SanDisk Desk Drive is an external SSD with a 18W power adapter, and it has been in the market for a few months now. Initially launched in capacities up to 8 TB, Western Digital had promised a 16 TB version before the end of the year. It appears that the product is coming to retail quite soon. One aspect to note is that this drive has been using TLC for the SKUs that are currently in the market, so it appears unlikely that the 16 TB version would be QLC. The units (at least up to the 8 TB capacity point) come with two SN850XE drives. Given the recent introduction of the 8 TB SN850X, an 'E' version with tweaked firmware is likely to be present in the 16 TB Desk Drive.

The 16 TB portable SSD in the SanDisk Extreme housing was a technology demonstration. It is definitely the highest capacity bus-powered portable SSD demonstrated by any vendor at any trade show thus far. Given the 16 TB Desk Drive's imminent market introduction, it is just a matter of time before the technology demonstration of the bus-powered version becomes a retail reality.

Kioxia's booth at FMS 2024 was a busy one with multiple technology demonstrations keeping visitors occupied. A walk-through of the BiCS 8 manufacturing process was the first to grab my attention. Kioxia and Western Digital announced the sampling of BiCS 8 in March 2023. We had touched briefly upon its CMOS Bonded Array (CBA) scheme in our coverage of Kioxial's 2Tb QLC NAND device and coverage of Western Digital's 128 TB QLC enterprise SSD proof-of-concept demonstration. At Kioxia's booth, we got more insights.

Traditionally, fabrication of flash chips involved placement of the associate logic circuitry (CMOS process) around the periphery of the flash array. The process then moved on to putting the CMOS under the cell array, but the wafer development process was serialized with the CMOS logic getting fabricated first followed by the cell array on top. However, this has some challenges because the cell array requires a high-temperature processing step to ensure higher reliability that can be detrimental to the health of the CMOS logic. Thanks to recent advancements in wafer bonding techniques, the new CBA process allows the CMOS wafer and cell array wafer to be processed independently in parallel and then pieced together, as shown in the models above.

The BiCS 8 3D NAND incorporates 218 layers, compared to 112 layers in BiCS 5 and 162 layers in BiCS 6. The company decided to skip over BiCS 7 (or, rather, it was probably a short-lived generation meant as an internal test vehicle). The generation retains the four-plane charge trap structure of BiCS 6. In its TLC avatar, it is available as a 1 Tbit device. The QLC version is available in two capacities - 1 Tbit and 2 Tbit.

Kioxia also noted that while the number of layers (218) doesn't compare favorably with the latest layer counts from the competition, its lateral scaling / cell shrinkage has enabled it to be competitive in terms of bit density as well as operating speeds (3200 MT/s). For reference, the latest shipping NAND from Micron - the G9 - has 276 layers with a bit density in TLC mode of 21 Gbit/mm², and operates at up to 3600 MT/s. However, its 232L NAND operates only up to 2400 MT/s and has a bit density of 14.6 Gbit/mm².

It must be noted that the CBA hybrid bonding process has advantages over the current processes used by other vendors - including Micron's CMOS under array (CuA) and SK hynix's 4D PUC (periphery-under-chip) developed in the late 2010s. It is expected that other NAND vendors will also move eventually to some variant of the hybrid bonding scheme used by Kioxia.

At FMS 2024, Phison gave us the usual updates on their client flash solutions. The E31T Gen 5 mainstream controller has already been seen at a few tradeshows starting with Computex 2023, while the USB4 native flash controller for high-end PSSDs was unveiled at CES 2024. The new solution being demonstrated was the E29T Gen 4 mainstream DRAM-less controller. Phison believes that there is still performance to be eked out on the Gen 4 platform with a low-cost DRAM-less solution.

Phison NVMe SSD Controller Comparison
	E31T	E29T	E27T	E26	E18
Market Segment	Mainstream Consumer			High-End Consumer
Manufacturing Process	7nm	12nm	12nm	12nm	12nm
CPU Cores	2x Cortex R5	1x Cortex R5	1x Cortex R5	2x Cortex R5	3x Cortex R5
Error Correction	7th Gen LDPC	7th Gen LDPC	5th Gen LDPC	5th Gen LDPC	4th Gen LDPC
DRAM	No	No	No	DDR4, LPDDR4	DDR4
Host Interface	PCIe 5.0 x4	PCIe 4.0 x4	PCIe 4.0 x4	PCIe 5.0 x4	PCIe 4.0 x4
NVMe Version	NVMe 2.0	NVMe 2.0	NVMe 2.0	NVMe 2.0	NVMe 1.4
NAND Channels, Interface Speed	4 ch, 3600 MT/s	4 ch, 3600 MT/s	4 ch, 3600 MT/s	8 ch, 2400 MT/s	8 ch, 1600 MT/s
Max Capacity	8 TB	8 TB	8 TB	8 TB	8 TB
Sequential Read	10.8 GB/s	7.4 GB/s	7.4 GB/s	14 GB/s	7.4 GB/s
Sequential Write	10.8 GB/s	6.5 GB/s	6.7 GB/s	11.8 GB/s	7.0 GB/s
4KB Random Read IOPS	1500k	1200k	1200k	1500k	1000k
4KB Random Write IOPS	1500k	1200k	1200k	2000k	1000k

Compared to the E27T, the key update is the use of a newer LDPC engine that enables better SSD lifespan as well as compatibility with the latest QLC flash, along with additional power optimizations.

The company also had a U21 USB4 PSSD reference design (complete with a MagSafe-compatible casing) on display, along with the usual CrystalDiskMark benchmark results. We were given to understand that PSSDs based on the U21 controller are very close to shipping into retail.

Phison has been known for taking the lead in introducing SSD controllers based on the latest and greatest interface options - be it PCIe 4.0, PCIe 5.0, or USB4. The competition is usually in the form of tier-one vendors opting for their in-house solution, or Silicon Motion stepping in a few quarters down the line after the market takes off with a more power-efficient solution. With the E29T, Phison is aiming to ensure that they still have a viable play in the mainstream Gen 4 market with their latest LDPC engine and supporting the highest available NAND flash speeds.

Microchip recently announced the availability of their second PCIe Gen 5 enterprise SSD controller - the Flashtec 5016. Like the 4016, this is also a 16-channel controller, but there are some key updates:

• PCIe 5.0 lane organization: Operation in x4 or dual independent x2 / x2 mode in the 5016, compared to the x8, or x4, or dual independent x4 / x2 mode in the 4016.
• DRAM support: Four ranks of DDR5-5200 in the 5016, compared to two ranks of DDR4-3200 in the 4016.
• Extended NAND support: 2400 MT/s NAND in the 4016, compared to the 3200 MT/s NAND support in the 5016.
• Performance improvements: The 5016 is capable of delivering 3.5M+ random read IOPS compared to the 3M+ of the 4016.

Microchip's enterprise SSD controllers provide a high level of flexibility to SSD vendors by providing them with significant horsepower and accelerators. The 5016 includes Cortex-A53 cores for SSD vendors to run custom applications relevant to SSD management. However, compared to the Gen4 controllers, there are two additional cores in the CPU cluster. The DRAM subsystem includes ECC support (both out-of-band and inline, as desired by the SSD vendor).

At FMS 2024, the company demonstrated an application of the neural network engines embedded in the Gen5 controllers. Controllers usually employ a 'read-retry' operation with altered read-out voltages for flash reads that do not complete successfully. Microchip implemented a machine learning approach to determine the read-out voltage based on the health history of the NAND block using the NN engines in the controller. This approach delivers tangible benefits for read latency and power consumption (thanks to a smaller number of errors on the first read).

The 4016 and 5016 come with a single-chip root of trust implementation for hardware security. A secure boot process with dual-signature authentication ensures that the controller firmware is not maliciously altered in the field. The company also brought out the advantages of their controller's implementation of SR-IOV, flexible data placement, and zoned namespaces along with their 'credit engine' scheme for multi-tenant cloud workloads. These aspects were also brought out in other demonstrations.

Microchip's press release included quotes from the usual NAND vendors - Solidigm, Kioxia, and Micron. On the customer front, Longsys has been using Flashtec controllers in their enterprise offerings along with YMTC NAND. It is likely that this collaboration will continue further using the new 5016 controller.

Western Digital's FMS 2024 demonstrations included a preview of their upcoming PCIe 5.0 x4 M.2 2280 NVMe SSDs for mobile workstations and consumer desktops. The Gen 5 client SSD market has been dominated by solutions based on Phison's E26 controller. The first generation products launched with slower NAND flash, while the more recent ones have exceeded the 14 GBps barrier by utilizing Micron's 2400 MT/s 232L 3D TLC. Western Digital has been conservative over the last year or so by focusing more on the mainstream / mid-range market in terms of new product introductions (such as the WD Blue SN5000, WD_BLACK SN770M, and the WD Blue SN580). Their SSD lineup is due for an update with Gen 5 drives being sorely missed. The SSDs being demonstrated at FMS 2024 will end up doing just that.

Western Digital's technology demonstrations in this segment involved two different M.2 2280 SSDs - one for the performance segment, and another for the mainstream market. They both utilize in-house controllers - while the performance segment drive uses a 8-channel controller with DRAM for the flash translation layer, the mainstream one utilizes a 4-channel DRAM-less controller. Both drives being benchmarked live were equipped with BiCS8 218-layer 3D TLC.

Western Digital is touting the power efficiency of their platform as a key differentiator, promising south of 7W (performance drive) and 5W (mainstream DRAM-less drive) for the complete SSD under stressful traffic. This makes it suitable for use in mobile workstations, but a good fit for desktops as well.

Demonstrated performance numbers indicate almost 15 GBps sequential reads and 2M+ random read IOPS for the performance drive, and 10.7 GBps sequential reads for the mainstream version. Western Digital might have missed the Gen 5 bus as it started out slowly. However, the technology demonstrations with the in-house controller and NAND indicate that WD has caught up just as the Gen 5 market is about to take off.|

Silicon Motion's SM2320 native USB 3.2 Gen 2x2 controller for USB flash drives and portable SSDs has enjoyed great market success with a large number of design wins over the last few years. Silicon Motion proudly displayed a selection of products based on the SM2320 on the show floor at FMS 2024.

The SM2320 went into mass production in Q3 2021. Since then, the NAND flash market has seen considerable change. QLC is becoming more and more reliable and common, leading to the launch of high-capacity cost-effective 4 TB and 8 TB SSDs. Newer NAND generations with flash operating at higher speeds have also made an appearance.

The SM2320, fabricated in TSMC's 28nm node, supported four channels of NAND flash running at up to 800 MT/s. The new SM2322 uses the same process node and retains support for the same number of flash channels and chip enables (8 CEs per channel). However, the NAND can now operate at up to 1200 MT/s.

The SM2322 also improves the QLC support, thanks to the implementation of a better ECC scheme. While the SM2320 opted for a 2KB LDPC implementation, the SM2322 goes in for a 4KB LDPC solution. The use of a larger region enables extension of the NAND's useful life.

The SM2322 and SM2320 packages are similar in size, and Silicon Motion expects PSSD designs using the SM2320 to adopt the SM2322 with different NAND (higher capacity / speeds) using the same enclosure. Products based on the SM2322 are expected to appear in the market before the end of the year.

Silicon Motion has been teasing their SM2508 client SSD controller for more than a year now at various trade shows. The controller is finally set for mass production, just in time as the mainstream segment of the Gen 5 SSD market is poised to take off. Silicon Motion expects SSDs based on the SM2508 to be available for purchase by the end of the year.

At FMS 2024, the company was reusing the same information cards seen at Computex in June. The specifications of the SM2508 from our Computex coverage are reproduced here.

Silicon Motion NVMe Client SSD Controller Comparison
	SM2508	SM2264	SM2268XT2	SM2269XT
Market Segment	High-End		Mainstream
Manufacturing Process	6nm	12nm	12nm	12nm
CPU Cores	4x Cortex R8	4x Cortex R8	2x Cortex R8	2x Cortex R8
Error Correction	4K+ LDPC	4K LDPC	4K+ LDPC	4K LDPC
DRAM	DDR4, LPDDR4X	DDR4, LPDDR4X	No	No
Host Interface	PCIe 5.0 x4	PCIe 4.0 x4	PCIe 4.0 x4	PCIe 4.0 x4
NVMe Version	NVMe 2.0	NVMe 1.4	NVMe 2.0	NVMe 1.4
NAND Channels, Interface Speed	8 ch, 3600 MT/s	8 ch, 1600 MT/s	4 ch, 3600 MT/s	4 ch, 1600 MT/s
Sequential Read	14.5 GB/s	7.5 GB/s	7.4 GB/s	5.1 GB/s
Sequential Write	14 GB/s	7 GB/s	6.7 GB/s	4.8 GB/s
4KB Random Read IOPS	2500k	1300k	1200k	900k
4KB Random Write IOPS	2500k	1200k	1200k	900k

Current Gen 5 SSDs in the consumer client market are currently all based on Phison's E26 controller. The appearance of newer platform solutions for SSD vendors is bound to be good from both an end-user pricing and adoption perspective.

Solidigm's D5-P5336 61.44 TB enterprise QLC SSD released in mid-2023 has seen unprecedented demand over the last few quarters, driven by the insatiable demand for high-capacity storage in AI datacenters. Multiple vendors have recognized and started preparing products to service this demand, but Solidigm appears to have taken the lead in actual market availability.

At FMS 2024, Solidigm previewed a U.2 version of their upcoming 122 TB enterprise QLC SSD. The proof-of-concept Gen 4 drives were running live in a 2U server, and Solidigm is preparing them for an early 2025 release.

Given the capacity play, Solidigm will be relying on QLC technology. However, the company was coy about confirming the NAND generation used in the product.

Source: The Advantages of Floating Gate Technology (YouTube)

The 61.44 TB D5-P5336 currently utilizes Solidigm's 192L 3D QLC based on the floating gate architecture. This has a distinct advantage for QLC endurance compared to the charge trap architecture also available to Solidigm from SK hynix. That said, SK hynix's 238L NAND also has a QLC avatar, which gives Solidigm the flexibility to use either NAND for the production version of the 122 TB drive. Solidigm expects to confirm this by the end of year in preparation for volume shipment in the first half of 2025.

Solidigm's datacenter SSD lineup includes models targeting different performance, endurance, and cost tradeoffs. Last year, the company had introduced the D5-P5336 QLC drive as a low-cost high-capacity drive for read-heavy workloads, while also preparing the SLC-based D7-P5810 for extremely write-intensive workloads requiring high endurance. The D7-P5520 / D7-P5620 Gen 4 drives with Solidigm's own 144L 3D TLC have been the high-performance offerings for generic workloads over the last couple of years.

Solidigm is announcing the availability of the successor to the D7-P5x20 today - the new D7-PS1010 and D7-PS1030. Both of these NVMe drives use SK hynix's 176L 3D TLC NAND and come with a PCIe 5.0 interface. The third digit in the model number matches the DWPD rating, with the D7-PS1010 targeting mixed workloads with a 1 DWPD rating, and the D7-PS1030 targeting write-intensive use-cases with a 3 DWPD rating.

Compared to the previous generation D7-P5x20, the D7-PS10x0 series brings about the following upgrades:

• Move from PCIe 4.0 x4 to PCIe 5.0 x4
• Move from 144L floating gate 3D TLC (Solidigm) to 176L charge trap 3D TLC (SK hynix)
• 25% longer mean-time between failures (MTBF) at 2.5M hours
• 10x higher uncorrectable bit-error rate (UBER) at 1E-18
• 1.8x to 2.8x improvement in high queue-depth random access IOPS
• 2.0x to 2.2x improvement in high queue-depth sequential access throughput

The specifications of the two new SSD families are summarized in the table below.

Solidigm D7-PS1000 Series Enterprise SSDs
		D7-PS1030	D7-P1010
Form Factor		U.2 2.5" 15mm E3.S 7.5mm
Interface		PCIe 5.0 NVMe 2.0
Capacities		1.6TB 3.2TB 6.4TB 12.8TB	1.92TB 3.68TB 7.68TB 15.36TB
NAND		SK hynix 176L 3D TLC (Charge Trap Architecture)
Sequential Read (128 KB @ QD 128)		14500 MB/s
Sequential Write (128 KB @ QD 128)		4100 MB/s (1.6 TB / 1.92 TB) 8200 MB/s (3.2 TB / 3.84 TB) 9300 MB/s (6.4 TB / 7.68 TB / 12.8 TB / 15.36 TB)
Random Read (4 KB @ QD 512)		2.35 M (1.6 TB / 1.92 TB) 3.1 M (3.2 TB / 3.84 TB) 2.8 M (6.4 TB / 7.68 TB) 2.75 M (12.8 TB / 15.36 TB)
Random Write (4 kB)		0.35 M (1.6 TB) 0.716 M (3.2 TB) 0.8 M (6.4 TB / 12.8 TB)	0.15 M (1.92 TB) 0.315 M (3.84 TB) 0.4 M (7.68 TB) 0.38 M (15.36 TB)
Power	Sustained Write	13 W (1.6 TB / 1.92 TB) 18 W (3.2 TB / 3.84 TB) 23 W (6.4 TB / 7.68 TB / 12.8 TB / 15.36 TB)
	Sustained Read	17 W (1.6 TB / 1.92 TB) 19 W (3.2 TB / 3.84 TB) 22 W (6.4 TB / 7.68 TB) 23 W (12.8 TB / 15.36 TB)
	Peak	18 W (1.6 TB / 1.92 TB) 22 W (3.2 TB / 3.84 TB) 29 W (6.4 TB / 7.68 TB) 30 W (12.8 TB / 15.36 TB)
	Idle	5 W
Write Endurance		3 DWPD	1 DWPD
Warranty		5 years

Based on Solidigm's own internal testing, the D7-PS1010 compares very favorably against the Gen 5 datacenter SSDs already in the market from Samsung and Kioxia. However, the recently introduced Micron 9550 series may present a better challenge to Solidigm's claims.

Gen 5 SSDs are well-suited for the storage-intensive tasks in AI workloads. Every new product needs to tie itself to the AI buzzword currently, but we should excuse SSD manufacturers for doing the same - after all training and inference needs to move large amounts of data back and forth between the processing engine and underlying memory. Solidigm expects the D7-PS10x0 to be a good fit as direct-attached storage internal to GPU servers or as all-flash tier supporting a HDD-only object tier in the cloud. For on-premises GPU servers, the flash / HDD tiered storage can be replaced by an all-QLC object tier.

Solidigm claims better energy efficiency compared to the competitors' Gen 5 drives from last year for various AI workload traces. While the data ingest and archival processes require system designers to maximize the storage capacity per watt (the QLC-based Solidigm D5-P5336 is attractive here), the core processing steps require the optimization of performance per watt. The D7-PS10x0 have a natural fit in this segment.

Solidigm / Intel has been serving the datacenter SSD market since its inception. The company is well aware of the quality and reliability requirements in this space. The D7-PS1010 and D7-PS1030 include the usual enhanced PLI (power loss imminent) validation checks for data saved in the process of power loss / restoration. The critical SRAMs in the SSD controller also have ECC protection. UBER testing goes well beyond the suggested JEDEC specifications. The company also claims that its silent data corruption testing and modeling are better than its competitors.

The new D7-PS1010 and D7-PS1030 bring class-leading Gen 5 performance to the datacenter SSD market. They are available for purchase now in both U.2 and E3.S form factors, with capacities ranging from 1.6 TB - 12.8 TB (D7-PS1030) and 1.92 TB - 15.36 TB (D7-PS1010).

In an unexpected announcement during their quarterly earnings call this week, Western Digital revealed that it has begun sampling an upcoming 32TB hard drive. The nearline HDD is aimed at hyperscalers, and relies on a combination of Westen Digital's EAMR technology, as well as shingled magnetic recording (SMR) technology to hit their highest capacity figures to date.

Western Digital's 32TB HDD uses all of the company's most advanced technologies. Besides energy-assisted magnetic recording (EAMR/ePMR 2 to be more precise) technology, WD is also leveraging triple-stage actuators for better positioning of heads and two-dimensional (TDMR) read heads, OptiNAND for extra performance and reliability, distributed sector (DSEC) technology and a proprietary error correcting code (ECC) technology. And, most importantly, UltraSMR technology to provide additional capacity.

"We are shipping samples of our 32TB UltraSMR/ePMR nearline hard drives to select customers," said David Goeckeler, chief executive of Western Digital, at the earnings call. "These drives feature advanced triple-stage actuators and OptiNAND technology which are designed for seamless qualification, integration and deployment in hyperscale cloud and enterprise data centers while maintaining exceptional reliability."

Seagate is currently shipping its 30TB Exos HDDs based on heat-assisted magnetic recording (HAMR) platform called Modaic 3+ to select exascalers, and the company has implied that it can build a 32TB version of the drive using SMR. Therefore, from capacity point of view, Western Digital's announcement means that the company has caught up with its rival.

As with the comapny's other UltraSMR drives, the 32TB nearline drive is aimed at WD's enterprise customers, whose infrastructure can handle the additional management requirements that SMR imposes. As SMR in enterprise drives is not transparent, it's up to the host to manage many of the complexities that come with a hard drive that isn't suited for random writes. Though at least in WD's case, the upshot is that UltraSMR also offers a more significant density increase than other SMR implementations, using a larger number of SMR bands to increase HDD capacity by up to 20%.

Working backwards, that 20% capacity increase also means that WD's new drive is starting from 2.56TB CMR platters. And while 2.56TB makes for a very decent areal density, this would mean that WD is still behind rival Seagate in terms of areal density overall, as Seagate has 3TB CMR platters in its latest HAMR-based Exos drives.