ULINK Products Introduction for Supermicro

Creating Effective SSD
Test Suites
Joseph Chen, ULINK Technology, Inc.
August 12th, 2013
Flash Memory Summit 2013
Santa Clara, CA
SSD Testing Overview
Reliability
• MTBF > 1M Hours
• UBER 10-15
• Endurance TBW
Functional
Performance
• Protocol Test
• Regression Test
• Compatibility
Test
• IOPS and MBPS
• TRIM
• Quality of
Service
DVT/QA
Burn-in
Test
Final Test
• NAND
Component
Test
• Read/Write
Test
Production
Flash Memory Summit 2013
Santa Clara, CA
Page 2
SSD Testing Matrix
EVT/DVT
Functional
Protocol Test
Electrical/Power Test
EMI/Agency/Logo Tes
Reliability
Data Compare Test
POR/Power Cycle
Error Handling Test
Performance
IOPS MBPS Test
TRIM Performance Test
Quality of Service
Compatibility
System Install/Boot Test
Application SW Test
System Operation Test
Burn-in
RDT
Production
Final Test
MTBF Test
Endurance Test/TBW
UBER Test
NAND Component
Scan
Flash Memory Summit 2013
Santa Clara, CA
Page 3
Protocol Test Summary
DUT Sum
Errors
H1
5
H2
0
H3
10
H4
10
H5
7
H6
3
H7
3
H8
1
S1
55
S2
22
S3
3
S4 BRK
S5
7
S6
7
S7
18
S8
18
S9
5
S10
5
S11
7
S12
7
S13
2
S14
2
S15 13
S16 13
S17 12
S18
5
Legend
PCT1 PCT2 PCT3 PCT4 PCT5 PCT6 PCT7 PCT8 PCT9 PCT10 PCT11 PCT12 PCT13 PCT14 PCT15 PCT16 PCT17 PCT18 PCT19 PCT20
SEN IDF SFS MAN WCF RWB ATA DID DCS SMS SCT PWM PWS IPM SST SSP PHY DSM NCQ SEC
NS
5
NS
1
3
1
NS
5
1
3
1
NS
5
3
3
.
NS
1
.
NS
NS
NS
6
3
NS
NS
NS
4
12
NS
NS
NS
NS
6
5
19
1
3
3
3
1
11
1
2
NS
NS
BRK NT
2
5
2
5
NS
3
3
NS
7
5
NS
NS
3
3
NS
7
5
NS
NS
NS
5
NS
NS
5
NS
NS
2
5
NS
NS
2
5
NS
NS
2
NS
NS
2
1
3
7
1
3
7
1
3
7
1
1
1
2
0/26 0/26 0/26 0/26 0/26 1/26 10/26 1/18 2/18 4/25 9/23 3/25 1/25 0/22 0/24 2/24 0/24 7/25 17/24 5/23
NT Not Tested
BRK Broken

Total Fail Rate: 14% (64/480)
HDD
NS Not Supported
Flash Memory Summit 2013
Santa Clara, CA





Top Fail Tests: NCQ/SCT/ATA/SEC
Average Fail Items: 2.4 (64/26)
Average HDD Fail Count: 4.8 (39/8)
Average SSD Fail Count: 11.1 (201/18)
Broken: 1 SSD
Test
PTC1
PTC2
PTC3
PTC4
PTC5
PTC6
PTC7
PTC8
PTC9
PTC10
PTC11
PTC12
PTC13
PTC14
PTC15
PTC16
PTC17
PTC18
PTC19
PTC20
Abb
SEN
IDF
SFS
MAN
WCF
RWB
ATA
DID
DCS
SMS
SCT
PWM
PWS
IPM
SST
SSP
PHY
DSM
NCQ
SEC
Name
SecurityEraseNormal
IdfyInfo_SATA
SetFeature_SATA
MandatoryCmds
WrCacheFlushTime
RdWrBoundaryCk
ATACmds
DCO_IdfyInfo
DCO_SATA
SmartSet
SCT
PwrMgt
PowerState
IPM_Cmplt
SSPState
SSP
PhyEvntCnt
ATA_DSM
NCQ
SecuritySet
SSD
Good
4
0
Bad
4
17
Ugly
0
1
Page 4
SSD Reliability Test
Should SSDs and HDDs be tested the same way?
Test
Spec
HDD
SSD
1M Hours
Mechanical Spindle
Head/Media
NA
Load/Unload
60K
Mechanical Spindle
Head Stiction
NA
Power Cycle
50K
Spindle Motor
Ramp/Latch/Park
NA
Endurance
TBW
NA
NAND
5K
NA
FTL
MTBF
Power Interrupt
Flash Memory Summit 2013
Santa Clara, CA
Page 5
FTL in SSD FW Block Diagram
Host
Cmds
NAND
Command Decode
Wear Leveling
Cache/Queue
Garbage Collection
Flash Memory Summit 2013
Santa Clara, CA
Page 6
FTL LBA Mapping
HDD Zone Map
SSD FTL Map
Fixed Zone Map
Map on cylinder/head/sector
Map created on factory Burn-in
Variable Mapping
High degree of variance
Complicate meta data management
Static Mapping
Map does not change on host
command
Dynamic Mapping
Map changed on host command
No Update
Map does not change on Idle
Continuous Update Mapping
Map changed on background tasks
(Wear Leveling/Garbage Collection)
Slow Mapping
Mechanical seek/latency (~200 IOPS,
5ms)
Fast/Ultra Fast Mapping
High IOPS (~50K, 20us) 0.4%
background mapping
Flash Memory Summit 2013
Santa Clara, CA
Page 7
How to Test FTL?
Error Handle Test
Aging Test
• Protocol Error
• Interface Error
• Temperature
• Growing Defects
• Read Disturbance
• Program Disturbance
Stress Test
Disruptive Test
• Speed Stress
• Access Stress
• S1/S3 Power State Stress
• Power Disruption
• Voltage Disruption
• Asynchronous Events
FTL
Stability
Flash Memory Summit 2013
Santa Clara, CA
Page 8
Power Interruption Test
(Unintended Shutdown Test)
Prepare
with
known
data
Power off
device
Power on
the
device
Verify
device
ready
timing
Verify
read and
write
function
Flash Memory Summit 2013
Santa Clara, CA
Page 9
Compare
data
Power Interrupt Test Summary
Capacity
Dead Device
Power Up
Ready
Timeout
Read/Write
Error
Good Data
Compare Error
(Shorn Write)
Prep Data
Compare Error
(Flying Write)
SSD1
128 GB
N
0
0
81
0
SSD2
400 GB
N
0
1
0
0
SSD3
128 GB
N
0
0
108
58
SSD4
128 GB
Y
BRK
BRK
BRK
BRK
SSD5
128 GB
N
0
0
0
0
SSD6
120 GB
N
10
0
0
0
SSD7
480 GB
N
0
0
0
0
SSD8
480 GB
N
0
0
0
0
Test for 100 Hours / 100 Loops
Flash Memory Summit 2013
Santa Clara, CA
Page 10
Power Interruption Test Log
Loop: 1, POR= 0.234 Sec, NumWrDmaCmd= 106, GoodDataMiscmp= 0/100, PwOffDataMiscmp(m): 6/006 [mmmmmm]
Prep Data Miscompare Count=0
Loop: 2, POR= 0.250 Sec, NumWrDmaCmd= 103, GoodDataMiscmp= 0/100, PwOffDataMiscmp(m): 3/003 [mmm]
Prep Data Miscompare Count=0
Loop: 3, POR= 0.249 Sec, NumWrDmaCmd= 107, GoodDataMiscmp= 0/100, PwOffDataMiscmp(m): 7/007 [mmmmmmm]
Prep Data Miscompare Count=0
Loop: 4, POR= 0.234 Sec, NumWrDmaCmd= 103, GoodDataMiscmp= 0/100, PwOffDataMiscmp(m): 3/003 [mmm]
Prep Data Miscompare Count=0
Loop: 5, POR= 0.234 Sec, NumWrDmaCmd= 108, ERROR: Buffer Miscompare
READ Buffer: LBA= 122547520 (0x74DED40) Offset: 0x8
00000 40 ED 4D 07 00 00 00 00 42 A0 59 12 42 A0 59 12 @.M.....B.Y.B.Y.
00010 42 A0 59 12 42 A0 59 12 42 A0 59 12 42 A0 59 12 B.Y.B.Y.B.Y.B.Y.
00020 42 A0 59 12 42 A0 59 12 42 A0 59 12 42 A0 59 12 B.Y.B.Y.B.Y.B.Y.
00030 42 A0 59 12 42 A0 59 12 42 A0 59 12 42 A0 59 12 B.Y.B.Y.B.Y.B.Y.
WRITE Buffer: LBA= 23 (0x17) Offset: 0x8
00000 40 ED 4D 07 00 00 00 00 4D 1B 60 F5 4D 1B 60 F5 @.M.....M.`.M.`.
00010 4D 1B 60 F5 4D 1B 60 F5 4D 1B 60 F5 4D 1B 60 F5 M.`.M.`.M.`.M.`.
00020 4D 1B 60 F5 4D 1B 60 F5 4D 1B 60 F5 4D 1B 60 F5 M.`.M.`.M.`.M.`.
00030 4D 1B 60 F5 4D 1B 60 F5 4D 1B 60 F5 4D 1B 60 F5 M.`.M.`.M.`.M.`.
MISCOMPARE ERROR: Good Data Miscompare Error: Command#: 83, Error LBA: 074DED40h, Code:1003, Pointer: 8h
Flash Memory Summit 2013
Santa Clara, CA
Page 11
SSD Endurance Spec Variations
Name
Descriptions
Workload
Specified
DWPD
(Drive Write Per Day)
• Ten full drive write per day, 5 year warranty (Seagate)
• Ten drive write per day for 5 years (Intel/Seagate)
No
GBPD
(Giga Byte Per Day)
• Minimum of three years of useful life under typical client 3 years
workloads with up to 20 GB of host writes per day (Intel)
• 20GB/day of host writes for 5 years under typical client workloads (OCZ)
Typical
Client
Workload
TBW
(Tera Byte Write)
• TBW: Total bytes written (Micron/Kingston/WD)
• TBW: Tera byte write (Sandisk)
No
PBW
(Peta Byte Write)
• Lifetime endurance (8KB): Up to 14 PB (Intel)
• 4 kilobytes (KB) write endurance of up to 1.5 petabytes (PB) with 20
percent over-provisioning (Intel)
Yes,
4KB/8KB
JEDEC 218/219
• JEDEC TBW: JEDEC 218/219 TBW rating with client/enterprise workload
Yes
Note: Information quoted from the public specifications of the list companies for the purpose of illustration
Flash Memory Summit 2013
Santa Clara, CA
Page 12
Endurance Test Workload
Endurance TBW definition issue:
Not a real world workload
TBW is based on WAF, WAF is based Workload
(  ∗  ) ∗ ( + )
 =
 ∗  
 =     
 
 =  =
−
 
 =   
 =      
Solution: Use JEDEC Workload for Endurance Tests
Flash Memory Summit 2013
Santa Clara, CA
Page 13
Single Port Endurance Test
IOPS
(example)
Time/
JEDEC 218
Client
TBW/
JEDEC 218
Client
TBW / Day
Days /
72 TBW
1 Port
12K
3 Hr
0.8
6.4
11.5
2 Ports
6K
6 Hr
0.8
3.2
22.5
4 Ports
3K
12 Hr
0.8
1.6
45.0
Single port test system significantly reduce test time
Flash Memory Summit 2013
Santa Clara, CA
Page 14
Four Corner Performance Test?
Four Corner: 4K Random R/W and Sequential R/W
SNIA Standardized SSD Performance
Issues – “Unreal” test condition
Not a real world workload
No TRIM performance tested
Solution – Use JEDEC 218 Client Workload
Real world workload
TRIM commands supported
Flash Memory Summit 2013
Santa Clara, CA
Page 15
Performance Test Summary
JEDEC Wkld Performance
6000
5000
IOPS
4000
3000
2000
1000
0
1
2
3
4
5
6
7
8
DUT
JEDEC 218A Client Workload IOPS
9
10
11
12
13
DUT
S1
S2
S3
S4
Capacity
128 GB
256 GB
100 GB
160 GB
IOPS
1186
1879
5241
2783
S5
S6
S7
S8
S9
80 GB
80 GB
120 GB
128 GB
250 GB
2442
813
1632
1997
1363
S10
S11
S12
S13
250 GB
480 GB
240 GB
120 GB
1050
3489
2023
1856
Tested with JEDEC Client Workload with 38M Commands Issued
Flash Memory Summit 2013
Santa Clara, CA
Page 16
Trim Performance Comparison
Trim Performance Comparison
6000
5000
IOPS
4000
3000
2000
1000
0
1
2
3
4
5
6
7
8
9
10
11
12
13
DUT
S1
S2
S3
S4
S5
S6
S7
S8
S9
S10
S11
S12
S13
GB
128 GB
256 GB
100 GB
160 GB
80 GB
80 GB
120 GB
128 GB
250 GB
250 GB
480 GB
240 GB
120 GB
IOPS
no Trim
1186
1879
5241
2783
2442
813
1632
1997
1363
1050
3489
2023
1856
DUT
No TRIM
TRIM
Tested with JEDEC Client Workload with 38M Commands Issued
Flash Memory Summit 2013
Santa Clara, CA
Page 17
IOPS
Trim
772
667
4615
3368
2944
1559
2430
2542
1185
1132
2490
1753
1644
% Diff
65.09%
35.50%
88.06%
121.02%
120.56%
191.76%
148.90%
127.29%
86.94%
107.81%
71.37%
86.65%
88.58%
PCIe AHCI SSD Testing
System
PCIe
HBA
(AHCI)
SATA
SATA
Controller
SATA SSD
DM SATA
Pro
System
DM SATA
Pro
NAND
PCIe
AHCI Controller w/
SATA Command Set
NAND
PCIe AHCI SSD
DriveMaster SATA Pro is compatible with both SATA SSD and PCIe AHCI SSD
Flash Memory Summit 2013
Santa Clara, CA
Page 18
PCIe AHCI Test Tool
AHCI Compliance Test Software
• Provide utility to walk though every command and protocol.
AHCI Hardware and Protocol Debug Tool
• Tool help debug and find the problem.
AHCI Traffic Generator for Device Interface Test
• Issuing commands with different combinations of commands,
corner cases, and error conditions. Decipher all of the
complicated traffic and review the result.
DriveMaster provides consistent, repeatable, automated, and comprehensive
PCIe AHCI test solution.
Flash Memory Summit 2013
Santa Clara, CA
Page 19
PCIe AHCI Register Structure
System Memory
AHCI HBA Registers
0x000
PCI Configuration
Space
0x000
Generic Host Control
CMD: Command Register
0x0100
Command 5
GHC: Global Host Control
Command 6
BAR0
Port 2 Ctl Reg
IS: Interrupt Status
Command 7
BAR1
Port 3 Ctl Reg
BAR2
Port 4 Ctl Reg
HTYPE:
In Self Test
Header Type
MLT:
CLS:
Master Latency
Timer
Cache Line
Size
Memory Mapped
Registers
0x030
Port 0 Ctl Reg
Port 5 Ctl Reg
0x0100
BAR4
0000
009C
Command 13
Port 8 Ctl Reg
00A0
Command 14
SS: Subsystem ID
Port 9 Ctl Reg
Port 11 Ctl Reg
Reserved
MLAT
:
Max Latency
(Opt)
MGNT:
Min Grant (Opt)
00FC
0x0C00
0x1000
0x0010
Reserved
Cyl High
Cyl Low
Sector
Num
Features
Exp
Cyl High
Exp
Cyl Low
Exp
Sector
Num Exp
Control
Rsv (0)
Sector
Cnt Exp
Sector
Cnt
Rsv (0)
Rsv (0)
Rsv (0)
Rsv (0)
Port 14 Ctl Reg
0100
Byte3
Byte2
Byte1
Byte0
Byte7
Byte6
Byte5
Byte4
CFIS: Command
FIS
Byte11
Byte10
Byte9
Byte8
ACMD: ATAPI
Cmd
Reserved
0x0060
Reserved
Port 15 Ctl Reg
0x0080
DBA: Data Base Addr (word aligned)
Command 17
DBAU: Data Base Addr Upper 32b
Command 18
Reserved
Rsv
0x0090
Command 20
DBC: Byte Count [21:00]
DBA: Data Base Addr (word aligned)
Command 21
DBAU: Data Base Addr Upper 32b
0104
PxCLB: Cmd List Base Addr
PxCLBU: CLB Addr Up32b
Command 22
Reserved
Port 16 Ctl Reg
0108
PxFB: FIS Base Addr
Command 23
Port 17 Ctl Reg
010C
PxFBU: FIS Base Addr Up32b
Command 24
Port 18 Ctl Reg
0110
0114
PxIS: Interrupt Status
PxIE: Interrupt Enable
Command 25
Port 19 Ctl Reg
Port 20 Ctl Reg
0118
PxCMD: Command
Port 21 Ctl Reg
011C
RSV: Reserved
Port 22 Ctl Reg
0120
PxTFD: Task File Data
0x3C0
Command 30
Port 23 Ctl Reg
0124
PxSIG: Signature
0x3E0
Command 31
Port 24 Ctl Reg
0128
PxSTSS: SStatus (SCR0)
Port 25 Ctl Reg
012C
PxSCTL: SControl (SCR2)
Port 26 Ctl Reg
0130
PxSERR: SError (SCR1)
Port 27 Ctl Reg
Port 28 Ctl Reg
Port 29 Ctl Reg
0134
PxSACT: SActive (SCR3)
0138
PxCI: Cmd Issue
Rsv
0x00A0
Command 26
Command 27
Command 28
0x00B0
Command 29
Port 0-31
Port 0-31 Received FIS
Structure
0x00
DSFIS: DMA Setup FIS
RSV
0x20
013C
Port 30 Ctl Reg
PSFIS: PIO Setup FIS
RSV
Port 31 Ctl Reg
RSV: Reserved
0x40
RFIS: D2H Register FIS
0x58
0x60
SDBFIS: Set Dev Bits FIS
016C
0170
UFIS: Unknown FIS
PxVS: Version
0xA0
017C
Reserved
Port 0-31
0xFF
Port 0-31
Flash Memory Summit 2013
Santa Clara, CA
C R R
Rsv
FIS Type
(27)
Command
Dev/Head
Reserved
0x0040
Command 19
Port 0-31 Ctl Register
Port 13 Ctl Reg
0x0800
RSV
CTBA_U0: Cmd Tbl Base Addr Up32b
Command 16
Port 12 Ctl Reg
INTR: Interrupt Info
Memory Mapped
Registers
Features
Command 15
Vendor Specific
Port 10 Ctl Reg
Capability
Ptr
0x0000
Command 12
Port 7 Ctl Reg
CAP:
CFA
Command 11
Reserved
Reserved
Reserved
7
R C B R P WA
Command 10
0020
ABAR: AHCI Base Addr BAR5
EROM: Exp ROM Base Addr
PMP
Command 9
VS: Version
Port 6 Ctl Reg
15
0x010
Command 8
PI: Ports Implemented
Details
23
CTBA0: Cmd Tbl Base Addr
Command 3
Command 4
PRDTL
PRDBC: PRD Byte Count
Command 2
Generic Host Control
Registers
CAP: Host Capabilities
RID: Revision ID
BIST: Built
BAR3
0x020
Details
Command Table
31
0x000
Command 1
Port 1 Ctl Reg
CC: Class Code
0x010
0x0000
ID: Identifier
STS: Device Status
0x020
HBA Memory Registers
Port 0-31 Cmd List
Structure
Command 0
Page 20
DBC: Byte Count [21:00]
0
1
0
1
PRDT: PRD
Table, (Physical
Region
Descriptor Table)
Up to 65,536
entries
PCIe AHCI Testing Support
Interactive AHCI Register Window
Manual Command Activation
Ball-of-Wax AHCI Validation
PCIe
Compliance
Suite
PCIe Protocol
Suite
PCIe Regression
Suite
PCIe
Performance
Suite
Flash Memory Summit 2013
Santa Clara, CA
Page 21
Summary
Creating Effective SSD Test Suites
Functional
Compliance Test (SATA-IO)
Protocol Test
Reliability
Regression Test for FTL
JEDEC Workload Endurance Test
Performance
Benchmark JEDEC Real-Life Workload IOPS
Benchmark TRIM Performance
ULINK SSD Testing Suites fulfil the need!
Flash Memory Summit 2013
Santa Clara, CA
Page 22
THANK YOU!
ULINK Technology, Inc.
3120 De La Cruz Blvd, Ste 117
Santa Clara, CA 95054
1-408-446-8455
www.ulinktech.com
Contact (at) ulinktech.com
Flash Memory Summit 2013
Santa Clara, CA
Page 23
Download PDF
Similar pages