PM1633A (mzils7t6hmls/01ej993) performance

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
A

Andrew Barnes

New Member
Apr 9, 2020
13
1
3
Hi,

I've just received 6x IBM 01ej993, which are Samsung PM1633A in the 7.68TB capacity, and i'm confused about the read and write performance.
firmware(MS17)
health(OK)
usage(1%)
LBA(512).

(HBA is LSI 9305, IT Mode, no BIOS. lspci reports link speed matches link capability (PCIe 3.0 x8). Kernel reports SAS drives negotiated link speed is 12Gbits)

Bash: 
#read test
fio --name=seqread --rw=read --bs=1M --iodepth=64 --numjobs=12 --size=50G --directory=/mnt/tank/benchmark --runtime=120 --group_reporting --direct=1 --ioengine=libaio

#write test
fio --name=seqwrite --rw=write --bs=1M --iodepth=64 --numjobs=12 --size=50G --directory=/mnt/tank/benchmark --runtime=120 --group_reporting --direct=1 --ioengine=libaio

The maximum I have seen is 600MB/s read, and 450MB/s write using BTRFS raid10 with 3 mirrored pairs, benchmarked with fio iodepth(64), numjobs(12).
iostat reported 100% utilisation during these tests.

According to Samsung (https://image.semiconductor.samsung...hure-pm1633a-25-sas-ssd/PM1633a_SAS_SSD-0.pdf):
1746786648667.png

According to Lenovo specs (Lenovo PM1633a Enterprise Capacity 12Gb SAS SSDs Product Guide (withdrawn product)):

1746786187100.png

My drive aligns very closely with Lenovos spec.

Is that that then? or is there a way to see the speeds samsung seems to suggest (although vaguely).
Even Lenovo's specs raise an eyebrow, because I would have thought higher capacity drives should have equal if not more performance if the number of parallel transactions are not increased by having more NAND (rather than reduced!).

I understand enterprise markets may benefit by segregating performance tiers and capacity tiers, so I wonder if there is a *way* to unlock performance tier?

I think these might be popular drives in current homelabs... hoping someone out there can share their experience, or knowledge in these.
Thanks
 
CyklonDX

CyklonDX

Well-Known Member
Nov 8, 2022
1,590
547
113
its hard to say for sure, it could be that it was meant for different sector size *typically its 520; or the firmware that comes with ibm is locked.

Test single disk first, and decide how much worse they are.
 
D

DarkServant

Member
Apr 5, 2022
71
74
18
With the capacity/performance, i've seen this over and over in datasheets, there is mostly some sweet spot. But why, i don't know... can be limited channels of the controller, or different NAND sub-generations (Samsung uses slightly older NAND on the highest capacity drives)?
On SAS i had only seen on some drives (old HGST SSD's) that the maximum is only obtained by using dual channel, which probably needs a special cable. I this case with the RAID it makes no sense of course.

I never worked with SAS since a very long time ago, switched to SATA with the uprising of solid state drives and then NVMe, in the times before, the fastest drives where 15k rpm SAS HDD's.
 
A

Andrew Barnes

New Member
Apr 9, 2020
13
1
3
Thanks for the replies.

I haven't seen better performance using a single drive EXT4 mounted (relatime, stripe=4095) and the same fio test.

I've reformatted a drive to remove type 2 protection using sg_format. I found that the command would not accept LBA of 4096, only 512. but the reformat did remove type 2 protection.

I then trimmed the drive:
Code: 
sudo fstrim -v /mnt/tank
/mnt/tank: 5.8 TiB (6331278827520 bytes) trimmed
and enabled WCE:
Code: 
sudo sdparm --set=WCE  /dev/disk/by-id/wwn-0x5002538a16c2ce8
All combined, the read performance of a single drive is 632MB/s, write is 484MB/s. No notable changes.

I don't understand how comparing two drives, both say LBA 512, but one that says Type 2 protection and one without still present as identical in terms of the amount of usable storage.

I don't understand how a drive can be both Type 2 and 512, as I thought 520 would represent the 8 bytes per block required for Type 2.

With Type 2 protection:
Code: 
=== START OF INFORMATION SECTION ===
Vendor:               IBM-D051
Product:              MZILS7T6HMLS
Revision:             MS17
Compliance:           SPC-4
User Capacity:        7,681,501,126,656 bytes [7.68 TB]
Logical block size:   512 bytes
Physical block size:  4096 bytes
Formatted with type 2 protection
8 bytes of protection information per logical block
LU is resource provisioned, LBPRZ=1
Rotation Rate:        Solid State Device
Form Factor:          2.5 inches
Logical Unit id:      0x5002538a0763d7b0
Serial number:        G1Y8109A0J600307
Device type:          disk
Transport protocol:   SAS (SPL-4)
Local Time is:        Thu May 15 14:07:05 2025 BST
SMART support is:     Available - device has SMART capability.
SMART support is:     Enabled
Temperature Warning:  Enabled
Without Type 2 protection:
Code: 
=== START OF INFORMATION SECTION ===
Vendor:               IBM-D051
Product:              MZILS7T6HMLS
Revision:             MS17
Compliance:           SPC-4
User Capacity:        7,681,501,126,656 bytes [7.68 TB]
Logical block size:   512 bytes
Physical block size:  4096 bytes
LU is resource provisioned, LBPRZ=1
Rotation Rate:        Solid State Device
Form Factor:          2.5 inches
Logical Unit id:      0x5002538a16c2ce80
Serial number:        G1Y570ZR0HC01550
Device type:          disk
Transport protocol:   SAS (SPL-4)
Local Time is:        Thu May 15 14:07:43 2025 BST
SMART support is:     Available - device has SMART capability.
SMART support is:     Enabled
Temperature Warning:  Enabled
I would like to try LBA of 4096, anyone idea why that gives this error:

Code: 
sudo sg_format -v --format --fmtpinfo=0 --pfu=0 --size=4096 /dev/disk/by-id/wwn-0x5002538a16c2ce80 --six --early
    IBM-D051  MZILS7T6HMLS      MS17   peripheral_type: disk [0x0]
      PROTECT=1
      << supports protection information>>
      Unit serial number: G1Y570ZR0HC01550 
      LU name: 5002538a16c2ce80
    mode sense(6) cdb: [1a 00 01 00 fc 00]
Mode Sense (block descriptor) data, prior to changes:
block count maxed out, set <<longlba>>
    mode sense(10) cdb: [5a 10 01 00 00 00 00 00 fc 00]
  <<< longlba flag set (64 bit lba) >>>
  Number of blocks=15002931888 [0x37e3e92b0]
  Block size=512 [0x200]
    mode select(10) cdb: [55 11 00 00 00 00 00 00 24 00]
mode select(10):
Descriptor format, current; Sense key: Illegal Request
Additional sense: Invalid field in parameter list
  Descriptor type: Sense key specific: Field pointer:
        Error in Data parameters: byte 20
  Descriptor type: Field replaceable unit code: 0xe
  Descriptor type: Vendor specific [0x80]
    00 0e
  Descriptor type: Vendor specific [0x81]
    ff ff ff ff ff ff
mode select, bad parameter list: see manufacturer's manual
MODE SELECT command: Illegal request, Invalid field in parameter list, type: sense key + asc,ascq=0x
I get the same error with 520 and 528. So I am unable to test the suggested original value. @CyklonDX

Any thoughts? would love to hear them
 
Last edited:
A

Andrew Barnes

New Member
Apr 9, 2020
13
1
3
CyklonDX said:
Its fine if its not the case.

Are you connecting those disks over dual port sas3?
*(here's example of connector)
Click to expand...
It is an interesting question, I've not discovered much information about dual port. your post on that link is the most I've seen so far.

according to smartctl, the second port is not connected:

Code: 
Protocol Specific port log page for SAS SSP
relative target port id = 1
  generation code = 5
  number of phys = 1
  phy identifier = 0
    attached device type: SAS or SATA device
    attached reason: unknown
    reason: loss of dword synchronization
    negotiated logical link rate: phy enabled; 12 Gbps
    attached initiator port: ssp=1 stp=1 smp=1
    attached target port: ssp=0 stp=0 smp=0
    SAS address = 0x5002538a0763d7b2
    attached SAS address = 0x500062b203ce7bd1
    attached phy identifier = 17
    Invalid DWORD count = 321
    Running disparity error count = 319
    Loss of DWORD synchronization count = 1
    Phy reset problem count = 0
    Phy event descriptors:
     Received ERROR count: 3
     Received address frame error count: 0
     Received abandon-class OPEN_REJECT count: 0
     Received retry-class OPEN_REJECT count: 19555
     Received SSP frame error count: 0
relative target port id = 2
  generation code = 5
  number of phys = 1
  phy identifier = 1
    attached device type: no device attached
    attached reason: unknown
    reason: power on
    negotiated logical link rate: phy enabled; unknown
    attached initiator port: ssp=0 stp=0 smp=0
    attached target port: ssp=0 stp=0 smp=0
    SAS address = 0x5002538a0763d7b3
    attached SAS address = 0x0
    attached phy identifier = 0
    Invalid DWORD count = 0
    Running disparity error count = 0
    Loss of DWORD synchronization count = 0
    Phy reset problem count = 0
    Phy event descriptors:
     Received ERROR count: 0
     Received address frame error count: 0
     Received abandon-class OPEN_REJECT count: 0
     Received retry-class OPEN_REJECT count: 0
     Received SSP frame error count: 0
Yet i'm using a supermicro M28SACB mobile rack: CSE-M28SACB-OEM | Mobile Racks | Accessories | Super Micro Computer, Inc.
connected to an LSI 9305-16i that I purchased pre flashed configured for ZFS etc from ebay. (mentioning that incase I ought to review the HBA firmware - LSI 9305-16i SAS HBA - IT Mode Unraid NAS ZFS TrueNAS Proxmox JBOD - NO BIOS | eBay UK).
Cables are: https://www.amazon.co.uk/dp/B0DQ1BLYX8

if dual port is part of sas3 spec, then I don't know what more to do to enable it
 
CyklonDX

CyklonDX

Well-Known Member
Nov 8, 2022
1,590
547
113
i think step 1 would be checking the plug on the disk itself.

if its like this, then you don't have it.

Samsung PM1643a 2.5'' SAS SSD 12Gb\s 15.36TB MZILT15THALA-00007


*this is how the connector looks on dual-link sas3 disks
1747326241301.png


but i would lean its ibm/lenovo firmware or samsung made it to much lower spec.
 

Attachments

Last edited:
A

Andrew Barnes

New Member
Apr 9, 2020
13
1
3
CyklonDX said:
i think step 1 would be checking the plug on the disk itself.

if its like this, then you don't have it.

Samsung PM1643a 2.5'' SAS SSD 12Gb\s 15.36TB MZILT15THALA-00007'' SAS SSD 12Gb\s 15.36TB MZILT15THALA-00007


*this is how the connector looks on dual-link sas3 disks
View attachment 43679


but i would lean its ibm/lenovo firmware or samsung made it to much lower spec.
Click to expand...
I can confirm I do not! There are some similar pins on the other side of the PCB. but no, the drive from the bottom looks exactly like the PM1643a that you have pictured here.

That is okay, I had no expectations of more than 12GBits bus speed. would have been cool if it did however.
 
CyklonDX

CyklonDX

Well-Known Member
Nov 8, 2022
1,590
547
113
Well no, sas2(6gbits) standard goes to 750MB/s

*you can try looking through your boot log syslog/messages whatever your system uses. You should see 'link up' logs for every disk (potentially). Alternatively you should see the controller in your bios/efi menu should see the disk link speed there.
If you go there try disabling or enabling cache it may enable/disable it on the disk itself. There are also commands to enable/disable that cache from os level.

i believe its this command *1 to enable, 0 to disable
hdparm -W 0 /dev/sdX

readahead parm *while may not show if you use hdparm -a /dev/sdX it may show you its actually turned on.
hdparm -A 0 /dev/sdx

on my sas3 ssd's write cache is not supported, but i recall 9300-16i was allowing to enable it in bios. Its worth checking and disabling or enabling if supported on your disks *see the differences.
 
Last edited:
N

nexox

Well-Known Member
May 3, 2023
1,510
729
113
You're compating SAS and U.2 (or possibly U.3) ports here, the second SAS port pins are on the opposite side of the connector from the power and primary port pins, where they're usually difficult to see.

I have occasionally looked and never found a cable to connect both ports, only backplanes and disk shelves with dual expanders.

Here's a disk that I've used in multipath mode in a dual-expander shelf, so I know it has both ports:
sas_pins.jpg

CyklonDX said:
i think step 1 would be checking the plug on the disk itself.

if its like this, then you don't have it.

Samsung PM1643a 2.5'' SAS SSD 12Gb\s 15.36TB MZILT15THALA-00007'' SAS SSD 12Gb\s 15.36TB MZILT15THALA-00007


*this is how the connector looks on dual-link sas3 disks
View attachment 43679


but i would lean its ibm/lenovo firmware or samsung made it to much lower spec.
Click to expand...
 
A

Andrew Barnes

New Member
Apr 9, 2020
13
1
3
Ok. So if they are dual port sas drives. And that aligns with the spec sheets I provided in first post.

What reasons are there then for the second port to appear as not connected? And multipathd doesn't seem to discover any multipath devices
 
CyklonDX

CyklonDX

Well-Known Member
Nov 8, 2022
1,590
547
113
  • Like
Reactions: nexox
CyklonDX

CyklonDX

Well-Known Member
Nov 8, 2022
1,590
547
113
well yeah, i just compared with intel p4600 u.2 and it does seem to be u.2 connector. Thus it means the 2nd signal channel can be present below or on top.
i'd presume the traces above are not being used on sas3 wd or toshiba and they only made it to streamline process or ?something? *the pcie u.2 traces are connected on px05s sas ssd.
 
You must log in or register to reply here.
Top Bottom