I can't understand why this is happening. I have a Broadcom 9600W-16e controller. There are two disk shelves, and the controller is connected with two SAS 4 cables to the disk shelf expander. From disk shelf 1, another shelf is connected with two SAS 4 cables. In disk shelf 2, there are 11 Samsung SAS 4 SSDs installed, and configured to JBOD.
Problem: When I put a load on each disk using fio with bs=1M, numjobs=1, iodepth=32 on 8 disks in the second shelf, the total performance is 14500 MB/s. However, when I run the test on 9 disks, the total performance drops by 8% to 13500 MB/s; for 10 disks, it drops to 12700 MB/s; and for 11 disks, it further drops to 11600 MB/s.
If I connect this shelf directly to the server, the overall speed for 8-9-10-11 disks remains consistent at around 16500 MB/s.
What I noticed is that when connecting any shelf to the controller, using storcli2 /c0 show in the Enclosure List shows: State = Faulty, but I don't see any issues in operation.
Operating system: Ubuntu.
Controller firmware version: 8.10.1
Driver: mpi3mr
