In the Lab: Mellanox SX1024 Switch Configuration

In today's look into the StorageReview Lab, we are working on the initial configuration of a new Mellanox 10/40Gb Ethernet switch, the SX1024. Offering 48 10Gb SFP+ ports and 12 40/56Gb QSFP+ ports, it adds an immense number of ports to our top-of-rack connectivity for high-speed storage arrays and host connections. To get the switch up and running in our lab environment, we make a number of changes from the stock shipping configuration, which in most cases is a blank switch.

In today's look into the StorageReview Lab, we are working on the initial configuration of a new Mellanox 10/40Gb Ethernet switch, the SX1024. Offering 48 10Gb SFP+ ports and 12 40/56Gb QSFP+ ports, it adds an immense number of ports to our top-of-rack connectivity for high-speed storage arrays and host connections. To get the switch up and running in our lab environment, we make a number of changes from the stock shipping configuration, which in most cases is a blank switch.

Once the management IP address has been sorted out, either through finding its initial DHCP address or connecting over a serial connection (those still exist!) we start the configuration process. In our lab we use flow control on our switches, which for the Mellanox gear you can enable both send and receive flow control separately. We also take advantage of the Mellanox fan-out port configuration, which allows us to use some of the QSFP+ ports to connect 4 SFP+ devices to it.

To configure the Mellanox SX1024 switch, users must be familiar with the MLNX-OS switch software, as well pretty handy with a Putty session. Once logged into the switch you enable the configuration mode and start addressing each port.  The "module-type qsfp-split-4 force" command allows users to split certain QSFP+ ports into four SFP+ connections, which different switches in the Mellanox lineup have limits on. On the SX1024 two ports can be configured in this mode, with disable the ports next to them.

To show where we ended up after a few minutes of setup, I've pasted some of the console output below. The main things addressed were enabling send and receive flow control on all 10G ports, and splitting two ports into 4 10Gb SFP+ connections.

SR-1024 [standalone: master] # show configuration

   interface ethernet 1/59 module-type qsfp-split-4 force
   interface ethernet 1/60 module-type qsfp-split-4 force

   interface ethernet 1/1 flowcontrol receive on force
   interface ethernet 1/1 flowcontrol send on force
   interface ethernet 1/2 flowcontrol receive on force
   interface ethernet 1/2 flowcontrol send on force
   interface ethernet 1/3 flowcontrol receive on force
   interface ethernet 1/3 flowcontrol send on force
   interface ethernet 1/4 flowcontrol receive on force
   interface ethernet 1/4 flowcontrol send on force
   interface ethernet 1/5 flowcontrol receive on force
   interface ethernet 1/5 flowcontrol send on force
   interface ethernet 1/6 flowcontrol receive on force
   interface ethernet 1/6 flowcontrol send on force
   interface ethernet 1/7 flowcontrol receive on force
   interface ethernet 1/7 flowcontrol send on force
   interface ethernet 1/8 flowcontrol receive on force
   interface ethernet 1/8 flowcontrol send on force
   interface ethernet 1/9 flowcontrol receive on force
   interface ethernet 1/9 flowcontrol send on force
   interface ethernet 1/10 flowcontrol receive on force
   interface ethernet 1/10 flowcontrol send on force
   interface ethernet 1/11 flowcontrol receive on force
   interface ethernet 1/11 flowcontrol send on force
   interface ethernet 1/12 flowcontrol receive on force
   interface ethernet 1/12 flowcontrol send on force
   interface ethernet 1/13 flowcontrol receive on force
   interface ethernet 1/13 flowcontrol send on force
   interface ethernet 1/14 flowcontrol receive on force
   interface ethernet 1/14 flowcontrol send on force
   interface ethernet 1/15 flowcontrol receive on force
   interface ethernet 1/15 flowcontrol send on force
   interface ethernet 1/16 flowcontrol receive on force
   interface ethernet 1/16 flowcontrol send on force
   interface ethernet 1/17 flowcontrol receive on force
   interface ethernet 1/17 flowcontrol send on force
   interface ethernet 1/18 flowcontrol receive on force
   interface ethernet 1/18 flowcontrol send on force
   interface ethernet 1/19 flowcontrol receive on force
   interface ethernet 1/19 flowcontrol send on force
   interface ethernet 1/20 flowcontrol receive on force
   interface ethernet 1/20 flowcontrol send on force
   interface ethernet 1/21 flowcontrol receive on force
   interface ethernet 1/21 flowcontrol send on force
   interface ethernet 1/22 flowcontrol receive on force
   interface ethernet 1/22 flowcontrol send on force
   interface ethernet 1/23 flowcontrol receive on force
   interface ethernet 1/23 flowcontrol send on force
   interface ethernet 1/24 flowcontrol receive on force
   interface ethernet 1/24 flowcontrol send on force
   interface ethernet 1/25 flowcontrol receive on force
   interface ethernet 1/25 flowcontrol send on force
   interface ethernet 1/26 flowcontrol receive on force
   interface ethernet 1/26 flowcontrol send on force
   interface ethernet 1/27 flowcontrol receive on force
   interface ethernet 1/27 flowcontrol send on force
   interface ethernet 1/28 flowcontrol receive on force
   interface ethernet 1/28 flowcontrol send on force
   interface ethernet 1/29 flowcontrol receive on force
   interface ethernet 1/29 flowcontrol send on force
   interface ethernet 1/30 flowcontrol receive on force
   interface ethernet 1/30 flowcontrol send on force
   interface ethernet 1/31 flowcontrol receive on force
   interface ethernet 1/31 flowcontrol send on force
   interface ethernet 1/32 flowcontrol receive on force
   interface ethernet 1/32 flowcontrol send on force
   interface ethernet 1/33 flowcontrol receive on force
   interface ethernet 1/33 flowcontrol send on force
   interface ethernet 1/34 flowcontrol receive on force
   interface ethernet 1/34 flowcontrol send on force
   interface ethernet 1/35 flowcontrol receive on force
   interface ethernet 1/35 flowcontrol send on force
   interface ethernet 1/36 flowcontrol receive on force
   interface ethernet 1/36 flowcontrol send on force
   interface ethernet 1/37 flowcontrol receive on force
   interface ethernet 1/37 flowcontrol send on force
   interface ethernet 1/38 flowcontrol receive on force
   interface ethernet 1/38 flowcontrol send on force
   interface ethernet 1/39 flowcontrol receive on force
   interface ethernet 1/39 flowcontrol send on force
   interface ethernet 1/40 flowcontrol receive on force
   interface ethernet 1/40 flowcontrol send on force
   interface ethernet 1/41 flowcontrol receive on force
   interface ethernet 1/41 flowcontrol send on force
   interface ethernet 1/42 flowcontrol receive on force
   interface ethernet 1/42 flowcontrol send on force
   interface ethernet 1/43 flowcontrol receive on force
   interface ethernet 1/43 flowcontrol send on force
   interface ethernet 1/44 flowcontrol receive on force
   interface ethernet 1/44 flowcontrol send on force
   interface ethernet 1/45 flowcontrol receive on force
   interface ethernet 1/45 flowcontrol send on force
   interface ethernet 1/46 flowcontrol receive on force
   interface ethernet 1/46 flowcontrol send on force
   interface ethernet 1/47 flowcontrol receive on force
   interface ethernet 1/47 flowcontrol send on force
   interface ethernet 1/48 flowcontrol receive on force
   interface ethernet 1/48 flowcontrol send on force
   interface ethernet 1/59/1 flowcontrol receive on force
   interface ethernet 1/59/1 flowcontrol send on force
   interface ethernet 1/59/2 flowcontrol receive on force
   interface ethernet 1/59/2 flowcontrol send on force
   interface ethernet 1/59/3 flowcontrol receive on force
   interface ethernet 1/59/3 flowcontrol send on force
   interface ethernet 1/59/4 flowcontrol receive on force
   interface ethernet 1/59/4 flowcontrol send on force
   interface ethernet 1/60/1 flowcontrol receive on force
   interface ethernet 1/60/1 flowcontrol send on force
   interface ethernet 1/60/2 flowcontrol receive on force
   interface ethernet 1/60/2 flowcontrol send on force
   interface ethernet 1/60/3 flowcontrol receive on force
   interface ethernet 1/60/3 flowcontrol send on force
   interface ethernet 1/60/4 flowcontrol receive on force
   interface ethernet 1/60/4 flowcontrol send on force

The SX1024 is now ready to deploy and get to work in the lab. 

Product Page

Discuss This Story