Part of building my ESX cluster in my garage was the addition of a 10 gigabit network for the storage layer. This storage layer allows all of the ESX hosts to talk to my FreeNAS iSCSI target which will host VMs and whatnot. The basic components of this storage fabric are a Fujitsu XG2000C 16-port 10gig Ethernet switch and four HP Infiniband 4X DDR Dual Port HBAs which are basically rebranded Mellanox ConnectX Infiniband adapters – model number MT25418.
I flashed the HP HBA with the following steps to turn it into a stock Mellanox MT25418 card with the following steps run from a windows 10 box. Briefly it goes something like:
Install Mellanox Firmware Tools (MFT) - WinMFT x64 4.9.0.38.exe Install MLNX_VPI - Mellanox Virtual Protocol Interconnect (VPI Drivers) ##get device name: Open Command Prompt mst status ## Firmware to Mellanox 2.9.1 flint -d mt25418_pciconf0 -i fw-25408-2_9_1000-MHGH28-XTC_A2-A3.bin -nofs -allow_psid_change burn
Then install the drivers. The drivers are the Mellanox mlx4_en bundle v1.6.1.2 that I have copied to my cloudfront distribution here.
To install the drivers, I first had to turn on SSH on the host, which requires a few steps:
– Download an install the vSphere client locally
– launch the client and point it to the ESX host
– Under Configuration – Under Security Profile – Under Service, click properties
– click SSH, click options, click start click ok, click ok
Then copy the driver bundle to /tmp/ on the host using WinSCP.
Now launch a putty session to the esx host and run the following command:
esxcli software vib install -d /tmp/mlx4_en-mlnx-1.6.1.2-offline_bundle-471530.zip
Then reboot.
When the host comes back up, there will be new entries for the Mellanox adapter
