VMware 6.5 and A2SDi-16C-HLN4F (cluster 2-node)

[Marco Neri]

Hello guys
I want to create a laboratory environment using No. 2 A2SDi-16C-HLN4F and VMware 6.5 (cluster). I wanted to know if any of you have found compatibility problems with this combination.
I have read that the x553 network cards are not recognized.
another thing the cluster should handle around 20 vm. the cpu is able to handle these loads?

Thank you

 

[Peanuthead]

You can inject the drivers into the ISO. That would solve your first issue. In regard to CPU load no one can answer that. We have no idea what those virtual machines will be doing or will not be doing.

 

[Marco Neri]

You can inject the drivers into the ISO. That would solve your first issue. In regard to CPU load no one can answer that. We have no idea what those virtual machines will be doing or will not be doing.

Thanks Peanuthead,
you're right the loads of vm are low, with few requests.
in short, the roles will be these:
1 vcenter HA (3vm)
2 domain controller 2016
1 exchange 2016
1 WDS & Veeam
and 10 vm linux to test various services, nagios, ocs inventory, owncloud.

I searched the internet for the vib related to x553 but nothing. I only found this but we talk about the x552.

How to download and install the Intel Xeon D 10GbE X552/X557 driver/VIB for VMware ESXi 6.x, works with the X540 PCIe card too | TinkerTry IT @ Home

sorry for my english

Thank you

 

[Craig Thomson]

I searched the internet for the vib related to x553 but nothing. I only found this but we talk about the x552.

I am also looking for X553 drivers for ESXi 6.5. Like Marco, I have only been able to find references to X552 and X557.

The VMware Compatibility Guide lists no results when searching for X553. However, searching for X552 or X557 does produce results.

Does this mean no X553 drivers are available for ESXi at present?

 

[Marco Neri]

I am also looking for X553 drivers for ESXi 6.5. Like Marco, I have only been able to find references to X552 and X557.

The VMware Compatibility Guide lists no results when searching for X553. However, searching for X552 or X557 does produce results.

Does this mean no X553 drivers are available for ESXi at present?

Hi Craig,
I found these drivers that refer to the model x552 / 557 I am attaching the driver zip.

from what I read around the x553 should take advantage of the same igbx driver related to 10gb cards.

at this moment I can not try to install the vib, because I'm still waiting to receive the 2 motherboards.

let me know thanks

 

[Peanuthead]

The other drivers may work. Here is what I found out. Same nic, different board.

https://www.servethehome.com/day-0-with-intel-atom-c3000-getting-nics-working/

 

[Craig Thomson]

Hi Craig,
I found these drivers that refer to the model x552 / 557 I am attaching the driver zip.

from what I read around the x553 should take advantage of the same igbx driver related to 10gb cards.

at this moment I can not try to install the vib, because I'm still waiting to receive the 2 motherboards.

Hi Marco,

Thanks for the attachment. Unfortunately, I am also waiting on delivery of my motherboard, so at this moment I cannot try to install the vib either.

I have read the same thing as you - that the ixgbe driver should work for X553. It is supposed to be the same as the X552/X557 but just with a different device ID. With that in mind, I decided to look through the source code within the attachment to see what I could find.

The source README contains the following:

The driver in this release is compatible with devices based on the following:
  * Intel(R) Ethernet Controller 82598
  * Intel(R) Ethernet Controller 82599
  * Intel(R) Ethernet Controller X540
  * Intel(R) Ethernet Controller x550
  * Intel(R) Ethernet Controller X552
  * Intel(R) Ethernet Controller X553

This seems to suggest that the X553 is supported by v4.5.3 of the ixgbe driver. However, when I looked a little deeper what I found suggested that the README may be wrong.

The Linux Kernel Driver DataBase lists the device IDs for the ixgbe driver which I have listed below.

X552: 15aa, 15ab, 15ac, 15ad, 15ae, 15b0
X553: 15c2, 15c3, 15c4, 15c6, 15c7, 15c8, 15ce, 15e4, 15e5

When I checked the source code for these device IDs I found the following:

[851] craig@server2:~/ixgbe/source/sandbox/587143/src/ixgbe > grep -ilP '15(a[a-e]|b0)' *
ixgbe_type.h
ixgbe.xml
pci.updates
[852] craig@server2:~/ixgbe/source/sandbox/587143/src/ixgbe > grep -ilP '15(c[234678e]|e[45])' *
[853] craig@server2:~/ixgbe/source/sandbox/587143/src/ixgbe > echo $?
1
[854] craig@server2:~/ixgbe/source/sandbox/587143/src/ixgbe >

As you can see, the device IDs for the X552 are found within the source code, but the device IDs for the X553 are not.

I also checked for direct references to X552 and X553:

[882] craig@server2:~/ixgbe/source/sandbox/587143/src/ixgbe > grep -il 'X552' *
ixgbe_type.h
ixgbe_x550.c
ixgbe.xml
pci.updates
README
[883] craig@server2:~/ixgbe/source/sandbox/587143/src/ixgbe > grep -il 'X553' *
README
[884] craig@server2:~/ixgbe/source/sandbox/587143/src/ixgbe >

Although this result looks bad we really can't be sure unless we actually try to install the vib.

 

[Craig Thomson]

The other drivers may work. Here is what I found out. Same nic, different board.

https://www.servethehome.com/day-0-with-intel-atom-c3000-getting-nics-working/

I found that link too, Peanuthead. Did you actually read it?

I appreciate you trying to help, but that link contains nothing relating to solving this problem. It only restates the problem.

 

[Marco Neri]

Hi Marco,

Thanks for the attachment. Unfortunately, I am also waiting on delivery of my motherboard, so at this moment I cannot try to install the vib either.

I have read the same thing as you - that the ixgbe driver should work for X553. It is supposed to be the same as the X552/X557 but just with a different device ID. With that in mind, I decided to look through the source code within the attachment to see what I could find.

The source README contains the following:

The driver in this release is compatible with devices based on the following:
  * Intel(R) Ethernet Controller 82598
  * Intel(R) Ethernet Controller 82599
  * Intel(R) Ethernet Controller X540
  * Intel(R) Ethernet Controller x550
  * Intel(R) Ethernet Controller X552
  * Intel(R) Ethernet Controller X553

This seems to suggest that the X553 is supported by v4.5.3 of the ixgbe driver. However, when I looked a little deeper what I found suggested that the README may be wrong.

The Linux Kernel Driver DataBase lists the device IDs for the ixgbe driver which I have listed below.

X552: 15aa, 15ab, 15ac, 15ad, 15ae, 15b0
X553: 15c2, 15c3, 15c4, 15c6, 15c7, 15c8, 15ce, 15e4, 15e5

When I checked the source code for these device IDs I found the following:

[851] craig@server2:~/ixgbe/source/sandbox/587143/src/ixgbe > grep -ilP '15(a[a-e]|b0)' *
ixgbe_type.h
ixgbe.xml
pci.updates
[852] craig@server2:~/ixgbe/source/sandbox/587143/src/ixgbe > grep -ilP '15(c[234678e]|e[45])' *
[853] craig@server2:~/ixgbe/source/sandbox/587143/src/ixgbe > echo $?
1
[854] craig@server2:~/ixgbe/source/sandbox/587143/src/ixgbe >

As you can see, the device IDs for the X552 are found within the source code, but the device IDs for the X553 are not.

I also checked for direct references to X552 and X553:

[882] craig@server2:~/ixgbe/source/sandbox/587143/src/ixgbe > grep -il 'X552' *
ixgbe_type.h
ixgbe_x550.c
ixgbe.xml
pci.updates
README
[883] craig@server2:~/ixgbe/source/sandbox/587143/src/ixgbe > grep -il 'X553' *
README
[884] craig@server2:~/ixgbe/source/sandbox/587143/src/ixgbe >

Although this result looks bad we really can't be sure unless we actually try to install the vib.

Hi Craig, good job, your reasoning is correct, the certainty we can have only trying the vib. it seems strange to me that, after one year from the release, the network driver is still not available. I have already equipped myself with a gigabit i350-t4 card but anyway my goal is to make it work in some way. the delivery of the cards is scheduled for the end of the month, let's update. Hello

 

[Marco Neri]

Hi Craig, I got my motherboards, I tried to install the drivers without positive results. network cards are not recognized by esxi. I'm doing further research but I think that for the moment there is no solution. I am currently using an i350-t4, I hope that soon VMware will make the new drivers available. the only possible way to use the cards is through the directpath to the virtual machines, but obviously it is a solution that goes against the benefits of virtualization. Does anyone on the forum have a solution? Thank you

 

[SlinkingAnt]

Almost the same issue here, I've bought the A2SDV-16C-TLN5F, with 4x X557-10GbE, but these also won't show up in ESXi 6.5 U1.

I tried loading the latest drivers from the vmware-site, and although the X557 should be supported with this driver, the DeviceID of the NIC is not in the list of supported devices. Other X557-devices are supported.

You can check the Device ID in ESXi with this command: 'esxcli hardware pcu list'
The supported devices are listed in this file after installation of the net-ixgbe driver: '/usr/share/hwdata/driver.pciids.d/ixgbe.ids'

Currently trying to figure out how to manually add my DeviceID to the list of supported devices, but so far no luck/

 

[Marco Neri]

in my case the device id is 0x15e4 as verified by craig.

Honestly I do not know how to modify these parameters to make the driver functional.

 

[Craig Thomson]

Sorry for the late reply. For some reason I'm not receiving email notifications when this thread is updated.

Unfortunately, my supplier shipped me the wrong motherboard. I sent it back, but now I'm waiting on the correct motherboard to be shipped, so I could be waiting for some time.

Marco, thanks for the update on your testing. It's disappointing that it did not work (especially when the README says X553 is supported).

The next step I will try is to create a custom vib. I will modify the ixgbe source code to support the X553 device IDs and recompile. The ixgbe driver should work for X553, I just have to make ESXi recognise the X553 device IDs. I've successfully compiled code for ESXi 6.0 before, but I've never created a vib before, so I'll have to do some reading for that part. Hopefully, I will get some time during the next couple of weekends.

SlinkingAnt, if you post your device IDs I'll include them in the code update.

Once I've created the custom vib I'll need to test it. My current ESXi 6.0U1 server is using very old hardware (it only has a single Intel 82566DC NIC), so I will be able to test installation of the vib but I won't be able to test the driver itself. The only option for that will be to wait until my correct motherboard arrives, or if either of you are willing to test the vib I can post it here - let me know.

But, let's see how I go creating the custom vib first.

 

[Marco Neri]

Sorry for the late reply. For some reason I'm not receiving email notifications when this thread is updated.

Unfortunately, my supplier shipped me the wrong motherboard. I sent it back, but now I'm waiting on the correct motherboard to be shipped, so I could be waiting for some time.

Marco, thanks for the update on your testing. It's disappointing that it did not work (especially when the README says X553 is supported).

The next step I will try is to create a custom vib. I will modify the ixgbe source code to support the X553 device IDs and recompile. The ixgbe driver should work for X553, I just have to make ESXi recognise the X553 device IDs. I've successfully compiled code for ESXi 6.0 before, but I've never created a vib before, so I'll have to do some reading for that part. Hopefully, I will get some time during the next couple of weekends.

SlinkingAnt, if you post your device IDs I'll include them in the code update.

Once I've created the custom vib I'll need to test it. My current ESXi 6.0U1 server is using very old hardware (it only has a single Intel 82566DC NIC), so I will be able to test installation of the vib but I won't be able to test the driver itself. The only option for that will be to wait until my correct motherboard arrives, or if either of you are willing to test the vib I can post it here - let me know.

But, let's see how I go creating the custom vib first.

Hi Craig, thank you very much, once you have created the vib, post it also that the testing on my esxi. thank you

 

[SlinkingAnt]

If you can include the DeviceID 8086:15C8 for the X557-AT4 card....

 

[Marco Neri]

Hi Craig,
here are some links that might be useful.

VMware Front Experience: ESXi Community Packaging Tools
Creating Custom VIBs For ESXi 5.0 & 5.1 with VIB Author Fling
How to create your own .vib files - Yellow Bricks

bye

 

[Craig Thomson]

I have bad news unfortunately. Simply adding the missing device IDs will not work.

I spent the weekend reviewing the VMware source code v4.5.3 and compared it to the original Intel source code on which it is based.

Here is what I found:

1. The VMware source code v4.5.3 appears to be based on the Intel source code v5.1.3 which was released 18 May 2017. That means the VMware driver is 10 months behind the Intel driver. The Intel source code is now at v5.3.6.

2. The Intel source (both v5.1.3 and v5.3.6) contains a lot of code that is specific to the X553 and X557. Each NIC type (X552/3/7) supports different feature sets, modes, responses, etc. which the code deals with. That code is completely missing from the VMware source v4.5.3. So, even if I add the missing device IDs the driver itself will not work with X553/7.

3. The X553/7 code in the Intel source v5.1.3 was intentionally removed by VMware when they ported it to v4.5.3. Although they removed the code they forgot to update the README (that's why it still lists X553). The fact that VMware removed this code worries me. They must have had a good reason to remove it. Attempting to re-insert the code may not work.

4. The VMware source v4.5.3 has been significantly modified to enable it to work on ESXi. When compared against the Intel source v5.1.3 there are significant changes.

So, no matter which way I decide to approach this it's going to be a big job, and there's no guarantee I will be able to make it work.

I can either:
1. Try to re-insert the X553/7 code from Intel v5.1.3 back into VMware v4.5.3
2. Try to add the VMware modifications from VMware v4.5.3 into Intel v5.1.3
3. Try to add the VMware modifications from VMware v4.5.3 into Intel v5.3.6

I think option 1 would be the easiest but that still means we'll be using a 10 month old driver. I'd prefer to do option 3 but I think that will most likely be the hardest to accomplish.

No matter which option I choose, it's going to take a lot of time (which I don't have much of right now) and I will only be able to do it once my own motherboard finally arrives.

To be honest, I think the best option would be to put pressure on VMware to update the driver themselves.

 

[Marco Neri]

I have bad news unfortunately. Simply adding the missing device IDs will not work.

I spent the weekend reviewing the VMware source code v4.5.3 and compared it to the original Intel source code on which it is based.

Here is what I found:

1. The VMware source code v4.5.3 appears to be based on the Intel source code v5.1.3 which was released 18 May 2017. That means the VMware driver is 10 months behind the Intel driver. The Intel source code is now at v5.3.6.

2. The Intel source (both v5.1.3 and v5.3.6) contains a lot of code that is specific to the X553 and X557. Each NIC type (X552/3/7) supports different feature sets, modes, responses, etc. which the code deals with. That code is completely missing from the VMware source v4.5.3. So, even if I add the missing device IDs the driver itself will not work with X553/7.

3. The X553/7 code in the Intel source v5.1.3 was intentionally removed by VMware when they ported it to v4.5.3. Although they removed the code they forgot to update the README (that's why it still lists X553). The fact that VMware removed this code worries me. They must have had a good reason to remove it. Attempting to re-insert the code may not work.

4. The VMware source v4.5.3 has been significantly modified to enable it to work on ESXi. When compared against the Intel source v5.1.3 there are significant changes.

So, no matter which way I decide to approach this it's going to be a big job, and there's no guarantee I will be able to make it work.

I can either:
1. Try to re-insert the X553/7 code from Intel v5.1.3 back into VMware v4.5.3
2. Try to add the VMware modifications from VMware v4.5.3 into Intel v5.1.3
3. Try to add the VMware modifications from VMware v4.5.3 into Intel v5.3.6

I think option 1 would be the easiest but that still means we'll be using a 10 month old driver. I'd prefer to do option 3 but I think that will most likely be the hardest to accomplish.

No matter which option I choose, it's going to take a lot of time (which I don't have much of right now) and I will only be able to do it once my own motherboard finally arrives.

To be honest, I think the best option would be to put pressure on VMware to update the driver themselves.

I would say try with the solution 1. when you have time obviously. otherwise we have to wait for vmware to update the driver to x553.

 

[Marco Neri]

pFsense has updated the version to 2.4.3, making the atom c3000 series compatible.
2.4.3 New Features and Changes - pfSense Documentation


I hope that it will soon be VMware's turn

 

[Craig Thomson]

I've been told that my motherboard has finally landed in the country (Australia). I should have it in about 1 week.

I'll try to find some time to do the code updates soon.

Marco - Good to know that pFsense has been updated. As you say, let's hope VMware follow soon.