Search results

  1. C

    Confusing Mellanox Mezzanine Cards for C6320

    Here's a table I'm starting with information on various Dell cards. I **THINK** these are all mezzanine cards. I'm having trouble finding firmware for my 3CYRK so maybe I will try to force 1965W update if I can find that firmware as they are as close as you can get to the same thing AFAIK...
  2. C

    Swtiching over Multiple Uplinks on VMware VDS

    Came across this info and wanted to cross link it under my original question for any others trying to understand why a vSwitch is not a switch... https://blog.ipspace.net/2010/11/vmware-virtual-switch-no-need-for-stp.html I wonder if there is a way to override some of this default behavior but...
  3. C

    Direct Connect vSphere 7 with 3 or 4 Hosts Possible?

    So I have caved in and purchased a Mikrotik 10G switch but I am STILL wanting to configure some direct connections between hosts. The main thing about having hardware switches is that I don't want to pay for an extra 1U at my data center. So I'm actually going to rig up the switch in an empty...
  4. C

    Is "vSAN Recovery" during yellow boot screen normal?

    I just want to confirm that "vSAN Recovery" during boot is not a normal thing? If it were not BAD then I would assume they'd use "initializing" or some other nomenclature that didn't suggest something had gone awry. Is this called PLOG recovery? Just want to make sure that this is...
  5. C

    Confusing Mellanox Mezzanine Cards for C6320

    I am continuing my saga with these cards and trying to update firmware. So far cannot find it but have inquired on Dell forums. Here's a repost from that thread of what I have encountered so far (so this info will be archived for others here as well). To run these updates on linux, you...
  6. C

    Swtiching over Multiple Uplinks on VMware VDS

    SOME IDEAS ON MOVING FORWARD FROM HERE..... I'm going to try a hybrid of direct connections and soft swtiches (Open vSwtich, routerOS, pfSense, etc). GOAL: Since directly connecting 2 nodes DOES work and is a very robust connection, and can provide the best line speeds, it would be a shame to...
  7. C

    Swtiching over Multiple Uplinks on VMware VDS

    Couple of points for anyone considering trying this.... don't overlook/underestimate the chicken and egg problems in a setup like this, especially with vSAN. I got a setup working at like 2.5gbps with Mikrotik Cloud Hosted Router (no optimizations or jumbo frames) and then mindlessly made...
  8. C

    Swtiching over Multiple Uplinks on VMware VDS

    Shifting gears to investigate options such as RouterOS, pfSense, vyOS etc as I am going to need more than just a port on a vDS anyway. Maybe Open vSwitch. There is a nice configuration detailed on Proxmox wiki where it looks like an instance is hosted on every physical host and then there is...
  9. C

    Swtiching over Multiple Uplinks on VMware VDS

    This is frustrating. Why can't I add a physical nic to a vDS and get it to function like a normal switch and switch port? Back in 2015, someone on VMware boards did some extensive attempts to get something like this working. He thought he had it working by using several vDS's instead of one...
  10. C

    Swtiching over Multiple Uplinks on VMware VDS

    So I'm trying to cross connect my physical ESXi hosts with 40GbE direct connect cables because that's the only way I can get 40GbE.... (and I don't want to pay data center to host a 1U switch that would be only 1GbE or 10GbE max anyway). I do know that VMware only officially supports 2 node...
  11. C

    C6100/C6220 Mellanox 40Gbe mezzanine cards $69.99 BO'd 4 @ $25

    Sorry to bump this old thread but wanted to add a little info.... both 3CYRK and Y5WKX work with C6320 sleds (using 4NWN5 bridge card) but what is odd is that the 3CYRK/BCAA is also coming up as PRO model whereas I thought FCAA is the better card with VPI? Or is with/without VPI completely...
  12. C

    Confusing Mellanox Mezzanine Cards for C6320

    I've purchased a couple mezzanine cards off ebay and am a bit confused. The BCAA/3CYRK I believe was originally sold by Dell for the C6220 and I bought 3 of these to try in my C6320 sleds. THEY DO WORK (use 4NWN5 bridge card) and they come up as Connectx-3 PRO and MT27520 family. Since I...
  13. C

    The Dell PowerEdge C6320/C6300 Mega Thread

    Let's talk about MEZZANINE CARDS for a moment.... there are a bunch of interposer/bridge cards out there that LOOK like they may be interchangable out there but they are NOT compatible! JKM5M Printed Wiring Assy.Daughterboard.Serial Attached Scsi.Bridge.X3.Data Center Solutions HH4P1...
  14. C

    Direct Connect vSphere 7 with 3 or 4 Hosts Possible?

    Looks like VMWare does NOT recommend placing witness on another 2 node cluster, or on another stretched cluster. https://storagehub.vmware.com/t/vmware-r-vsan-tm-network-design/2-node-vsan-deployments-common-config-questions/ Kind of sucks.... seemed like it might be a nice solution for a...
  15. C

    Direct Connect vSphere 7 with 3 or 4 Hosts Possible?

    Interesting, thanks. I see some posts on Mellanox forums that seem to confirm what you are saying at least for their cx3 and earlier products but this is interesitng on Intel forums.... looks like some of their NIC's can handle it...
  16. C

    Direct Connect vSphere 7 with 3 or 4 Hosts Possible?

    From wikipedia..... "QSFP+ is an evolution of QSFP to support four 10 Gbit/s channels carrying 10 Gigabit Ethernet, 10GFC FiberChannel, or QDR InfiniBand.[45] The 4 channels can also be combined into a single 40 Gigabit Ethernet link."
  17. C

    Direct Connect vSphere 7 with 3 or 4 Hosts Possible?

    I am not sure..... right now they are not a primary option I'm considering but in the cursory resarch I've done, it sounded like all 40gbe actually has 4x 10gb "lanes" underneath. If that is the case then it just seemed natural that the connection could be split into these lanes and that's why...
  18. C

    Direct Connect vSphere 7 with 3 or 4 Hosts Possible?

    This was my initial plan until I started reading today that people really frowned on trying to direct connect 3 nodes. I really don't get why it's a problem when the whole point of virtualization was to move away from physical equipement and vSphere is suppose to have robust networking built...
  19. C

    Direct Connect vSphere 7 with 3 or 4 Hosts Possible?

    Actually there are 2x QSFP+ AND 2x SFP+ ports. I don't have any onboard standard RJ45 so will need to have WAN and LAN uplinks with some of them. I could possibly add in some quad port 1gbase-t nics for uplinks then use the 4x sfp/qsfp ports on each node to max speed for inter-node connections...
  20. C

    Direct Connect vSphere 7 with 3 or 4 Hosts Possible?

    Looks like VMWare already advises the use of "stretched" topologies in a single location, for example in this document for "rack awareness" https://storagehub.vmware.com/t/vmware-r-vsan-tm-network-design/stretched-cluster-deployments/ THe other thing in more recent versions is moving away from...