ESXi and Cisco VIRL home Lab - Hardware Options

[bishun]

Hey all, I'm a network engineer by trade and have played with ESXi in the past. I have a HP ProLiant N40L that I don't often turn on anymore because it really isn't enough machine to do much, luckily it wasn't a huge investment either.

This is my first post here, hopefully this is an OK sub-forum for this topic.

I recently started looking at Cisco VIRL which is a virtual environment for Cisco's networking OS's, and it supports some of them that GNS3 can't (Nexus, and ASA devices). So here I am looking at server options again for the first time in a few years.

After some brief searching, the Supermicro SYS-E300-8D caught my eye (Tenet's E300-8D thread brought me here). I have a 13U rack which is now mostly empty due to selling off Cisco lab hardware, so I'm leaning towards a rack-mountable server.

Hoping to hear some possible suggestions for the following requirements, basically in this order of importance:

1. Quiet, I need to run this in my apartment, and it will be in or near the living room. It can be above a whisper of course, but this is the reason I'm not buying a used Dell off of eBay or something.

2. Powerful enough to run ESXi and VIRL (bare-metal) without being disappointed by performance. I'll boot into either as needed. I'm thinking at any given time if I was booting into ESXi to play, I'd want to be able to spin up at least 4 VMs - don't need fancy graphics, these would probably be a mixture of WIN/*NIX.

3. Cost / Power consumption - I'm willing to throw some money at something if it's meeting the above requirements. I'm also hoping not to spend more than $2,000, at least not immediately. The cost is less important than having a sufficiently quiet/powerful virtual lab.

Whatever option I go with it will be an Intel CPU. The Supermicro SYS-E300-8D may be the right choice, but I'm not sure how quiet I can get it. I don't have a fear of swapping out parts in servers, or in other words whatever solution I come up with doesn't need to be a perfect one right out of the box.

By all means ask me more questions about needs or make recommendations. Links to other posts, here, elsewhere, blogs, etc are all appreciated.

 

[CreoleLakerFan]

There is some good information on VIRL performance in this article:

My Experiences With Cisco's VIRL - Packet Pushers -

I have a couple VIRL instances which I power up and down. One instance runs on my home ESX All-In-One server, which is an E5-2670 v1 w/64 GB RAM. I allocate four cores and 32GB of RAM to VIRL ... which is somewhat limited when it comes to larger CCIE topologies, but I'm honestly not there yet - I'm still working at the CCNP level. This server was built for quiet and tolerable power draw since it sits in a room inside of my house.

I have another VIRL server at the office - it's a dual Westmere 12/24 w/96 GB - it runs everything I've been able to dream up so far, including the INE CCIE v5 Topologies, which I put together for giggles since again - not really at CCIE level yet in my Cisco studies. Eventually I am going to relocate my office chassis server into a rack-mounted chassis housed in my 24U rack in my garage, so I can integrate VIRL/QinQ with my physical Cisco/Juniper switches and routers (frame relay) plus Juniper gear.

In my experience with VIRL RAM is more of a limiting factor than CPU performance. Again, highly dependent on what you are using VIRL for - if you start getting into CCIE topologies you will find that those IOSXr (CSR1000v) images suck up huge amounts of RAM. That said, while the D1518 you are looking can take as much as 128GB, but it might be a little light in the pants when it comes to CPU grunt for CCIE VIRL topologies.

**edited for grammar**

 

[bishun]

Thanks CreoleLakerFan, I'll dig through that blog link.

 

[John Moreno]

Hey all, I'm a network engineer by trade and have played with ESXi in the past. I have a HP ProLiant N40L that I don't often turn on anymore because it really isn't enough machine to do much, luckily it wasn't a huge investment either.

This is my first post here, hopefully this is an OK sub-forum for this topic.

I recently started looking at Cisco VIRL which is a virtual environment for Cisco's networking OS's, and it supports some of them that GNS3 can't (Nexus, and ASA devices). So here I am looking at server options again for the first time in a few years.

After some brief searching, the Supermicro SYS-E300-8D caught my eye (Tenet's E300-8D thread brought me here). I have a 13U rack which is now mostly empty due to selling off Cisco lab hardware, so I'm leaning towards a rack-mountable server.

Hoping to hear some possible suggestions for the following requirements, basically in this order of importance:

1. Quiet, I need to run this in my apartment, and it will be in or near the living room. It can be above a whisper of course, but this is the reason I'm not buying a used Dell off of eBay or something.

2. Powerful enough to run ESXi and VIRL (bare-metal) without being disappointed by performance. I'll boot into either as needed. I'm thinking at any given time if I was booting into ESXi to play, I'd want to be able to spin up at least 4 VMs - don't need fancy graphics, these would probably be a mixture of WIN/*NIX.

3. Cost / Power consumption - I'm willing to throw some money at something if it's meeting the above requirements. I'm also hoping not to spend more than $2,000, at least not immediately. The cost is less important than having a sufficiently quiet/powerful virtual lab.

Whatever option I go with it will be an Intel CPU. The Supermicro SYS-E300-8D may be the right choice, but I'm not sure how quiet I can get it. I don't have a fear of swapping out parts in servers, or in other words whatever solution I come up with doesn't need to be a perfect one right out of the box.

By all means ask me more questions about needs or make recommendations. Links to other posts, here, elsewhere, blogs, etc are all appreciated.

 

[John Moreno]

Hi,

I bought a Dell R610 Server on eBay that comes with 20 x Virtual Routers (CSR-1000v) already pre-configured and ready to use.

I paid only $500, so I think that it was a good deal! This is the Link:

www.ebay.com/itm/271495342739

The Dell R610 is not noisy at all, so it should be good for your Apt.
My Server came with a couple of Xeon E5620 CPU's and 64GB RAM which is a lot!
The E5620 are not the most up to date Processors, but they are good enough for what I need.

This Seller offered me a RAM upgrade to 96GB, but I didn't think that I would need that much RAM.
He also said that he could upgrade the Server to Dell R620, which supports E5-26XX Processors.
The R620 is much more expensive though, but since your budget is $2000, you should be able to afford a Dell R620 if you think that the R610 is not good enough for you.

But whatever the Server you use, CSR-1000v Virtual Routers is way to go!

VIRL is not good for CCIE. It may be good enough for CCNA/CCNP, but not for CCIE.

INE uses CSR-1000v Virtual Routers, so this is definitely the best option.

I know that VIRL also includes CSR-1000v Routers, but managing these Routers inside VIRL is a real pain in the neck!
In addition, VIRL does not support Layer 3 Switches! It only supports L2 devices, which won't cut it.

So I would encourage you to check it out.

I hope this helps. Take care!!

 

[bishun]

John thanks for the information. I especially appreciate the information about VIRL. I can read documentation all day, but I really respect opinions from those with the direct experience. I've never used INE's labs or training, but I've always heard great things about them.

The route I'm going perhaps is leaning toward a pure ESXi lab with Cisco lab time handled another way.

I'm now looking at TinkerTry's MicroServers page - which has plenty of information on some of the new SuperMicro mini-towers, with Intel embedded processors.

I'm specifically looking at Supermicro Mini Tower Intel Xeon D-1567, a bit spendy but I think it could fit my needs moving into the future. It's quiet, and with swapped out fans can be more so. It's fairly powerful for home use, the price is acceptable, and its power efficient which counts in the long run.

I'll have to dig around more in regards to what storage makes the most sense for my intended use case.

 

[Marsh]

@bishun

Since you mention this is the beginning of your long journey. I'll give you my bare minimum hardware that you couldrun the INE 10 Cisco virtual routers lab.
In my actual homelab, I am always curious what is the bare minimum resource that produce acceptable performance ( It was drilled into my head by my VP when I worked in Oracle in the late 80's).

Are you ready?
I have two bare metal box that would and still run the INE lab ( 10 routers INE lab config ).

First box is a 10years old Dell insprion 530, with a dual Core2 duo processor, 8gb RAM, 32GB SSD
Second box is a newer Shuttle XG41 , Intel E8400 CPU, 8gb RAM , 32GB SSD

The setup consists of 10 vIOS router ( filename is vios-adventerprisek9-m.vmdk ), each virtual Cisco router only use less than 512mb, therefore it really require 5GB ram + Ubuntu OS QEMU/KVM server 512mb to 1gb ram.
Of course, the same lab would run my Dual E5 + 196gb ram host as well.

Few years back, I transferred the Cisco INE lab to Vmware worstation, but kept the whole bare metal host setup for keepsake.
I powered up the Shuttle XG41 lga775 box once a year to remind myself that one do not need the latest dual E5 server to get the job done.

 

[TLN]

I'm able to run about 10x Cisco routers on Unetlab on my HP G8 Microserver. 16 gigs of memory, and Win7 desktop (via Gpu pasthrough) and NAS running on same ESXi.