Yeah you really get less per $ with the E5 alternative, however going with the E5 gives great room for later upgrades.The E5 system I built had a $1000 budget as well. I ended up with an ASUS Z9PE-D16, 1x E5-2620, and 24GB of ram. $1000 won't go very far into an E5 build compared to the Dell deal.
Your first alternative is quite similar to what i have today (my whitebox), and an E3 upgrade is not that big a difference im afraid.I think your choice depends on what you really want to end up with. They are really different outcomes...
If you want a "practical" server to run "home production" kinds of VMs (simple web server, SIP-PBX, TV tuner, etc) go with the E3-based server. You can afford a faster CPU, it will run lower power, you can make it quieter, etc.
If you want a "real" lab and want to play with VM alternatives (ESXi vs KVM) and want maximum capability per $ (or Euro) go with the C6100. However, do understand that this will end up with a system that is power hungry, hot and loud. Without mods it is not very friendly in a home/apartment environment. Don't forget its also one to two generations behind in technology - its cheap because its getting a bit old. But it is the best deal on the market right now for a small server cluster.
If you want something to play with higher-power solutions go with the E5 (though I'd advise waiting a few months for IB). Just understand that this is probably a choice focused on future upgrades and growth. Unless you are made of money you'll have to settle for a slower CPU than you get for the some money in the E3 lineup.
The real lab sounds tempting. I thought about going with 2x nodes running for my VMs and ZFS, this should only be ~240 W (my whitebox todays does 160W).
I'm also going to remove the 20000 RPM fans like you did with yours. I think doing this should make it possible to run it in a bigger closet i have in my current apartment.
Me and my partner are also planing to move now that we are both graduates and getting a dedicated server space is on the list of priorities !
I'm actually looking into the E5 alternative since it gives me room to expand later on, more memory and maybe Ivy cpus.
I googled around and checked benchmarks this morning. However the Opteron family just seems to be too power hungry and deliver poor performance in comparison with the E5 alternative. To some extent i feel that i can spend the extra bucks to get the better intel performance.I have been looking at the C6100's and other cheap options for my home lab as I have outgrown my 32GB, E3 1230V2 box, and thought I would throw in an AMD build
Asus KGPE-D16 motherboard - dual socket G34 with 16 dimm slots (ASUS - KGPE-D16)
They seem to go for $350 to $450 depending on if you go new or used
2 x AMD opteron 6128 - 8 core processors - $30-50 each on ebay
16 x 4GB SDRAM - $450 or so brand new, but I am sure you can find cheaper on ebay
RAM is limited to 256GB RDIMM or 64GB with standard UDIMM's
CPU's can be upgraded right through to the latest Opteron 6386SE which is a 16 core 140W monster
The good thing for you is the shipping costs will be low as these are all small packages, and then you can pick up most of the parts locally.
The motherboard is SSIEEB which is EATX size so it will fit in a lot of mid/full tower cases.
PSU can just be a high quality unit like Seasonic etc.
Thought I would throw it out there for consideration
Is it possible to use a raid-card (M1015 should fit) in one C6100 node and connect it to the 3.5'' hotswap enclosures? For instance use 6x3.5'' slots for one node? I tried looking at the pictures and it looks like those ordinary sata cables going to the hdd-enclosure could be wired to go to a raid-card instead.