One other consideration is that some people say that ES steppings might have a short life span and bad reliability. To those with long years experience with them. How true is has that been in the past?
Were there particular ES series that started to become unreliable, unstable in time, as i they were aging? My experience with normal hardware is that usually when i buy it new i test it hard with some torture test, and if it survives, it usually survives the rest of my usage. But maybe that does not apply to ES cpus?
Main problem with ES is that I think changelog (list of bugs) fixed in each stepping was distributed under NDA to customers who were designing their systems. Each stepping means that something was fixed in between, but what exactly and how important that is - you probably won't know (unless some one accidentally publishes changelog for the stepping or will tell you in private what to expect).
It can be reliability problem, it can be short life span, or it can be reduced performance or it can be something in a case which you'll never use in your system and you won't even notice a difference between D0 and E5 for your particular workload.
Then question is if you want to risk that or not.
I myself think that for homelab/test setup that is acceptable risk (as there is no other way to get enough cores, etc for that amount of money, and in worst case you'll have a downtime of that setup, so what if it doesn't cost you reputation and revenue?). If I'd be a startup that need HW for something - probably I'll consider the risks of getting ES system (especially on early stages). If I'd be a serious company - I'd probably wouldn't go for buying ES now.
You can tell that probably later steppings should have more bugs fixed and if you'll have something it probably will be a performance issues under some conditions and not stability problems, just based on a common sense, as it was said above - ES2 and later were used by devs to develop motherboards, drivers, etc. so they should be stable enough for that under most common conditions otherwise their job would've been a nightmare (but that is common sense, not experience).
The choice is always yours. But you just have to be aware that there are risks involved here.