yep they work just fine for pure compute, if you have storage elsewhere. Can even toss in 2x SATA3 SSDs if you like. The big limitation is only one PCIe slot, 3.0 x16, and no onboard SFP+. E5v1/v2 are getting long in the tooth, but still workhorses, and DDR3 RDIMMs are cheap.
The stock fans, 5x 40mm, have a high-pitched whine even at idle, not what I'd want in my office.