The cost of data

[alex_stief]

I recently stumbled upon a memo about "ROT data". The point it was trying to convey was: try to minimize the amount of disk space you occupy needlessly.
Now I get the point to some extent, IT costs can be a significant factor. But that article was riddled with some obvious bs claims, presumably to convince tech illiterate folks about the importance of the issue.

But one thing stood out for me: it was claimed that the average annual cost to maintain 1TB of data was 20k-30k euros.
Now I am fully aware that there is more to the cost than servers and hard drives. But that number seems outrageously high to me. What do you think?

 

[StevenDTX]

That totally depends on speed, availability, etc. My company says 1TB of mostly static NAS data costs around $10k USD per year to maintain. Highly-available VM datastores are significantly higher than that.

 

[Evan]

While I can see some compose get better economy of scale etc but I could imagine that’s probably a true figure. The thing is if that is made up of a lot of more or less fixed costs like staff salary then reducing the data will only make the value per TB go up but the total cost remain about the same.

btw i think your using per TB but they mean per PB.

 

[alex_stief]

Evan, you can either agree that it is a reasonable figure. Or you can argue that it is off by three orders of magnitude (TB vs PB). Which one is it

 

[Blinky 42]

20-30k/yr is way too high per TB unless you are time-warped in the past or justifying the cost to create the data or move it around.

I can lease a full cabinet + power and bw in London for < $2k/mo. = 24k/year (1/4 that or less elsewhere or in the States, but just for argument sake since it is one of the most expensive sites I have for power currently)
Initial investment for hardware of (just list prices w/ online configurators)

• $38K for 1U head-end server with 150TB raw NVMe (10x15.36TB)
  
• $55K for 4U JBOD chassis with 1PB usable ( 90x 16T SAS drives, RAID6 or Z2 or whatever + spares, still more than 1PB)

Assume you spend say 10% annually on HW churn to keep it running for 5 years. ($93k * 10% = 9.3k/y) = $46.5 over 5y

Servers = $93k
Colo 5y = $120k
Upkeep 5y= $46.5k
Total = 259.5k / 5year for 1PB of bulk storage + 150TB of NVMe = 50k/PB/yr.

Creating that data costs some $, but the above will keep it up and running sitting pretty for probably 10+yr depending on the rate you chew thru the NVMe endurance. If you double the power / colo fees to fill tat cabinet with 9 additional 4U storage shelves:
Servers = $588k / 10PB + 150TB NVMe
Colo 5y = $240k
Upkeep 5yr = $294k
Total = 1.122M / 5yr = 224k/yr = $22.4k/PB/yr

 

[Evan]

Evan, you can either agree that it is a reasonable figure. Or you can argue that it is off by three orders of magnitude (TB vs PB). Which one is it

I was thinking per PB it was a reasonable annual figure to maintain. When i think maintain I was thinking more headcount than electrical. If you consider maintain as floor space , kwhr, staff, investment cost etc it’s still way to high per TB and somewhat too low per PB.

While I am viewing it as a large company even the smallest company per TB would be USD$ 60 per year for 1TB onedrive on Microsoft cloud (even includes mails and sharepoint, teams etc in that price)
Based on that seems to be per TB and even then Microsoft can find a profit and be 5-6 times below their benchmark.