Identifying issues running FreeNAS without ECC memory

[Alfa147x]

Hey folks!

My FreeNAS server has been running like a dream for the last 2 months. I have migrated everything off of both ReadyNAS appliances and have 2 jails running.

• Supermicro X8ST3-F
  
• Intel Xeon X5650 running
  
• 2x Corsair Vengeance LP DDR3 8GB PC3 12800

I spent ages trying to accommodate an affordable method to using ECC memory but decided against it. The NAS houses mostly media and triplicate copies of "mission critical" data. I'm not attached to the media files but I am the smaller subsection of data deemed "mission critical" which is why I have it mirrored to DropBox and Google Drive before it hits the NAS.

So my question is: How do I identify issues (corruption?) that isn't being handled by ECC memory? I want to know if I should spend the cash to upgrade to ECC or not.

Thanks!

 

[Jeggs101]

I'd replace your memory under two scenarios - you want to expand memory or you are losing sleep. I were spending time on it, I would probably just replace. Since you have an X58 you need more expensive ECC UDIMMs 4GB DDR3 1333 UDIMM ECC ROHS RAM 4GB for $20 seems lower end of pricing.

At some point, if you wanted to get like 48GB, swapping the motherboard out would become cost effective as RDIMMs are cheaper. Getting 16GB 2x8GB DDR3 is $35: Cisco 2X 8GB 15-12288-01 DDR3 8GB 2Rx4 PC3L1333 ECC REG 16GB TOTAL

 

[CreoleLakerFan]

I have a FreeNAS server configured with an E3-1220L and 8GB total ECC UDIMMs. I'm serving media to my home via an 8 x 4TB RAIDZ2 volume. I have no problem saturating a 2 x 1GbE LAG with sequential reads/writes (uncompressed BRD rips). I started out with 16GB, but ended up downgrading and putting the additional RAM into other projects.

Let me guess - you have been reading the FreeNAS forums where they recommend 'as much RAM as you can afford' for your build.

Your board is triple channel, so you could acquire 12 GB for $60-$70. You probably could go with 6GB and be fine for serving media, but your jails might suffer. Have you considered moving your FreeNAS into a VM under ESXi? You'd probably get more use out of your RAM since ESXi is inherently great at memory de-duplication. Assign the 1068E to your FreeNAS VM. You could use the ICH10R ports to host your FreeNAS OS and a couple of VMs on a small SSD or two.

I know this isn't what you asked for - sorry for the unsolicited input on your build.

 

[CreoleLakerFan]

how much ram does zfs really need

This is one of my favorite threads of all time. I have it bookmarked for reference anytime this topic comes up.

 

[Alfa147x]

I'd replace your memory under two scenarios - you want to expand memory or you are losing sleep. I were spending time on it, I would probably just replace. Since you have an X58 you need more expensive ECC UDIMMs 4GB DDR3 1333 UDIMM ECC ROHS RAM 4GB for $20 seems lower end of pricing.

At some point, if you wanted to get like 48GB, swapping the motherboard out would become cost effective as RDIMMs are cheaper. Getting 16GB 2x8GB DDR3 is $35: Cisco 2X 8GB 15-12288-01 DDR3 8GB 2Rx4 PC3L1333 ECC REG 16GB TOTAL

Yup so I found out that all ECC memory isn't the same: Thread where I realized I bought the wrong memory.

I have a FreeNAS server configured with an E3-1220L and 8GB total ECC UDIMMs. I'm serving media to my home via an 8 x 4TB RAIDZ2 volume. I have no problem saturating a 2 x 1GbE LAG with sequential reads/writes (uncompressed BRD rips). I started out with 16GB, but ended up downgrading and putting the additional RAM into other projects.

Let me guess - you have been reading the FreeNAS forums where they recommend 'as much RAM as you can afford' for your build.

Your board is triple channel, so you could acquire 12 GB for $60-$70. You probably could go with 6GB and be fine for serving media, but your jails might suffer. Have you considered moving your FreeNAS into a VM under ESXi? You'd probably get more use out of your RAM since ESXi is inherently great at memory de-duplication. Assign the 1068E to your FreeNAS VM. You could use the ICH10R ports to host your FreeNAS OS and a couple of VMs on a small SSD or two.

I know this isn't what you asked for - sorry for the unsolicited input on your build.

Sooo this build started as my VMware server and I had all my storage on two Netgear ReadyNAS appliances. But now I get a solid amount of AWS credits every month so I moved all my VMs to the cloud.

In attempt to downsize servers and increase spindle space I converted my Vmware server into a NAS. The two jails are for syncing with various cloud storage providers and Plex so their fairly NAS specific.

how much ram does zfs really need

This is one of my favorite threads of all time. I have it bookmarked for reference anytime this topic comes up.

So with the 16GB in the box today it sits at ~15GB of utilization but performance from daily use doesn't seem impacted.

 

[Alfa147x]

So I guess I my question is:
How do I confirm that my data isn't being screwed up by not using ECC?

 

[CreoleLakerFan]

I have never found an answer to that question. I don't think there is an easy one. If you have the original data that you know has not been affected you could checksum it and compare the results to your current data set (this is essentially how ZFS works). But I don't know how you would accomplish that without having a reliable data storage mechanism in the first place, which would involve having a system with ReFS, ZFS, or btrfs - with ECC memory. Kind of circular-catch22.

But there's this:

End-to-end Data Integrity for File Systems: A ZFS Case Study (PDF)

DRAM Errors in the Wild: A Large-Scale Field Study (PDF)

Exploring the impact of Bit Rot (PDF)

Library of Congress discussion covering impact of flipped bits on video files

TL;DR - Use ECC if your data is important to you. Bit flips have a statistically high probability of occurrence, and certain digital media (pictures) is demonstrably impacted by the effects of flipped bits. Personally the data I care most about is A) photos/video of family, B) professional related data. A is irreplaceable, B involves a lot of work to recreate and potential reputation loss where data corruption impacts business/clients.

 

[Aluminum]

How do I confirm that my data isn't being screwed up by not using ECC?

You don't, ECC is required to correct 1 bit errors and detect 1-2 bit errors.