> In a 6 node, single pool NL cluster(8TB drives), the DPL will be set to 3d1n1d. Specific directories with billions of small files will be set to 2x mirroring.  > Since everything is replicated to an alternate site with the same protection scheme, would I be sufficiently protected from data loss?  Going from +3d:1n1d to (essentially) +1n is a huge difference, and those directories will be at risk.  Two drive failures (within the same failure domain) means data loss.  Since you have the data copied to a second location, perhaps one could argue that the files really aren't 2x, but 3x.  That's mostly true, however dealing with data loss on a cluster isn't fun and can cause other issues not directly related to missing data, e.g. Job Engine jobs that fail when they encounter inaccessible files, workloads that expect files to be in a certain place (and fail when they're not), etc.  So it's not necessarily as easy as "just restore the damaged file from the DR site".  Since you said you spoke with your account team, what did they say about the supportability of such a plan?  Are you sure that Isilon Support will help you in the event you do actually lose data (I'm not saying you will, but let's think worst case)?  Will they help clean up the filesystem?  I don't know these answers nor your threshold for risk, but you should fully understand the ramifications of doing what you're suggesting.  > Is a small file guaranteed to be placed on 2 different nodes and not just on 2 different disks?  Yes.  2x, or +1n, will guarantee the loss of a single device without loss of data.  > After a node outage, will the files be created/copied to a different node to keep the 2x status?  Yes, this is what the FlexProtect Job Engine job does.  Understand, though, that this is a cluster-wide job and while it's running (scanning your billions of files) you're in a "window of risk" where a second device failure means data loss (for your under-protected data).  > If both copies of the file are gone( double node outage )  To be clear here, you have data loss when 2 nodes OR 2 drives fail (assuming those drives are in the same failure domain [disk pool]).  Drive failures are much more common than node failures.  So don't think that this is just about node failures.  > will the system try to recover from the replicated data set?  No.  At 2x with two failures, there are parts of filesystem that are lost.  Fortunately your directories should all be intact since we protect directories one level higher than the protection policy (so directories will be 3x ).  But any file data within that 2x directory tree that had data on both of the failed devices will be inaccessible.  OneFS will not be able to repair these files, and will NOT take any action to restore these files either from tape backup or DR cluster.  Both of those processes would require manual intervention by you.
Hello,  Please see my previous answered question for a bit of history: Best, most efficient, protection level for small files - 55KB   After the help and feedback I received from the community and my EMC reps, we are looking at setting the protection level to 2x mirroring on specific directories to cope with our overhead issues. Before we pull the trigger, I wanted to run it by the experts here.  For example:  In a 6 node, single pool NL cluster(8TB drives), the DPL will be set to 3d1n1d. Specific directories with billions of small files will be set to 2x mirroring  Since everything is replicated to an alternate site with the same protection scheme, would I be sufficiently protected from data loss?  Is a small file guaranteed to be placed on 2 different nodes and not just on 2 different disks? After a node outage, will the files be created/copied to a different node to keep the 2x status? If both copies of the file are gone( double node outage ) will the system try to recover from the replicated data set?  Is this feasible?  Regards, Oz
Small file (55KiB) Mirroring instead of FEC