Q:

ZFS pool degraded on reboot

I have a Ubuntu server set up with a 14-disk ZFS raidz2 pool.
About 80% of the time, on reboot, I will end up with a degraded pool with two of the disks marked as faulted.  The drives that are faulted aren't always the same, but it is always exactly two drives.  For example:
$ sudo zpool status
  pool: tank
 state: DEGRADED
status: One or more devices could not be used because the label is missing or
        invalid.  Sufficient replicas exist for the pool to continue
        functioning in a degraded state.
action: Replace the device using 'zpool replace'.
   see: http://zfsonlinux.org/msg/ZFS-8000-4J
  scan: resilvered 4K in 0h0m with 0 errors on Sun Sep 30 23:08:51 2018
config:

        NAME                      STATE     READ WRITE CKSUM
        tank                      DEGRADED     0     0     0
          raidz2-0                DEGRADED     0     0     0
            sde                   ONLINE       0     0     0
            sdc                   ONLINE       0     0     0
            sdd                   ONLINE       0     0     0
            sda                   ONLINE       0     0     0
            sdh                   ONLINE       0     0     0
            11521322863231878081  FAULTED      0     0     0  was /dev/sdf1
            15273938560620494453  FAULTED      0     0     0  was /dev/sdg1
            sdb                   ONLINE       0     0     0
            sdi                   ONLINE       0     0     0
            sdj                   ONLINE       0     0     0
            sdk                   ONLINE       0     0     0
            sdl                   ONLINE       0     0     0
            sdm                   ONLINE       0     0     0
            sdn                   ONLINE       0     0     0

errors: No known data errors

I can export and re-import the pool, and the disks are no longer faulted.  For example:
$ sudo zpool export tank
$ sudo zpool import tank
$ sudo zpool status
  pool: tank
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://zfsonlinux.org/msg/ZFS-8000-9P
  scan: resilvered 4K in 0h0m with 0 errors on Sun Sep 30 23:08:51 2018
config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          raidz2-0  ONLINE       0     0     0
            sde     ONLINE       0     0     0
            sdc     ONLINE       0     0     0
            sdd     ONLINE       0     0     0
            sda     ONLINE       0     0     0
            sdh     ONLINE       0     0     0
            sdg     ONLINE       0     0     1
            sdf     ONLINE       0     0     0
            sdb     ONLINE       0     0     0
            sdi     ONLINE       0     0     0
            sdj     ONLINE       0     0     0
            sdk     ONLINE       0     0     0
            sdl     ONLINE       0     0     0
            sdm     ONLINE       0     0     0
            sdn     ONLINE       0     0     0

errors: No known data errors

The HBA being used has worked correctly in another server.
Anything else I can try to avoid these faulted drives on reboot?  I have another HBA I could swap in.

A:

You should not use /dev/sdX names for your pool configuration.
Any change in SCSI enumeration, say inserting a CDROM or a USB drive, could cause the device names to change, resulting in the errors you're experiencing.
You have an option of using the /dev/disk/by-id names.
Do this with a zpool export tank and zpool import -d /dev/disk/by-id tank

