Storage and RAID Errors
RAID and LVM errors are the highest-stakes Linux issues — data integrity is on the line. Most of them follow a pattern: a disk returns errors, the array degrades, the rebuild begins, and either succeeds (good) or fails halfway with another disk dropping (bad). The ten errors below are what the on-call sees when storage hardware misbehaves.
#81 – RAID array degraded
Description: mdadm reports one or more drives have dropped out of an md array.
Solution: cat /proc/mdstat shows status; mdadm --detail /dev/md0 for full diagnosis; failed disk: mdadm /dev/md0 --remove /dev/sdc && mdadm /dev/md0 --add /dev/sde.
#82 – RAID rebuild failed (second disk dropped during recovery)
Description: Worst-case scenario; array is now in inconsistent state.
Solution: Stop writes immediately. Image the failing disk with ddrescue if possible. Restore from backup. Don’t guess — this is when you call the storage vendor.
#83 – LVM: Volume group not found
Description: vgs / lvs show no volume groups or report missing PVs after a reboot or disk swap.
Solution: vgscan --cache; vgchange -ay vg_name to activate; check pvs — missing physical volume?
#84 – LVM thin pool 100% full
Description: Filesystem and OS appear fine but writes fail.
Solution: lvs shows pool usage; immediate fix: extend the pool with lvextend -L+50G vg/pool; long-term: enable autoextend in /etc/lvm/lvm.conf.
#85 – SMART: Pre-fail attribute
Description: smartctl -a /dev/sdX shows reallocated_sector_ct or pending_sector_count incrementing.
Solution: Replace the disk preemptively. SMART pre-fail is the warning shot before catastrophic failure.
#86 – multipath device: failed all paths
Description: Every path to a SAN LUN is dead and I/O is hanging.
Solution: multipath -ll; check fabric (FC switches, iSCSI portals); verify LUN is still mapped on the array; iscsiadm -m session -R to rescan.
#87 – LVM snapshot exceeded its CoW pool
Description: Snapshot becomes invalid because it ran out of space tracking changes.
Solution: Snapshot is unusable when full. Either extend it (lvextend) before changes accumulate, or accept the loss and lvremove.
#88 – mdadm: insufficient devices to start array
Description: Two disks lost from a RAID 5; not enough left for parity.
Solution: Force assemble at risk: mdadm --assemble --force /dev/md0 /dev/sda1 /dev/sdb1 /dev/sdd1. Last resort — data integrity not guaranteed.
#89 – iSCSI session lost
Description: Initiator loses its session to the iSCSI target and dependent block devices disappear.
Solution: journalctl -u iscsid; iscsiadm -m session to list; iscsiadm -m node -T target -p portal --login to reconnect.
#90 – fstrim / discard not supported
Description: SSDs need TRIM to maintain performance; fstrim fails on certain filesystems/drivers.
Solution: Verify with lsblk -d -o NAME,DISC-GRAN,DISC-MAX; for thin-provisioned LVM: enable in /etc/lvm/lvm.conf (issue_discards = 1).
Conclusion
- Monitor
mdadm --detailoutput via Prometheus/check_mk; a degraded array is silent without monitoring. - Replace SMART pre-fail disks BEFORE they fail completely.
- Test backup restores quarterly. The first time you find your backup is broken should not be during an outage.
- Use
ddrescuenotddwhen imaging dying disks. - RAID is not backup. Snapshots aren’t backups either. Real backups go off-host.