You are currently viewing 볼륨의 크기가 0바이트입니다.

볼륨의 크기가 0바이트입니다.

비프음이 울리고 파일이 보이지 않아요.

갑자기 나스에서 비프음이 울리더니 공유 폴더에 저장한 파일이 보이지 않습니다. 저장소 관리자에서 확인해 보니 볼륨 1이 충돌했다고 나옵니다. 또한 볼륨 1의 크기와 사용한 용량이 모두 0바이트로 나옵니다. 나스에는 스토리지 풀이 한 개 있습니다. 이 스토리지 풀의 용량을 나눠서 볼륨1과 볼륨 2로 사용하고 있는데요. 볼륨2는 정상적으로 작동하고 있습니다.

스토리지 풀의 정보

디스크 4개를 사용해서 RAID10을 사용하고 있습니다. 할당 상태와 상태 모두 정상으로 나옵니다. 이는 디스크 – 스토리지 풀 – 볼륨의 상관관계에서 디스크와 스토리지 풀은 이상이 없다는 의미입니다.

볼륨2는 정상

하나의 스토리지 풀에 생성한 볼륨1과 볼륨2 중에서 볼륨1만 충돌했다면 이는 볼륨1 자체에 문제가 생겼음을 의미합니다.

소요시간 : 3일    어려움 : ★★★★★

데이터 복구 작업을 진행합니다.

볼륨의 크기가 0바이트로 보이는 이유는 볼륨1이 마운트 되지 않기 때문입니다. 리눅스를 설치한 컴퓨터에서는 볼륨1이 마운트됩니다. 하지만 일부 폴더가 보이지 않습니다. 이런 증상의 원인은 파일시스템에 오류가 생겼기 때문으로 생각됩니다. 어쩔수 없이 최후의 방법인 데이터 복구 작업을 통해서 볼륨1의 파일을 복구했습니다.

볼륨1의 파일을 복구한 이후에는 볼륨1을 제거 후 다시 생성합니다. 그리고 공유 폴더를 생성하고 복구한 파일을 원래의 공유 폴더 안에 다시 넣어줍니다. 

데이터 복구에 소요되는 시간은 디스크 용량과 개수, 레이드 유형, 데이터 용량에 따라서 결정되는데요. 이번 사례는 4TB 디스크 4개, 레이드 10, 파일의 용량은 5TB 정도이고 소요된 시간은 3일입니다.

복구천사 화면

복구천사 프로그램이 btrfs 파일시스템도 아주 잘 복구해 줍니다.

궁금하면 ↓↓↓↓

root@DS918p:~# cat /proc/mdstat
Personalities : [raid1] [raid10]
md2 : active raid10 sata3p3[0] sata2p3[3] sata1p3[2] sata4p3[1]
      7804393088 blocks super 1.2 64K chunks 2 near-copies [4/4] [UUUU]

md1 : active raid1 sata3p2[0] sata2p2[3] sata1p2[2] sata4p2[1]
      2097088 blocks [4/4] [UUUU]

md0 : active raid1 sata3p1[0] sata2p1[3] sata1p1[2] sata4p1[1]
      2490176 blocks [4/4] [UUUU]

unused devices: 
root@DS918p:~# mdadm -D /dev/md2
/dev/md2:
        Version : 1.2
  Creation Time : Tue Sep  1 13:02:59 2020
     Raid Level : raid10
     Array Size : 7804393088 (7442.85 GiB 7991.70 GB)
  Used Dev Size : 3902196544 (3721.42 GiB 3995.85 GB)
   Raid Devices : 4
  Total Devices : 4
    Persistence : Superblock is persistent

    Update Time : Fri Jan  6 09:09:46 2023
          State : clean
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0

         Layout : near=2
     Chunk Size : 64K

           Name : NAS:2
           UUID : 0cb160f8:90b38940:55e79ee8:5a004888
         Events : 62

    Number   Major   Minor   RaidDevice State
       0       8       35        0      active sync set-A   /dev/sata3p3
       1       8       51        1      active sync set-B   /dev/sata4p3
       2       8        3        2      active sync set-A   /dev/sata1p3
       3       8       19        3      active sync set-B   /dev/sata2p3
root@DS918p:~# mdadm -E /dev/sata1p3
/dev/sata1p3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 0cb160f8:90b38940:55e79ee8:5a004888
           Name : NAS:2
  Creation Time : Tue Sep  1 13:02:59 2020
     Raid Level : raid10
   Raid Devices : 4

 Avail Dev Size : 7804393120 (3721.42 GiB 3995.85 GB)
     Array Size : 7804393088 (7442.85 GiB 7991.70 GB)
  Used Dev Size : 7804393088 (3721.42 GiB 3995.85 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
   Unused Space : before=1968 sectors, after=32 sectors
          State : clean
    Device UUID : 9e1aa1a8:44f15335:2d571127:aa71284b

    Update Time : Fri Jan  6 09:09:46 2023
       Checksum : 6ff85bc4 - correct
         Events : 62

         Layout : near=2
     Chunk Size : 64K

   Device Role : Active device 2
   Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
root@DS918p:~# mdadm -E /dev/sata2p3
/dev/sata2p3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 0cb160f8:90b38940:55e79ee8:5a004888
           Name : NAS:2
  Creation Time : Tue Sep  1 13:02:59 2020
     Raid Level : raid10
   Raid Devices : 4

 Avail Dev Size : 7804393120 (3721.42 GiB 3995.85 GB)
     Array Size : 7804393088 (7442.85 GiB 7991.70 GB)
  Used Dev Size : 7804393088 (3721.42 GiB 3995.85 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
   Unused Space : before=1968 sectors, after=32 sectors
          State : clean
    Device UUID : 08af097f:4970fad3:7275955b:2623f65d

    Update Time : Fri Jan  6 09:09:46 2023
       Checksum : 2c593ef6 - correct
         Events : 62

         Layout : near=2
     Chunk Size : 64K

   Device Role : Active device 3
   Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
root@DS918p:~# mdadm -E /dev/sata3p3
/dev/sata3p3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 0cb160f8:90b38940:55e79ee8:5a004888
           Name : NAS:2
  Creation Time : Tue Sep  1 13:02:59 2020
     Raid Level : raid10
   Raid Devices : 4

 Avail Dev Size : 7804393120 (3721.42 GiB 3995.85 GB)
     Array Size : 7804393088 (7442.85 GiB 7991.70 GB)
  Used Dev Size : 7804393088 (3721.42 GiB 3995.85 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
   Unused Space : before=1968 sectors, after=32 sectors
          State : clean
    Device UUID : 1ad4bc4b:a98b5d10:4c37e68c:b6b866f5

    Update Time : Fri Jan  6 09:09:46 2023
       Checksum : fe30d6ce - correct
         Events : 62

         Layout : near=2
     Chunk Size : 64K

   Device Role : Active device 0
   Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
root@DS918p:~# mdadm -E /dev/sata4p3
/dev/sata4p3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 0cb160f8:90b38940:55e79ee8:5a004888
           Name : NAS:2
  Creation Time : Tue Sep  1 13:02:59 2020
     Raid Level : raid10
   Raid Devices : 4

 Avail Dev Size : 7804393120 (3721.42 GiB 3995.85 GB)
     Array Size : 7804393088 (7442.85 GiB 7991.70 GB)
  Used Dev Size : 7804393088 (3721.42 GiB 3995.85 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
   Unused Space : before=1968 sectors, after=32 sectors
          State : clean
    Device UUID : 4910f366:26389037:901a222e:15eb36e5

    Update Time : Fri Jan  6 09:09:46 2023
       Checksum : d1a5d51e - correct
         Events : 62

         Layout : near=2
     Chunk Size : 64K

   Device Role : Active device 1
   Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
root@DS918p:~# smartctl -A /dev/sata1 -d sat
smartctl 6.5 (build date Oct  7 2021) [x86_64-linux-4.4.180+] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME                                                   FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate                                              0x000b   100   100   016    Pre-fail  Always       -       0
  2 Throughput_Performance                                           0x0005   129   129   054    Pre-fail  Offline      -       104
  3 Spin_Up_Time                                                     0x0007   253   253   024    Pre-fail  Always       -       245 (Average 68)
  4 Start_Stop_Count                                                 0x0012   099   099   000    Old_age   Always       -       6950
  5 Reallocated_Sector_Ct                                            0x0033   100   100   005    Pre-fail  Always       -       0
  7 Seek_Error_Rate                                                  0x000b   100   100   067    Pre-fail  Always       -       0
  8 Seek_Time_Performance                                            0x0005   128   128   020    Pre-fail  Offline      -       18
  9 Power_On_Hours                                                   0x0012   099   099   000    Old_age   Always       -       11500
 10 Spin_Retry_Count                                                 0x0013   100   100   060    Pre-fail  Always       -       0
 12 Power_Cycle_Count                                                0x0032   099   099   000    Old_age   Always       -       6679
192 Power-Off_Retract_Count                                          0x0032   094   094   000    Old_age   Always       -       7219
193 Load_Cycle_Count                                                 0x0012   094   094   000    Old_age   Always       -       7219
194 Temperature_Celsius                                              0x0002   253   253   000    Old_age   Always       -       23 (Min/Max 9/46)
196 Reallocated_Event_Count                                          0x0032   100   100   000    Old_age   Always       -       0
197 Current_Pending_Sector                                           0x0022   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable                                            0x0008   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count                                             0x000a   200   200   000    Old_age   Always       -       0

root@DS918p:~# smartctl -A /dev/sata2 -d sat
smartctl 6.5 (build date Oct  7 2021) [x86_64-linux-4.4.180+] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME                                                   FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate                                              0x000b   100   100   016    Pre-fail  Always       -       0
  2 Throughput_Performance                                           0x0005   129   129   054    Pre-fail  Offline      -       104
  3 Spin_Up_Time                                                     0x0007   253   253   024    Pre-fail  Always       -       239 (Average 51)
  4 Start_Stop_Count                                                 0x0012   099   099   000    Old_age   Always       -       7009
  5 Reallocated_Sector_Ct                                            0x0033   100   100   005    Pre-fail  Always       -       0
  7 Seek_Error_Rate                                                  0x000b   100   100   067    Pre-fail  Always       -       0
  8 Seek_Time_Performance                                            0x0005   128   128   020    Pre-fail  Offline      -       18
  9 Power_On_Hours                                                   0x0012   099   099   000    Old_age   Always       -       11486
 10 Spin_Retry_Count                                                 0x0013   100   100   060    Pre-fail  Always       -       0
 12 Power_Cycle_Count                                                0x0032   099   099   000    Old_age   Always       -       6739
192 Power-Off_Retract_Count                                          0x0032   094   094   000    Old_age   Always       -       7252
193 Load_Cycle_Count                                                 0x0012   094   094   000    Old_age   Always       -       7252
194 Temperature_Celsius                                              0x0002   253   253   000    Old_age   Always       -       23 (Min/Max 9/46)
196 Reallocated_Event_Count                                          0x0032   100   100   000    Old_age   Always       -       0
197 Current_Pending_Sector                                           0x0022   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable                                            0x0008   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count                                             0x000a   200   200   000    Old_age   Always       -       0

root@DS918p:~# smartctl -A /dev/sata3 -d sat
smartctl 6.5 (build date Oct  7 2021) [x86_64-linux-4.4.180+] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME                                                   FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate                                              0x000b   100   100   016    Pre-fail  Always       -       0
  2 Throughput_Performance                                           0x0005   128   128   054    Pre-fail  Offline      -       108
  3 Spin_Up_Time                                                     0x0007   253   253   024    Pre-fail  Always       -       118 (Average 146)
  4 Start_Stop_Count                                                 0x0012   099   099   000    Old_age   Always       -       6912
  5 Reallocated_Sector_Ct                                            0x0033   100   100   005    Pre-fail  Always       -       0
  7 Seek_Error_Rate                                                  0x000b   100   100   067    Pre-fail  Always       -       0
  8 Seek_Time_Performance                                            0x0005   128   128   020    Pre-fail  Offline      -       18
  9 Power_On_Hours                                                   0x0012   099   099   000    Old_age   Always       -       11500
 10 Spin_Retry_Count                                                 0x0013   100   100   060    Pre-fail  Always       -       0
 12 Power_Cycle_Count                                                0x0032   099   099   000    Old_age   Always       -       6641
192 Power-Off_Retract_Count                                          0x0032   095   095   000    Old_age   Always       -       7156
193 Load_Cycle_Count                                                 0x0012   095   095   000    Old_age   Always       -       7156
194 Temperature_Celsius                                              0x0002   253   253   000    Old_age   Always       -       23 (Min/Max 9/42)
196 Reallocated_Event_Count                                          0x0032   100   100   000    Old_age   Always       -       0
197 Current_Pending_Sector                                           0x0022   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable                                            0x0008   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count                                             0x000a   200   200   000    Old_age   Always       -       0

root@DS918p:~# smartctl -A /dev/sata4 -d sat
smartctl 6.5 (build date Oct  7 2021) [x86_64-linux-4.4.180+] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME                                                   FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate                                              0x000b   100   100   016    Pre-fail  Always       -       0
  2 Throughput_Performance                                           0x0005   129   129   054    Pre-fail  Offline      -       104
  3 Spin_Up_Time                                                     0x0007   253   253   024    Pre-fail  Always       -       211 (Average 70)
  4 Start_Stop_Count                                                 0x0012   099   099   000    Old_age   Always       -       6966
  5 Reallocated_Sector_Ct                                            0x0033   100   100   005    Pre-fail  Always       -       0
  7 Seek_Error_Rate                                                  0x000b   100   100   067    Pre-fail  Always       -       0
  8 Seek_Time_Performance                                            0x0005   128   128   020    Pre-fail  Offline      -       18
  9 Power_On_Hours                                                   0x0012   099   099   000    Old_age   Always       -       11510
 10 Spin_Retry_Count                                                 0x0013   100   100   060    Pre-fail  Always       -       0
 12 Power_Cycle_Count                                                0x0032   099   099   000    Old_age   Always       -       6696
192 Power-Off_Retract_Count                                          0x0032   094   094   000    Old_age   Always       -       7218
193 Load_Cycle_Count                                                 0x0012   094   094   000    Old_age   Always       -       7218
194 Temperature_Celsius                                              0x0002   253   253   000    Old_age   Always       -       23 (Min/Max 9/46)
196 Reallocated_Event_Count                                          0x0032   100   100   000    Old_age   Always       -       0
197 Current_Pending_Sector                                           0x0022   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable                                            0x0008   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count                                             0x000a   200   200   000    Old_age   Always       -       0

스토리지 풀에 문제가 없다고 판단하는 근거는 /proc/mdstat 파일의 내용과 mdadm 명령어의 결과에서 특별한 점이 없기 때문입니다. 하드디스크에 문제가 없다고 판단하는 근거는 smartctl 명령어의 결과에서 특별한 점이 없기 때문입니다.

# 나스가 부팅과정에서 볼륨1을 마운트하지 못함. mount 과정에서 생기는 기록.
2023-01-08T10:57:53+09:00 DS918p spacetool[9266]: volume_mount.c:209 Failed to mount '/dev/mapper/cachedev_0' on '/volume1'
2023-01-08T10:57:53+09:00 DS918p spacetool[9266]: volume_mount.c:295 Failed to do volume mount: [/volume1]
2023-01-08T10:57:53+09:00 DS918p kernel: [   59.208080] BTRFS: open_ctree failed
2023-01-08T10:57:55+09:00 DS918p kernel: [   61.247355] parent transid verify failed on 1306660798464 wanted 321488 found 321486
2023-01-08T10:57:55+09:00 DS918p kernel: [   61.338706] parent transid verify failed on 1306660798464 wanted 321488 found 321486
2023-01-08T10:57:55+09:00 DS918p kernel: [   61.574302] parent transid verify failed on 1306660798464 wanted 321488 found 321486
2023-01-08T10:57:55+09:00 DS918p kernel: [   61.582978] BTRFS error (device dm-3): BTRFS: dm-3 failed to repair parent transid verify failure on 1306660798464, mirror = 1
2023-01-08T10:57:56+09:00 DS918p kernel: [   61.606043] parent transid verify failed on 1306660798464 wanted 321488 found 321486
2023-01-08T10:57:56+09:00 DS918p kernel: [   61.924474] BTRFS error (device dm-3): BTRFS: dm-3 failed to repair parent transid verify failure on 1306660798464, mirror = 2
2023-01-08T10:57:56+09:00 DS918p kernel: [   61.939290] BTRFS error (device dm-3): failed to load usrquota subtree 257, ret=-5
2023-01-08T10:57:56+09:00 DS918p kernel: [   61.947740] BTRFS error (device dm-3): usrquota disabled due to faield to load tree
2023-01-08T10:57:57+09:00 DS918p kernel: [   63.531197] btree_readpage_end_io_hook: 2 callbacks suppressed

# 부팅 후에 수동으로 mount 명령어를 사용해도 비슷한 기록이 생김.
2023-01-08T11:11:17+09:00 DS918p kernel: [  863.834395] btree_readpage_end_io_hook: 8 callbacks suppressed
2023-01-08T11:11:17+09:00 DS918p kernel: [  863.840926] parent transid verify failed on 1306660798464 wanted 321488 found 321486
2023-01-08T11:11:17+09:00 DS918p kernel: [  863.932305] parent transid verify failed on 1306660798464 wanted 321488 found 321486
2023-01-08T11:11:17+09:00 DS918p kernel: [  864.167567] parent transid verify failed on 1306660798464 wanted 321488 found 321486
2023-01-08T11:11:17+09:00 DS918p kernel: [  864.176213] BTRFS error (device dm-3): BTRFS: dm-3 failed to repair parent transid verify failure on 1306660798464, mirror = 1
2023-01-08T11:11:17+09:00 DS918p kernel: [  864.199769] parent transid verify failed on 1306660798464 wanted 321488 found 321486
2023-01-08T11:11:17+09:00 DS918p kernel: [  864.290975] parent transid verify failed on 1306660798464 wanted 321488 found 321486
2023-01-08T11:11:18+09:00 DS918p kernel: [  864.526057] parent transid verify failed on 1306660798464 wanted 321488 found 321486
2023-01-08T11:11:18+09:00 DS918p kernel: [  864.534695] BTRFS error (device dm-3): BTRFS: dm-3 failed to repair parent transid verify failure on 1306660798464, mirror = 2
2023-01-08T11:11:18+09:00 DS918p kernel: [  864.549557] BTRFS error (device dm-3): failed to load usrquota subtree 257, ret=-5
2023-01-08T11:11:18+09:00 DS918p kernel: [  864.558006] BTRFS error (device dm-3): usrquota disabled due to faield to load tree
2023-01-08T11:11:18+09:00 DS918p kernel: [  864.586864] BTRFS: open_ctree failed

볼륨1의 파일시스템에 오류가 있다고 판단하는 근거는 mount 명령어를 사용하면 여러가지 오류메시지가 발생하기 때문입니다.

root@PC-MORANG-debian:/mount-point/md127$ ls -al 폴더_A/자식폴더/
ls: cannot access 폴더_A/자식폴더/폴더_C: 입력/출력 오류
ls: cannot access 폴더_A/자식폴더/폴더_D: 입력/출력 오류
ls: cannot access 폴더_A/자식폴더/폴더 E: 입력/출력 오류
합계 0
drwxrwxrwx 1 1027 users  298  1월  2 13:58  .
drwxr-xr-x 1 root root   824  1월  2 13:58  ..
drwxrwxrwx 1 1028 users  398  9월  7 10:20 폴더_B
d????????? ? ?    ?        ?             ? 폴더_C
d????????? ? ?    ?        ?             ? 폴더_D
d????????? ? ?    ?        ?             ? 폴더_E
drwxrwxrwx 1 1027 users 1328 10월 13 09:38 폴더_F
drwxrwxrwx 1 1027 users  480  2월 18  2022 폴더_G
root@PC-MORANG-debian:/mount-point/md127$

리눅스를 설치한 컴퓨터에는 볼륨1이 마운트 됩니다. 하지만 일부 폴더에서 이름만 정상적으로 보이고 권한, 소유자, 그룹, 수정일 등은 물음표로 표기되는 증상이 발생합니다. 또한 그 폴더에 액세스하면 시놀로지 나스에서발생한 오류와 똑같은 ‘BTRFS error (device dm-1): parent transid verify failed on 1306660798464 wanted 321488 found 321486’ 내용이 kern.log에 쌓입니다.

답글 남기기