md0 ドライブの一つが動いていなかった @2017-6-11†
- Ubuntu の ディスクメニューで md0 が赤色表示 → md0 が Degrade 状態となっていた。 動作確認
- Superblock の persistency は確保されているが、1台のドライブ(/dev/sda)が切り離されていた
munakata@mythen:~ (master #)$ sudo mdadm -D /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Sun Dec 11 23:04:19 2011
Raid Level : raid1
Array Size : 3906885632 (3725.90 GiB 4000.65 GB)
Used Dev Size : 3906885632 (3725.90 GiB 4000.65 GB)
Raid Devices : 2
Total Devices : 1
Persistence : Superblock is persistent
Update Time : Sat Jun 10 18:28:18 2017
State : clean, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 0
Spare Devices : 0
Name : mythen:0
UUID : 4cd693e9:dd3ad1a9:3a5a23a9:62ce3a05
Events : 280089
Number Major Minor RaidDevice State
0 0 0 0 removed
2 8 17 1 active sync /dev/sdb1
- /dev/sda にエラーが発生しているかを smartctrl の簡易テストで確認
munakata@mythen:~ (master #)$ sudo smartctl -t short /dev/sda
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.13.0-48-generic] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
Testing has begun.
Please wait 2 minutes for test to complete.
Test will complete after Sat Jun 10 18:27:37 2017
Use smartctl -X to abort test.
- テストの結果 /dev/sda 自体に障害が発生していないことを確認(completed without error)
munakata@mythen:~ (master #)$ sudo smartctl -l selftest /dev/sda
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.13.0-48-generic] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed without error 00% 22649 -
- md0 に /dev/sda を再アッタチ → 自動的に rebuild がスタートする(rebuild には数時間がかかる)
munakata@mythen:~ (master #)$ sudo mdadm /dev/md0 --add /dev/sda1
mdadm: added /dev/sda1
munakata@mythen:~ (master #)$ sudo mdadm -D /dev/md0
/dev/md0:
Version : 1.2munakata@mythen:~ (master #)$ sudo mdadm /dev/md0 --add /dev/sda1
mdadm: added /dev/sda1
Creation Time : Sun Dec 11 23:04:19 2011
Raid Level : raid1
Array Size : 3906885632 (3725.90 GiB 4000.65 GB)
Used Dev Size : 3906885632 (3725.90 GiB 4000.65 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Sat Jun 10 19:49:34 2017
State : clean, degraded, recovering
Active Devices : 1
Working Devices : 2
Failed Devices : 0
Spare Devices : 1
Rebuild Status : 14% complete
Name : mythen:0
UUID : 4cd693e9:dd3ad1a9:3a5a23a9:62ce3a05
Events : 282261
Number Major Minor RaidDevice State
3 8 1 0 spare rebuilding /dev/sda1
2 8 17 1 active sync /dev/sdb1
munakata@mythen:~ (master #)$ sudo mdadm -D /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Sun Dec 11 23:04:19 2011
Raid Level : raid1
Array Size : 3906885632 (3725.90 GiB 4000.65 GB)
Used Dev Size : 3906885632 (3725.90 GiB 4000.65 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Sun Jun 11 07:40:39 2017
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Name : mythen:0
UUID : 4cd693e9:dd3ad1a9:3a5a23a9:62ce3a05
Events : 296999
Number Major Minor RaidDevice State
3 8 1 0 active sync /dev/sda1
2 8 17 1 active sync /dev/sdb1
munakata@mythen:~ (master #)$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid1 sda1[3] sdb1[2]
3906885632 blocks super 1.2 [2/2] [UU]
dtv_recipe が動作不安定 @2017-6-11†
- HDD 録画したファイルの中で特定のファイルがリードエラー(再生中に停止)する状況になった
- SMART のログを確認
- 重要なのは Current_Pending_Sector = 4 となっている点、正常な状態ではない
munakata@mythen:~ (master #)$ sudo smartctl -a /dev/sdf
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.13.0-48-generic] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Device Model: WDC WD40EZRZ-00WN9B0
Serial Number: WD-WCC4E3JH7YV9
LU WWN Device Id: 5 0014ee 261da88a0
Firmware Version: 80.00A80
User Capacity: 4,000,787,030,016 bytes [4.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 5400 rpm
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: ACS-2 (minor revision not indicated)
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Sun Jun 11 08:07:16 2017 JST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 121) The previous self-test completed having
the read element of the test failed.
Total time to complete Offline
data collection: (53760) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 537) minutes.
Conveyance self-test routine
recommended polling time: ( 5) minutes.
SCT capabilities: (0x7035) SCT Status supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 912
3 Spin_Up_Time 0x0027 185 179 021 Pre-fail Always - 7741
4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 16
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0
9 Power_On_Hours 0x0032 081 081 000 Old_age Always - 14096
10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 10
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 2
193 Load_Cycle_Count 0x0032 025 025 000 Old_age Always - 527334
194 Temperature_Celsius 0x0022 110 104 000 Old_age Always - 42
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 4 <------- ここ
198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 3
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 208
SMART Error Log Version: 1
No Errors Logged
- dtv_recipe (/dev/sdf) をテストする
munakata@mythen:~ (master #)$ sudo smartctl -t short /dev/sdf
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.13.0-48-generic] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION ===
Sending command: "Execute SMART Short self-test routine immediately in off-line mode".
Drive command "Execute SMART Short self-test routine immediately in off-line mode" successful.
Testing has begun.
Please wait 2 minutes for test to complete.
Test will complete after Sun Jun 11 08:06:42 2017
Use smartctl -X to abort test.
- テスト結果を見る
- 上から最新のテスト結果
- read_failure で終了していて、エラーが出た LBA の先頭の番地が記録されている
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed: read failure 90% 14096 140218489
# 2 Extended offline Completed: read failure 90% 14082 140218489
# 3 Conveyance offline Completed: read failure 90% 14082 140218488
# 4 Short offline Completed: read failure 90% 14082 140218488
- 今回はディスクを交換することにした。
- 暫定的に不良ブロックを使わない 設定の紹介 もあった(が、不良ブロックが1つとは限らないだろう)