※ このエントリは、はてなグループ終了に伴う、サブブログからの引越エントリ(2010/11, 2011/02, 2012/05, 06, 07)です。
※ 情報が古い可能性もありますので、ご留意ください。
Fusion-io ioDrive Duo で "device busy."
検証サーバに ioDrive Duo を刺して触ろうとしていた時のこと。
早速、デタッチして一応フォーマットしようと思ったら、、、怒られる。
# fio-detach /dev/fct0 Detaching: [====================] (100%) / Error: failed to detach /dev/fct0: device busy.
あぁ、↓の方で記載している "Failed to communicate with device." が出た時のパターンかなと思いきや、それも違った。
ちょっと調べると、、、おや？
# ls /dev/md* md/ md127
見慣れぬ RAID デバイスが・・・。昔誰かが使った時に組まれたものっぽい。
# mdadm --detail /dev/md127
/dev/md127:
Version : 0.90
Creation Time : Thu Feb 2 11:13:34 2012
Raid Level : raid0
Array Size : 625001472 (596.05 GiB 640.00 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 127
Persistence : Superblock is persistent
Update Time : Thu Feb 2 11:13:34 2012
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Chunk Size : 256K
UUID : b734xxxx:b4b4xxxx:1fb1xxxx:91c5xxxx
Events : 0.1
Number Major Minor RaidDevice State
0 252 0 0 active sync /dev/fioa
1 252 16 1 active sync /dev/fiobほほう。Duo だからね。誰かが組んだんだろう。
# mdadm --manage /dev/md127 --stop mdadm: stopped /dev/md127
というわけで、stop させて、
# fio-detach /dev/fct0 Detaching: [====================] (100%) /
デタッチしましたとさ。ここから fio-format してスッキリ。
Fujitsu RX200 S7 で Fusion-io ioDrive Duo
PCI バスの帯域幅的にもいけそうだ！
# hdparm -t /dev/md0 /dev/md0: Timing buffered disk reads: 4744 MB in 3.00 seconds = 1580.78 MB/sec
# fio-pci-check
Root Bridge PCIe 8750 MB/sec needed max
Bridge 00:02.02 (04-08)
Needed 2000 MB/sec Avail 4000 MB/sec
Bridge 04:00.00 (05-08)
Needed 2000 MB/sec Avail 4000 MB/sec
* Correctable Error(s): Detected
* Unsupported Request(s): Detected
Clearing Errors
Slot Power limit: 25.0W (25000mw)
Bridge 05:05.00 (07-07)
Needed 1000 MB/sec Avail 1000 MB/sec
* Correctable Error(s): Detected
* Unsupported Request(s): Detected
Clearing Errors
ioDrive 07:00.0 (1005) Firmware 107053
Slot Power limit: 25.0W (25000mw)
Bridge 05:06.00 (08-08)
Needed 1000 MB/sec Avail 1000 MB/sec
* Correctable Error(s): Detected
* Unsupported Request(s): Detected
Clearing Errors
ioDrive 08:00.0 (1005) Firmware 107053
Slot Power limit: 25.0W (25000mw)
Fusion-io ioDrive2 を自動でアタッチ (attach) させないようにする
"/etc/modprobe.d/iomemory-vsl.conf" を編集して、以下を設定すると、自動アタッチが無効になる。
options iomemory-vsl auto_attach=0
Fusion-io ioDrive/ioDrive2 使用時、PCI スロットの速度確認
fio-pci-check コマンドで。
ioDrive Duo (2.2.3 build 66)
# fio-pci-check
Root Bridge PCIe 8000 MB/sec needed max
Bridge 00:07.00 (06-0a)
Needed 2000 MB/sec Avail 4000 MB/sec
Bridge 06:00.00 (07-0a)
Needed 2000 MB/sec Avail 4000 MB/sec
Slot Power limit: 25.0W (25000mw)
Bridge 07:05.00 (09-09)
Needed 1000 MB/sec Avail 1000 MB/sec
ioDrive 09:00.0 Firmware 101583
Slot Power limit: 25.0W (25000mw)
Bridge 07:06.00 (0a-0a)
Needed 1000 MB/sec Avail 1000 MB/sec
ioDrive 0a:00.0 Firmware 101583
Slot Power limit: 25.0W (25000mw)
ioDrive2 (3.1.1 build 172)
# fio-pci-check
Root Bridge PCIe 14000 MB/sec needed max
ERROR: invalid link speed (value 0x0)
ERROR: invalid link width (value 0x0)
Bridge 00:03.00 (04-04)
Needed 2000 MB/sec Avail 2000 MB/sec
ERROR: invalid maximum link speed (0x3)
ioDrive 04:00.0 Firmware 107004
Slot Power limit: 25.0W (25000mw)なんか足りないのか奇麗に取りきれていない・・・が。
Fusion-io ioDrive2 で "Failed to communicate with device." が出た時
デバイスをデタッチしようとした時などに
# fio-detach /dev/fioa Detaching: [====================] (100%) Error: Device '/dev/fioa' had an error while detaching. Failed to communicate with device.
と出る時は、それ物理ボリュームのデバイスが指定できてないかも。
# fio-detach /dev/fct0 Detaching: [====================] (100%) / fioa - detached.
こんな感じで指定しましょう。
でも、まだ出ちゃう時とかは、、、
# fio-detach /dev/fct0 Detaching: [====================] (100%) Error: Device '/dev/fct0' had an error while detaching. Internal failure.
まだマウントしたままとかで、使われている状態のままかも。umount できているか確認しましょう。
Fusion-io ioDrive2 の fio-status
# fio-status -a
Found 1 ioMemory device in this system
Fusion-io driver version: 3.1.1 build 172
Adapter: Single Controller Adapter
Fusion-io ioDrive2 1.205TB, Product Number:F00-001-1T20-CS-0001, FIO SN:XXXXXXX91
ioDrive2 Adapter Controller, PN:XXXXXXXXX09
External Power: NOT connected
PCIe Bus voltage: avg 12.00V
PCIe Bus current: avg 0.77A
PCIe Bus power: avg 9.20W
PCIe slot available power: 25.00W
PCIe negotiated link: 4 lanes at 5.0 Gt/sec each, 2000.00 MBytes/sec total
Connected ioMemory modules:
fct0: Product Number:F00-001-1T20-CS-0001, SN:XXXXXXX91
fct0 Attached as 'fioa' (block device)
ioDrive2 Adapter Controller, Product Number:F00-001-1T20-CS-0001, SN:XXXXXXX91
ioDrive2 Adapter Controller, PN:XXXXXXXXX09
SMP(AVR) Versions: App Version: 1.0.9.0, Boot Version: 0.0.3.1
Powerloss protection: protected
PCI:04:00.0, Slot Number:2
Vendor:1aed, Device:2001, Sub vendor:1aed, Sub device:2001
Firmware v6.0.0, rev 107004 Public
1205.00 GBytes block device size
Format: v500, 2353515625 sectors of 512 bytes
PCIe slot available power: 25.00W
PCIe negotiated link: 4 lanes at 5.0 Gt/sec each, 2000.00 MBytes/sec total
Internal temperature: 54.14 degC, max 56.11 degC
Internal voltage: avg 1.01V, max 1.02V
Aux voltage: avg 2.48V, max 2.49V
Reserve space status: Healthy; Reserves: 100.00%, warn at 10.00%
Rated PBW: 17.00 PB, 99.99% remaining
Lifetime data volumes:
Physical bytes written: 904,009,537,672
Physical bytes read : 999,422,573,856
RAM usage:
Current: 13,050,432 bytes
Peak : 13,050,432 bytes
Fusion-io ioDrive Duo (MLC 640GB)で ベンチマーク (hdparm -t)
とりあえず、"hdparm -t" で簡易的に3回繰り返す感じで実行。
# hdparm -t /dev/md0 /dev/md0: Timing buffered disk reads: 3870 MB in 3.00 seconds = 1289.98 MB/sec # hdparm -t /dev/md0 /dev/md0: Timing buffered disk reads: 3976 MB in 3.00 seconds = 1325.24 MB/sec # hdparm -t /dev/md0 /dev/md0: Timing buffered disk reads: 3760 MB in 3.00 seconds = 1252.90 MB/sec
"Fusion-io ioDrive Duo MLC 640GB" で fio-status
fio-status -a
# fio-status -a
Found 2 ioDrives in this system with 1 ioDrive Duo
Fusion-io driver version: 2.1.0 build 240
Adapter: ioDrive Duo
HP 640GB MLC PCIe ioDrive Duo for ProLiant Servers, Product Number:600282-B21 SN:xxxxx
ioDrive Duo HL, PN:001xxxxxxxx, Mfr:004, Date:20100919
External Power: NOT connected
Powerloss protection: available
PCIE Bus voltage: avg 11.90V, min 11.75V, max 11.98V
PCIE Bus current: avg 0.00A, max 2.90A
PCIE Bus power: avg 0.00W, max 0.00W
PCIE Power limit threshold: 24.75W
PCIE slot available power: 25.00W
PCIE bus errors: correctable, unsupported request
PCIE negotiated link: 8 lanes at 5.00 Gbits/sec each, 4000 MBytes/sec total
Connected ioDimm modules:
fct0: HP 640GB MLC PCIe ioDrive Duo for ProLiant Servers, Product Number:600282-B21 SN:xxxxx
fct1: HP 640GB MLC PCIe ioDrive Duo for ProLiant Servers, Product Number:600282-B21 SN:xxxxx
fct0 Attached as 'fioa' (block device)
HP 640GB MLC PCIe ioDrive Duo for ProLiant Servers, Product Number:600282-B21 SN:xxxxx
HP ioDIMM 320GB, PN:002xxxxxxxx, Mfr:004, Date:20100917
Located in 0 Upper slot of ioDrive Duo SN:xxxxxx
Powerloss protection: protected
PCI:07:00.0, Slot Number: Unavailable
Vendor:1aed, Device:1005, Sub vendor:103c, Sub device:178e
Firmware v5.0.3, rev 43247
322.46 GBytes block device size, 396 GBytes physical device size
Error correction: 11 bits per 240 bytes
Format v004, 629,806,848 sectors of 512 bytes
FPGA ID:0 Format UID:00000000908b0132b735004210daa400
PCIE slot available power: 25.00W
PCIE negotiated link: 4 lanes at 2.50 Gbits/sec each, 1000 MBytes/sec total
Internal temperature: avg 50.7 degC, max 56.1 degC
Board temperature: 40 degC
Internal voltage: avg 1.028V, max 1.031V
Aux voltage: avg 2.461V, max 2.464V
Media status: Healthy; Reserves: 100.00%, warn at 10.00%
Lifetime data volumes:
Physical bytes written: 2,213,319,448,568
Physical bytes read : 2,189,074,722,536
fct1 Attached as 'fiob' (block device)
HP 640GB MLC PCIe ioDrive Duo for ProLiant Servers, Product Number:600282-B21 SN:xxxxx
HP ioDIMM 320GB, PN:002xxxxxxxx, Mfr:004, Date:20100917
Located in 1 Lower slot of ioDrive Duo SN:xxxxxx
Powerloss protection: protected
PCI:08:00.0, Slot Number: Unavailable
Vendor:1aed, Device:1005, Sub vendor:103c, Sub device:178e
Firmware v5.0.3, rev 43247
322.46 GBytes block device size, 396 GBytes physical device size
Error correction: 11 bits per 240 bytes
Format v004, 629,806,848 sectors of 512 bytes
FPGA ID:0 Format UID:0000000115630132b735004210daa400
PCIE slot available power: 25.00W
PCIE negotiated link: 4 lanes at 2.50 Gbits/sec each, 1000 MBytes/sec total
Internal temperature: avg 49.2 degC, max 54.1 degC
Board temperature: 39 degC
Internal voltage: avg 1.014V, max 1.014V
Aux voltage: avg 2.458V, max 2.461V
Media status: Healthy; Reserves: 100.00%, warn at 10.00%
Lifetime data volumes:
Physical bytes written: 2,211,806,766,816
Physical bytes read : 2,185,788,095,984
fio-status --help
# fio-status --help Fusion-io fio-status utility (2.1.0.240 optimusprime@4a5881616570) General information utility for Fusion-io devices. Copyright 2006-2010 Fusion Multisystems, Inc., dba Fusion-io. All rights reserved. usage: fio-status [<options>] [<dev>] <dev> is a pathname to the control device (fctx). Options: -a --all Report all available information. -c --count fio-board-count mode: Report the number of Fusion-io boards installed. -d --data-volume Report the volume of data read and written. -fk --format key Force alternate 'key=value' format output. -v --version Print version information -h --help Print the help menu
