服务器磁盘卷组缓存cache设置介绍

工具1： storcli

a. 确认软件包是否安装

[root@localhost ~]#rpm -qa | grep storcli

storcli-1.21.06-1.noarch

备注：若检索结果为空，需要安装对应的软件安装包。安装命令如下：

#rpm -ivh storcli-xx-xx-1.noarch.rpm

b. 查看逻辑卷读/写缓存状态

备注：请关注标红部分示例为：NRWTD, 参数解释如下：

NR: No Read Ahead（关闭预读，即读缓存关闭）

R:Read Ahead(开启预读，即开启读缓存）

WB: Write Back（开启回写，即开启写缓存）

AWB: Always Write Back（强制开启回写，强制开启写缓存）

WT: Write Through(直写模式，关闭回写/写缓存）

[root@localhost ~]# /opt/MegaRAID/storcli/storcli64 /c0/v1 show

Controller = 0

Status = Success

Description = None

Virtual Drives :

==============

--------------------------------------------------------------

DG/VD TYPE State Access Consist Cache Cac sCC Size Name

--------------------------------------------------------------

1/1 RAID10 Optl RW No NRWTD - ON 2.180 TB

--------------------------------------------------------------

Cac=CacheCade|Rec=Recovery|OfLn=OffLine|Pdgd=Partially Degraded|Dgrd=Degraded

Optl=Optimal|RO=ReadOnly|RW=ReadWrite|HD=Hidden|TRANS=TransportReady|B=Blocked| Consist=Consistent|R=ReadAheadAlways|NR=NoReadAhead|WB=WriteBack| AWB=AlwaysWriteBack|WT=WriteThrough|C=CachedIO|D=DirectIO|sCC=ScheduledCheck Consistency

c. 读缓存打开/关闭

语法如下：

storcli /cx/vx set rdcache=RA|NoRA

参数说明：

cx: x为数字，从0开始，代表控制器编号

vx: x为数字，从0开始，代表逻辑卷编号

RA: Read Ahead

NoRA: No Read Ahead

示例：

[root@localhost ~]# /opt/MegaRAID/storcli/storcli64 /c0/v1 set rdcache=ra

Controller = 0

Status = Success

Description = None

Detailed Status :

===============

---------------------------------------

VD Property Value Status ErrMsg ErrCd

---------------------------------------

1 rdCache RA Success - 0

---------------------------------------

[root@localhost ~]# /opt/MegaRAID/storcli/storcli64 /c0/v1 set rdcache=nora

Controller = 0

Status = Success

Description = None

Detailed Status :

===============

---------------------------------------

VD Property Value Status ErrMsg ErrCd

---------------------------------------

1 rdCache NoRA Success - 0

---------------------------------------

d. 写缓存打开/关闭

语法如下：

storcli /cx/vx set wrcache=WT|WB|AWB

参数说明：

cx: x为数字，从0开始，代表控制器编号

vx: x为数字，从0开始，代表逻辑卷编号

WT: Write Through

WB: Write Back

AWB: Always Write Back

示例：

[root@localhost ~]# /opt/MegaRAID/storcli/storcli64 /c0/v1 set wrcache=wt

Controller = 0

Status = Success

Description = None

Detailed Status :

===============

---------------------------------------

VD Property Value Status ErrMsg ErrCd

---------------------------------------

1 wrCache WT Success - 0

---------------------------------------

[root@localhost ~]# /opt/MegaRAID/storcli/storcli64 /c0/v1 set wrcache=wb

Controller = 0

Status = Success

Description = None

Detailed Status :

===============

---------------------------------------

VD Property Value Status ErrMsg ErrCd

---------------------------------------

1 wrCache WB Success - 0

---------------------------------------

[root@localhost ~]# /opt/MegaRAID/storcli/storcli64 /c0/v1 set wrcache=awb

Controller = 0

Status = Success

Description = None

Detailed Status :

===============

---------------------------------------

VD Property Value Status ErrMsg ErrCd

---------------------------------------

1 wrCache AWB Success - 0

---------------------------------------

工具2：megacli

a. 确认软件包是否安装

[root@localhost ~]# rpm -qa | grep -i MegaCli

MegaCli-8.07.10-1.noarch

备注：若检索结果为空，需要安装对应的软件安装包。安装命令如下：

#rpm -ivh MegaCli-xx-xx-1.noarch.rpm

b. 查看逻辑卷读/写缓存状态

请关注标红部分示例为： WriteBack, ReadAheadNone, Direct, Write Cache OK if Bad BBU, 参数解释如下：

NORA: No Read Ahead（关闭预读，即读缓存关闭）

RA:Read Ahead(开启预读，即开启读缓存）

WB: Write Back（开启回写，即开启写缓存）

ForcedWB: Forced Write Back（强制开启回写，强制开启写缓存）

WT: Write Through(直写模式，关闭回写/写缓存）

示例：

[root@localhost ~]# /opt/MegaRAID/MegaCli/MegaCli64 -ldinfo -l1 -a0

Adapter 0 -- Virtual Drive Information:

Virtual Drive: 1 (Target Id: 1)

Name :

RAID Level : Primary-1, Secondary-0, RAID Level Qualifier-0

Size : 2.180 TB

Sector Size : 512

Is VD emulated : Yes

Mirror Data : 2.180 TB

State : Optimal

Strip Size : 64 KB

Number Of Drives per span:2

Span Depth : 5

Default Cache Policy: WriteBack, ReadAheadNone, Direct, Write Cache OK if Bad BBU

Current Cache Policy: WriteBack, ReadAheadNone, Direct, Write Cache OK if Bad BBU

Default Access Policy: Read/Write

Current Access Policy: Read/Write

Disk Cache Policy : Disk's Default

Ongoing Progresses:

Check Consistency : Completed 26%, Taken 493 min.

Encryption Type : None

Bad Blocks Exist: No

PI type: No PI

Is VD Cached: No

Exit Code: 0x00

c. 读/写缓存打开/关闭

语法如下：

MegaCli -LDSetProp {-Name LdNamestring} | -RW|RO|Blocked|RemoveBlocked | WT|WB|ForcedWB [-Immediate] |RA|NORA|ADRA |

DsblPI

-Lx|-L0,1,2|-Lall -aN|-a0,1,2|-aALL

参数说明：

aN: N为数字，从0开始，代表控制器编号

Lx: x为数字，从0开始，代表逻辑卷编号

NORA: No Read Ahead（关闭预读，即读缓存关闭）

RA:Read Ahead(开启预读，即开启读缓存）

WB: Write Back（开启回写，即开启写缓存）

ForcedWB: Forced Write Back（强制开启回写，强制开启写缓存）

WT: Write Through(直写模式，关闭回写/写缓存）

读缓存修改示例：

[root@localhost ~]# /opt/MegaRAID/MegaCli/MegaCli64 -LdSetProp RA -L1 -a0

Set Read Policy to ReadAhead on Adapter 0, VD 1 (target id: 1) success

Exit Code: 0x00

[root@localhost ~]# /opt/MegaRAID/MegaCli/MegaCli64 -LdSetProp NORA -L1 -a0

Set Read Policy to NoReadAhead on Adapter 0, VD 1 (target id: 1) success

Exit Code: 0x00

读缓存修改示例：

[root@localhost ~]# /opt/MegaRAID/MegaCli/MegaCli64 -LdSetProp WB -L1 -a0

Set Write Policy to WriteBack on Adapter 0, VD 1 (target id: 1) success

Exit Code: 0x00

[root@localhost ~]#/opt/MegaRAID/MegaCli/MegaCli64 -LdSetProp WT -L1 -a0

Set Write Policy to WriteThrough on Adapter 0, VD 1 (target id: 1) success

Exit Code: 0x00

[root@localhost ~]#/opt/MegaRAID/MegaCli/MegaCli64 -LdSetProp ForcedWB -L1 -a0

Set Write Policy to Forced WriteBack on Adapter 0, VD 1 (target id: 1) success

Exit Code: 0x00

工具3：hpssacli/ssacli

说明：

1. hpssacli工具为旧工具，目前大部分线上服务器都安装有对应的安装包，ssacli为新工具，在旧工具无法正常执行的情况下，建议使用新工具，语法完全一致。

2. ssdsmartpath（固态硬盘智能路径）和caching(写缓存）只能打开一个，在一个打开的状态下，另外一个的值必定是关闭的,例如：

caching：Enable的情况下，ssdsmartpath的值必定为disabled.

a. 确认安装包是否安装

[root@localhost ~]# rpm -qa | grep hpssacli

hpssacli-2.20-11.0.x86_64

备注：若检索结果为空，需要安装对应的软件安装包。安装命令如下：

#rpm -ivh hpssacli-xx-xx.x86_64.rpm

#rpm -ivh ssacli-xx-xx.x86_64.rpm

b. 确认硬盘控制器所在slot插槽

说明：如下所示，阵列卡型号为Smart Array P840ar, 插槽位置为Slot 0, 务必查询信息，插槽位置后面的命令要用。

[root@localhost ~]# hpssacli ctrl all show

Smart Array P840ar in Slot 0 (Embedded) (sn: PVYKH0BRH710CO)

c. 确认逻辑卷写缓存(caching)状态

参数说明：

ctrl: Controller 简写

slot=0: 硬盘控制器所在PCIe插槽位置

ld 2: ld(logical drive 逻辑卷）,后面的数字为逻辑卷编号，顺序从1开始

show: 信息展示

说明：请关注示例中caching的值，示例中为Disabled(即关闭）

[root@localhost ~]# hpssacli ctrl slot=0 ld 2 show

Smart Array P840ar in Slot 0 (Embedded)

array B

Logical Drive: 2

Size: 2.2 TB

Fault Tolerance: 1+0

Heads: 255

Sectors Per Track: 32

Cylinders: 65535

Strip Size: 256 KB

Full Stripe Size: 1280 KB

Status: OK

MultiDomain Status: OK

Caching: Disabled

Unique Identifier: 600508B1001C0D8EB99EB2A7EE5060E4

Disk Name: /dev/sdb

Mount Points: /opt/uredis 2.2 TB Partition Number 1

Logical Drive Label: 06172624PVYKH0BRH710COB70A

Mirror Group 1:

physicaldrive 1I:1:5 (port 1I:box 1:bay 5, Solid State SATA, 480.1 GB, OK)

physicaldrive 1I:1:6 (port 1I:box 1:bay 6, Solid State SATA, 480.1 GB, OK)

physicaldrive 1I:1:7 (port 1I:box 1:bay 7, Solid State SATA, 480.1 GB, OK)

physicaldrive 1I:1:8 (port 1I:box 1:bay 8, Solid State SATA, 480.1 GB, OK)

physicaldrive 1I:1:4 (port 1I:box 1:bay 4, Solid State SATA, 480.1 GB, OK)

Mirror Group 2:

physicaldrive 1I:1:3 (port 1I:box 1:bay 3, Solid State SATA, 480.1 GB, OK)

physicaldrive 2I:2:1 (port 2I:box 2:bay 1, Solid State SATA, 480.1 GB, OK)

physicaldrive 2I:2:2 (port 2I:box 2:bay 2, Solid State SATA, 480.1 GB, OK)

physicaldrive 2I:2:3 (port 2I:box 2:bay 3, Solid State SATA, 480.1 GB, OK)

physicaldrive 2I:2:4 (port 2I:box 2:bay 4, Solid State SATA, 480.1 GB, OK)

Drive Type: Data

LD Acceleration Method: HP SSD Smart Path

d. 确认逻辑卷ssdsmartpath状态

参数说明：

ctrl: Controller 简写

slot=0: 硬盘控制器所在PCIe插槽位置

array b: ld(array, 磁盘阵列，也即逻辑卷）,后面的字母为逻辑卷编号，顺序从a开始

show: 信息展示

说明：请关注示例中ssdsmartpath的值，示例中为Enable(即关闭）

[root@localhost ~]# hpssacli ctrl slot=0 array b show

Smart Array P840ar in Slot 0 (Embedded)

Array: B

Interface Type: Solid State SATA

Unused Space: 0 MB (0.0%)

Used Space: 4.4 TB (100.0%)

Status: OK

MultiDomain Status: OK

Array Type: Data

HP SSD Smart Path: enable

e. 关闭固态硬件智能路径并打开写缓存示例

[root@localhost ~]# hpssacli ctrl slot=0 array b modify ssdsmartpath=disable

[root@localhost ~]# hpssacli ctrl slot=0 ld 2 modify caching=enable

f. 关闭写缓存并打开固态硬盘智能路径

[root@localhost ~]# hpssacli ctrl slot=0 ld 2 modify caching=disable

[root@localhost ~]# hpssacli ctrl slot=0 array b modify ssdsmartpath=enable

工具4： arcconf

a. 确认软件包是否安装

[root@localhost ~]# rpm -qa | grep Arcconf

Arcconf-2.04-22665.x86_64

备注：若检索结果为空，需要安装对应的软件安装包。安装命令如下：

#rpm -ivh Arcconf-xx-xx.x86_64.rpm

b. 确认硬盘控制器信息

说明：控制器编号从1开始，所以示例中控制器ID为1:

[root@localhost ~]# /usr/Arcconf/arcconf list

Controllers found: 1

----------------------------------------------------------------------

Controller information

----------------------------------------------------------------------

Controller ID : Status, Slot, Mode, Name, SerialNumber, WWN

----------------------------------------------------------------------

Controller 1: : Optimal, Slot 2147483647, RAID (Expose RAW), , FB488000, 560DA833FB488000

c. 查看逻辑卷读/写缓存状态

备注：

1. 请关注标红部分示例为：参数说明如下：

Read-cache setting : Enabled(读缓存打开)

Write-cache setting : Enabled(写缓存打开）

2. 示例命令行参数说明（/usr/Arcconf/arcconf getconfig 1 ld 1）

getconfig: prints controller information(打印控制器相关信息）

第一个1：控制器编号

ld:logical drive(逻辑卷，即磁盘阵列）

第二个1：逻辑卷编号，从0开始，示例是第二个逻辑卷，所以为1.

[root@localhost ~]# /usr/Arcconf/arcconf getconfig 1 ld 1

Controllers found: 1

----------------------------------------------------------------------

Logical device information

----------------------------------------------------------------------

Logical Device number 1

Logical Device name : data

Block Size of member drives : 512 Bytes

RAID level : 10

Unique Identifier : 6CDE0BB1

Status of Logical Device : Optimal

Additional details : Quick initialized

Size : 1014 MB

Parity space : 1024 MB

Stripe-unit size : 256 KB

Interface Type : Serial ATA

Device Type : HDD

Read-cache setting : Enabled

Read-cache status : On

Write-cache setting : Enabled

Write-cache status : On

Partitioned : No

Protected by Hot-Spare : No

Bootable : No

Failed stripes : No

Power settings : Disabled

--------------------------------------------------------

Logical Device segment information

--------------------------------------------------------

Group 0, Segment 0 : Present (3815447MB, SATA, HDD, Enclosure:0, Slot:2) K4KZMMKL

Group 0, Segment 1 : Present (3815447MB, SATA, HDD, Enclosure:0, Slot:3) K7G2L3SL

Group 1, Segment 0 : Present (3815447MB, SATA, HDD, Enclosure:0, Slot:4) K4KYTSLL

Group 1, Segment 1 : Present (3815447MB, SATA, HDD, Enclosure:0, Slot:5) K7G183TL

d. 读/写缓存设置语法

语法如下：

Usage: SETCACHE <Controller#> LOGICALDRIVE <LogicalDrive#> <logical mode> [noprompt] [nologs]

Usage: SETCACHE <Controller#> DEVICE <Channel# ID#> <physical mode> [nologs]

Usage: SETCACHE <Controller#> DEVICEALL <policy> [noprompt] [nologs]

Example: SETCACHE 1 LOGICALDRIVE 0 ron

Example: SETCACHE 1 DEVICE 0 0 wt

Example: SETCACHE 1 DEVICEALL drivespecific

===================================================================================

Changes a device's cache mode.

LOGICALDRIVE parameters

LogicalDrive# : Number of the logical device whose cache will be altered

Logical Modes : ron - read cache enabled

roff - read cache disabled

wt - write through

wb - write back

wbb - write back with battery/ZMM

con - cache enabled

coff - cache disabled

DEVICE parameters

Channel# ID# : Channel and ID of the device whose cache will be altered

Physical Modes : wt - write through

wb - write back

DEVICEALL parameters

Policy : disable - write back for all physical drives

enable - write through for all physical drives

drivespecific - user can set for an individual physical drive

部分参数说明：

rron: read cache enabled(打开读缓存)

roff: read cache disabled(关闭读缓存)

wt: write through(只写模式，即关闭写缓存）

wb:write back(回写模式，即打开写缓存）

con: cache enabled(打开逻辑卷缓存（打开读/写缓存前提是逻辑卷缓存已打开，在逻辑卷关闭的情况下，无法打开读/写缓存））

coff: cache disabled(关闭逻辑卷缓存）

noprompt：no prompt(没有提示，按照默认y执行，部分命令需要和用户交互，使用noprompt参数可按照默认yes答复执行）

e. 读缓存打开/关闭示例

[root@localhost ~]# /usr/Arcconf/arcconf setcache 1 logicaldrive 1 roff

Controllers found: 1

Command completed successfully.

[root@localhost ~]# /usr/Arcconf/arcconf setcache 1 logicaldrive 1 ron

Controllers found: 1

Command completed successfully.

f. 写缓存打开/关闭示例

[root@localhost ~]# /usr/Arcconf/arcconf setcache 1 logicaldrive 1 wt

Controllers found: 1

Command completed successfully.

You have new mail in /var/spool/mail/root

[root@localhost ~]# /usr/Arcconf/arcconf setcache 1 logicaldrive 1 wb

Controllers found: 1

WARNING: Power failure without battery/ZMM support will lead to data loss.

Do you wish to continue?

Press y, then ENTER to continue or press ENTER to abort: y

Command completed successfully.

[root@localhost ~]# /usr/Arcconf/arcconf setcache 1 logicaldrive 1 wbb

Controllers found: 1

Command completed successfully