之前打算装 Oracle 11gR2 的 RAC 的。 后来因为资源不够，就改装 oracle 10g 了。
raw 是用 openfiler 配置 加 Multipath 多路径配置的。
raw 的配置参考我的 Blog ：
Linux 5.4 LVM RAW 设备 配置的深入研究
http://blog.csdn.net/tianlesoftware/archive/2010/11/26/6036263.aspx
这种情况下配置有些注意事项。 否则会在执行 root.sh 命令的时候，报错：
[root@rac1 init.d]# /u01/app/oracle/product/crs/root.sh
WARNING: directory '/u01/app/oracle/product' is not owned by root
WARNING: directory '/u01/app/oracle' is not owned by root
WARNING: directory '/u01/app' is not owned by root
WARNING: directory '/u01' is not owned by root
Checking to see if Oracle CRS stack is already configured
Setting the permissions on OCR backup directory
Setting up NS directories
PROT-1: Failed to initialize ocrconfig
Failed to upgrade Oracle Cluster Registry configuration
一般出现这个问题的原因有几种：
（ 1 ） Bug
这种情况参考：
Executing root.sh errors with "Failed To Upgrade Oracle Cluster Registry Configuration" [ID 466673.1]
http://blog.csdn.net/tianlesoftware/archive/2010/11/26/6037141.aspx
（ 2 ） raw 设备的权限问题
[root@rac1 raw]# pwd
/dev/raw
[root@rac1 raw]# ll
total 0
crw-r----- 1 root oinstall 162, 1 Nov 25 15:34 raw1
crw-r----- 1 root oinstall 162, 2 Nov 25 15:34 raw2
crw-r--r-- 1 oracle oinstall 162, 3 Nov 25 13:48 raw3
crw-r--r-- 1 oracle oinstall 162, 4 Nov 25 13:48 raw4
crw-r--r-- 1 oracle oinstall 162, 5 Nov 25 13:48 raw5
权限要设置成如下：
chown root.oinstall /dev/raw/raw[1-2]
chown oracle.oinstall /dev/raw/raw[3-5]
chmod 775 /dev/raw/raw[1-2]
chmod 775 /dev/raw/raw[3-5]
可以在 udev 的规则里设置， 也可以把权限添加到 /etc/rc.d/rc.local 中。
（ 3 ） raw 设备读写问题
这种情况参考：
Placement of Voting disk and OCR Files in Oracle RAC 10g and 11gR1 [ID 293819.1]
http://blog.csdn.net/tianlesoftware/archive/2010/11/26/6037108.aspx
清空一下 RAW:
[root@rac1 raw]# dd if=/dev/zero of=/dev/mapper/crsp6 bs=1M count=200
dd: writing `/dev/mapper/crsp6': No space left on device
197+0 records in
196+0 records out
205599744 bytes (206 MB) copied, 21.1817 seconds, 9.7 MB/s
说明：
这里有个提示信息，磁盘空间不足 : No space left on device 。 这里的情况是， 我的每个 raw 是 200M 。 但是实际容量并没有这么多。 所以我清空写 200M 的时候，会报空间不足。 这种情况会按照实际大小进行清空。 从下面我们可以该 raw 设备实际大小是 197M 。
[root@rac1 raw]# dd if=/dev/zero of=/dev/mapper/crsp6 bs=1M count=190
190+0 records in
190+0 records out
199229440 bytes (199 MB) copied, 21.9521 seconds, 9.1 MB/s
[root@rac1 raw]# dd if=/dev/zero of=/dev/mapper/crsp5 bs=1M count=195
195+0 records in
195+0 records out
204472320 bytes (204 MB) copied, 22.0362 seconds, 9.3 MB/s
[root@rac1 raw]# dd if=/dev/zero of=/dev/mapper/crsp7 bs=1M count=195
195+0 records in
195+0 records out
204472320 bytes (204 MB) copied, 19.9692 seconds, 10.2 MB/s
[root@rac1 raw]# dd if=/dev/zero of=/dev/mapper/crsp8 bs=1M count=195
195+0 records in
195+0 records out
204472320 bytes (204 MB) copied, 18.5526 seconds, 11.0 MB/s
[root@rac1 raw]# dd if=/dev/zero of=/dev/mapper/crsp9 bs=1M count=195
195+0 records in
195+0 records out
204472320 bytes (204 MB) copied, 21.9439 seconds, 9.3 MB/s
我这次遇到问题的 Log 如下：
[root@rac1 client]# pwd
/u01/app/oracle/product/crs/log/rac1/client
[root@rac1 client]# ls
ocrconfig_7527.log
[root@rac1 client]# cat ocrconfig_7527.log
Oracle Database 10g CRS Release 10.2.0.1.0 Production Copyright 1996, 2005 Oracle. All rights reserved.
2010-11-25 19:54:14.694: [ OCRCONF][1189568]ocrconfig starts...
2010-11-25 19:54:14.696: [ OCRCONF][1189568]Upgrading OCR data
2010-11-25 19:54:15.434: [ OCRRAW][1189568]propriogid:1: INVALID FORMAT
2010-11-25 19:54:15.464: [ OCRRAW][1189568]propriogid:1: INVALID FORMAT
2010-11-25 19:54:15.464: [ OCRRAW][1189568]proprioini: both disks are not OCR formatted
2010-11-25 19:54:15.464: [ OCRRAW][1189568]proprinit: Could not open raw device
2010-11-25 19:54:15.465: [ default][1189568]a_init:7!: Backend init unsuccessful : [26]
2010-11-25 19:54:15.465: [ OCRCONF][1189568]Exporting OCR data to [OCRUPGRADEFILE]
2010-11-25 19:54:15.465: [ OCRAPI][1189568]a_init:7!: Backend init unsuccessful : [33]
2010-11-25 19:54:15.465: [ OCRCONF][1189568]There was no previous version of OCR. error:[PROC-33: Oracle Cluster Registry is not configured]
2010-11-25 19:54:16.072: [ OCRRAW][1189568]propriogid:1: INVALID FORMAT
2010-11-25 19:54:16.075: [ OCRRAW][1189568]propriogid:1: INVALID FORMAT
2010-11-25 19:54:16.075: [ OCRRAW][1189568]proprioini: both disks are not OCR formatted
2010-11-25 19:54:16.075: [ OCRRAW][1189568]proprinit: Could not open raw device
2010-11-25 19:54:16.076: [ default][1189568]a_init:7!: Backend init unsuccessful : [26]
2010-11-25 19:54:16.231: [ OCRRAW][1189568]propriogid:1: INVALID FORMAT
2010-11-25 19:54:16.232: [ OCRRAW][1189568]propriogid:1: INVALID FORMAT
2010-11-25 19:54:16.265: [ OCRRAW][1189568]ibctx:1:ERROR: INVALID FORMAT
2010-11-25 19:54:16.265: [ OCRRAW][1189568]proprinit:problem reading the bootblock or superbloc 22
2010-11-25 19:54:16.646: [ OCRRAW][1189568]propriogid:1: INVALID FORMAT
2010-11-25 19:54:16.647: [ OCRRAW][1189568]propriogid:1: INVALID FORMAT
2010-11-25 19:54:16.757: [ OCRRAW][1189568]propriowv: Vote information on disk 0 [/dev/raw/raw1] is adjusted from [0/0] to [1/2]
2010-11-25 19:54:16.762: [ OCRRAW][1189568]propriowv: Vote information on disk 1 [/dev/raw/raw2] is adjusted from [0/0] to [1/2]
2010-11-25 19:54:17.193: [ OCRRAW][1189568]propriniconfig:No 92 configuration
2010-11-25 19:54:17.193: [ OCRAPI][1189568]a_init:6a: Backend init successful
2010-11-25 19:54:18.263: [ OCRCONF][1189568]Initialized DATABASE keys in OCR
2010-11-25 19:54:18.478: [ OCRCONF][1189568]csetskgfrblock0: clsfmt returned with error [4].
2010-11-25 19:54:18.478: [ OCRCONF][1189568]Failure in setting block0 [-1]
2010-11-25 19:54:18.478: [ OCRCONF][1189568]OCR block 0 is not set !
2010-11-25 19:54:18.479: [ OCRCONF][1189568]Exiting [status=failed]...
这个是 Multipath 的 bug 问题，就是我们遇到的第一种情况：
（ 1 ） 用 clsfmt 检查 raw 设备
[root@rac1 bin]# ./clsfmt.bin ocr /dev/raw/raw1
clsfmt: Failure trying to resize OCR file from 205567488 bytes to 205570048 bytes
clsfmt: Received unexpected error 4 from skgfifi
skgfifi: Additional information: -2
Additional information: 205567488
这里没有成功， 就是 bug 。
（ 2 ）用 Pathch 中的新文件替换 clsfmt.bin 文件。
Patch 是： p4679769 。 这个需要 Oracle 帐号才能下载。 所以我将 Linux 平台下 32 位和 64 位的 path 都上传到 csdn 的下载里了。 有遇到相同问题的朋友自己去下载。 下载地址：
http://download.csdn.net/source/2862150
（ 3 ）再次测试：
[root@rac1 bin]# ./clsfmt ocr /dev/raw/raw1
clsfmt: Failure trying to resize OCR file from 205567488 bytes to 205570048 bytes
clsfmt: successfully initialized file /dev/raw/raw1
[root@rac1 bin]# ./clsfmt ocr /dev/raw/raw2
clsfmt: Failure trying to resize OCR file from 205599744 bytes to 205602816 bytes
clsfmt: successfully initialized file /dev/raw/raw2
[root@rac1 bin]# ./clsfmt ocr /dev/raw/raw3
clsfmt: Failure trying to resize OCR file from 205599744 bytes to 205602816 bytes
clsfmt: successfully initialized file /dev/raw/raw3
[root@rac1 bin]# ./clsfmt ocr /dev/raw/raw4
clsfmt: Failure trying to resize OCR file from 205599744 bytes to 205602816 bytes
clsfmt: successfully initialized file /dev/raw/raw4
[root@rac1 bin]# ./clsfmt ocr /dev/raw/raw5
clsfmt: Failure trying to resize OCR file from 205599744 bytes to 205602816 bytes
clsfmt: successfully initialized file /dev/raw/raw5
bug 修复成功了。
（ 4 ） 清空 raw 设备
因为我们已经执行过 root.sh 命令，会有相关信息写到 raw 设备里。 用 dd 命令清空这些信息。 最好留一点空间。 因为在 Solaris 平台上有预留 1M 空间的做法，不然会出现问题。
[root@rac1 rac1]# dd if=/dev/zero of=/dev/raw/raw1 bs=10M count=10
10+0 records in
10+0 records out
104857600 bytes (105 MB) copied, 17.4534 seconds, 6.0 MB/s
[root@rac1 rac1]# dd if=/dev/zero of=/dev/raw/raw2 bs=10M count=10
10+0 records in
10+0 records out
104857600 bytes (105 MB) copied, 11.4307 seconds, 9.2 MB/s
[root@rac1 rac1]# dd if=/dev/zero of=/dev/raw/raw3 bs=10M count=10
10+0 records in
10+0 records out
104857600 bytes (105 MB) copied, 18.4136 seconds, 5.7 MB/s
[root@rac1 rac1]# dd if=/dev/zero of=/dev/raw/raw4 bs=10M count=10
10+0 records in
10+0 records out
104857600 bytes (105 MB) copied, 17.821 seconds, 5.9 MB/s
[root@rac1 rac1]# dd if=/dev/zero of=/dev/raw/raw5 bs=10M count=10
10+0 records in
10+0 records out
104857600 bytes (105 MB) copied, 17.3289 seconds, 6.1 MB/s
如果不清空 raw 设备，直接执行 root.sh 命令，还是会报错误：
[root@rac1 bin]# /u01/app/oracle/product/crs/root.sh
WARNING: directory '/u01/app/oracle/product' is not owned by root
WARNING: directory '/u01/app/oracle' is not owned by root
WARNING: directory '/u01/app' is not owned by root
WARNING: directory '/u01' is not owned by root
Checking to see if Oracle CRS stack is already configured
Setting the permissions on OCR backup directory
Setting up NS directories
PROT-1: Failed to initialize ocrconfig
Failed to upgrade Oracle Cluster Registry configuration
错误的 log 日志如下：
#cat /u01/app/oracle/product/crs/log/rac1/client/ocrconfig_16159.log
Oracle Database 10g CRS Release 10.2.0.1.0 Production Copyright 1996, 2005 Oracle. All rights reserved.
2010-11-25 23:27:25.468: [ OCRCONF][2118112]ocrconfig starts...
2010-11-25 23:27:25.469: [ OCRCONF][2118112]Upgrading OCR data
2010-11-25 23:27:25.558: [ OCRRAW][2118112]propriogid:1: INVALID FORMAT
2010-11-25 23:27:25.559: [ OCRRAW][2118112]proprioini: disk 0 (/dev/raw/raw1) doesn't have enough votes (1,2)
2010-11-25 23:27:25.559: [ OCRRAW][2118112]proprinit: Could not open raw device
2010-11-25 23:27:25.560: [ default][2118112]a_init:7!: Backend init unsuccessful : [26]
2010-11-25 23:27:25.560: [ OCRCONF][2118112]Exporting OCR data to [OCRUPGRADEFILE]
2010-11-25 23:27:25.560: [ OCRAPI][2118112]a_init:7!: Backend init unsuccessful : [33]
2010-11-25 23:27:25.560: [ OCRCONF][2118112]There was no previous version of OCR. error:[PROC-33: Oracle Cluster Registry is not configured]
2010-11-25 23:27:25.746: [ OCRRAW][2118112]propriogid:1: INVALID FORMAT
2010-11-25 23:27:25.746: [ OCRRAW][2118112]proprioini: disk 0 (/dev/raw/raw1) doesn't have enough votes (1,2)
2010-11-25 23:27:25.746: [ OCRRAW][2118112]proprinit: Could not open raw device
2010-11-25 23:27:25.747: [ default][2118112]a_init:7!: Backend init unsuccessful : [26]
2010-11-25 23:27:26.112: [ OCRRAW][2118112]propriogid:1: INVALID FORMAT
2010-11-25 23:27:26.112: [ OCRRAW][2118112]proprioini: disk 0 (/dev/raw/raw1) doesn't have enough votes (1,2)
2010-11-25 23:27:26.112: [ OCRRAW][2118112]proprinit: Could not open raw device
2010-11-25 23:27:26.120: [ OCRAPI][2118112]a_init:6b!: Backend init unsuccessful : [26]
2010-11-25 23:27:26.120: [ OCRCONF][2118112]Failed to initialized OCR context. error:[PROC-26: Error while accessing the physical storage]
2010-11-25 23:27:26.120: [ OCRCONF][2118112]Exiting [status=failed]...
（ 5 ）清空 raw 设备后，在执行 /u01/app/oracle/product/crs/root.sh 命令
这次成功执行。
------------------------------------------------------------------------------
Blog ： http://blog.csdn.net/tianlesoftware
网上资源： http://tianlesoftware.download.csdn.net
相关视频： http://blog.csdn.net/tianlesoftware/archive/2009/11/27/4886500.aspx
DBA1 群： 62697716( 满 ); DBA2 群： 62697977( 满 )
DBA3 群： 62697850 DBA 超级群： 63306533;
聊天 群： 40132017
-- 加群需要在备注说明 Oracle 表空间和数据文件的关系，否则拒绝申请
