記錄處理過程
1. 重新 mount ACFS 發現有 error
[root@db1 ~]# sh /u01/app/grid/cfgtoollogs/asmca/scripts/acfs_script.sh
/bin/mkdir: cannot create directory ‘/ACFS’: File exists
mount.acfs: CLSU-00107: operating system function: open64; failed with error data: 2; at location: OOF_1
mount.acfs: CLSU-00101: operating system error message: No such file or directory
mount.acfs: CLSU-00104: additional error information: open64 (/dev/ofsctl)
mount.acfs: ACFS-00502: Failed to communicate with the ACFS driver. Verify the ACFS driver has been loaded.
ACFS /ACFS mounted successfully.2. 確認 ACFS module 是否有正常讀取
[root@db1 ~]# lsmod | grep oracle
oracleafd 214072 1 3. 重新讀取 ACFS module
[root@db1 ~]# acfsload start
ACFS-9391: Checking for existing ADVM/ACFS installation.
ACFS-9392: Validating ADVM/ACFS installation files for operating system.
ACFS-9393: Verifying ASM Administrator setup.
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9154: Loading 'oracleoks.ko' driver.
ACFS-9154: Loading 'oracleadvm.ko' driver.
ACFS-9154: Loading 'oracleacfs.ko' driver.
Use of uninitialized value $storage_mode in string ne at /u01/app/19.0.0/grid/lib/osds_acfslib.pm line 1175.
ACFS-9325: Driver OS kernel version = 3.10.0-862.el7.x86_64.
ACFS-9326: Driver build number = 190222.
ACFS-9212: Driver build version = 19.0.0.0.0 (19.3.0.0.0).
ACFS-9547: Driver available build number = 190222.
ACFS-9548: Driver available build version = 19.0.0.0.0 (19.3.0.0.0).
ACFS-9549: Kernel and command versions.
Kernel:
Build version: 19.0.0.0.0
Build full version: 19.3.0.0.0
Build hash: 9256567290
Bug numbers: NoTransactionInformation
Commands:
Build version: 19.0.0.0.0
Build full version: 19.3.0.0.0
Build hash: 9256567290
Bug numbers: NoTransactionInformation
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
ACFS-9156: Detecting control device '/dev/ofsctl'.
ACFS-9294: updating file /etc/sysconfig/oracledrivers.conf
ACFS-9322: completed4. 重新確認 ACFS module 是否有正常讀取
[root@db1 ~]# lsmod | grep oracle
oracleacfs 5581810 0
oracleadvm 1231385 0
oracleoks 721311 2 oracleacfs,oracleadvm
oracleafd 214072 1 5. 重新 mount ACFS ( 發現 volume /dev/asm/acfs-20 不存在 )
[root@db1 ~]# sh /u01/app/grid/cfgtoollogs/asmca/scripts/acfs_script.sh
/bin/mkdir: cannot create directory ‘/ACFS’: File exists
mount.acfs: CLSU-00107: operating system function: open64; failed with error data: 2; at location: OOF_1
mount.acfs: CLSU-00101: operating system error message: No such file or directory
mount.acfs: CLSU-00104: additional error information: open64 (/dev/asm/acfs-20)
mount.acfs: ACFS-02017: Failed to open volume /dev/asm/acfs-20. Verify the volume exists.
Error occurred when mounting /ACFS.6. 切換 grid 身份執行 asmcmd，確認 State 為 ENABLED
ASMCMD> volinfo --all
Diskgroup Name: DATA
Volume Name: ACFS
Volume Device: /dev/asm/acfs-20
State: DISABLED
Size (MB): 101376
Resize Unit (MB): 64
Redundancy: UNPROT
Stripe Columns: 8
Stripe Width (K): 1024
Usage: ACFS
Mountpath: /ACFS 7. 啟用 volume
ASMCMD> volinfo --show_diskgroup /dev/asm/acfs-20
data
ASMCMD> volenable -G data ACFS
ASMCMD> volinfo --all
Diskgroup Name: DATA
Volume Name: ACFS
Volume Device: /dev/asm/acfs-20
State: ENABLED
Size (MB): 101376
Resize Unit (MB): 64
Redundancy: UNPROT
Stripe Columns: 8
Stripe Width (K): 1024
Usage: ACFS
Mountpath: /ACFS 8. 重新 mount ACFS
[root@db1 ~]# /bin/mount -t acfs /dev/asm/acfs-20 /ACFS
＊＊＊＊＊ Re mount ACFS Hang ＊＊＊＊＊
a. 如果重新 mount ACFS 有卡住，請重啟 cluster has 服務
[root@db1 ~]# /u01/app/19.0.0/grid/bin/crsctl stop has -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'db1'
CRS-2673: Attempting to stop 'ora.orcl1.db' on 'db1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'db1'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'db1' succeeded
CRS-2677: Stop of 'ora.orcl1.db' on 'db1' succeeded
CRS-2673: Attempting to stop 'ora.evmd' on 'db1'
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'db1'
CRS-2677: Stop of 'ora.DATA.dg' on 'db1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'db1'
CRS-2677: Stop of 'ora.evmd' on 'db1' succeeded
CRS-2677: Stop of 'ora.asm' on 'db1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'db1'
CRS-2677: Stop of 'ora.cssd' on 'db1' succeeded
CRS-2673: Attempting to stop 'ora.driver.afd' on 'db1'
CRS-2677: Stop of 'ora.driver.afd' on 'db1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'db1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@db1 ~]# /u01/app/19.0.0/grid/bin/crsctl start has
CRS-4123: Oracle High Availability Services has been started.b. 並重新讀取 ACFS module
[root@db1 ~]# acfsload start
ACFS-9391: Checking for existing ADVM/ACFS installation.
ACFS-9392: Validating ADVM/ACFS installation files for operating system.
ACFS-9393: Verifying ASM Administrator setup.
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9325: Driver OS kernel version = 3.10.0-862.el7.x86_64.
ACFS-9326: Driver build number = 190222.
ACFS-9212: Driver build version = 19.0.0.0.0 (19.3.0.0.0).
ACFS-9547: Driver available build number = 190222.
ACFS-9548: Driver available build version = 19.0.0.0.0 (19.3.0.0.0).
ACFS-9549: Kernel and command versions.
Kernel:
Build version: 19.0.0.0.0
Build full version: 19.3.0.0.0
Build hash: 9256567290
Bug numbers: NoTransactionInformation
Commands:
Build version: 19.0.0.0.0
Build full version: 19.3.0.0.0
Build hash: 9256567290
Bug numbers: NoTransactionInformation
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
ACFS-9156: Detecting control device '/dev/ofsctl'.
ACFS-9294: updating file /etc/sysconfig/oracledrivers.conf
ACFS-9322: completedc. 並重新對 acfs 空間做 file check
[root@db1 ~]# /sbin/fsck -a -v -y -t acfs /dev/asm/acfs-20
fsck from util-linux 2.23.2
version = 19.0.0.0.0
*****************************
********** Pass: 1 **********
*****************************
Oracle ASM Cluster File System (ACFS) On-Disk Structure Version: 49.0
ACFS file system created at: Thu Dec 17 10:06:21 2020
checking primary file system
Files checked in primary file system: 5%
Files checked in primary file system: 100%
Checking if any files are orphaned...
0 orphans found
Checker completed with no errors.d. 重新 mount ACFS
[root@db1 ~]# /bin/mount -t acfs /dev/asm/acfs-20 /ACFS
＊＊＊＊＊ Something Path & Log ＊＊＊＊＊
Log:
$GRID_HOME/log/acfs
$GRID_BASE/cfsdata/`hostname`/acfs2021-10-05 17:55:51.313 (27040075.567) : asm_vdbg_+asm[9697] ADVMK-0017: The ASM instance terminated unexpectedly. All ADVM volumes will be taken offline.
2021-10-05 17:55:51.313 (27040075.567) : asmShutdown[9700] ACFSK-0029: Volume /dev/asm/acfs-20 is now offline
2021-10-05 17:55:51.313 (27040075.567) : asmShutdown[9700] ACFSK-0030: ASM Instance has Failed.
2021-10-05 17:55:51.313 (27040075.567) : asmShutdown[9700] SHUTDOWN Starting
2021-10-05 17:55:51.663 (27040075.917) : asmShutdown[9700] ADVMK-0019: Volume acfs-20 in disk group DATA was taken offline, all attempts to read or write to this volume will fail.
2021-10-05 17:55:51.663 (27040075.917) : asmShutdown[9700] AsmVolStateShutdown: DATA.acfs-20: is now shutdown.
2021-10-05 17:55:51.663 (27040075.917) : asmShutdown[9700] AsmDgStateShutdown: DATA: shutdown complete with status 0.
2021-10-05 17:55:51.666 (27040075.920) : asmShutdown[9700] AsmRootStateShutdown: ADVM shutdown done.
2021-10-05 17:55:51.666 (27040075.920) : asmShutdown[9700] SHUTDOWN completed, status 0
2021-10-05 17:55:54.808 (27040079.062) : umount[14135] unmounted ACFS file system F0 dev_path:/dev/asm/acfs-20 mountPoint:/ACFS
2021-10-05 17:55:55.396 (27040079.650) : umount[14135] VOLCLOSE Starting
2021-10-05 17:55:55.408 (27040079.662) : umount[14135] ADVMK-0006: Volume acfs-20 in diskgroup DATA disabled.
2021-10-05 17:55:55.408 (27040079.662) : umount[14135] asmDeleteVolume: DATA.acfs-20: deleted.
2021-10-05 17:55:55.408 (27040079.662) : umount[14135] VOLCLOSE completed, status 0
＊＊＊＊＊Solution＊＊＊＊＊
1. 新增開機在執行 oracle-ohasd.service 前先即執行 acfsload service
[root@db1 ~]# vim /etc/systemd/system/oracle-acfsload.service#
#
# Oracle ACFS Drivers
[Unit]
Description=Load ACFS Drivers during boot
After=syslog.target oracle-ohasd.service
[Service]
ExecStart=/u01/app/19.0.0/grid/bin/acfsload start -s >/dev/null 2>&1 </dev/null
Type=simple
[Install]
WantedBy=multi-user.target graphical.target2. Reload Daemon
[root@db1 ~]# systemctl daemon-reload
參考來源：
ORA-15477: cannot communicate with the volume driver (DBD ERROR: OCIStmtExecute)
ADVM Driver Not installed / ORA-15032 ORA-15489 delete waiting for last close
Mount ACFS after Reboot On Standalone GI
已轉向 blogger 記錄
https://slowlife-notes.blogspot.com