Configure the Oracle ASM Redhat 6.x in the VMware

1> Enabling Virtual Disk Sharing

(http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1034165)
To configure a shared virtual disk by setting the multi-writer flag:
  1. Create a virtual disk to be shared using any acceptable approach. For example, through the vSphere Client:

    1. Create a new disk for a virtual machine on a VMFS data store:


    2. Select Support clustering features such as Fault Tolerance. This creates the new virtual disk in the required format, eager zeroed thick.



      Note: ESXi 5.0 provides Thick Provision Lazy Zeroed, Thick Provision Eager Zeroed, and Thin Provision options under Disk Provisioning. Be sure to select Thick Provision Eager Zeroed. Also, note that the documentation may refer to this as Flat pre-initialized.

    3. Choose the appropriate Virtual Device Node setting, such as SCSI (1:0). Make a note of it, as it is required later.



      Note: There are command-line alternatives to perform the previous operation, but this instruction is beyond the scope of this article.
  2. Add the multi-writer flag to configure sharing. There are two ways to achieve this:

    Note: Do not change the SCSI Controller's SCSI Bus Sharing options. Keep the default selection, None.

    1. Power off the virtual machine.
    2. In the . vmx file that defines the virtual machine, add an entry similar to:

      scsiX:Y.sharing = "multi-writer"
      where X is the controller ID and Y is the disk ID on that controller. The setting screen of a virtual machine shows these values.



      Add this setting for each virtual disk that you want to share. For example, to share four disks, the configuration file entries look like this:

      scsi1:0.sharing = "multi-writer"
      scsi1:1.sharing = "multi-writer"
      scsi1:2.sharing = "multi-writer"
      scsi1:3.sharing = "multi-writer"
    3. Save the .vmx file and power on the virtual machine.
    4. In the vSphere Client, power off the virtual machine, go to Edit Settings > Options > Advanced > General > Configuration Parameters. Add rows for each of the shared disks and set their value to multi-writer.

      Example:


  3. Add this disk to another virtual machine:

    1. In the vSphere Client inventory, right-click the virtual machine and click Edit Settings.
    2. Click the Hardware tab and click Add.
    3. Select Hard Disk and click Next.
    4. Select Use an Existing Virtual Disk.

      Note: You do not need to create a disk because the disk is already created.
  4. Enable sharing on this virtual machine, using one of the approaches listed in step 2.

UDEV SCSI Rules Configuration for ASM in Redhat 6

Please make anote, Oracle supports ASMLib before the Redhat 6 (you have upgrade the ASNlib when you upgrade the kernel).  Please make a note for this article, the udevtest does not exist in the RHEL 6

(The following artical comes from: http://www.oracle-base.com/articles/linux/udev-scsi-rules-configuration-in-oracle-linux-5-and-6.php)

For Oracle Automatic Storage Manager (ASM) to use disks, it needs to be able to identify the devices consistently and for them to have the correct ownership and permissions. In Linux you can use ASMLib to manage these tasks, but it is seen as an additional layer of complexity and has never really gained any popularity. Instead, many people use the Linux device manager "udev" to perform these tasks. This article presents a brief overview of setting up udev rules with respect to disks for use with ASM in Oracle 11g. The examples are all done using Oracle Linux 5 and 6, so they will be consistent with RHEL and CentOS 5 and 6.

Background

Essentially, what udev does is apply rules defined in files in the "/etc/udev/rules.d" directory to the device nodes listed in the "/dev" directory. The rules can be defined in a variety of ways, but what we need to do is identify the device and say what we want udev to do with it.
In this case I know all my disk devices are named "/dev/sd?1", where the "?" represents a letter from a-d, so I can identify the devices of interest using the following rule parameters.
KERNEL=="sd?1", BUS=="scsi"
I want to tie each specific device to an alias, so it is always identified the same way, regardless of the device name Linux assigns it. So I need to be able to test each device that matches the previous pattern to see if it is the disk I am interested in. Each disk has a unique SCSI ID, so I can place a test into the rule, telling it how to perform the test, and the result it should return for a positive match. The following rule parameters explain how to test the device and what result constitutes a match in Oracle Linux 5.
PROGRAM=="/sbin/scsi_id -g -u -s /block/$parent", RESULT=="SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_"
The scsi_id command works a little differently in Oracle Linux 6, so for that the following test works better.
PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_"
Once we have identified the specific device of interest, we need to indicate what actions should be performed on it. The following parameters specify an alias, the ownership and the permissions for the device.
NAME="asm-disk1", OWNER="oracle", GROUP="dba", MODE="0660"
So the whole rule for each disk will look something like this in Oracle Linux 5.
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s /block/$parent", RESULT=="SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_", NAME="asm-disk1", OWNER="oracle", GROUP="dba", MODE="0660"
Or this in Oracle Linux 6.
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_", NAME="asm-disk1", OWNER="oracle", GROUP="dba", MODE="0660"
This means that the device pointing to the partition "sd*1" on the disk with the SCSI ID of "SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_" will always be called "/dev/asm-disk1", regardless of the letter "?" Linux assigns when the device is discovered. In addition, the device will have the correct ownership and permissions for ASM.
There are a number of wildcards and matching patterns that can be used if you don't want to write device-specific rules.
Now we know roughly what we are trying to achieve, we will look at each step necessary for setting up the disks for ASM to use.

Identify the Disks (/sbin/scsi_id)

We are going to write device-specific rules, so we need to be able to identify each device consistently, irrespective of the order in which Linux discovers it. To do this we are going to use the SCSI ID for each disk (not the partition), which we get using the scsi_id command. The "-s" option makes the paths relative to the "/sys" directory. For Oracle Linux 5, use the following command.
# /sbin/scsi_id -g -u -s /block/sdb
SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_
# /sbin/scsi_id -g -u -s /block/sdc
SATA_VBOX_HARDDISK_VB46dec7e0-192e8000_
# /sbin/scsi_id -g -u -s /block/sdd
SATA_VBOX_HARDDISK_VBce8c63bb-ac67a172_
# /sbin/scsi_id -g -u -s /block/sde
SATA_VBOX_HARDDISK_VB7437a3b7-95b199cd_
#
The "-s" is not available in Oracle Linux 6, so you must use the following syntax.
# /sbin/scsi_id -g -u -d /dev/sdb
SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_
# /sbin/scsi_id -g -u -d /dev/sdc
SATA_VBOX_HARDDISK_VB46dec7e0-192e8000_
# /sbin/scsi_id -g -u -d /dev/sdd
SATA_VBOX_HARDDISK_VBce8c63bb-ac67a172_
# /sbin/scsi_id -g -u -d /dev/sde
SATA_VBOX_HARDDISK_VB7437a3b7-95b199cd_
#

Make SCSI Devices Trusted

Add the following to the "/etc/scsi_id.config" file to configure SCSI devices as trusted. Create the file if it doesn't already exist.
options=-g

Create UDEV Rules File

Create the "/etc/udev/rules.d/99-oracle-asmdevices.rules" file.
# vi /etc/udev/rules.d/99-oracle-asmdevices.rules
The file should contain the following lines for Oracle Linux 5. The PROGRAM parameter must match the command you used to retrieve the SCSI ID, and the RESULT parameter must match the value returned from your disks.
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s /block/$parent", RESULT=="SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_", NAME="asm-disk1", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s /block/$parent", RESULT=="SATA_VBOX_HARDDISK_VB46dec7e0-192e8000_", NAME="asm-disk2", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s /block/$parent", RESULT=="SATA_VBOX_HARDDISK_VBce8c63bb-ac67a172_", NAME="asm-disk3", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s /block/$parent", RESULT=="SATA_VBOX_HARDDISK_VB7437a3b7-95b199cd_", NAME="asm-disk4", OWNER="oracle", GROUP="dba", MODE="0660"
The equivalent for Oracle Linux 6 is shown below.
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_", NAME="asm-disk1", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="SATA_VBOX_HARDDISK_VB46dec7e0-192e8000_", NAME="asm-disk2", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="SATA_VBOX_HARDDISK_VBce8c63bb-ac67a172_", NAME="asm-disk3", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="SATA_VBOX_HARDDISK_VB7437a3b7-95b199cd_", NAME="asm-disk4", OWNER="oracle", GROUP="dba", MODE="0660"

Load Updated Block Device Partitions (/sbin/partprobe)

Load updated block device partition tables.
# /sbin/partprobe /dev/sdb1
# /sbin/partprobe /dev/sdc1
# /sbin/partprobe /dev/sdd1
# /sbin/partprobe /dev/sde1

Test Rules (udevtest)

Test the rules are working as expected.
# udevtest /block/sdb/sdb1
# udevtest /block/sdc/sdc1
# udevtest /block/sdd/sdd1
# udevtest /block/sde/sde1
The output from the first disk should look something like this.
# udevtest /block/sdb/sdb1
main: looking at device '/block/sdb/sdb1' from subsystem 'block'
udev_rules_get_name: add symlink 'disk/by-id/scsi-SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3-part1'
udev_rules_get_name: add symlink 'disk/by-path/pci-0000:00:0d.0-scsi-1:0:0:0-part1'
run_program: '/lib/udev/vol_id --export /dev/.tmp-8-17'
run_program: '/lib/udev/vol_id' returned with status 4
run_program: '/sbin/scsi_id -g -u -s /block/sdb/sdb1'
run_program: '/sbin/scsi_id' (stdout) 'SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3_'
run_program: '/sbin/scsi_id' returned with status 0
udev_rules_get_name: rule applied, 'sdb1' becomes 'asm-disk1'
udev_device_event: device '/block/sdb/sdb1' already in database, validate currently present symlinks
udev_node_add: creating device node '/dev/asm-disk1', major = '8', minor = '17', mode = '0660', uid = '1100', gid = '1200'
udev_node_add: creating symlink '/dev/disk/by-id/scsi-SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3-part1' to '../../asm-disk1'
udev_node_add: creating symlink '/dev/disk/by-path/pci-0000:00:0d.0-scsi-1:0:0:0-part1' to '../../asm-disk1'
main: run: 'socket:/org/kernel/dm/multipath_event'
main: run: 'socket:/org/kernel/udev/monitor'
main: run: '/lib/udev/udev_run_devd'
main: run: 'socket:/org/freedesktop/hal/udev_event'
main: run: '/sbin/pam_console_apply /dev/asm-disk1 /dev/disk/by-id/scsi-SATA_VBOX_HARDDISK_VBd306dbe0-df3367e3-part1 /dev/disk/by-path/pci-0000:00:0d.0-scsi-1:0:0:0-part1'
#

Restart UDEV Service

Restart the UDEV service.
# /sbin/udevcontrol reload_rules
# /sbin/start_udev

Check Ownership and Permissions

Check the disks are now available with the "asm-disk*" alias and the correct ownership and permissions.
# cd /dev
# ls -al asm-disk*
brw-rw---- 1 oracle dba 8, 17 Apr  8 22:47 asm-disk1
brw-rw---- 1 oracle dba 8, 33 Apr  8 22:47 asm-disk2
brw-rw---- 1 oracle dba 8, 49 Apr  8 22:47 asm-disk3
brw-rw---- 1 oracle dba 8, 65 Apr  8 22:47 asm-disk4
#
So the ASM_DISKSTRING initialization parameter in the ASM instance can be set to '/dev/asm-disk*' to identify the ASM disks.
For more information see:

Comments

Post a Comment

Popular posts from this blog

VIOS TIPs

HMC image locates: /extra/viosimages/2.2.3.4 To install VIO server from HMC: installios To erase the previous setting: installios -u hscroot@P7504HMC:~> cat /etc/nimol.conf # nimol.conf file NIMOL_SERVER 9.22.217.30 ORIGINAL_FILES_DIR /var/tmp/nimol_original RPCBIND_STARTED yes NFS_STARTED yes REMOTE_ACCESS_METHOD /usr/bin/rsh NIMOL_TFTPBOOT /tftpboot TFTP_ENABLED yes REMOTE_SYSLOG_ENABLED yes HMC_SYSLOG_CONF_MODIFIED yes NIMOL_SYSLOG_FACILITY local2 NIMOL_SUBNET 9.22.217.0 NIMOL_BOOTREPLYD yes LABEL default1 /extra/default1 processing hscroot@P7504HMC:~>  installios -F -e -R default1 Logging session output to /tmp/installios.24173.log. nimol_config WARNING: error from command /bin/awk nimol_config MESSAGE: Executed /usr/sbin/exportfs -ua. nimol_config MESSAGE: Executed /usr/sbin/exportfs -a. nimol_config MESSAGE: Removed /tftpboot/default1.chrp.mp.ent. nimol_config MESSAGE: Removed /extra/default1. nimol_config MESSAGE: Removed "LABEL default
Read more

Configure Solaris 10 LDOM on Solaris 11.4

root@comglv1:~# ldm add-memory 16G htcsun1 root@comglv1: # zfs destroy zones cannot destroy 'zones': operation does not apply to pools use 'zfs destroy -r zones' to destroy all datasets in the pool use 'zpool destroy zones' to destroy the pool itself root@comglv1: # zfs destroy -r zones root@comglv1: # zpool create LDOM mirror c0t5000CCA03C0E89D8d0s0 c0t5000CCA03C2F3BB4d0s0 root@comglv1: # zfs create LDOM/htcsun1 root@comglv1: # zfs create -V 150g LDOM/htcsun1/disk0 root@comglv1: # ldm rm-vdisk vdisk0 htcsun1 root@comglv1: # ldm rm-vdsdev htcsun1-vd0@primary-vds0 root@comglv1: # ldm rm-vdsdev  vol0@primary-vds0 root@comglv1: # ldm rm-vdsdev iso@primary-vds0 root@comglv1: # add-vdsdev options=ro sol-10-u11-ga-sparc-dvd.iso htcsun1-iso@primary-vds0 root@comglv1: # ldm add-vdsdev /dev/zvol/dsk/LDOM/htcsun1/disk0 htcsun1-vd0@primary-vds0 root@comglv1: # ldm add-vdisk vdisk0 htcsun1-vd0@primary-vds0 htcsun1
Read more

Change P410i from HBA mode to Raid mdoe

fs0:\> dir Directory of: fs0:\   03/07/13  10:44a            1,492,736  0126.bin   03/07/13  10:44a                  953  fwupdate.nsh   03/07/13  10:44a            4,194,304  sandman_578b0.bin   03/07/13  10:44a              452,096  saupdate.efi   12/03/12  10:12a               12,792  .psafe3   03/07/13  10:38a <DIR>          4,096  .Trash-500           5 File(s)   6,152,881 bytes           1 Dir(s) fs0:\> fwupdate.nsh fwupdate.nsh> echo -off ******************************************************************************** ****                                                                        **** ****                        IO CARD FIRMWARE UPDATE                         **** ****            (c) Copyright Hewlett-Packard Company, 2001-2012            **** ****                          All rights reserved.                          **** ****                                                                        **** ****                                         
Read more