Friday, June 26, 2009

ASM Disk Controller numbers can get changed if the SAN Switch is replaced

Introduction


We have setup our Oracle E-Business Suite database to be highly available by using Oracle 10g RAC technology using the ASM feature. We use EMC Symmetrix for storage and a Brocade device for the SAN switch. Our SAN fabric was recently upgraded and among many changes, the legacy Brocade SAN switches were replaced by Cisco switches.


Problem


Due to this change, we rebooted both the Oracle RAC database nodes. But the database would not start because the ASM diskgroup could not be mounted. We have hardcoded the asm_diskstring parameter to a specific set of disk controller numbers. While mounting a diskgroup, the ASM instance will discover only those devices that match the asm_diskgroup parameter. Upon further investigation, we found out that the disk controller numbers got changed after the new switch installation.



Note the changed disk controller numbers in the below image



Solution


We use EMC Symmetrix as our SAN storage solution. During the initial ASM conversion project, we managed to save the syminq numbers of all our ASM devices. Because the syminq numbers do NOT change even upon a switch replacement, we could use the same syminq numbers to find out the new disk controller numbers. We then updated the asm_diskgroup initialization parameter with these new controller numbers and then the ASM diskgroup was successfully mounted and the database started.

Sunday, June 7, 2009

How to avoid username conflict in OID during AD synchronization

Introduction

Our master source of truth for authenticating Single Sign On (SSO) users for access to the E-Business Suite is the Active Directory (AD). We periodically syncrhonize data from AD to the Oracle Internet Directory (OID) using the syncrhonization profiles.

Because Oracle SSO can work with only OID, we cannot afford any failures/mismatches between AD and OID and hence we do not let the syncrhonization to continue if an error is reported. The 'Continue after error' is set to FALSE.

One common problem in a directory is managing usernames for people with identical names. For example, an organization can have two employees with the same name viz. John Smith and John Smith respectively. The AD administrator creates these accounts as JSMITH and JSMITH1 respectively because the username has to be unique across the organization. During initial propagation to OID, these accounts will have two seperate entries in OID as the same JSMITH and JSMITH1 respectively. When JSMITH leaves the organization, the other John Smith can request for a change in his username from JSMITH1 to JSMITH (JSMITH looks good compared to JSMITH1).


Problem

The update of JSMITH1 to JSMITH generates a new change number in AD and the next syncrhonization cycle will attempt to make the corresponding change in OID too. We had such an update in AD yesterday and OID failed to process the change. The following is the error that is reported in the syncrhonization profile logfile.



Exception Doing ModRDN operation : javax.naming.NameAlreadyBoundException: [LDAP: error code 68 - Entry Already Exists]; remaining name 'cn=jsmith1,ou=us,dc=mycompany,dc=com'
Ignore modrdn.

Solution

At the time of processing the update, OID had the following two entries

'cn=jsmith1,ou=us,dc=mycompany,dc=com'
'cn=jsmith,ou=us,dc=mycompany,dc=com'

AD had only one entry JSMITH (after the username change by the administrator).

'cn=jsmith,ou=us,dc=mycompany,dc=com'

Because OID has an jsmith entry already, the update of JSMITH1 to JSMITH fails with the [LDAP: error code 68 - Entry Already Exists] error.

I solved the problem by deleting both the entries jsmith and jsmith1 in OID and making a fake update in AD (removed the telephone number of jsmith). The next syncrhonization job picked up the telephone number change in AD and created a new jsmith entry in OID. Once the OID processing has become successful, I corrected the telephone number in AD.