Wednesday, February 25, 2009

OID connection not available. OID server may be down

Introduction


In addition to the SSO and MetaData Repository components, we are also configuring the OID server to be highly available, thus enabling all the 3 components Viz. SSO webserver, MetaData Repository, OID server for High Availability. As part of the OID server high availability setup, I have installed a new Infrastructure node oid_serv2 with the OID and DIP components along with 'High Availability and Replication' in the installation option list.

Problem


A good way to test whether the new node serves Single Sign-On (SSO) requests is to shutdown the other existing OID server oid_serv1 in the Identity Management (IM) cluster. In addition, the OID server name has to be changed from oid_serv1 to oid_serv2 in the MetaData Repository database by running $ORACLE_HOME/sso/admin/plsql/sso/ssooconf.sql on the SSO server. However, I started recieving 'Error:Internal Server Error. Please try the operation later'.

Upon further investigation, I found the following errors in the $ORACLE_HOME/sso/log/ssoServer.log

Wed Feb 25 15:44:14 CST 2009 [ERROR] AJPRequestHandler-ApplicationServerThread-7 DirContextPool: OID connection not available. OID Server, oid_serv1.mycompany.com:636 may be down ...

Wed Feb 25 15:44:14 CST 2009 [ERROR] AJPRequestHandler-ApplicationServerThread-7 Communication Exception received. Cleaning up the stale connectionjavax.naming.CommunicationException: Could not get the connection. OID Server may be down

Solution


Changing the OID server name in $ORACLE_HOME/sso/admin/plsql/sso/ssooconf.sql alone does not seem to be enough for the SSO webserver to recognize the new OID server. I searched the configuration files in the SSO ORACLE_HOME and found a couple of additional references to the existing OID server, needing replacement with the newly installed OID server oid_serv2.

On the SSO ORACLE_HOME, change the following files, replacing oid_serv1 with oid_serv2 and then bouncing the processes using the opmnctl command.

  • $ORACLE_HOME/config/ias.properties (OIDhost)
  • $ORACLE_HOME/ldap/admin/ldap.ora (DIRECTORY_SERVERS)

Conclusion


The SSO Administration Guide mentions that running ssooconf.sql alone is enough to recognize the new OID server. However, as mentioned above, the SSO server does not seem to read the change from the database, but instead from its configuration files to get this working.

Friday, February 20, 2009

ONS Port and CRS Installation

Introduction

Most installations performed by the Oracle Universal Installer (OUI) selects port numbers only that are freely available on the host. However, one service that always fixes its port number during installation is the ONS service (Oracle Notification Service). As part of setting up the High Availability features for the Single Sign-On system, I have installed Oracle Cluster Ready Services (CRS) on Linux platform to convert the existing Metadata repository into a highly available database cluster. The installation completed successfully, however configuring the Virtual IPs was a problem (vipca). The ons service on the Node2 refuses to start.
Upon further investigation, I found the OUI assigned 6200 as the ONS port ( $CRS_ORACLE_HOME/opmn/conf/ons.config ), which is already in use by a different Oracle software on the same host.
Solution
  • Login to the cluster node2 where ONS service did not start
  • srvctl stop nodeapps -n node2
  • $CRS_ORACLE_HOME/bin/racgons.bin remove_config node2:6200
  • $CRS_ORACLE_HOME/bin/racgons.bin add_config node2:6299 ( non-default port )
  • Update $CRS_ORACLE_HOME/opmn/conf/ons.config and replace the remoteport=6200 with remoteport=6299
  • Run $CRS_ORACLE_HOME/bin/vipca again (It should complete successfully this time)
  • $CRS_ORACLE_HOME/crs_stat -t ( Verify ONS service is ONLINE )
Conclusion
As a best practice, one should install new Oracle software with all existing applications up and running so that the OUI only chooses port numbers that are not in use. However, it does not seem to be the case for the ONS port, because the OUI always sticks to the 6200. As a good practice, one can avoid problems during future Oracle installations on a given host by changing the ONS port from the default 6200 to a non-default value in the current installation.