Tuesday, August 18, 2015

Apache Spark Training in DBA University

We have included Apache Spark topics as part of our Big Data with Hadoop training course. Spark is the most active Big Data open source project and has become a top level Apache project in Feb 2014. In our course, you get an opportunity to process large real world JSON data sets using Apache Spark both on a single node pseudo-distributed Hadoop cluster as well as three node Hadoop cluster in the cloud.

The Apache Spark topics along with lab work are below

1). How to use the Spark API in Python.
2). How to process large JSON data sets using the Spark SQL context in the spark shell. (Scala-based shell)
3). How to obtain data from external databases using the JDBC built-in data source library in Spark.
4) How to add the Spark service in a three node Hadoop cluster. 


If you are interested, please register at http://dbauniversity.webex.com/

The DBA University difference

1) All faculty based in USA and training through video enabled web conferencing software.
2) Custom written ebooks (About 300 pages training materials per course).
3) Cloud based remote lab server access for each student.
3) Our faculty and student video portal - http://video.dbauniversity.com
4) Taught using interactive white board technology.

Sunday, August 2, 2015

Oracle RMAN Active Database Duplication example

Introduction

Oracle RMAN Active Database Duplication is a new feature introduced in Oracle 11g R1. This option helps perform database copies without the need for relying on the source database backup. Oracle RMAN makes a copy of the source database (active database) at run time without the need of using its backup for the duplicate exercise. This option is very attractive when one is tasked with a “test database” to “test database” copy exercise (cloning exercise) as source test databases do not usually have backup jobs scheduled. Though it can be used for “production database” to “test database” copies as well, this method is not recommended because the RMAN duplicate command channels are launched on the source database host, thereby causing high load on the production database server.

Because the active database duplicate option does not rely on a backup, it cannot be used to perform Database Point-in-Time Recovery (“DBPITR”). Additionally, the source database must be in archive log mode if the duplication has to be performed without causing a downtime to the source database. Otherwise, the source database must be shutdown immediate and then startup mount.

Prerequisites

1) The source database must be in archivelog mode if one wants to duplicate it without shutting it down

2) If source database were a test database and enabling archive log mode is not necessary, then shutdown immediate and startup mount the source database.

3) When RMAN is connected to the auxiliary instance (destination to-be-duplicated instance), it must be provided with a net service name of it. This requirement applies even if the auxiliary instance is on the local host. Therefore, please add the tns entry  of the destination to-be-duplicated instance in the tnsnames.ora of the source database.

4) The source database and auxiliary instances must use the same SYS user password, which means that both instances must have password files.

5) The dynamic service registration cannot be used for the destination database while the RMAN active duplicate command is being executed. Hence only static service registration will work for the RMAN active duplicate to be successful.

6) Firewall access must be opened from source database server to destination database server  on the destination database's listener port number. (if they are on two different servers)

Assumptions

1) sys password is sys on both source and destination (to-be-duplicated) databases.
2) SRCDB and DESTDB are the names of the respective source and destination databases.
3) SRCDB has an spfile (server parameter file).

Prepare destination database (to-be-duplicated database)

1)  Login to oracle@target database server.

2) Create new directories on this server that exactly matches the source database's dignostic_dest folder and fast recovery area folder as the RMAN active database duplicate command in the next section specifically looks for the exact directory name on destination server too.

3)  Start the listener process.
4)   cd  $ORACLE_HOME/dbs
5)   rm  -f  orapwDESTDB     (Let us remove any existing password file. Replace DESTDB with the actual destination database name).

6)  orapwd  password=sys   file=orapwDESTDB     (Replace DESTDB with the actual destination database name).

7) cd $ORACLE_HOME/dbs

8) Remove  any spfile or pfile for the destination database (if exists)

9) Create a new initDESTDB.ora with just one parameter. ( (Replace DESTDB with the actual destination database name).

db_name = DESTDB

10) sqlplus   ‘/   as   sysdba’
11) startup   nomount;
12) exit;

Run the active database duplication command on source database server

1) Login to oracle@source database server
2) cd  $ORACLE_HOME/dbs

3) orapwd  password=sys   file=orapwSRCDB (If it does not exist already)
4) sqlplus   ‘/   as  sysdba’
5) alter   user   sys  identified  by  sys;
6) exit;

Note:  Please replace  DESTDB with the actual destination database name in the active duplicate command below.

7) tnsping   DESTDB   (make sure that the destination database listener can be accessible through SQL*Net from the source database server.  If not add an entry in tnsnames.ora and also make sure its listener is started on the destination server).
8) rman
9) connect   target   sys/sys@SRCDB;
10) connect   auxiliary   sys/sys@DESTDB;    
11)

run
{
allocate channel d1 type disk;
allocate channel d2 type disk;
allocate channel d3 type disk;
allocate channel d4 type disk;
allocate auxiliary channel a1 type disk;
allocate auxiliary channel a2 type disk;
set newname for database to '/u01/data/DESTDB/%b';
set newname for tempfile 1 to '/u01/data/DESTDB/temp01.tmp';
DUPLICATE TARGET DATABASE
  to DESTDB
  FROM ACTIVE DATABASE
  PASSWORD FILE
  SPFILE
  PARAMETER_VALUE_CONVERT 'SRCDB','DESTDB'
  SET CONTROL_FILES='/u01/data/DESTDB/c1.ctl'
  LOGFILE
  GROUP   1  ('/u01/data/DESTDB/redo1.log')   SIZE    50M,
  GROUP   2  ('/u01/data/DESTDB/redo2.log')   SIZE    50M ;
}


12) exit;