Saturday, October 1, 2016

To Monitor TOP Latches Statistics & Wait Information

Display Latch Statistics
WITH latch AS (
    SELECT name,
           ROUND(gets * 100 / SUM(gets) OVER (), 2) pct_of_gets,
           ROUND(misses * 100 / SUM(misses) OVER (), 2) pct_of_misses,
           ROUND(sleeps * 100 / SUM(sleeps) OVER (), 2) pct_of_sleeps,
           ROUND(wait_time * 100 / SUM(wait_time) OVER (), 2)  pct_of_wait_time
      FROM v$latch)
SELECT * FROM latch
WHERE pct_of_wait_time > .1 OR pct_of_sleeps > .1
ORDER BY pct_of_wait_time DESC;
Display Latch Statistics for Active Session History
WITH ash_query AS ( SELECT event, program, h.module, h.action,   object_name,
            SUM(time_waited)/1000 time_ms, COUNT( * ) waits, username, sql_text,
            RANK() OVER (ORDER BY SUM(time_waited) DESC) AS time_rank,
            ROUND(SUM(time_waited) * 100 / SUM(SUM(time_waited)) OVER (), 2)  pct_of_time
      FROM  v$active_session_history h
      JOIN  dba_users u  USING (user_id)
      LEFT OUTER JOIN dba_objects o
           ON (o.object_id = h.current_obj#)
      LEFT OUTER JOIN v$sql s USING (sql_id)
     WHERE event LIKE '%latch%' or event like '%mutex%'
     GROUP BY event,program, h.module, h.action,
         object_name,  sql_text, username)
SELECT event,module, username,  object_name, time_ms,pct_of_time, sql_text
FROM ash_query
WHERE time_rank < 11
ORDER BY time_rank;
Display information about Top Latches
SELECT l.latch#,  l.name, l.gets, l.misses, l.sleeps,
l.immediate_gets, l.immediate_misses, l.spin_gets
FROM   v$latch l
WHERE  l.misses > 0
ORDER BY l.misses DESC;
Display Latches wait information
SELECT  latch#,  name, gets, misses, sleeps
FROM    v$latch
WHERE   sleeps>0
ORDER   BY misses, sleeps;
Display Latche Children information
select  addr, latch#, gets, misses,sleeps
FROM    v$latch_children
WHERE   sleeps>0 and  latch# in (select unique p2                
from  v$session_wait                
where event ='latch free')
ORDER   BY sleeps;
Display the information of Latches has been waiting for
select * from    v$latchname
WHERE latch# in (select unique p2    
from  v$session_wait                
where event ='latch free');

Frequently used OS (Linux/Solaris/AIX) Command for DBA

As a DBA you need to use frequent OS command or alt least how to query the OS and its hardware. Usually we do it before fresh install upgrade, migrate of database/operating system. Here is some of the useful frequently used day to day OS command for DBA.
To find and delete files older than N number of days:
find . -name ‘*.*’ -mtime +[N in days] -exec rm {} \;
Example : find . -mtime +5 -exec rm {} \;
The above command is specially useful to delete log, trace, tmp file
To list files modified in last N days:
find . -mtime - -exec ls -lt {} \;
Example: find . -mtime +3 -exec ls -lt {} \;1
The above command will find files modified in last 3 days
To sort files based on Size of file:
ls -l | sort -nk 5 | more
useful to find large files in log directory to delete in case disk is full
To find files changed in last N days :
find -mtime -N –print
Example: find -mtime -2 -print
To find CPU & Memory detail of linux:
cat /proc/cpuinfo (CPU)
cat /proc/meminfo (Memory)
Linux: cat /proc/cpuinfo|grep processor|wc -l
HP: ioscan -fkn -C processor|tail +3|wc -l
Solaris: psrinfo -v|grep "Status of processor"|wc –l
psrinfo -v|grep "Status of processor"|wc –l
lscfg -vs|grep proc | wc -l
To find if Operating system in 32 bit or 64 bit:
ON Linux: uname -m
On 64-bit platform, you will get: x86_64 and on 32-bit patform , you will get:i686
On HP: getconf KERNEL_BITS
On Solaris: /usr/bin/isainfo –kv
On 64-bit patform, you will get: 64-bit sparcv9 kernel modules and on 32-bit, you will get: 32-bit sparc kernel modules. For solaris you can use directly: isainfo -v
If you see out put like: "32-bit sparc applications" that means your O.S. is only 32 bit but if you see output like "64-bit sparcv9 applications" that means youe OS is 64 bit & can support both 32 & 64 bit applications.
To find if any service is listening on particular port or not:
netstat -an | grep {port no}
Example: netstat -an | grep 1523
To find Process ID (PID) associated with any port:
This command is useful if any service is running on a particular port (389, 1521..) and that is run away process which you wish to terminate using kill command
lsof | grep {port no.} (lsof should be installed and in path)
How to kill all similar processes with single command:
ps -ef | grep opmn |grep -v grep | awk ‘{print $2}’ |xargs -i kill -9 {}
Locating Files under a particular directory:
find . -print |grep -i test.sql
To remove a specific column of output from a UNIX command:
For example to determine the UNIX process Ids for all Oracle processes on server (second column)
ps -ef |grep -i oracle |awk '{ print $2 }'
Changing the standard prompt for Oracle Users:
Edit the .profile for the oracle user
PS1="`hostname`*$ORACLE_SID:$PWD>"
Display top 10 CPU consumers using the ps command:
/usr/ucb/ps auxgw | head -11
Show number of active Oracle dedicated connection users for a particular ORACLE_SID
ps -ef | grep $ORACLE_SID|grep -v grep|grep -v ora_|wc -l
Display the number of CPU’s in Solaris:
psrinfo -v | grep "Status of processor"|wc -l
Display the number of CPU’s in AIX:
lsdev -C | grep Process|wc -l
Display RAM Memory size on Solaris:
prtconf |grep -i mem
Display RAM memory size on AIX:
First determine name of memory device: lsdev -C |grep mem
then assuming the name of the memory device is ‘mem0’ then the command is: lsattr -El mem0
Swap space allocation and usage:
Solaris : swap -s or swap -l
Aix : lsps -a
Total number of semaphores held by all instances on server:
ipcs -as | awk '{sum += $9} END {print sum}'
View allocated RAM memory segments:
ipcs -pmb
Manually deallocate shared memeory segments:
ipcrm -m ''
Show mount points for a disk in AIX:
lspv -l hdisk13
Display occupied space (in KB) for a file or collection of files in a directory or sub-directory:
du -ks * | sort -n| tail
Display total file space in a directory:
du -ks .
Cleanup any unwanted trace files more than seven days old:
find . *.trc -mtime +7 -exec rm {} \;
Locate Oracle files that contain certain strings:
find . -print | xargs grep rollback
Locate recently created UNIX files:
find . -mtime -1 -print
Finding large files on the server:
find . -size +102400 -print
Crontab Use:
To submit a task every Tuesday (day 2) at 2:45PM
45 14 2 * * /opt/oracle/scripts/tr_listener.sh > /dev/null 2>&1
To submit a task to run every 15 minutes on weekdays (days 1-5)
15,30,45 * 1-5 * * /opt/oracle/scripts/tr_listener.sh > /dev/null 2>&1
To submit a task to run every hour at 15 minutes past the hour on weekends (days 6 and 0)
15 * 0,6 * * opt/oracle/scripts/tr_listener.sh > /dev/null 2>&1

How to recover or re-create temporary tablespace in 10g

In database you may discover that your temporary tablespace is deleted from OS or it might get corrupted. In order to get it back you might think about recover it. The recovery process is simply recover the temporary file from backup and roll the file forward using archived log files if you are in archivelog mode.
Another solution is simply drop the temporary tablespace and then re-create a new one and assign new one as a default tablespace to the database users.
SQL> Select File_Name, File_id, Tablespace_name from DBA_Temp_Files;
FILE_NAME                     FILE_ID TABLESPACE_NAME
----------------------------- ------- ----------------
D:\ORACLE\ORADATA\SADHAN\TEMP02.DBF 1     TEMPMake the affected temporary files offline and create new TEMP tablespace and assign it default temporary tablespace:SQL> Alter database tempfile 1 offline;
SQL> Create temporary tablespace TEMP1 tempfile 'D:\ORACLE\ORADATA\SADHAN\TEMP02.DBF' size 1500M;
SQL> alter database default temporary tablespace TEMP1;
Check the users who are not pointed to default temp tablespace and assign them externally then finally drop the old tablespace.
SQL> Select temporary_tablespace, username from dba_users where temporary_tablespace<>'TEMP';
TEMPORARY_TABLESPACE       USERNAME
--------------------       ---------
TEMP                       SH1
TEMP                       SH2

SQL>alter user SH1 temporary tablespace TEMP1;
SQL>alter user SH2 temporary tablespace TEMP1;
SQL>Drop tablespace temp;

How to Migrate SQL Profiles from One database to Another database

You can migrate SQL profile using export and import from one database to another database just like stored outline. Prior to oracle 10g you can migrate SQL profiles with the dbms_sqltune.import_sql_profile procedure where as in oracle 10g release 2 and beyond using dbms_sqltunepackage. In both case you have to create a staging table on the source database and populate that staging table with the relevant data. Below is the step to migrate SQL profile in 10g release 2.
Step1. Create the staging table to store SQL Profiles in source databaseSQL> sys/oracle@sadhan as sysdba
SQL> BEGIN
DBMS_SQLTUNE.CREATE_STGTAB_SQLPROF
(table_name => ‘SQL_PROFILES’,schema_name=>’HRMS’);
     END;
/
PL/SQL procedure successfully completed.
Step2. Now Copy SQL profiles from SYS to the Staging tableSQL> BEGIN
DBMS_SQLTUNE.PACK_STGTAB_SQLPROF
(profile_category => ‘%’,
staging_table_name => ‘SQL_PROFILES’,
staging_schema_owner=>’HRMS’);
END;
/
PL/SQL procedure successfully completed.
Note: As you need to copy all SQL profiles on my database ‘%’ value for profile_category was the best option.
Step3. Export the staging table at sourceSQL> select count(*) from HRMS.sql_profiles;
COUNT(*)
-------------
3
expdp system/***** dumpfile=expdp_sql_profiles.dmp TABLES=HRMS.SQL_PROFILES DIRECTORY=DPUMP
Step4. Restore the database with the backup taken before all SQL profiles were generated and import the staging table at target database.impdp system/***** dumpfile=expdp_sql_profiles.dmp TABLES=HRMS.SQL_PROFILES DIRECTORY=DPUMP TABLE_EXISTS_ACTION=REPLACE
Note: Do not forget to create staging table on destination database. Use replace = TRUE if you need to have same SQL_Profiles on both the database.
Step5. Finally Unpack the SQL profiles from the staging table on destination database.SQL> BEGIN
DBMS_SQLTUNE.UNPACK_STGTAB_SQLPROF
(staging_table_name => ‘SQL_PROFILES’,
staging_schema_owner=>’HRMS’, replace=>FALSE);
END;
/
PL/SQL procedure successfully completed.

Getting description of all tables in the database

set feedback off
set verify off
set echo off
prompt This script is to get description of all tables excluding some schemas.
set termout off
set pages 500
set heading off
set linesize 150
spool table_definition.sql
select 'spool table_def_output.log;' from dual;
select 'DESC ' || A.OWNER ||'.'||A.TABLE_NAME DESC_SCRIPT from dba_tables a where
OWNER NOT IN ('SYS','SYSTEM','SYSMAN','MGMT_VIEW','TSMSYS','WMSYS','EXP_DBA','OUTLN','ORACLE_OCM','DBSNMP', 'MDSYS','EXFSYS', 'CTXSYS', 'OLAPSYS');
select 'exit;' from dual;
set termout on
prompt Running Script now to get description
set termout off
@table_definition.sql;
exit

Compare & Differentiate Statspack & AWR

The AWR report is a great tool for monitoring day by day database activities for DBA. The use of Statspack/AWR report helps the DBA quickly identify the possible cause or database load. The AWR report mainly contains the following sections.
·        Database instance details
·        Database Memory statistics.
·        Top 5 wait events.
·        Top SQL’s order by execution time and elapsed time.
·        SQL’s order by physical reads.
·        SQL’s order by buffer reads and gets.
·        Tablespace information and hit ratio on the table spaces.
·        Initialization Parameter’s
The purpose of these two tools is the same but why DBA prefer to use AWR than Statspack.
1.      Statspack needs to be installed manually while AWR is installed configured and managed by default in a standard manner.
2.      The AWR repository holds all the statistics available in STATSPACK as well as some additional statistics which are not.
3.      Statspack analysis is complex and needs a skilled eyes and an adequate level of experience to detect problems. AWR along with ADDM runs continuously and generates alerts or perform analysis automatically.
4.      Statspack impose a reasonable load during snaps collection where as AWR collection occurs continuously (offloaded to selected Background process) allowing for smoother, less perceptible and less disruptive progress. 
5.      Statspack is not accessible via GUI such as OEM for viewing or management, where as AWR is accessible both via the OEM as well as SQL & PL/SQL for viewing or management the report.
6.      Statspack gather information from V$SQL for high load SQL based on certain criteria such as number of logical and physical I/O per SQL stored SQL statements where as AWR recognize high load SQL as it occurs rather than collecting high-load SQL from V$SQL (which may not be accurate at this time as it captured before, may be outside of the snapshot period).
7.      Statspack does not store the active session history (ASH) statistics which are available in AWR dba_hist_active_sess_history view.
8.      The Statspack does not store history for new metric statistics introduced in oracle 10g such as the dba_hist_sysmetric_history anddba_hist_sysmetric_summary. The AWR also contains views such as dba_hist_service_stat, dba_hist_service_wait_class dba_hist_service_name views to store history for cumulative performance statistics.
9.      STATSPACK data is stored in the PERFSTAT schema in any designated tablespace, while AWR data is stored in the SYS schema in the new SYSAUX tablespace in 10g.
10.  Statspack snapshot purges manually where as AWR snapshots are purged automatically by MMON (default, keeping 7 days snapshots available you can modify it). If AWR detects that the SYSAUX tablespace is in danger of running out of space, it will free space in SYSAUX by automatically deleting the oldest set of snapshots.

ORA-01102: cannot mount database in EXCLUSIVE mode

ORA-01102: cannot mount database in EXCLUSIVE mode

While starting the oracle database instance it fails with following error.
Connected to an idle instance.
ORACLE instance started.
Total System Global Area 877574740 bytes
Fixed Size 651436 bytes
Variable Size 502653184 bytes
Database Buffers 263840000 bytes
Redo Buffers 10629120 bytes
ORA-01102: cannot mount database in EXCLUSIVE mode
Cause:
This ORA-01102 error indicates an instance tried to mount the database in exclusive mode, but some other instance has already mounted the database in exclusive or parallel mode. By default a database is started in EXCLUSIVE mode. Sometimes it cause due to setting incorrect ORACLE_SID environment specially for newly created database check you have set the correct CASE of ORACLE_SID in .base_profile. The real cause of ORA-01102 would be found in the alert log file where you will find additional information. Thus it always suggesting after getting error you must check the alert.log first.
The common reasons causing error ORA-01102 as follows.
·        The processes for Oracle (PMON, SMON, LGWR and DBWR) still exist. You can search them by ps -ef |grep “db_name”
·        Shared memory segments and semaphores still exist even though the database has been shutdown.
·        There exists a file named ORACLE_HOME\dbs\lk “db_name" where db_name is your actual database name.
·        A file named ORACLE_HOME\dbs\sgadef{sid}.dbf exists where SID is your actual database SID.
·        If you have two databases the same host and one is already started then if you try to start the other one will cause the issue.
·        While setting the ORACLE_SID environment variable, remember the point it is case sensitive in .bash_profile.
Solution:
·        Verify that there are no background processes owned by "oracle" $ ps -ef | grep ora_ | grep $ORACLE_SID. If background process exists, kill them.
·        Verify that no shared memory segments and semaphores that are owned by "oracle" still exist, if so, remove them $ ipcs –b
$ ipcrm -m Shared_Memory_ID_Number
$ ipcrm -s Semaphore_ID_Number
·        Verify that file $ORACLE_HOME\dbs\lk{db_name} does not exist where db_name is your actual database name.
·        Verify that file $ORACLE_HOME\dbs\sgadef{sid}.dbf does not exist where sid is your actual database SID.
·        Check you have set the correct case of ORACLE_SID in .base_profile.
Note: The "lk{db_name}" and "sgadef{sid}.dbf" files are used for locking shared memory. It may happen that even though no memory is allocated, Oracle thinks memory is still locked. By removing the "sgadef" and "lk" files you remove any knowledge oracle has of shared memory that is in use. So after removing those two file you can try to startup database.
·        If you see you have several databases in your machine and both of them uses have same entry in the parameter control_files and db_name then use correct values belonging to the individual databases.

·        From alert log if you see error related to permission denied" then ensure that in the file/directory oracle has permission and ensure that oracle is owner of the file. With chmod and chown you can change permission and ownership respectively.