Archive for the ‘scripts’ Category

shell script to clean old oracle trace and log files

Friday, February 26th, 2010

This code cleans up old trace files, log files, core dumps, etc. It is designed to be run from cron. It takes a somewhat brutal approach by deleting files after just 7 days – good for e.g. dev/test servers, but in production you would probably want to modify this to keep files for longer.

For this example all oracle files of interest were under directory “/ora” – that would need to be changed to suit other sites.

# 1) Remove old oracle-owned aud, trc, trw, core files
#    Also remove old recv error files (these have format ".err_[0-9]")
#    Also remove access_log., error_log., emoms.log. files - which are generated in large numbers
#    Only core files named "core." are removed: because many other files with "core" in their name are not core dumps, are used by oracle.

echo "*** REMOVE OLD ORACLE AUDIT FILES and OLD RECV Error Files and OLD TRACE FILES ***"
find /ora -mtime +7 -user oracle \( \
  -name '*.aud' -o \
  -name '*.trc*' -o \
  -name '*.trw' -o \
  -name 'core.[0-9]*' -o \
  -name '*.err_[0-9]*' -o \
  -name 'access_log.[0-9]*' -o \
  -name 'error_log.[0-9]*' -o \
  -name 'emoms.log.[0-9]*' \) -exec rm {} \;

# 2) Cut down alert logs and listener logs, and also access_log and event_log (webcache/portal/etc.)
#    Old log files are ignored - only log files modified in the last 7 days are worked on.
#    Small log files are ignored - only files bigger than 3mb (=6144*512 byte blocks) are worked on. 3mb is approximately 30,000 lines of text.

echo "*** CUT DOWN THE ALERT LOGS and ORACLE LISTENER.LOG FILE ***"
for FILE in `find /ora -mtime -7 -size +6144 -user oracle \( \
  -name 'alert_*.log' -o \
  -name 'listener*.log' -o \
  -name 'access_log' -o \
  -name 'event_log' -o \
  -name 'http-web-access.log' -o \
  -name 'server.log' \)`
do
  echo "*** cutting $FILE ***"
  cp $FILE $FILE.tmp
  tail -10000 $FILE.tmp > $FILE
  rm $FILE.tmp
done

echo "*** CUT DOWN MESSAGES and WARN FILES  ***"
for FILE in `find /var/log/ -mtime -7 -size +6144 \( \
  -name 'messages' -o \
  -name 'warn'  \)`
do
  echo "*** cutting $FILE ***"
  cp $FILE $FILE.tmp
  tail -10000 $FILE.tmp > $FILE
  rm $FILE.tmp
done

#End of file.

Move script for ORA-03297: file contains used data beyond requested RESIZE value

Thursday, March 5th, 2009

Attempting to shrink files which have a lot of empty space can fail with this error:

SQL> alter database datafile '/ora1data/TEST/TEST_ts_data1_f1.dbf' resize 2g ;

ORA-03297: file contains used data beyond requested RESIZE value

Provided you can get an outage, one fix for this is to temporarily move segments to another tablespace using a scripts like this. (Note that this script includes the commands to move LOBs).

set pages 9999 lines 132
spool go.tmp
select 'alter table '||owner||'.'||table_name||' move tablespace ts_data2 ;'
from dba_tables
where tablespace_name = 'TS_DATA1'
order by 1
/
select 'alter index '||owner||'.'||index_name||' rebuild tablespace ts_data2 ;'
from dba_indexes
where tablespace_name = 'TS_DATA1'
and index_type != 'LOB'
order by 1
/
select 'alter table '||owner||'.'||table_name||' move tablespace ts_data2 lob ('||column_name||') store as (tablespace ts_data2) ;'
from dba_lobs
where tablespace_name = 'TS_DATA1'
order by 1
/
spool off
ed go.tmp

They can be moved back afterwards, once the original tablespace has been shrunk, if desired.

One thing to watch out for is user tablespace quotas during the moves. If you get errors like this:

ORA-01950: no privileges on tablespace 'TS_DATA2'

You either need to temporarily grant unlimited tablespace privilege to the user affected, or grant them a quota on the new tablespace.

Using tar with compress or gzip

Thursday, March 5th, 2009

1) To tar files up into a tarball:

tar cvfp - file1 file2 file3 > tarfile.tar

And to untar:

tar xvfp tarfile.tar

2) Now, same thing but with compress added in:

tar cvfp - file1 file2 file3 | compress > tarfile.tar.Z
zcat tarfile.tar.Z | tar xvfp -

3) And, the same thing using gzip rather than compress:

tar cvfp - file1 file2 file3 | gzip > tarfile.tar.gz
gunzip -c tarfile.tar.gz | tar xvfp -

(Or, if you prefer, “gzip -cd” is the same as “gunzip -c”)

Gzip compresses down in size slightly better than compress, but tends to take slightly longer to do so.

Installing dbms_profiler

Friday, June 22nd, 2007

I used the following to install dbms_profiler, using a central set of system owned tables rather than each user having its own tables:

conn / as sysdba
@?/rdbms/admin/profload.sql
conn system/password
alter user system default tablespace users ;
-- [or any other reasonable tablespace]
@?/rdbms/admin/proftab.sql
GRANT all ON plsql_profiler_runnumber TO PUBLIC;
GRANT all ON plsql_profiler_data TO PUBLIC;
GRANT all ON plsql_profiler_units TO PUBLIC;
GRANT all ON plsql_profiler_runs TO PUBLIC;
CREATE PUBLIC SYNONYM plsql_profiler_runs FOR plsql_profiler_runs;
CREATE PUBLIC SYNONYM plsql_profiler_units FOR plsql_profiler_units;
CREATE PUBLIC SYNONYM plsql_profiler_data FOR plsql_profiler_data;
CREATE PUBLIC SYNONYM plsql_profiler_runnumber FOR plsql_profiler_runnumber;
alter user system default tablespace system  ;

Based on Tim Hall’s post.

What SQL and sessions are running?

Friday, April 20th, 2007

GUIs like TOAD and Enterprise Manager can show this better, but they’re not always to hand. Every session active now or within the last minute is shown.

set pages 9999 lines 132
col username form a20
col osuser form a20
col program form a50
col minutes form 999
alter session set nls_date_format = 'Dy DD-Mon-YYYY HH24:MI:SS'
/
select distinct
s.sid, s.username, s.osuser, s.program||' '||s.module program,
s.status, s.last_call_Et/60 minutes, s.logon_time, w.state, w.event, w.seconds_in_wait ,
q.sql_text
from v$session s, v$sql q, v$session_wait w
where ( s.status = 'ACTIVE' or last_call_et < 60 )
and s.sql_hash_value = q.hash_value
and s.sql_address = q.address
and s.sid = w.sid
order by s.sid
/
set pages 9999 lines 80 head on feed 6

Bug? with wrong results from all_objects in stored plsql procedures

Friday, March 2nd, 2007

Wouldn’t it be nice if sql code produced the same result, regardless of whether it is inside a stored procedure or is a standalone plsql block? But that’s not what happens when the all_ views are used.

Run this code to see the anomaly for yourself:

set serverout on
declare
  var1 number ;
begin
  select count(*) into var1 from all_objects ;
  dbms_output.put_line('all_objects: '||var1) ;
end ;
/
create or replace procedure af_temp as
  var1 number ;
begin
  select count(*) into var1 from all_objects ;
  dbms_output.put_line('all_objects: '||var1) ;
end ;
/
exec af_temp ;
drop procedure af_temp ;

It’s the exact same plsql each time, but very different results reported. For example:

all_objects: 13501

PL/SQL procedure successfully completed.

Procedure created.

all_objects: 4929

PL/SQL procedure successfully completed.

Procedure dropped.

This behaviour appears to occur on all versions – I tested from 7.3 through 10gR2. The biggest discrepancies in results seem to be with DBA users other than SYS, but all users show some discrepancy.

This issue doesn’t occur with the dba_ views, so best to use them instead in stored plsql objects. That does mean explicitly granting select privileges on the dba_ views being used in stored procedures, which isn’t necessary for the all_ views.

So, is this a bug? I can’t see any mention of it all in metalink.

It’s not just an academic issue either: it caused this code of mine to fail to find and drop the objects it was meant to drop.

10046 tracing in another session

Wednesday, February 14th, 2007

To switch 10046 tracing on in another session (first setting on timed statistics and making max_dump_file_size very large):

set pages 9999 verify off
col serial new_value serial noprint
select serial# serial from v$session where sid = &1 ;
exec sys.dbms_system.set_bool_param_in_session(&1,&serial,'timed_statistics',true)
exec sys.dbms_system.set_int_param_in_session(&1,&serial,'max_dump_file_size',999999999)
exec sys.dbms_system.set_ev(&1,&serial,10046,8,'')

And to switch it off:

set pages 9999 verify off
col serial new_value serial noprint
select serial# serial from v$session where sid = &1 ;
exec sys.dbms_system.set_ev(&1,&serial,10046,0,'')

Note that you can run new (9i+) tkprof against old (8i and below) 10046 trace files in order to report the wait information. It is also possible to cat multiple trace files together before tkprof’ing.

Based on Alexander Bubernak’s post at http://www.dbasupport.com/oracle/ora10g/10046event.shtml

Clone a database

Friday, February 9th, 2007

The below SQL generates a SQL script that can be used to clone a database, putting tablespaces into hot backup mode one at a time. Run it on the source database, and edit the output to specify new target file and directory names, and change cp to rcp or scp if cloning to another server.

Database clones can also be done with rman, which has the advantage of avoiding the performance draining hot backup mode.

set pages 9999 lines 132 serverout on size 99999
BEGIN
  FOR t IN ( select distinct tablespace_name from dba_data_files )
  LOOP
    dbms_output.put_line('alter tablespace '||t.tablespace_name||' begin backup ;') ;
    FOR f IN ( select file_name from dba_data_files where tablespace_name = t.tablespace_name )
    LOOP
      dbms_output.put_line('host cp '||f.file_name||' '||f.file_name ) ;
    END LOOP ;
    dbms_output.put_line('alter tablespace '||t.tablespace_name||' end backup ;') ;
  END LOOP ;
END ;
/
set lines 80
select 'alter tablespace '||tablespace_name||' add tempfile '||file_name||' size '||bytes/1024/1024||' m ;'
from dba_temp_files
/
prompt alter system switch logfile ;;
prompt alter system backup controlfile to trace ;;

Slow SQL report

Wednesday, February 7th, 2007

Lists slow SQL in library cache for tuning investigations:

set pages 9999
spool c.lst
select elapsed_time/1000000 secs, executions,
  elapsed_time/1000000/greatest(executions,1) secs_per_exec,
sql_text from v$sql
where executions > 50
and elapsed_time/1000000/greatest(executions,1) > 1
order by 3 desc
/
spool off
ed c.lst

For older versions of oracle, use buffer_gets or disk_reads in place of elapsed_time.

Using 10g datapump and scheduler to copy schemas

Thursday, February 1st, 2007

Update 13-Feb-07: Setting the datapump table_exists_action to replace does not overwrite views, sequences, plsql objects. They have to be dropped separately before datapump is called, see gotchas and code below.

Update 02-Mar-07: dba_ views rather than all_ views have to be used to identify what objects to drop prior to datapump, for reasons explained in this post.

For a nightly process to copy small schemas from one database to another, the normal, or old style, way to do that is with export/import in a cron controlled shell script, which would include a sql-from-sql script to drop all the target schema objects before doing the import.

But that presents problems in RAC, which is what I was working on – which node does the cron script reside on? What happens if that node is down?

An option would be to have the script run on another machine, one unrelated to the RAC cluster, but that introduces more points of failure, and means sending data back and forward over sql*net.

So instead I used the oracle’s 10g scheduler and datapump, as below. This data pumps straight over a database link, without having to write to a dump file in between, which is nice. And because it is all in the database, it ought to be unaffected by particular RAC nodes being down.

(more…)