Hello,
Not easy without knowning the structure of you tables or the number of rows.
First thing I would look into is the sequential scans. But as the cost is
very low, I assume the tables have only a couple of rows, so this should not
make any difference.
If you say the snippet repeats for each record you work on, I assume you are
using no prepared statements (Cursors).
This could be a great enhancement to you program, because Informix does not
have to compile again and again the sql.
In case the source is not accessible, this is a problem.
In your config, I do not see any temp Dbspaces. These could be useful for
sort operations.
The huge cost on insert operations I have also seen often on big databases,
it shows that there are indices to update (maybe triggers which perform
other work ?).
You could try using more CPU VPs than active cpus (Informix will perform
even better in most cases).
The best information would be a measuring trace of your program (at which
operation most of the time is spent really).
You could try to get this info also by executing the same commands once with
a small programm or a script (maybe only for one record).
Is the database transactional ? Maybe time is taken at commit time or in
checkpoints (how long are these when the operation takes place ?).
Sorry, no more hints I can think of now with the information given.
Marcus
-----Original Message-----
From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of Doug
Fossmeyer
Sent: Friday, May 26, 2006 7:20 PM
To: ids@iiug.org
Subject: Performance issue after 732 to 94fc6 upgrade [6819]
Hello,
We finally upgraded from 7.32 to 9.40FC6. Some of our batch processes and
most user OLTP are performing as well or better than the 732 version.
However 3 of our large batch processes are now taking 30-40% longer to
complete. For example a process that used to take 7 hrs is now taking 11.5
hrs. I am seeking some advice on what may be occurring. I have included our
onconfig and an sqexplain for the process in question. One factor that seems
strange in the sqexplain is the cost of the insert statement. I am unsure if
that is just how sqexplain reports the data or if that is the issue.
We are on hp-ux 11.11 with a 4 way 8 gig N class server. IDS 64 bit engine
and
32 bit tools/network. The program in question is 4gl. Machine notes were
followed and kernel params slightly modified. The machine does not appear to
be taxed, not paging, cpu ok, etc. No excessive check points and onstat -p,
g ioq, g iof, g iov are OK.
We have run Art's dostats for the database in question and manually update
stats for the system tables recommended by IBM. We did a dbexport and
dbimport for the upgrade. We have rebuilt most of the indexes in question
(but not all). PDQ is not enabled, no fragmentation scheme, did not specify
detached indexes. (During the install we did have one major faux pas, we
installed the
64 bit tools, 64 bit engine, 32 bit network. We realised afterwards that we
grabbed the wrong tools cd (64 bit), and needed the 32 bit tools. Our
application vendor and IDS reseller stated we could just re-install the 32
bit tools over the top w/o having to reinstall the engine and network.)
I want to rule out IDS and server issues before address the business rule
set up or the vendor's code. So any advice or critique is fine.
Thanks in advance,
Doug
ROOTNAME rootdbs # Root dbspace name
ROOTPATH /dbms/links/rootdbs9 # Path for device containing root dbspace
ROOTOFFSET 0 # Offset of root dbspace into device (Kbytes) ROOTSIZE 30000 #
Size of root dbspace (Kbytes)
# Physical Log Configuration
PHYSDBS plogdbs94 # Location (dbspace) of physical log PHYSFILE 127000 #
Physical log file size (Kbytes)
# Logical Log Configuration
LOGFILES 101 # Number of logical log files LOGSIZE 2000 # Logical log size
(Kbytes) TABLSPACE_STATS 0 # Maintain tblspace statistics
# System Configuration
SERVERNUM 1 # Unique id corresponding to a OnLine instance DBSERVERNAME
online9 # Name of default database server DBSERVERALIASES test9 # List of
alternate dbservernames NETTYPE ipcshm,1,100,CPU NETTYPE soctcp,2,100,NET
DEADLOCK_TIMEOUT 120 # Max time to wait of lock in distributed env.
RESIDENT 1 # Forced residency flag (Yes = 1, No = 0) MULTIPROCESSOR 1 # 0
for single-processor, 1 for multi-processor VPCLASS CPU,num=3,aff=1-3,noage
VPCLASS AIO,num=2,aff=1-3 SINGLE_CPU_VP 0 # If non-zero, limit number of cpu
vps to one
# Shared Memory Parameters
LOCKS 200000 # Maximum number of locks
BUFFERS 900000 # Maximum number of shared buffers PHYSBUFF 64 # Physical log
buffer size (Kbytes) LOGBUFF 64 # Logical log buffer size (Kbytes) CLEANERS
8 # Number of buffer cleaner processes SHMBASE 0x0L # Shared memory base
address SHMVIRTSIZE 327680 SHMADD 32768 # Size of new shared memory segments
(Kbytes) SHMTOTAL 0 # Total shared memory (Kbytes). 0=>unlimited CKPTINTVL
300 # Check point interval (in sec) LRUS 128 # Number of LRU queues
LRU_MAX_DIRTY 10.000000 # LRU percent dirty begin cleaning limit
LRU_MIN_DIRTY 5.000000 # LRU percent dirty end cleaning limit TXTIMEOUT
0x12c # Transaction timeout (in sec) STACKSIZE 64 # Stack size (Kbytes)
# DYNAMIC_LOGS:
DYNAMIC_LOGS 0
LTXHWM 40
LTXEHWM 50
# OFF_RECVRY_THREADS:
OFF_RECVRY_THREADS 10 # Default number of offline worker threads
ON_RECVRY_THREADS 1 # Default number of online worker threads
# Backup/Restore variables
BAR_ACT_LOG /dbms/informix9/log/bar_act.log BAR_DEBUG_LOG
/dbms/informix9/log/informix/bar_dbug.log
# ON-Bar Debug Log - not in /tmp please BAR_MAX_BACKUP 0 BAR_RETRY 1
BAR_NB_XPORT_COUNT 10 BAR_XFER_BUF_SIZE 31 RESTARTABLE_RESTORE on
BAR_PROGRESS_FREQ 0
# Read Ahead Variables
RA_PAGES 32 # Number of pages to attempt to read ahead RA_THRESHOLD 30 #
Number of pages left before next group
DBSPACETEMP tempdbs6:tempdbs7:tempdbs8:tempdbs9:tempdbs10
FILLFACTOR 90 # Fill factor for building indexes
USEOSTIME 0 # 0: use internal time(fast), 1: get time from O
# Parallel Database Queries (pdq)
MAX_PDQPRIORITY 90 # Maximum allowed pdqpriority DS_MAX_QUERIES # Maximum
number of decision support queries DS_TOTAL_MEMORY # Decision support memory
(Kbytes) DS_MAX_SCANS 1048576 # Maximum number of decision support scans
DATASKIP off
# OPTCOMPIND
OPTCOMPIND 1 # To hint the optimizer
DIRECTIVES 1 # Optimizer DIRECTIVES ON (1/Default) or OFF (0) ONDBSPACEDOWN
2 # Dbspace down option: 0 = CONTINUE, 1 = ABORT,
# HETERO_COMMIT (Gateway participation in distributed transactions)
HETERO_COMMIT 0 SBSPACENAME # Default smartblob space name - this is where b
SYSSBSPACENAME # Default smartblob space for use by the Informi
BLOCKTIMEOUT 3600 # Default timeout for system block SYSALARMPROGRAM
/dbms/informix9/etc/evidence.sh # System Alarm program path
# Optimization goal: -1 = ALL_ROWS(Default), 0 = FIRST_ROWS OPT_GOAL -1
ALLOW_NEWLINE 0 # embedded newlines(Yes = 1, No = 0 or anything
START OF PROCESS
QUERY:
------
delete from retwahst where empid = ?
Estimated Cost: 7
Estimated # of Rows Returned: 36
1) bsidba.retwahst: INDEX PATH
(1) Index Keys: empid (Serial, fragments: ALL)
Lower Index Filter: bsidba.retwahst.empid = '101289 '
QUERY:
------
select count ( * ) from hr_retirewa , hr_pe_mstr where id = ? and hr_pe_id =
?
and ( currbeg <= ? or extractbeg is null or extractbeg = " " ) and ( currend
>= ? or extractend is null or extractend = " " ) and ( retirestat = "A"
>or
retirestat = "O" or retirestat = "G" or retirestat = "F" )
Estimated Cost: 4
Estimated # of Rows Returned: 1
1) bsi.hr_pe_mstr: INDEX PATH
(1) Index Keys: hr_pe_id (Key-Only) (Serial, fragments: ALL)
Lower Index Filter: bsi.hr_pe_mstr.hr_pe_id = '101289 '
2) bsidba.hr_retirewa: INDEX PATH
Filters: ((((bsidba.hr_retirewa.currend >= 05/01/2006 OR
bsidba.hr_retirewa.extractend IS NULL ) OR bsidba.hr_retirewa.extractend = )
AND (((bsidba.hr_retirewa.retirestat = 'A' OR bsidba.hr_retirewa.retirestat
= 'O' ) OR bsidba.hr_retirewa.retirestat = 'G' ) OR
bsidba.hr_retirewa.retirestat = 'F' ) ) AND ((bsidba.hr_retirewa.currbeg <=
05/31/2006 OR bsidba.hr_retirewa.extractbeg IS NULL ) OR
bsidba.hr_retirewa.extractbeg = ) )
(1) Index Keys: id currbeg currend (Serial, fragments: ALL)
Lower Index Filter: bsidba.hr_retirewa.id = '101289 '
NESTED LOOP JOIN
QUERY:
------
select retirestat , currbeg , currend , start_per , reportbeg , extractend ,
currend , currsys , currplan , currrate , reportbeg , reportend , hr_pe_mstr
.
* from hr_retirewa , hr_pe_mstr where id = ? and hr_pe_id = ? and ( currbeg
<= ? or extractbeg is null or extractbeg = " " ) and ( currend >= ? or
extractend is null or extractend = " " ) and ( retirestat = "A" or
retirestat = "O" or retirestat = "G" or retirestat = "F" )
Estimated Cost: 7
Estimated # of Rows Returned: 1
1) bsi.hr_pe_mstr: INDEX PATH
(1) Index Keys: hr_pe_id (Serial, fragments: ALL)
Lower Index Filter: bsi.hr_pe_mstr.hr_pe_id = '101289 '
2) bsidba.hr_retirewa: INDEX PATH
Filters: ((((bsidba.hr_retirewa.currend >= 05/01/2006 OR
bsidba.hr_retirewa.extractend IS NULL ) OR bsidba.hr_retirewa.extractend = )
AND (((bsidba.hr_retirewa.retirestat = 'A' OR bsidba.hr_retirewa.retirestat
= 'O' ) OR bsidba.hr_retirewa.retirestat = 'G' ) OR
bsidba.hr_retirewa.retirestat = 'F' ) ) AND ((bsidba.hr_retirewa.currbeg <=
05/31/2006 OR bsidba.hr_retirewa.extractbeg IS NULL ) OR
bsidba.hr_retirewa.extractbeg = ) )
(1) Index Keys: id currbeg currend (Serial, fragments: ALL)
Lower Index Filter: bsidba.hr_retirewa.id = '101289 '
NESTED LOOP JOIN
QUERY:
------
select syscode from hr_rettblwa where system = ? and plan = ?
Estimated Cost: 2
Estimated # of Rows Returned: 1
1) bsidba.hr_rettblwa: SEQUENTIAL SCAN
Filters: (bsidba.hr_rettblwa.plan = 3 AND bsidba.hr_rettblwa.system = 'SRS '
)
QUERY:
------
select min ( extractbeg ) from hr_retirewa where id = ? and ( retirestat =
"A"
or retirestat = "O" or retirestat = "G" or retirestat = "F" )
Estimated Cost: 3
Estimated # of Rows Returned: 1
1) bsidba.hr_retirewa: INDEX PATH
Filters: (((bsidba.hr_retirewa.retirestat = 'A' OR
bsidba.hr_retirewa.retirestat = 'O' ) OR bsidba.hr_retirewa.retirestat = 'G'
) OR bsidba.hr_retirewa.retirestat = 'F' )
(1) Index Keys: id currbeg currend (Serial, fragments: ALL)
Lower Index Filter: bsidba.hr_retirewa.id = '101289 '
QUERY:
------
select reportbeg , reportend , currbeg , currend , currsys , currplan ,
currrate , extractbeg , extractend , retirestat , ovrtype from hr_retirewa
where id = ?
Estimated Cost: 3
Estimated # of Rows Returned: 1
1) bsidba.hr_retirewa: INDEX PATH
(1) Index Keys: id currbeg currend (Serial, fragments: ALL)
Lower Index Filter: bsidba.hr_retirewa.id = '101289 '
QUERY:
------
select drs_period [ 1 , 4 ] , drs_period [ 5 , 6 ] , earned_dol , adj_dollar
, report_dol , report_hrs , drs_period from retwa_reported where ( empssn =
?
and retwa_reported . warn <> "NoDf" ) order by drs_period
Estimated Cost: 1
Estimated # of Rows Returned: 1
Temporary Files Required For: Order By
1) bsidba.retwa_reported: SEQUENTIAL SCAN
Filters: (bsidba.retwa_reported.empssn = 'xxxxxxxx ' AND
bsidba.retwa_reported.warn != 'NoDf' )
QUERY:
------
select * from retwa_b where recstatus = "FC" and ssn = ?
Estimated Cost: 22
Estimated # of Rows Returned: 80
1) bsidba.retwa_b: INDEX PATH
Filters: bsidba.retwa_b.recstatus = 'FC'
(1) Index Keys: ssn (Serial, fragments: ALL)
Lower Index Filter: bsidba.retwa_b.ssn = 'xxxxxxxx '
QUERY:
------
select pyt_per_cc, py_batch_name, pyt_date01, pyt_hrs_no01, pyt_hrs01,
pyt_rt01, pyt_amt01, type, statuscd, system, plan, deferflag,
py_per_check_dt, py_per_end, RetirePB, RetireHB, pyt_num_cd from
pyt_hrs_dtl, cdhtmp25001, py_per_mstr where hr_pe_id = ? and (py_batch_name
like 'SYSTM%' or py_batch_name like 'DRS%') and (pyt_status = 'DS' or
pyt_status = 'DM' or pyt_status = 'DT') and (pyt_date01 <= ?) and
(pyt_date01 >= ?) and pyt_hrs_no01= cdhtmp25001.CdhNo and
((cdhtmp25001.RetirePB > ' ' and cdhtmp25001.RetirePB is not NULL) or
(cdhtmp25001.RetireHB > ' ' and cdhtmp25001.RetireHB is not NULL)) and
(cdhtmp25001.Type <> 'P') and ((py_per_check_dt >= cdhtmp25001.dtBeg or
cdhtmp25001.dtBeg is NULL or cdhtmp25001.dtBeg = ' ') and (py_per_check_dt
<= cdhtmp25001.dtEnd or cdhtmp25001.dtEnd is NULL or cdhtmp25001.dtEnd = '
')) and py_per_cc = pyt_per_cc
Estimated Cost: 306
Estimated # of Rows Returned: 1
1) bsi.pyt_hrs_dtl: INDEX PATH
Filters: ((((bsi.pyt_hrs_dtl.py_batch_name LIKE 'SYSTM%' OR
bsi.pyt_hrs_dtl.py_batch_name LIKE 'DRS%' ) AND bsi.pyt_hrs_dtl.pyt_date01
>=
02/01/2001 ) AND bsi.pyt_hrs_dtl.pyt_date01 <= 05/31/2006 ) AND
((bsi.pyt_hrs_dtl.pyt_status = 'DS' OR bsi.pyt_hrs_dtl.pyt_status = 'DM' )
OR bsi.pyt_hrs_dtl.pyt_status = 'DT' ) )
(1) Index Keys: hr_pe_id (Serial, fragments: ALL)
Lower Index Filter: bsi.pyt_hrs_dtl.hr_pe_id = '101289 '
2) dougf.cdhtmp25001: INDEX PATH
Filters: ((dougf.cdhtmp25001.retirepb > ' ' AND dougf.cdhtmp25001.retirepb
IS NOT NULL ) OR (dougf.cdhtmp25001.retirehb > ' ' AND
dougf.cdhtmp25001.retirehb IS NOT NULL ) )
(1) Index Keys: cdhno type retirepb retirehb dtbeg dtend (Key-First)
(Serial,
fragments: ALL)
Lower Index Filter: bsi.pyt_hrs_dtl.pyt_hrs_no01 = dougf.cdhtmp25001.cdhno
Key-First Filters: (dougf.cdhtmp25001.type != 'P' ) NESTED LOOP JOIN
3) bsi.py_per_mstr: INDEX PATH
Filters: (((bsi.py_per_mstr.py_per_check_dt <= dougf.cdhtmp25001.dtend OR
dougf.cdhtmp25001.dtend IS NULL ) OR dougf.cdhtmp25001.dtend = ) AND
((bsi.py_per_mstr.py_per_check_dt >= dougf.cdhtmp25001.dtbeg OR
dougf.cdhtmp25001.dtbeg IS NULL ) OR dougf.cdhtmp25001.dtbeg = ) )
(1) Index Keys: py_per_cc (Serial, fragments: ALL)
Lower Index Filter: bsi.py_per_mstr.py_per_cc = bsi.pyt_hrs_dtl.pyt_per_cc
NESTED LOOP JOIN ...SNIP REPEATS 25 times for array ....SNIP
QUERY:
------
select pyh_per_cc, pyh_no02, pyh_amt02,pyh_ck_dt, py_per_end, type, system,
plan, statuscd, py_per_beg, RetirePB, RetireHB, DeferFlag from pyh_hst_dtl,
cdhtmp25001, py_per_mstr where pyh_hst_dtl.hr_pe_id = ? and
pyh_hst_dtl.pyh_no02 = cdhtmp25001.CdhNo and ((cdhtmp25001.RetirePB > ' '
and cdhtmp25001.RetirePB is not NULL)) and ((pyh_ck_dt >= cdhtmp25001.dtBeg
or cdhtmp25001.dtBeg is NULL or cdhtmp25001.dtBeg = ' ') and (pyh_ck_dt <=
cdhtmp25001.dtEnd or cdhtmp25001.dtEnd is NULL or cdhtmp25001.dtEnd = ' '))
and (cdhtmp25001.Type <> 'E') and py_per_beg <= ? and py_per_end >= ? and
pyh_per_cc = py_per_cc and (pyh_hst_dtl.pyh_ck_note in ('DP','DT') or
(pyh_hst_dtl.pyh_ck_note in ('WP','WT') and pyh_per_cc = ?))
Estimated Cost: 87
Estimated # of Rows Returned: 1
1) dougf.cdhtmp25001: SEQUENTIAL SCAN
Filters: ((dougf.cdhtmp25001.retirepb > ' ' AND dougf.cdhtmp25001.type !=
'E'
) AND dougf.cdhtmp25001.retirepb IS NOT NULL )
2) bsi.pyh_hst_dtl: INDEX PATH
(1) Index Keys: hr_pe_id (Serial, fragments: ALL)
Lower Index Filter: bsi.pyh_hst_dtl.hr_pe_id = '101289 '
DYNAMIC HASH JOIN
Dynamic Hash Filters: bsi.pyh_hst_dtl.pyh_no02 = dougf.cdhtmp25001.cdhno
Other Join Filters: ((((bsi.pyh_hst_dtl.pyh_ck_dt >= dougf.cdhtmp25001.dtbeg
OR dougf.cdhtmp25001.dtbeg IS NULL ) OR dougf.cdhtmp25001.dtbeg = ) AND
((bsi.pyh_hst_dtl.pyh_ck_dt <= dougf.cdhtmp25001.dtend OR
dougf.cdhtmp25001.dtend IS NULL ) OR dougf.cdhtmp25001.dtend = ) ) AND
(bsi.pyh_hst_dtl.pyh_ck_note IN ('DP' , 'DT' )OR
(bsi.pyh_hst_dtl.pyh_ck_note IN ('WP' , 'WT' )AND bsi.pyh_hst_dtl.pyh_per_cc
= 601105 ) ) )
3) bsi.py_per_mstr: INDEX PATH
Filters: (bsi.py_per_mstr.py_per_beg <= 05/31/2006 AND
bsi.py_per_mstr.py_per_end >= 02/01/2001 )
(1) Index Keys: py_per_cc (Serial, fragments: ALL)
Lower Index Filter: bsi.pyh_hst_dtl.pyh_per_cc = bsi.py_per_mstr.py_per_cc
NESTED LOOP JOIN ...snip REPEATS 10 times for array ....snip
QUERY:
------
select unique earningper [ 1 , 4 ] , earningper [ 5 , 6 ] , systemcode ,
plancode , statuscode , typecd from retwa_b where ( recstatus = "FC" ) and (
ssn = ? )
Estimated Cost: 22
Estimated # of Rows Returned: 40
1) bsidba.retwa_b: INDEX PATH
Filters: bsidba.retwa_b.recstatus = 'FC'
(1) Index Keys: ssn (Serial, fragments: ALL)
Lower Index Filter: bsidba.retwa_b.ssn = 'xxxxxxxxx '
QUERY:
------
select * from retwaadj,hr_pe_mstr where ssn = hr_pe_ssn and hr_pe_id = ?
Estimated Cost: 1
Estimated # of Rows Returned: 1
1) bsidba.retwaadj: SEQUENTIAL SCAN
2) bsi.hr_pe_mstr: INDEX PATH
(1) Index Keys: hr_pe_ssn hr_pe_id (Serial, fragments: ALL)
Lower Index Filter: (bsidba.retwaadj.ssn = bsi.hr_pe_mstr.hr_pe_ssn AND
bsi.hr_pe_mstr.hr_pe_id = '101289 ' ) NESTED LOOP JOIN
QUERY:
------
select * from retwaadj,hr_pe_mstr where ssn = hr_pe_ssn and hr_pe_id = ?
Estimated Cost: 1
Estimated # of Rows Returned: 1
1) bsidba.retwaadj: SEQUENTIAL SCAN
2) bsi.hr_pe_mstr: INDEX PATH
(1) Index Keys: hr_pe_ssn hr_pe_id (Serial, fragments: ALL)
Lower Index Filter: (bsidba.retwaadj.ssn = bsi.hr_pe_mstr.hr_pe_ssn AND
bsi.hr_pe_mstr.hr_pe_id = '101289 ' ) NESTED LOOP JOIN
QUERY:
------
insert into retwahst values ( ? , ? , ? , ? , ? , ? , ? , ? , ? , ? , ? , ?
, ? , ? , ? , ? , ? , ? , ? , ? , ? , ? , ? , ? , ? )
Estimated Cost: 18598
Estimated # of Rows Returned: 163966
.......SNIP REPEATS 38 times for all data
Process then continues for the next ID (9000 total id's)
****************************************************************************
***
Forum Note: Use "Reply" to post a response in the discussion forum.
