IDS Forum

Re: Random Server Slowdown "Redone"

Posted By: Art Kagel
Date: Wednesday, 13 January 2010, at 11:37 a.m.

In Response To: Random Server Slowdown "Redone" (JAVIER HERNáNDEZ)

I'm going to suggest that you change your NETTYPE for ipcshm from NET to CPU
and from a single poll thread to 4 (one for each CPU VP) if there are a
significant number of users connecting via shared memory (if this connection
type is for maintenance only then make the change but you can get away with
only two poll threads). Search the forum history or the Informix FAQ for my
reasoning on this.

Also you have 8 LRU queues but only 4 CLEANERS. You should have the greater
of at least one cleaner thread for each LRU queue or 1.25 cleaner threads
per chunk. Adjust accordingly.

Art

Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (art@iiug.org)

See you at the 2010 IIUG Informix Conference
April 25-28, 2010
Overland Park (Kansas City), KS
www.iiug.org/conf

Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Advanced DataTools, the IIUG, nor any other
organization with which I am associated either explicitly, implicitly, or by
inference. Neither do those opinions reflect those of other individuals
affiliated with any entity with which I am affiliated nor those of the
entities themselves.

2010/1/13 JAVIER HERNáNDEZ <javier_hdzt@yahoo.com.mx>

> Hi,
>
> In mid December I posted about a problem we have with an instance slowing
> down
> at random times/days. Since that we upgraded the instance to IDS 11.5
> 64-bit
> to better use the hardware assigned to Informix.
>
> Well, the problem has returned and neded assistance in solving the puzzle.
>
> At random days/time we experience slow downs in the responce of the
> instance,
> checking the processor utilization shows no problem 10% average (spikes of
> 100% util in 1 or 2 processor of 4)and there is plenty of available memory
> (the server has 8GB and IDS is consuming no more the 3GB), the most strange
> issue is that we are monitoring this server with OAT v2.26 and when the
> slow
> down occurs it can't connect, also, the production environment is using a
> mix
> of clients: Oracle Application server, a compiled .NET app using ODBC (8
> Machines).
>
> The maximun number of conections we have seen is 370.
>
> Below I include some info about the server that can be of interest.
>
> My server is using RedHat 4 ES update 5
> [informix@saih respaldo_conf]$ uname -a
> Linux saih.inr.gob.mx 2.6.9-55.ELsmp #1 SMP Fri Apr 20 16:36:54 EDT 2007
> x86_64 x86_64 x86_64 GNU/Linux
>
> We are using IDS 11.5 FC5
> [informix@saih respaldo_conf]$ onstat -
>
> IBM Informix Dynamic Server Version 11.50.FC5 -- On-Line -- Up 14:44:21 --
> 3296300 Kbytes
>
> [informix@saih ~]$ onstat -p
>
> IBM Informix Dynamic Server Version 11.50.FC5 -- On-Line -- Up 15:02:58 --
> 3296300 Kbytes
>
> Profile
> dskreads pagreads bufreads %cached dskwrits pagwrits bufwrits %cached
> 803051 5116508 1317398798 99.94 863679 1063515 7082328 87.97
>
> isamtot open start read write rewrite delete commit rollbk
> 974690064 1029061 1914834 937852565 2519305 12637 73794 8051 4
>
> gp_read gp_write gp_rewrt gp_del gp_alloc gp_free gp_curs
> 0 0 0 0 0 0 0
>
> ovlock ovuserthread ovbuff usercpu syscpu numckpts flushes
> 0 0 0 3512.02 49.25 302 151
>
> bufwaits lokwaits lockreqs deadlks dltouts ckpwaits compress seqscans
> 58848 2 2656077359 0 0 1 19463 70959
>
> ixda-RA idx-RA da-RA RA-pgsused lchwaits
> 32181 2342 400009 434365 46228
>
> And finaly I include an edited copy of my onconfig (to remove comments)
>
> ROOTNAME rootdbs
> ROOTPATH $INFORMIXDIR/liga_dbspaces/dbs_rootweb
> ROOTOFFSET 0
> ROOTSIZE 2008092
> MIRROR 0
> MIRRORPATH $INFORMIXDIR/tmp/demo_on.root_mirror
> MIRROROFFSET 0
> PHYSFILE 50000
> PLOG_OVERFLOW_PATH $INFORMIXDIR/tmp
> PHYSBUFF 128
> LOGFILES 20
> LOGSIZE 10000
> DYNAMIC_LOGS 2
> LOGBUFF 64
> LTXHWM 70
> LTXEHWM 80
> MSGPATH $INFORMIXDIR/inrserver.log
> CONSOLE $INFORMIXDIR/inrserver.con
> TBLTBLFIRST 0
> TBLTBLNEXT 0
> TBLSPACE_STATS 1
> DBSPACETEMP dbs_tempweb
> SBSPACETEMP
> SBSPACENAME
> SYSSBSPACENAME
> ONDBSPACEDOWN 2
> SERVERNUM 0
> DBSERVERNAME ol_inrserver
> DBSERVERALIASES ol_inrservershm
> NETTYPE ipcshm,1,50,NET
> NETTYPE soctcp,3,150,NET
> LISTEN_TIMEOUT 60
> MAX_INCOMPLETE_CONNECTIONS 1024
> FASTPOLL 1
> MULTIPROCESSOR 1
> VPCLASS cpu,num=4,noage
> VPCLASS shm,num=1,noage
> VPCLASS soc,num=3,noage
> VP_MEMORY_CACHE_KB 0
> SINGLE_CPU_VP 0
> #VPCLASS aio,num=1
> CLEANERS 4
> AUTO_AIOVPS 1
> DIRECT_IO 0
> LOCKS 20000
> DEF_TABLE_LOCKMODE page
> RESIDENT 2
> SHMBASE 0x44000000L
> SHMVIRTSIZE 262144
> SHMADD 32768
> EXTSHMADD 32768
> SHMTOTAL 0
> SHMVIRT_ALLOCSEG 0,3
> SHMNOACCESS
> CKPTINTVL 300
> AUTO_CKPTS 1
> RTO_SERVER_RESTART 0
> BLOCKTIMEOUT 3600
> TXTIMEOUT 300
> DEADLOCK_TIMEOUT 60
> HETERO_COMMIT 0
> TAPEDEV /dev/IBMtape0
> TAPEBLK 32
> TAPESIZE 0
> LTAPEDEV /dev/null
> LTAPEBLK 32
> LTAPESIZE 0
> BAR_ACT_LOG $INFORMIXDIR/tmp/bar_act.log
> BAR_DEBUG_LOG $INFORMIXDIR/tmp/bar_dbug.log
> BAR_DEBUG 0
> BAR_MAX_BACKUP 0
> BAR_RETRY 1
> BAR_NB_XPORT_COUNT 20
> BAR_XFER_BUF_SIZE 31
> RESTARTABLE_RESTORE ON
> BAR_PROGRESS_FREQ 0
> BAR_BSALIB_PATH
> BACKUP_FILTER
> RESTORE_FILTER
> BAR_PERFORMANCE 0
> ISM_DATA_POOL ISMData
> ISM_LOG_POOL ISMLogs
> DD_HASHSIZE 31
> DD_HASHMAX 10
> DS_HASHSIZE 31
> DS_POOLSIZE 127
> PC_HASHSIZE 31
> PC_POOLSIZE 127
> STMT_CACHE 0
> STMT_CACHE_HITS 0
> STMT_CACHE_SIZE 512
> STMT_CACHE_NOLIMIT 0
> STMT_CACHE_NUMPOOL 1
> USEOSTIME 0
> STACKSIZE 128
> ALLOW_NEWLINE 0
> USELASTCOMMITTED NONE
> FILLFACTOR 90
> MAX_FILL_DATA_PAGES 0
> BTSCANNER num=1,threshold=5000,rangesize=-1,alice=6,compression=default
> ONLIDX_MAXMEM 5120
> MAX_PDQPRIORITY 20
> DS_MAX_QUERIES
> DS_TOTAL_MEMORY 2048
> DS_MAX_SCANS 1048576
> DS_NONPDQ_QUERY_MEM 512
> DATASKIP
> OPTCOMPIND 2
> DIRECTIVES 1
> EXT_DIRECTIVES 0
> OPT_GOAL -1
> IFX_FOLDVIEW 0
> AUTO_REPREPARE 1
> RA_PAGES 64
> RA_THRESHOLD 16
> EXPLAIN_STAT 1
> SQLTRACE level=med,ntraces=5000,size=10240,mode=global
> #DBCREATE_PERMISSION informix
> #DB_LIBRARY_PATH
> IFX_EXTEND_ROLE 1
> SECURITY_LOCALCONNECTION
> UNSECURE_ONSTAT
> ADMIN_USER_MODE_WITH_DBSA
> ADMIN_MODE_USERS
> SSL_KEYSTORE_LABEL
> PLCY_POOLSIZE 127
> PLCY_HASHSIZE 31
> USRC_POOLSIZE 127
> USRC_HASHSIZE 31
> STAGEBLOB
> OPCACHEMAX 0
> ENCRYPT_HDR
> ENCRYPT_SMX
> ENCRYPT_CDR 0
> ENCRYPT_CIPHERS
> ENCRYPT_MAC
> ENCRYPT_MACFILE
> ENCRYPT_SWITCH
> CDR_EVALTHREADS 1,2
> CDR_DSLOCKWAIT 5
> CDR_QUEUEMEM 4096
> CDR_NIFCOMPRESS 0
> CDR_SERIAL 0
> CDR_DBSPACE
> CDR_QHDR_DBSPACE
> CDR_QDATA_SBSPACE
> CDR_MAX_DYNAMIC_LOGS 0
> CDR_SUPPRESS_ATSRISWARN
> DRAUTO 0
> DRINTERVAL 30
> DRTIMEOUT 30
> HA_ALIAS
> DRLOSTFOUND $INFORMIXDIR/etc/dr.lostfound
> DRIDXAUTO 0
> LOG_INDEX_BUILDS
> SDS_ENABLE
> SDS_TIMEOUT 20
> SDS_TEMPDBS
> SDS_PAGING
> UPDATABLE_SECONDARY 0
> FAILOVER_CALLBACK
> TEMPTAB_NOLOG 0
> DELAY_APPLY 0
> STOP_APPLY 0
> LOG_STAGING_DIR
> ON_RECVRY_THREADS 1
> OFF_RECVRY_THREADS 10
> DUMPDIR $INFORMIXDIR/tmp
> DUMPSHMEM 1
> DUMPGCORE 0
> DUMPCORE 0
> DUMPCNT 1
> ALARMPROGRAM $INFORMIXDIR/etc/alarmprogram.sh
> ALRM_ALL_EVENTS 0
> STORAGE_FULL_ALARM 600,3
> SYSALARMPROGRAM $INFORMIXDIR/etc/evidence.sh
> RAS_PLOG_SPEED 25000
> RAS_LLOG_SPEED 0
> EILSEQ_COMPAT_MODE 0
> QSTATS 0
> WSTATS 0
> #VPCLASS jvp,num=1
> JVPJAVAHOME $INFORMIXDIR/extend/krakatoa/jre
> JVPHOME $INFORMIXDIR/extend/krakatoa
> JVPPROPFILE $INFORMIXDIR/extend/krakatoa/.jvpprops
> JVPLOGFILE $INFORMIXDIR/jvp.log
> #JDKVERSION 1.5
> JVPJAVALIB /bin/j9vm
> JVPJAVAVM jvm
> #JVPARGS -verbose:jni
> #JVPCLASSPATH
>
> $INFORMIXDIR/extend/krakatoa/krakatoa_g.jar:$INFORMIXDIR/extend/krakatoa/jdbc_g.jar
> JVPCLASSPATH
>
> $INFORMIXDIR/extend/krakatoa/krakatoa.jar:$INFORMIXDIR/extend/krakatoa/jdbc.jar
>
> BUFFERPOOL
>
> default,buffers=10000,lrus=8,lru_min_dirty=50.000000,lru_max_dirty=60.500000
> BUFFERPOOL
>
> size=2K,buffers=1200000,lrus=8,lru_min_dirty=50.000000,lru_max_dirty=60.000000
> AUTO_LRU_TUNING 1
>
> Hope you can help me.
>
> Javier Hernández
>
>
>
> *******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>

--001517473392d15fa0047d0e6059

Messages In This Thread

• Random Server Slowdown "Redone"
  JAVIER HERNáNDEZ -- Wednesday, 13 January 2010, at 11:01 a.m.
  • Re: Random Server Slowdown "Redone"
    JAVIER HERNáNDEZ -- Wednesday, 13 January 2010, at 11:03 a.m.
    • Re: Random Server Slowdown "Redone"
      Art Kagel -- Wednesday, 13 January 2010, at 11:44 a.m.
    • Re: Random Server Slowdown "Redone"
      CESAR MARTINS -- Wednesday, 13 January 2010, at 7:39 p.m.
  • Re: Random Server Slowdown "Redone"
    Alexandre Marini -- Wednesday, 13 January 2010, at 11:16 a.m.
  • Re: Random Server Slowdown "Redone"
    Art Kagel -- Wednesday, 13 January 2010, at 11:37 a.m.
  • Re: Random Server Slowdown "Redone"
    JAVIER HERNáNDEZ -- Friday, 15 January 2010, at 10:53 a.m.
    • Re: Random Server Slowdown "Redone"
      JAVIER HERNáNDEZ -- Friday, 15 January 2010, at 11:07 a.m.
    • Re: Random Server Slowdown "Redone"
      CESAR MARTINS -- Friday, 15 January 2010, at 11:54 a.m.
    • Re: Random Server Slowdown "Redone"
      Art Kagel -- Friday, 15 January 2010, at 12:44 p.m.
  • Re: Random Server Slowdown "Redone"
    JAVIER HERNáNDEZ -- Wednesday, 27 January 2010, at 11:35 a.m.
    • Re: Random Server Slowdown "Redone"
      Obnoxio The Clown -- Wednesday, 27 January 2010, at 11:41 a.m.
    • Re: Random Server Slowdown "Redone"
      Adam Tauno Williams -- Wednesday, 27 January 2010, at 11:49 a.m.

IDS Forum is maintained by Administrator with WebBBS 5.12.