IDS Forum

RE: Re.slow response

Posted By: Dan Michaelis
Date: Thursday, 10 June 2004, at 5:29 p.m.

In Response To: RE: Re.slow response (Sushil Shirodkar)

For what it's worth;

I always look at I/O. While it's possible to have a problem elsewhere, and
I've heard of people having them, my experience has been that there's ALWAYS
an I/O bottleneck to be found when you're talking about performance issues.

My troubleshooting approach moves from the O/S upwards. I tend to start
with an iostat or a sar to determine which disks are particularly "hot". On
my Solaris box here, I use

iostat -xCn 5 100

I am particularly interested in the %w number; that's the percentage of time
that there are "transactions" (not necessarily database transactions)
waiting on the disk, regardless of %b (busy). The idea is that even if the
disk is not busy, if you have stuff waiting on it, you're still in trouble.
Likewise, if the disk is busy, but generally nobody is waiting on it, that
might not be such a bad thing (though I'd probably still want to distribute
data off of it if I could).

Once you've done that, use the device column in the iostat output to go back
to your dbspaces (this can sometimes take some work, depending on your
volume mananger -- I have a hard time reading Veritas Volume Manager
output), and determine which dbspaces are your hot spots. You might be
surprised that these are not your big tables, but contain things like your
logfiles, temp space, etc. Once you know what dbspaces you have issues
with, you can start to resolve your problems. If you've got I/O problems
with temp space, consider either striping the disk at the O/S or hardware
level, or increasing the number of temp spaces (Informix should round-robin
requests across the different temp spaces). If the issue is with your
logfiles, some sort of striping seems like the right solution to me. If the
issue is with your table data, consider doing something other than
round-robin fragmentation, and across multiple disks. If the issue is with
indexes, definately consider some sort of fragmentation there.

From an application perspective, make sure that you'e not doing a "select *"
where selecting individual columns will do. Remember that if you select
only columns that are included in an index, you'll get all of you data
without going to the table at all, which really reduces your I/O. To that
end, you might want to consider adding some columns to your existing
indexes, even if they're not included in your where clause, so that you can
do more index-only reads to retrieve your data. Be careful, though, you do
incurr some "cost" with that.

For what it's worth; I'm not a big fan of round-robin fragmentation. To be
honest, I think that current operating systems and/or hardware do that kind
of "striping" faster and better, and it seems to me not to be worthwhile
just to emulate disk striping at the row level. I'm sure that someone much
smarter than I will disabuse me of that notion, but I don't think I've ever
done or recommended round-robin fragmentation.

Feel free to write me directly if you have questions or something that I've
said doesn't make sense. If it doesn't make sense, it's likely wrong... <G>
But you never know... <G>

Thanks.


Dan Michaelis



>From: "Sushil Shir...." <sushilps@hotmail.com>
>To: ids@iiug.org
>Subject: RE: Re.slow response [3099] Date: Thu, 10 Jun 2004 14:25:37 -0400
>(EDT)
>
>Hi,
>
>Here is the output of onstat -p, buffers read/write looks good. Checkpoint
>is always 0-2 secs,
>and checked couple of other things nowhere I can find the problem/issue. I
>do have perf. manual for reference.
>We have fiber disk with cache 512 mb, we are planning to put 2 gb cache its
>under consideration.
>O/S looks good except the oninit process which is eating 80-90 % of the
>power, this happens whenever we try to open window for more business, and
>start getting bigger volume of transactions into the database(irrespective
>of the time of the day)
>Backup takes place during night time.
>--------------------------------------------------------------------------------------------------------------------------
>Profile
>
>dskreads pagreads bufreads Êched dskwrits pagwrits bufwrits Êched
>4593473 5000224 2424607883 99.81 2036594 3058089 113185276 98.20
>
>isamtot open start read write rewrite delete commit
>rollbk
>2418731304 35461283 17699289 2117912282 15758216 1343207 44413 1181780
>5
>
>gp_read gp_write gp_rewrt gp_del gp_alloc gp_free gp_curs
>0 0 0 0 0 0 0
>
>ovlock ovuserthread ovbuff usercpu syscpu numckpts flushes
>0 0 0 35322.40 4795.57 19 7245
>
>bufwaits lokwaits lockreqs deadlks dltouts ckpwaits compress seqscans
>525818 1 140500918 0 0 254 7932843 276854
>
>ixda-RA idx-RA da-RA RA-pgsused lchwaits
>437940 834684 402338 1674814 4470596
>---------------------------------------------------------------------------------------------------------------------
>Sushil...
>
> >From: NormaJean.Sebastian@tellabs.com
> >To: forum.subscriber@iiug.org, ids@iiug.org, sushilps@hotmail.com
> >Subject: RE: Re.slow response [3097]
> >Date: Thu, 10 Jun 2004 12:45:51 -0500
> >MIME-Version: 1.0
> >Received: from tellabs.com ([204.154.129.57]) by mc1-f9.hotmail.com with
> >Microsoft SMTPSVC(5.0.2195.6824); Thu, 10 Jun 2004 10:47:27 -0700
> >Received: from ([172.23.207.11])by mx4.tellabs.com with ESMTP ;Thu, 10
>Jun
> >2004 12:45:51 -0500
> >Received: from localhost (root@localhost)by mailw01.hq.tellabs.com
> >(8.11.1/8.8.6) with ESMTP id i5AHjp302710;Thu, 10 Jun 2004 12:45:51 -0500
> >(CDT)
> >X-Message-Info: JGTYoYF78jEeFDEaX/cCYoJC6GHZkbHF
> >X-OpenMail-Hops: 1
> >Message-Id: <H00016dd15cfcf99.1086889550.mail@MHS>
> >Return-Path: normajean.sebastian@tellabs.com
> >X-OriginalArrivalTime: 10 Jun 2004 17:47:27.0782 (UTC)
> >FILETIME=[FEE7B460:01C44F12]
> >
> >
> >how are you buffer hit ratios, and other standard "perf tuning" things
> >(do you have a perf tuning manual to review)?
> >
> >how about you disk system.... do you have caching at that level... i.e.,
> >we have EMC disk storage with a GB or so of cache... that's an extra
> >performance level to check.
> >o/s system running fine to?
> >
> >is your slow response ALL the time or at certain times... what else is
> >happening on the system at those times? (both the DB system and O/S)...
> >maybe backup interference?
> >
> >
> >
> >
> >-----Original Message-----
> >From: sushilps@hotmail.com [mailto:sushilps@hotmail.com]
> >Sent: Thursday, June 10, 2004 11:35 AM
> >To: ids@iiug.org; forum.subscriber@iiug.org
> >Subject: RE: Re.slow response [3097]
> >
> >
> >Hi,
> >
> >Basically we are talking about 6 core tables which we think might be the
> >
> >problem, for these 6 tables we do have detached indexes in its own
> >dbspaces
> >and fragemented by round-robin which also has its own dbspace.
> >
> >sushil...
> >
> > >From: "Russell J. ...." <rclancy@rotech.com>
> > >To: ids@iiug.org
> > >Subject: RE: Re.slow response [3095] Date: Thu, 10 Jun 2004 11:28:37
> >
> > >-0400 (EDT)
> > >Received: from mc3-f18.hotmail.com ([64.4.50.154]) by
> >mc3-s3.hotmail.com
> > >with Microsoft SMTPSVC(5.0.2195.6824); Thu, 10 Jun 2004 08:52:50 -0700
> > >Received: from ace.iiug.org ([216.177.38.212]) by mc3-f18.hotmail.com
> >with
> > >Microsoft SMTPSVC(5.0.2195.6824); Thu, 10 Jun 2004 08:52:04 -0700
> > >Received: from ace.iiug.org (localhost [127.0.0.1])by ace.iiug.org
> > >(8.12.10-14/8.12.8) with ESMTP id i5AFUtFX017545;Thu, 10 Jun 2004
> >11:30:59
> > >-0400 (EDT)
> > >Received: (from nobody@localhost)by ace.iiug.org
> >(8.12.10-14/8.12.8/Submit)
> > >id i5AFSbGM017455;Thu, 10 Jun 2004 11:28:37 -0400 (EDT)
> > >X-Message-Info: jl7Vrt/mfsoDVNgfMplM1uxeMTbYoYxK
> > >Message-Id: <200406101528.i5AFSbGM017455@ace.iiug.org>
> > >Apparently-To: forum.subscriber@iiug.org
> > >Precedence: bulk
> > >Return-Path: nobody@ace.iiug.org
> > >X-OriginalArrivalTime: 10 Jun 2004 15:52:05.0132 (UTC)
> > >FILETIME=[E0AF24C0:01C44F02]
> > >
> > >FWIW, you may want to detach the indexes onto separate disks. If you
> >decide
> > >to do that and know what your indexed data will look like, you might
> >want
> > >to
> > >instead also fragment by expression to even the I/O over the disks.
> > >
> > >Russ
> > >
> > >-----Original Message-----
> > >From: forum.subscriber@iiug.org [mailto:forum.subscriber@iiug.org] On
> > >Behalf
> > >Of JHAYS2@sears.com
> > >Sent: Thursday, June 10, 2004 9:07 AM
> > >To: ids@iiug.org
> > >Subject: Re: Re.slow response [3093]
> > >
> > >Partition your most active tables over separate disks to spread out
> >the=20
> > >I/O.
> > >
> > >Also, could you furnish more info from your config file, especially
> >the=20
> > >area concerning System Config and Shared Memory?
> > >
> > >
> > >
> > >
> > >"Sushil Shir...." <sushilps@hotmail.com>
> > >Sent by: forum.subscriber@iiug.org
> > >06/09/2004 08:27 PM
> > >
> > >=20
> > > To: ids@iiug.org
> > > cc:=20
> > > Subject: Re.slow response [3090]
> > >
> > >
> > >Hello Everybody,
> > >
> > >Env: IDS 9.30 UC3,AIX 5.1. 4 cpu, 8gb ram IBM P660 with fiber channel.
> > >
> > >Our system is running fine with acceptable response, but since our
> >company
> > >=
> > >
> > >
> > >is growing we are getting more transactions/more hits on the database
> > >and=20
> > >started having slow response.
> > >
> > >1) All tables are within 2 extents
> > >2) Indexes/data is in good shape
> > >3) Good update statistics script
> > >
> > >Need suggestion/guidance from you folks as which area we should look
> >after
> > >=
> > >
> > >
> > >and what would
> > >be the problem.
> > >
> > >Thanks in advance,
> > >Sushil...
> > >
> > >=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5
> >F=5F=
> > >=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5
> >F=5F=
> > >=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
> > >Get fast, reliable Internet access with MSN 9 Dial-up ? now 3 months
> >FREE!
> > >=
> > >
> > >
> > >http://join.msn.click-url.com/go/onm00200361ave/direct/01/
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
> >_________________________________________________________________
> >Looking to buy a house? Get informed with the Home Buying Guide from MSN
> >
> >House & Home. http://coldwellbanker.msn.com/
> >
> >
> >
> >
> >
> >
> >-----------------------------------------
> >============================================================
> >The information contained in this message may be privileged
> >and confidential and protected from disclosure. If the
> >reader of this message is not the intended recipient, or an
> >employee or agent responsible for delivering this message to
> >the intended recipient, you are hereby notified that any
> >reproduction, dissemination or distribution of this
> >communication is strictly prohibited. If you have received
> >this communication in error, please notify us immediately by
> >replying to the message and deleting it from your computer.
> >
> >Thank you.
> >Tellabs
> >============================================================
>
>_________________________________________________________________
>MSN Toolbar provides one-click access to Hotmail from any Web page – FREE
>download! http://toolbar.msn.click-url.com/go/onm00200413ave/direct/01/
>
>

_________________________________________________________________
Looking to buy a house? Get informed with the Home Buying Guide from MSN
House & Home. http://coldwellbanker.msn.com/

Messages In This Thread

• Re.slow response
  Sushil Shirodkar -- Wednesday, 9 June 2004, at 9:27 p.m.
  • Re: Re.slow response
    Obnoxio The Clown -- Thursday, 10 June 2004, at 3:00 a.m.
  • Re: Re.slow response
    Sushil Shirodkar -- Thursday, 10 June 2004, at 8:36 a.m.
    • Re: Re.slow response
      Ruben Dutan -- Thursday, 10 June 2004, at 9:41 a.m.
  • Re: Re.slow response
    JHAYS2@sears.com -- Thursday, 10 June 2004, at 9:06 a.m.
    • RE: Re.slow response
      Russell J. Clancy -- Thursday, 10 June 2004, at 11:28 a.m.
      • RE: Re.slow response
        Sushil Shirodkar -- Thursday, 10 June 2004, at 12:35 p.m.
        • RE: Re.slow response
          NormaJean.Sebastian@tellabs.com -- Thursday, 10 June 2004, at 1:50 p.m.
          • Fw: Re.slow response
            Stephanie_Peltier@txnp.uscourts.gov -- Thursday, 10 June 2004, at 2:40 p.m.
            • RE: Fw: Re.slow response
              Sushil Shirodkar -- Friday, 11 June 2004, at 10:10 a.m.
        • RE: Re.slow response
          Sushil Shirodkar -- Thursday, 10 June 2004, at 2:25 p.m.
          • RE: Re.slow response
            Darren_Jacobs@carmax.com -- Thursday, 10 June 2004, at 3:58 p.m.
            • RE: Re.slow response
              Obnoxio The Clown -- Friday, 11 June 2004, at 5:45 a.m.
          • RE: Re.slow response
            JHAYS2@sears.com -- Thursday, 10 June 2004, at 4:18 p.m.
          • RE: Re.slow response
            keithfang@comcast.net -- Thursday, 10 June 2004, at 4:34 p.m.
          • RE: Re.slow response
            Dan Michaelis -- Thursday, 10 June 2004, at 5:29 p.m.
          • RE: Re.slow response
            Murray Wood (IList) -- Thursday, 10 June 2004, at 6:44 p.m.
          • RE: Re.slow response
            Sushil Shirodkar -- Friday, 11 June 2004, at 12:26 p.m.
            • RE: Re.slow response
              Ravi T -- Friday, 11 June 2004, at 3:25 p.m.
  • Re: Re.slow response
    Obnoxio The Clown -- Thursday, 10 June 2004, at 11:47 a.m.

IDS Forum is maintained by Administrator with WebBBS 5.12.