IDS Forum

SDS - Processes hang durning Checkpoint on Primary

Posted By: Jeff Filippi
Date: Wednesday, 16 September 2009, at 3:08 p.m.

I have a client who is running into issues on SDS instance during
checkpoints on primary. They are using non-blocking checkpoints.

Some of the processes hang until 8:12:26. The checkpoint on the Primary
started at 8:09:05 and finished at 8:15:19.

>From looking at "onstat -g ckp" there is under a second of wait time.

These processes run in sub-seconds.

They only see this issue when a checkpoint is occurring.

Informix: 11.50FC3

HP-UX 11.23 Itanium

Below is output from "onstat -g ckp", entries from online.log file and log
files from the application.

Has anyone saw this type of issue?

I opened a PMR but I did not get any answers.

onstat -g ckp

PRIMARY

AUTO_CKPTS=On RTO_SERVER_RESTART=900 seconds Estimated recovery time 24
seconds

Critical
Sections
Physical Log Logical Log

Clock Total
Flush Block # Ckp t Wait Long # Dirty Dskflu Total Avg
Total Avg
Interval Time Trigger LSN Time Time Time Waits
Time Time Time Buffers /Sec Pages /Sec Pages /Sec
1405 08:15:19 Llog 2230411:0x7331018 314.3 313.8 0.0 2
0.0 0.4 0.4 1522250 4850 1527246 42 2719627 75

SDS

1405 08:09:27 *Recovery 2230411:0x7331018 0.2 0.0 0.0 0
0.0 0.0 0.0 1497919 1497919 1503345 41 0 0

online log:

PRIMARY

08:15:19 Checkpoint Completed: duration was 314 seconds.

08:15:19 Fri Aug 28 - loguniq 2230411, logpos 0x7331018, timestamp:
0x96fbba Interval: 1405

08:15:19 Maximum server connections 69

08:15:19 Checkpoint Statistics - Avg. Txn Block Time 0.000, # Txns blocked
2, Plog used 1527246, Llog used 2719627

SDS:

08:09:28 Checkpoint Statistics - Avg. Txn Block Time 0.000, # Txns blocked
0, Plog used 1503345, Llog used 0

08:14:47 Checkpoint Completed: duration was 5 seconds.
(NOTE: NOT AN ENTRY IN OUTPUT OF onstat -g ckp???)

08:14:47 Fri Aug 28 - loguniq 2230411, logpos 0x7331018, timestamp:
0x972fd5 Interval: 1390

In the application, here are 3 processes running on the SDS server get stuck
during the checkpoint on the Primary.

These processes run in sub-seconds and here they all hung until 08:12:26.

1:****** NAME INFORMATION ****** 08/28/2009:08:10:23
1:TIME - before tscore 08:10:23
1:TIME - after tscore 08:10:23
1:TIME - before primary tst 08:12:26
1:Connected to 111.1.11.11 at 08:12:26
1:Successfully shutdown TCP connection at 08:12:26
1:Connected to 111.1.11.11 at 08:12:26
1:Successfully shutdown TCP connection at 08:12:26
1:TIME - after primary tst 08:12:26
1:TIME - before DEF 08:12:26
1:TIME - after DEF 08:12:26
1:TIME - before Build ABC 08:12:26
1:TIME - after Build ABC 08:12:26

-----------------

3:****** NAME INFORMATION ****** 08/28/2009:08:09:33
3:TIME - before tscore 08:09:33
3:TIME - after tscore 08:09:33
3:TIME - before Hard Copy 08:09:34
3:TIME - after Hard Copy 08:09:34
3:TIME - before DEF 08:09:34
3:TIME - after DEF 08:09:34
3:TIME - before Build ABC 08:09:34
3:TIME - after Build ABC 08:12:26

---------------------

4:****** NAME INFORMATION ****** 08/28/2009:08:09:52
4:TIME - before tscore 08:09:53
4:TIME - after tscore 08:09:53
4:TIME - before primary tst 08:09:53
4:Connected to 111.1.11.11 at 08:09:53
4:Successfully shutdown TCP connection at 08:09:55
4:Connected to 111.1.1.11 at 08:09:55
4:Successfully shutdown TCP connection at 08:09:55
4:TIME - after primary tst 08:09:55
4:TIME - before DEF 08:09:55
4:TIME - after DEF 08:09:55
4:TIME - before Build ABC 08:09:55
4:TIME - after Build ABC 08:12:26

Thanks, Jeff

Jeffrey J. Filippi

Integrated Data Consulting, LLC

104 Cottonwood Dr.

Streamwood, IL 60107

630-842-3608

630-497-1762 (Fax)

jeff.filippi@itdataconsulting.com

www.itdataconsulting.com

IBM BUSINESS PARTNER

Cert-Info-Mgmt_color

Messages In This Thread

• SDS - Processes hang durning Checkpoint on Primary
  Jeff Filippi -- Wednesday, 16 September 2009, at 3:08 p.m.
  • Re: SDS - Processes hang durning Checkpoint on Pri
    Madison Pruet -- Wednesday, 16 September 2009, at 4:52 p.m.

IDS Forum is maintained by Administrator with WebBBS 5.12.