IDS Forum

Re: VERY slow BTS index build

Posted By: Mark Ashworth
Date: Monday, 8 September 2014, at 3:44 p.m.

In Response To: Re: VERY slow BTS index build (MICHAEL HOFFMAN)

Hi Michael,

BTS is a little sensitive in performance to the vocabulary of the data
being indexed (ie how many distinct words). With a 300,000 word
vocabulary, I just tried an index build with 950,000 rows with a char(25)
column (containing 1 -4 random words from the vocabulary) on an idle linux
x86-64 VM with 11.50.FC8W4 with a demo server created by the install (ie
basically a default onconfig file) with on SBSBSPACE setup, one bts vp and
SBSPACETEMP set to bts_temp with no other special BTS options on the
create index statement. It took just over 7 minutes.

onspaces -c -S bts_temp -t -o 0 -s 400000 -p $DIR/bts_temp

select count(*) from bts_tab;

(count(*))

950000

IBM Informix Dynamic Server Version 11.50.FC8W4 -- On-Line -- Up 00:18:30
-- 64316 Kbytes

create index bts_idx on bts_tab (text bts_char_ops) using bts in
bts_index;
Index created.

real 7m2.699s

onstat -l has 7 logs of size 5000 and 30 logs of size 10000.

There were several BTS performance related defects addressed in fixpacks
after 11.50.xC6.

-- Mark.
Mark Ashworth
IBM Informix Extensibility Architect
Office phone: +1 (905) 413-5033
Alternate: +1 (905) 697-8094
Email: ashworth@ca.ibm.com
Check out my blog

From: "MICHAEL HOFFMAN" <mrh@panix.com>
To: ids@iiug.org,
Date: 09/08/2014 12:46 PM
Subject: Re: VERY slow BTS index build [33732]
Sent by: ids-bounces@iiug.org

Final outcom of the build: **25 hours***!! There has GOT to be a way to
speed
this up, right?
Checkpoints throughout the process ran in 2 seconds or less.
Overall buffer usage was 96% Data, 3.5% Other. So the estimate of 400000
Buffers was pretty accurate.

Any ideas what I should look to chance to make this index build run
quicker?
There is no way we'll be able to implement BTS if that is the performance
we
should expect. :-(

Thanks,
Michael

*******************************************************************************

Forum Note: Use "Reply" to post a response in the discussion forum.

Messages In This Thread

• VERY slow BTS index build
  MICHAEL HOFFMAN -- Friday, 5 September 2014, at 6:13 p.m.
  • Re: VERY slow BTS index build
    david@smooth1.co.uk -- Friday, 5 September 2014, at 8:35 p.m.
    • Re: VERY slow BTS index build
      MICHAEL HOFFMAN -- Friday, 5 September 2014, at 8:40 p.m.
      • Re: VERY slow BTS index build
        Mark Ashworth -- Monday, 8 September 2014, at 12:14 p.m.
  • Re: VERY slow BTS index build
    Mark Ashworth -- Monday, 8 September 2014, at 12:33 p.m.
    • Re: VERY slow BTS index build
      MICHAEL HOFFMAN -- Monday, 8 September 2014, at 12:41 p.m.
  • Re: VERY slow BTS index build
    MICHAEL HOFFMAN -- Monday, 8 September 2014, at 12:46 p.m.
    • Re: VERY slow BTS index build
      Mark Ashworth -- Monday, 8 September 2014, at 3:44 p.m.
      • Re: VERY slow BTS index build
        MICHAEL HOFFMAN -- Monday, 8 September 2014, at 6:44 p.m.
        • Re: VERY slow BTS index build
          Art Kagel -- Monday, 8 September 2014, at 7:48 p.m.
          • Re: VERY slow BTS index build
            MICHAEL HOFFMAN -- Monday, 8 September 2014, at 10:26 p.m.
            • Re: VERY slow BTS index build
              Art Kagel -- Tuesday, 9 September 2014, at 6:12 a.m.
          • Re: VERY slow BTS index build
            MICHAEL HOFFMAN -- Monday, 8 September 2014, at 10:36 p.m.
            • Re: VERY slow BTS index build
              Art Kagel -- Tuesday, 9 September 2014, at 6:15 a.m.
              • Re: VERY slow BTS index build
                MICHAEL HOFFMAN -- Tuesday, 9 September 2014, at 11:35 a.m.
                • Re: VERY slow BTS index build
                  Kate Bellsouth -- Tuesday, 9 September 2014, at 12:47 p.m.
                • Re: VERY slow BTS index build
                  Art Kagel -- Tuesday, 9 September 2014, at 2:04 p.m.
            • Re: VERY slow BTS index build
              Kate Bellsouth -- Tuesday, 9 September 2014, at 10:55 a.m.
              • Re: VERY slow BTS index build
                MICHAEL HOFFMAN -- Tuesday, 9 September 2014, at 11:40 a.m.
                • Re: VERY slow BTS index build
                  Kate Bellsouth -- Tuesday, 9 September 2014, at 12:44 p.m.
                  • Re: VERY slow BTS index build
                    Art Kagel -- Tuesday, 9 September 2014, at 2:21 p.m.
            • Re: VERY slow BTS index build
              MICHAEL HOFFMAN -- Tuesday, 9 September 2014, at 1:56 p.m.
              • Re: VERY slow BTS index build
                Mark Ashworth -- Wednesday, 10 September 2014, at 10:18 a.m.
                • Re: VERY slow BTS index build
                  MICHAEL HOFFMAN -- Wednesday, 10 September 2014, at 10:37 a.m.
                  • Re: VERY slow BTS index build
                    Mark Ashworth -- Wednesday, 10 September 2014, at 11:16 a.m.
        • Re: VERY slow BTS index build
          Mark Ashworth -- Tuesday, 9 September 2014, at 11:17 a.m.
          • Re: VERY slow BTS index build
            MICHAEL HOFFMAN -- Tuesday, 9 September 2014, at 11:30 a.m.
  • Re: VERY slow BTS index build -- MEMORY!!!
    MICHAEL HOFFMAN -- Wednesday, 10 September 2014, at 11:26 a.m.

IDS Forum is maintained by Administrator with WebBBS 5.12.