IDS Forum

RE: HPL Conversion (vomitted)... Question

Posted By: Schmitz, Rob B [EQ]
Date: Wednesday, 16 July 2008, at 12:43 p.m.

In Response To: Re: HPL Conversion (vomitted)... Question (MIKE MAGIE)

Mike,

I actually do have a script. It is not intended to cover all situations, just those our group has run into. It serves our needs. You might be able to use it as is or you might be able to start with it and then modify going forward. The script is named hpl_create.ksh and it uses a text file named hpl_create.list. The hpl_create.list file has one line per table. Each line contains the name of the table, the number of devices (unload files), the letter e or d indicating express or deluxe mode, and the root name of the job and device(s). The job name will be the root name with "_job" tacked on the end and the device name will be the root name with "_dvc" tacked on the end.

For example,

mytab 12 e mytab

Will create both a load and an unload job named mytab_job with 12 unload files defined by device name mytab_dvc. The jobs will use express mode.

There are restrictions on using express mode (row size, extended data types, etc.) and the script performs a few rudimentary checks and may change an express specification to deluxe if needed.

The script expects you to pass the instance name, the database name, and the path for the location of your unload files (device path). With all of our scripts, we source a script that sets our environment for a particular instance. Also, what we call an instance is essentially the extent name of the onconfig file and we use it for many things to identify an instance. It is different (usually shorter) than the INFORMIXSERVER value.

The first time a job is created for a table with this script, you will see several errors related to the "delete job" lines in the script. As I say, this is a somewhat crude script that meets our needs and it has never been worth our time to make it squeaky clean.

This script was not written to meet anyone's needs but our own; however, you may find some useful bits in it. (caveat lector)

Here is the text of the script:

-----------------------------------------------------------------------------------------

#!/usr/bin/ksh
#-------------------------------------------------------------------------------
# hpl_create.ksh
#
# Notes on job creation options:
# -flu means create both an unload job and a load job
# adding an "a" means treat data source as a device array
# adding a "c" means deluxe mode
# adding an "N" means it is a deluxe mode without replication
# adding an "n" means no-conversion (this is fastest, must be express)
# Without the "c" option, the mode will default to express
#-------------------------------------------------------------------------------

if [ "$#" -lt 3 ]
then

echo "USAGE: `basename $0` instance database device_path"

exit 1
else

export INST=$1

export DB=$2

export DEVICE_PATH=$3
fi

. ~informix/infx_env $INST

#-------------------------------------------------------------------------------
# Create the jobs
#-------------------------------------------------------------------------------

cat hpl_create.list | tr '{A-Z}' '{a-z}' \
| while read TAB NUM_DEVICES MODE JOBNAME_ROOT
do

echo " unload to hpl_create.tabinfo delimiter ' '

select rowsize

from systables

where tabname = '${TAB}'" \

| dbaccess $DB > /dev/null 2>&1

if [ `cat hpl_create.tabinfo | wc -l` -eq 0 ]

then

echo "Table $TAB does not exist in $DB"

else

export ROWSIZE=`cat hpl_create.tabinfo`

export PAGESIZE=`onstat -b|grep "buffer size"|awk '{print ($(NF-2))}'`

if [ "$ROWSIZE" -ge "$PAGESIZE" -a "$MODE" = "e" ]

then

echo "Changing the mode to Deluxe for $TAB because row size exceeds page size."

export MODE="d"

fi

export JOBNAME=${JOBNAME_ROOT}_job

export DVCNAME=${JOBNAME_ROOT}_dvc

onpladm delete device $DVCNAME

onpladm delete job $JOBNAME -fl -R

onpladm delete job $JOBNAME -fu -R

echo "BEGIN OBJECT DEVICEARRAY ${DVCNAME}" > ${TAB}_device_specfile

while [ "$NUM_DEVICES" -gt 0 ]

do

echo "BEGIN SEQUENCE" >> ${TAB}_device_specfile

echo "TYPE FILE" >> ${TAB}_device_specfile

echo "FILE $DEVICE_PATH/${TAB}.unl${NUM_DEVICES}" >> ${TAB}_device_specfile

echo "TAPEBLOCKSIZE 0" >> ${TAB}_device_specfile

echo "TAPEDEVICESIZE 0" >> ${TAB}_device_specfile

echo "PIPECOMMAND" >> ${TAB}_device_specfile

echo "END SEQUENCE" >> ${TAB}_device_specfile

let NUM_DEVICES=NUM_DEVICES-1

done

echo "END OBJECT" >> ${TAB}_device_specfile

onpladm create object -F ${TAB}_device_specfile

if [ "$MODE" = "d" ]

then

onpladm create job $JOBNAME -d ${DVCNAME} -fluaN -D $DB -t $TAB

else

onpladm create job $JOBNAME -d ${DVCNAME} -flua -n -D $DB -t $TAB

fi

fi
done

#-------------------------------------------------------------------------------
# cleanup the files
#-------------------------------------------------------------------------------

rm -f hpl_create.tabinfo
rm -r *_device_specfile

-----------------------------------------------------------------------------------------

Rob Schmitz
Embarq Data Management
913-534-3474
rob.b.schmitz@embarq.com
www.embarq.com

-----Original Message-----
From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of MIKE MAGIE
Sent: Wednesday, July 16, 2008 10:52 AM
To: ids@iiug.org
Subject: Re: HPL Conversion (vomitted)... Question [12726]

Never mind it's easier than I thought. Since I want delimited unload files I
just need to first create my device array - I use the spec file:

BEGIN OBJECT DEVICEARRAY maya

BEGIN SEQUENCE

TYPE FILE

FILE /users/jmagie/COOL_SCRIPTS/HPL/data/file.unl

TAPEBLOCKSIZE 0

TAPEDEVICESIZE 0

PIPECOMMAND

END SEQUENCE
END OBJECT

Then create my device:

onpladm create object maya.spec

Then create my job (-d device, -D database, -t table, -flu load and unload)

onpladm create job maya -d maya -D maya -t tab1 -flu

And Badda bing badda boom I am good to go...

I have not been drinking...

MM

*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.

Messages In This Thread

• HPL Conversion (vomitted)... Question
  MIKE MAGIE -- Wednesday, 16 July 2008, at 10:17 a.m.
  • Re: HPL Conversion (vomitted)... Question
    MIKE MAGIE -- Wednesday, 16 July 2008, at 11:51 a.m.
    • RE: HPL Conversion (vomitted)... Question
      Schmitz, Rob B [EQ] -- Wednesday, 16 July 2008, at 12:43 p.m.
  • RE: HPL Conversion (vomitted)... Question
    Walter Milan -- Wednesday, 16 July 2008, at 11:59 a.m.
  • Re: HPL Conversion (vomitted)... Question
    John Miller iii -- Wednesday, 16 July 2008, at 12:03 p.m.
    • Re: HPL Conversion (vomitted)... Question
      jack.parker4@verizon.net -- Wednesday, 16 July 2008, at 3:07 p.m.

IDS Forum is maintained by Administrator with WebBBS 5.12.