IDS Forum

Re: Restore + Logical log - caution

Posted By: Martin Fuerderer
Date: Tuesday, 24 January 2012, at 7:59 a.m.

In Response To: Re: Restore + Logical log - caution (Cesar Inacio Martins)

Yes. :)

... For complete restore (including log restore),
all NEEDED logical log backup tapes/files will be
requested by the engine.

Regards, Martin
--
Martin Fuerderer
IBM Informix Development Munich, Germany
Information Management

Read about the Informix Warehouse Accelerator:
http://tinyurl.com/the-iwa-blog

IBM Deutschland Research & Development GmbH
Chairman of the Supervisory Board: Martina Koederitz
Board of Management: Dirk Wittkopp
Corporate Seat: Boeblingen, Germany
Reg.-Gericht: Amtsgericht Stuttgart, HRB 243294

ids-bounces@iiug.org wrote on 01/24/2012 01:43:41 PM:
>
> Hi Martin,
>
> Thanks for your answer.
>
> If I understand right in few words you means:
>
> - The logical logs are included into archive 0 only for physical
> restore... not logical log restore...
>
> And this is just to keep the data integrity when back the database
> online.
>
> And this "saved" logs (inside of archive 0) isn't applicable when
> need to make a complete restore (physical + logical log).
>
> Then all logical logs backup tapes/files will be request by then engine.

>
> I'm Right?
>
> Regards
> Cesar
>
> On 24/1/2012 10:24, Martin Fuerderer wrote:
> > Hi,
> >
> > sorry that it took me quite a while to pick up on this
> > topic ... but here's an attempt to answer some of
> > the questions:
> >
> > - Cesar Inacio described the existing behavior correctly.
> > It is really the case that upon logical log restore the
> > oldest log file to be restored is the one that contains
> > the BEGIN WORK log record of the oldest transaction
> > that is (still) open at the time of the level-0 archive
> > checkpoint. As such, this log file can be much older
> > than the level-0 archive itself.
> > - This is not new behavior. It has been like that since a
> > long time (some say since 1988 and that's most
> > probably correct). At least I confirmed, that a 7.31.UD8
> > from 2004 had exactly this behavior.
> >
> > - For a level-0 archive to be restorable without logical log
> > restore, logical log records needed to make the restore
> > logically consistent are contained within the level-0 archive.
> > This is especially needed for the cases where logical logs
> > are never backed up but only discarded (think of LTAPEDEV
> > in the onconfig file set to /dev/null).
> > These needed logical log records are applied after the
> > physical restore (which is the copying of pages from tape
> > to chunks on disk) when transactions were open at the
> > time of the level-0 archive checkpoint. Applying these
> > logical log records rolls back those open transactions and
> > such makes the restore logically consistent.
> >
> > - If the physical restore of a level-0 archive is to be followed
> > by a logical restore, the space where logical log files reside
> > in the chunks on disk is cleared before doing the logical log
> > restore. This step is done to ensure that no obsolete log
> > record data or otherwise corrupt data is lingering in the log
> > space (in chunks on disk). After this clearing, all the needed
> > logical log files must be restored, including those that contain
> > log records of an open transaction that started before the
> > level-0 archive checkpoint.
> > [ It already has been discussed in this email thread, that
> >
> > these old log files may be needed to roll back the open
> >
> > transactions if they are not committed during the log roll
> >
> > forward. ]
> >
> > - The remaining question now is: Why then is this clearing
> > of the log space so important before starting the log restore?
> > The reason here is the way how the logical roll forward is
> > implemented (and this again is an implementation that always
> > was like this and has not changed recently). The logical roll
> > forward is processing the log records sequentially and after
> > each processed log record it is looking on disk whether there
> > are more log records to be processed. The end of the logical
> > roll forward is reached when the next logical log file page on
> > disk is all zero. And here is the catch: without the clearing of
> > logical log space before the restore of log files, some
> > "garbage data" may still be left over on disk (that also was not
> > overwritten by the physical restore phase). Such "garbage data"
> > may trick the logical roll forward into finding and applying
> > obsolete "stuff" after the last proper log record.
> > There are mainly three scenarios that can ensue when the
> > logical roll forward encounters "garbage data" on disk after the
> > last valid and correct log record:
> > - the log roll forward would recognize the garbage as garbage
> >
> > and correctly end the log roll forward. This is the good case.
> > - the log roll forward would think that there is a valid log
> >
> > record, but when attempting to apply it gets completely
> >
> > corrupted and cannot recover from this. In this case the
> >
> > log roll forward would just stop and leave the system in a
> >
> > hopelessly inconsistent state, making the whole restore a failure.
> > - the log rollforward would apply obsolete log records successfully.
> >
> > This probably would be a rare, but possible case, especially when
> >
> > there were old log files on that part of the disk before the restore.
> >
> > Even though the restore may well finish, this would probably be
> >
> > the worst case, as data has been corrupted without any notice.
> >
> > - As all this is not new behavior but was 'always' like this, there
> > is not much motivation to change it, at least not for the time
> > being. The current implementation is rather stable and reliable
> > (since quite a few years), an important attribute for the
> > backup and restore functionality. Improvements to avoid the
> > need of restoring those old log files would require medium level
> > redesign and implementation of otherwise reliably working code,
> > adding a real risk for introducing new problems and making the
> > code unstable (at least for some time) - a prospect that many
> > customers probably would not be happy with ...
> >
> > - Some years ago we also discussed the possibility of making the
> > checking for the validity of a log page found on disk much
> > stronger than it currently is. Also this would require redesign
> > and reimplementation, but additionally also diminish performance
> > during normal operation. The logging itself would have to create
> > more redundant data, adding it to the log records or the log
> > pages, so that later the checks can be more assuring. More
> > redundant data also means bigger log records and hence
> > more I/O when writing them (to disk). At the time of these
> > discussions, the costs involved were clearly deemed far
> > outweighing the possible benefits.
> >
> > Long response, I know ... :-)
> > But I hope it can clarify some things and give some background
> > info for better understanding ... which hopefully can help with
> > accepting the current implementation. At first sight it may look
> > quite dumb, but after all there is some thinking behind it. :)
> >
> > Regards, Martin
>
>
>
*******************************************************************************

> Forum Note: Use "Reply" to post a response in the discussion forum.
>

Messages In This Thread

• Re: Restore + Logical log - caution
  Martin Fuerderer -- Tuesday, 24 January 2012, at 7:24 a.m.
  • Re: Restore + Logical log - caution
    Cesar Inacio Martins -- Tuesday, 24 January 2012, at 7:43 a.m.
    • Re: Restore + Logical log - caution
      Martin Fuerderer -- Tuesday, 24 January 2012, at 7:59 a.m.

IDS Forum is maintained by Administrator with WebBBS 5.12.