Feedback:

Ability to pack a table while in use

173 votes
Vote 0 votes Vote Vote
Vote
Sign in
Check!
(thinking…)
Reset
or sign in with
  • facebook
  • google
    I agree to the terms of service

    You'll receive a confirmation email with a link to create a password (optional).

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    tmjac2tmjac2 shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

    17 comments

    Sign in
    Check!
    (thinking…)
    Reset
    or sign in with
    • facebook
    • google
      I agree to the terms of service

      You'll receive a confirmation email with a link to create a password (optional).

      Signed in as (Sign out)
      Submitting...
      • AlexAlex commented  ·   ·  Flag as inappropriate

        For sure, pack table could work with #TEMP tables at first place. Because in the moment those #TEMP functionality doesn't looks like complete for everyday work. In my view this functionality was clearly under priced.

      • Lev P.Lev P. commented  ·   ·  Flag as inappropriate

        Could the logic be similar to FLOCK() ? Then any of the following could be performed: PACK, ZAP, REINDEX, etc., followed by an equivalent of UNLOCK.

      • Rodd GrahamRodd Graham commented  ·   ·  Flag as inappropriate

        Should R/W ops be allowed to block during pack?

        If and only if the blocked op eventually succeeds without an error returned to the client AND the pack is implemented to block for no more than a few seconds at a time.

        Since ADS does not currently implement any blocked operation failures at the Sql layer besides long running transactions (we don't do those do we...), most ADS applications are not defensively programmed against Sql failures due to locking/blocking or written to provide user feedback during extraordinary delays. Even if long transactions are used, the code would know which transactions are at risk of failing unlike a PACK block which could occur against the simplest sql operation thereby requiring all operations to be defensive.

      • Rodd GrahamRodd Graham commented  ·   ·  Flag as inappropriate

        If the memo file has inverse pointers to the row data (recnos in the memo header), it should be scannable from bottom to top with a simple online move down compaction. No index maintenance required in this phase.
        Online row data and memo data compaction should be independent with system procedures to do one or both as desired.

        Online indexing is more difficult and schema changes a nightmare.

      • Rodd GrahamRodd Graham commented  ·   ·  Flag as inappropriate

        For packing, consideration might be given to just scanning from both ends of the table and moving rows from the top to the bottom with normal index operations. Assuming that concurrent inserts will use deleted rows first, when the two scans meet, the remainder of the physical table can be truncated in place.

      • Rodd GrahamRodd Graham commented  ·   ·  Flag as inappropriate

        These (online indexing/packing/structure mods) are not 'dangerous' operations as other SQL implementation support them just fine. However they are made more complicated by the physical (non-paged) nature of the underlying ISAM.

      • tmjac2tmjac2 commented  ·   ·  Flag as inappropriate

        I was thinking that you could almost do it with straight SQL script. If you could see the deleted rows, inside a transaction the db could move the data from the last record to the first deleted row. It might mean 1 transaction operation per moved row. It would be slow, but should be fairly safe if a server crash happens. Granted I don't know how this works inside ADS...

      • ma1ma1 commented  ·   ·  Flag as inappropriate

        With this kind of suggestions (pack a table while in use and add indexes to tables while in use) you may risk table integrity and index integrity. Please DO NOT implement these dangerous suggestions!

      • jeremym1234jeremym1234 commented  ·   ·  Flag as inappropriate

        It will be easier, but not as safe. If the server crashes during a backup, you don't lose any data, we store the old record data in memory, not the new data, it is already on disk. You are correct that we certainly can use some of the backup technology to address a live pack, however.

      • jeremym1234jeremym1234 commented  ·   ·  Flag as inappropriate

        Yeah, something like that would be possible. We would need to also try to minimize the potential for data loss.

      • joachimdjoachimd commented  ·   ·  Flag as inappropriate

        I mean something different...maybe I just didn't make it clear enough:
        - select a.*, newfield into newtable from oldtable
        + still allow datamodification on oldtable - but additionally keep track of it!
        - disallow modification on oldtable
        + put modifications into newtable
        + rename oldtable to .bak
        + rename newtable to oldtable
        - allow modifications of updated "oldtable"

      • jeremym1234jeremym1234 commented  ·   ·  Flag as inappropriate

        We could do that, but it would be prone to data loss if the server went down before in-memory changes were flushed. The backup is the opposite; it keeps OLD data in memory when tables are changed during a backup. The backup reads the old data, and if the server goes down it doesn't matter if you lose old data.

      • joachimdjoachimd commented  ·   ·  Flag as inappropriate

        What about using the same procedure as online backup uses? Create the new file(s) in the background and keep data modifications in memory to apply after the new file(s) was/where created. Then have just a short outage whilst renaming the file(s).

      Knowledge Base and Helpdesk