Problem with index size related to the number of tapes

Applies to

All Networker5 installations

Problem

Sometimes a very large index is created by Legato, without having a big amount of tapes. The index can grow for hundreds of Megabytes in a normal activity.Normally its recommended to run nsrck -F to verify the indexes and reclaim space. 
The user will notice the following:

    root@ripro1:/ 278 > nsrck -F ripro1

    nsrck: checking index for ripro1

    nsrck: compressing index for ripro1

    nsrck: completed checking 1 client(s)

    root@ripro1:/ 279 >

It was found , that the compression is not done all the time properly.

Especially after the indexes are moved or recovered, there seems to be a problem.

Cause

About nsrck:

Nsrck checks the consistency of of the file indexes. 

Normally, the nsrck program is automatically started by the nsrindexd

program as part of the nsrindexd startup. 

The .nsrck file is locked upon program execution, therefore, only one instance of nsrck can run on the server.

The nsrck program is restartable at any time during its execution. Therefore, it can survive system crashes or exhaustion of resources without losing data.

Index consistency checking is done in up to four phases:

     ¥ Phase zero determines if a client's index is necessary requires further investigation. This phase checks the internal state of the index, and if that state is consistent, avoids further passes. Phase zero also reports index names that appear suspicious (for example, indexes whose names do not map to valid network addresses).

     ¥ Phase one fixes any errors found in the database record file, db, and  rebuilds the b-tree indexes for the database, if necessary.

     ¥ If the -X option is specified, phase two cross checks the client file index with the media database. Records that do not have existing, browsable save set entries are deleted.

     ¥ If the database requires compression, either due to space freed by the previous phases or due to state flagged by a previous run, the index is compressed during phase three.

     Index compression is a two or three step process. First, the records of thedatabase are copied to a temporary database, db.CMP. When that operation completes, a flag file, db.SVC, is created, the old, uncompressed database is removed, and the compressed database is renamed to db.

Finally, the db.SVC file is removed. If there is not enough room on the

filesystem containing the db file to also include the temporary database, nsrck creates a temporary file on another writable filesystem, and stores a  pointer to this file in a file named db.PTR. In this case, an extra copy of the data is required, since the uncompressed database must first be removed before the data can be copied back to the correct place. After all of these steps are completed, the db.PTR file is removed.

     The following example describes the format and options available for the nsrck program:

     nsrck [ -qM ] | [ -T tempdir ] [ -X [ -x percent ]

     | -C | -F | -m ] [ clientname ... ]

     ¥ Use the -C option to force index compression on the named clients, or all clients if none are specified. Other phases of checking are only performed  if an error in a database is detected.

     ¥ Use the -F option to force a check on the listed client names. If no names are given, forced checks are performed for all client indexes. This option forces all phases of index checking. For backward compatibility, the -F option implies index compression, and may be used to force the index to be compressed. This option is typically only necessary when the browse policy is reduced (for example, if the browse policy is changed from 1 year to 6 months

     ¥ Use the -M option to use the nsrck program in master mode (not advised for manual operation). This option advises nsrck that nsrd or another NetWorker daemon invoked nsrck, and will log messages with

timestamps, as well as perform any other behavior expected by nsrd.

     ¥ Use the -m option to force nsrck to check and rebuild the media database b-tree indexes, instead of checking a clientױs online file index.

     ¥ Use the -q option to use nsrck in quiet mode. Quiet mode suppresses all advisory messages.

     ¥ Use the -T option to specify the directory for nsrck to use to hold the  temporary database during compression if there is not enough room in the filesystem containing the db file. If this option is used and there is insufficient space in the specified temporary directory, the nsrck program fails. This argument is ignored if there is sufficient space in the filesystem containing the db file.

     ¥ Use the -X option to tell nsrck to cross-check the save set ids in the index records with save sets found in the media database instead of checking the index databases (unless an error in phase zero occurs). Records that do not correspond to media save sets are discarded. If specific clients are listed, the cross-check is limited to those client indexes.

     Use the -x option to compress a database, after it has been cross-checked, if the databases uses less than the specified percent of the UNIX file. The unused pages are returned back to the filesystem. The default percentage for the -x option is 30.

Workaround/
Permanent Fix 

Check the index size after you ran nsrck -F.

If its still very big (2-3 GB), run the nsrck -C

Check again the index size ,it should be much smaller than before.

Note:Never run the 'nsrck -c' command by using a small 'c' !

This will recreate the index directory and delete all indexes.

Use only the capital 'C' !!!

Technical contacts

Frank Conrad x2723

Date submitted 

September 25, 2000

Last modified

September 25, 2000