Home | Contact Us | Submit Troubleshooting Info | Search | Help
Output | Software Solutions | Proofing | Color, Screening and Calibration | Scanning | Digital Photography | Consumables | POS
Home | Contact Us | Submit Troubleshooting Info | Search | Help
Output | Software Solutions | Proofing | Color, Screening and Calibration | Scanning | Digital Photography | Consumables | POS
This is a brief essay addressing three topics:
The entries for the media database are created as the write operation progresses and as information is compiled by nsrmmd. Statistics regarding the volume and location of save sets on the volume are buffered until the save set write is complete. At that time, the information is formatted and passed to nsrmmdbd for inclusion in the media database.
A major change in the NetWorker 5.0 for UNIX design is the redesign of the client file index and media database into a more traditional UNIX structure composed of a directory and files within the directory. In the redesign of the client file index, the single WiSS file per client machine translates into a per-client-machine UNIX directory, /nsr/index/clientname. A volume header file appears in every directory and functions as a directory structure for the files contained within. Within the directory are an unlimited number of files. (Each file is, by default, a 2-gigabyte file but the size of each file in the directory can be controlled through an environment variable.)
The filenames within the client file index directory include several different .suffix endings to identify the file types. The same structure is used for the volume entry in the media database, regardless of media type or family. The volume entry tracks the save sets that reside on the volume. A listing of some of the information contained in a volume entry in the media database follows:
volid, a unique internal volume ID
NetWorker’s client file index and media database management operations are executed by two NetWorker application programs: nsrim and nsrck. The nsrim program enforces the browse and retention policies in the client file index and the media database. After the nsrim program executes, it makes an RPC call to the nsrindexd service, which in turn spawns the nsrck program. It is nsrck that performs index consistency checking and repairs. In addition to maintaining the integrity of the client file index, nsrck also, when appropriate, compresses the index, returning unused space to the server for filesystem use.
The nsrim program executes under any one of four conditions:
Once invoked, nsrim makes three discrete passes through the NetWorker databases:
The nsrim program accesses each entry in the client file index to determine if the entry has exceeded the browse policy for the client. It is possible for one client to have multiple client resources, each with a different value specified for the browse policy attribute. It is therefore possible for a single save set to be subject to two different browse policies. The nsrim program handles issues of precedence according to precise calculations, but the strategy objective can be generalized as follows: nsrim acts conservatively and applies the value that represents the longest browse policy period. (This is also the case when multiple resources for a single client contain different values for the retention policy attribute.)
Any time an administrator changes the browse policy for a client, nsrim enforces the new browse policy on its next pass through the client file index. The new browse policy is applied automatically to existing file entries, but not until the next invocation of nsrim.
All file entries that were restored using the scanner program have a status of "scanned-in" and are exempt from consideration by nsrim.
If nsrim finds an entry that has exceeded its browse policy, nsrim marks it for deletion. When nsrim completes its pass through the client file index, it marks the index for later cross-checking by nsrindexd, NetWorker’s index service.
The nsrim program makes its first pass through the media database to determine if the status of any save set has changed. The entries in the media database are queried by client, enabling nsrim to efficiently apply each client resource’s retention policy to the save sets associated with the client. A save set’s status might change in any of three situations:
The nsrim program makes its second pass through the media database to query all the volume entries to determine if the mode of any volume has changed. A volume’s mode might change under many circumstances. After nsrim completes its second pass through the media database, status messages describing changes are written to the NetWorker messages log.
The nsrindexd service works with the nsrck program to delete entries from the index and to maintain consistency in the event of failure during this procedure. The nsrck program runs against all clients with save sets that have changed state. At the same time that nsrindexd is deleting entries from the client file index, nsrck is performing consistency checking.
The nsrck program is able to use available disk space on disks other than the one where the index directory checking is occurring. However, the free space must be on a local filesystem. If an administrator mounts a scratch disk before running nsrck, the newly mounted filesystem must have a corresponding entry in /etc/fstab for nsrck to use it.
After consistency checking completes, nsrck verifies the client file index utilization. If the value is below the utilization threshold percentage, nsrck automatically compresses the index and returns the reclaimed space to the operating system. Initially, the default utilization threshold percentage is 50 percent. However if an administrator removes the oldest complete backup cycle of a save set from client file index (nsrim -l) or initiates consistency checking (nsrim -X), the utilization threshold percentage changes from 50 percent to 30 percent. To change the percentage explicitly, an administrator executes nsrim -x percentage.
On the UNIX platform, a request to delete the oldest (complete) backup cycle from the client file index can actually cause the client file index to grow. When the "delete oldest cycles" request is issued, nsrim marks all entries in the client file index and the media database until only one full save set backup cycle remains. Because of the way that UNIX manages holey files, this operation can expand the index. The situation is immediately remedied if an administrator directs NetWorker to compress the index and return the space to the operating system by executing the command nsrck -F after the backup cycles are deleted.
Other NetWorker operations also have an effect on both client file index and media database management.
If an administrator invokes nsrmm to purge a save set or a volume from the client file index, nsrmm directs nsrindexd to remove any entries associated with the specified save set or volume from the client file index.
If an administrator invokes nsrmm to delete a volume from the NetWorker system, entries for save sets on the volume and for the volume itself are removed from the media database, and any remaining entries associated with save sets on the volume are deleted from the client file index.
If a client is removed from the NetWorker configuration, the client file index entries remain subject to nsrim operations and the client index file on the server remains until the client index file is itself manually deleted. If the file is not deleted and a client is created with the same name as the removed client, NetWorker will enter new file entries into the existing client index file.
Client file index compression generates temporary files roughly equivalent to the
original size of the index being compressed. For this reason, the nsrck program should not
be run manually until adequate free space on the local filesystem is available. From the
command line, execute the following:
nsrck -F client_name
It is also the case that running this same command as a troubleshooting technique can often resolve issues of index corruption. The complete regime for troubleshooting (index repair and compression) recommended by Legato Technical Support is the following:
nsrck -F client_name
nsrck -X client_name
nsrck -C client_name
Media database compression occurs every 22 days or whenever NetWorker detects the absence of a file included in the database named .comprssd. The .comprssd file ages automatically and every 22 days it is deleted, triggering a media database compression.
Through the information contained in the media database (for save sets) and in the client file index (for files and directories), NetWorker avoids having to read a volume record-by-record to locate a data location. Instead NetWorker maps the save set (or file or directory) to the correct volume, and within the volume, to the sought-for data’s closest media file mark number and record number. Since most devices can search for file marks far more quickly than they can scan records, NetWorker can quickly jump to the area on the volume where the data resides.
For example, when a user makes a recover request, the originating client is already known. A lookup in the client file index provides the data’s offset location within a specific savestream and the save set record in the media database. This information can be mapped to a volume ID, a media file mark, a record number, and a byte offset within the record. Once the volume is mounted, an nsrmmd service directs the device to position itself at the nearest file mark, and the volume is read until the precise byte location is found. The notation chunks added to the volume by nsrmmd during the write operation provide the device with further positioning instructions as needed.
The relationships among the information contained in the user’s initial recover request, the client file index entries, the media database save set entries, and the media database volume entries are sometimes indirect and sometimes iterative.
Against the background of the mechanics behind the consistency check- and repair- tools [nsrim] and [nsrck], it is recommended – in particular when handling large size savesets or numbers of clients – to run those tools proactively (i.e. once pro week) whenever your system management maintenance windows allows it.
Generally the following guidelines can be given when aiming for decreasing inconsistency:
# Make numbers in browse/retention policies as small as possible, meaning 2 decades in stead of 20 years, 2 quarters in stead of 26 weeks.
# The size of the disk where the client indexes are on should be at least twice as big as the largest client index
# If possible on a UNIX server do not put the indexes on a remotely mounted disk. For performance and stability reasons this is not recommended.
# Run a nsrck -F after every system and NetWroker crash.
The status of a save set may actually remain "recoverable" for some period of time after the save set formally exceeds its retention policy. The additional time, above and beyond the retention policy, is a consequence of data dependencies created by the schedule used to create the backup. No save set status will be changed if a younger save set (whose retention policy has not been exceeded) still depends on this save set. For example, the entry for a "full" backup of a save set /paris/finances will not change to "recyclable" if an incremental backup of /paris/finances has not exceeded its retention policy, and still depends on the full in the event of a recover request.