Microsoft KB Archive/822896

= Exchange Server 2003 data backup and Volume Shadow Copy services =

Article ID: 822896

Article Last Modified on 12/3/2007

-

APPLIES TO

 Microsoft Exchange Server 2003 Enterprise Edition Microsoft Exchange Server 2003 Standard Edition, when used with:  Microsoft Windows Server 2003, 64-Bit Datacenter Edition

 Microsoft Windows Server 2003, Enterprise x64 Edition

 Microsoft Windows Server 2003, Datacenter Edition (32-bit x86)

 Microsoft Windows Server 2003, Enterprise Edition (32-bit x86)</li></ul>

 Microsoft Windows Server 2003, Standard Edition (32-bit x86)</li></ul>

 Microsoft Windows Server 2003, Web Edition</li></ul> </li> Microsoft Windows Small Business Server 2003 Premium Edition</li> Microsoft Windows Small Business Server 2003 Standard Edition</li></ul>

-

<div class="notice_section">

<div class="summary_section">

SUMMARY
The Volume Shadow Copy service feature in Microsoft Windows Server 2003 can be used to create applications that back up and restore Microsoft Exchange Server 2003. The Windows Volume Shadow Copy Service (VSS) provides an infrastructure that enables third-party storage management programs, business programs, and hardware providers to cooperate in creating and managing shadow copies. Solutions based on this infrastructure can use the shadow copies (or mirrored copies) to backup and restore one or more Exchange Server 2003 databases.

The Volume Shadow Copy service coordinates communication between Requestors (backup applications), Writers (applications in Windows services like Exchange Server 2003, and SQL Server 2000), and Providers (system, software or hardware components that create the shadow copies). To use the Volume Shadow Copy service feature to backup Exchange Server 2003, the backup program must include an Exchange Server 2003 aware Volume Shadow Copy service requestor. Because the backup program that is bundled with Windows Server has no such requestor, organizations must use third-party backup applications.

In order to be compliant with Exchange Server 2003, VSS based backup applications must follow three basic requirements to ensure the integrity and recoverability of shadow copy backups. If these requirements are not followed, Microsoft Product Support Services (PSS) will consider the backup solution to be outside of the Exchange VSS framework and will not be able to troubleshoot backup and restore issues. Customers should verify with their backup vendors that the backup application fulfills the Exchange-compliant requirements listed in this Knowledge Base article. Details of the Exchange VSS requirements are presented in the &quot;More Information&quot; section of this article.

As with any third party solution, the vendor of the backup application is your primary support provider for backup and recovery issues. PSS can help you diagnose or analyze issues with the available database and transaction log file sets. However, Microsoft does not troubleshoot or debug third party products. PSS assistance is limited to advice about how to best continue to recover the available database and transaction log files.

For more information about how VSS based solutions are supported by PSS, click the following article number to view the article in the Microsoft Knowledge Base:

841696 Overview of the Microsoft third-party storage software solutions support policy

<div class="moreinformation_section">

The following list describes the Exchange Server 2003 backup with Volume Shadow Copy service process:

 * 1) The backup program (or agent) runs a scheduled job.
 * 2) The Volume Shadow Copy service Requestor in the backup program sends a command to the Volume Shadow Copy service to take a shadow copy of the selected Exchange Server 2003 storage groups.
 * 3) Volume Shadow Copy service communicates with the Exchange Server 2003 Writer to prepare for a snapshot backup.
 * 4) Volume Shadow Copy service communicates with the appropriate storage provider to create a shadow copy of the storage volume or the storage volumes that contain the Exchange Server 2003 storage group or storage groups.
 * 5) Volume Shadow Copy service releases Exchange Server 2003 to resume ordinary operations.
 * 6) The Volume Shadow Copy service Requestor verifies the integrity of the backup set prior to informing Exchange that the backup was successful.

For example, when a shadow copy request is received from an Exchange Server 2003 backup program that has Volume Shadow Copy service support (a requestor), Volume Shadow Copy service communicates with the Exchange Server 2003 Writer to prepare for the snapshot, at this point Exchange Server 2003 prohibits administrative actions against the storage group, checks volume dependencies and suspends all write operations to database and transaction log files while allowing read-only access. Volume Shadow Copy service then communicates with the appropriate storage provider to initiate the shadow copy process for the disk volumes that contain the Exchange Server 2003 data. The shadow copy typically takes couple of seconds which will be practically imperceptible to the end user. When the shadow copy has been taken, Volume Shadow Copy service communicates with the Exchange Server 2003 Writer that Exchange can resume ordinary operations. Backup program verifies the health of the shadow copy prior to informing Exchange that the backup was successful. At the end of a successful backup Exchange truncates the logs and records the time of the last backup for the database or databases.

For more information about Exchange Server 2003 backup with Volume Shadow Copy services, click the following article number to view the article in the Microsoft Knowledge Base:

842066 TechNet Support WebCast: Volume Shadow Copy for Exchange Server 2003

The following list describes the Exchange Server 2003 requirements that any shadow copy backup applications must follow to ensure the integrity and recoverability of Exchange databases:
The list below provides the specific Application event logs that identify if the Exchange requirements are being followed. Backup applications and the Exchange server may log other events associated with the backup and restore process. Confirming that the following events are logged during backup and restore will serve as the verification of compliance with Exchange VSS requirements. Currently, there is no certification program for any third party software solution running on Exchange. Compliance ensures the integrity and recoverability of shadow copy backups but makes no warranty on the performance or reliability of the third party solution. <ol>  Exchange database, transaction log and checkpoint files must be backed up exclusively through the Exchange Writer:

The following application events will be logged if the Exchange Writer is initiated during shadow copy backups. Event Type: Information

Event Source: ESE

Event Category: ShadowCopy

Event ID: 2005

Date: 6/17/2004

Time: 11:40:41 AM

User: N/A

Computer: EXCHSERVER

Description: Information Store (2180) Shadow copy instance 3 starting. This will be a “Backup Type”* shadow copy. * Where “Backup Type” is the type of backup performed (Full, Copy, Incremental or Differential). Event Type: Information

Event Source: MSExchangeIS

Event Category: Exchange VSS Writer

Event ID: 9608

Date: 6/17/2004

Time: 11:40:42 AM

User: N/A

Computer: EXCHSERVER

Description: Exchange VSS Snapshot prepared for Snapshot successfully. Event Type: Information

Event Source: MSExchangeIS

Event Category: Exchange VSS Writer

Event ID: 9610

Date: 6/17/2004

Time: 11:40:43 AM

User: N/A

Computer: EXCHSERVER

Description: Exchange VSS Snapshot has frozen the storage groups successfully. Event Type: Information

Event Source: MSExchangeIS

Event Category: Exchange VSS Writer

Event ID: 9612

Date: 6/17/2004

Time: 11:40:44 AM

User: N/A

Computer: EXCHSERVER

Description: Exchange VSS Snapshot has thawed the storage groups successfully. </li> The backup application must validate the integrity of the shadow copy backup set.

Microsoft recommends, but does not require, that this be done before the backup application notifies Exchange that backup has completed. The reason for this recommendation is that Exchange performs two important tasks after a successful backup:  Exchange updates the headers of the backed up databases to reflect the last successful backup time</li> Exchange removes (“prunes”) transaction logs from the server that are no longer required to roll forward from the last successful backup.</li></ul>

If a backup application postpones integrity verification until after these tasks have completed, then special care must be taken to preserve the last verified backup along with copies of all log files required by that backup. Although the backup may have already been reported as successful to Exchange, the backup should not be relied on until the backup application has actually completed integrity verification.

Please refer to the &quot;How to Perform Integrity Verification for VSS backups&quot; section of this article for complete details about how to perform the integrity checks and how to determine which database and transaction log files must be preserved until integrity verification has finished.</li>  Restores to original location** must be done exclusively with the Exchange Writer

Exchange Writer will log following events in the Application Event logs during a shadow copy restore process. Event Type: Information

Event Source: MSExchangeIS

Event Category: Exchange VSS Writer

Event ID: 9620

Date: 6/17/2004

Time: 1:49:59 PM

User: N/A

Computer: EXCHSERVER

Description: Exchange VSS Snapshot has processed pre-restore event successfully Event Type: Information

Event Source: MSExchangeIS

Event Category: Exchange VSS Writer

Event ID: 9618

Date: 6/17/2004

Time: 1:59:46 PM

User: N/A

Computer: EXCHSERVER

Description: Exchange VSS Snapshot has processed post-restore event successfully </li></ol>


 * &quot;Original location&quot; means an Exchange computer with the same server name and the same file path as the Exchange computer where the VSS backup was taken.

Restores to alternate locations cannot be achieved via the Exchange Writer as of Exchange Server 2003 SP1. VSS based backup applications can choose to provide manual or other programmatic methods to restore shadow copies of Exchange databases to alternate locations.

How to perform integrity verification for VSS backups
When a database is backed up using the Exchange streaming backup API, each page in the database is read in turn, and the checksum integrity of each page is verified during the backup process. The checksum integrity of transaction log files is also checked before they are backed up.

During a VSS backup, there is no opportunity for Exchange to read each database file in its entirety and to verify its checksum integrity. Therefore, database and transaction log file integrity must be verified by the backup application. This can be accomplished by running Eseutil as described at the end of this document.

If you do not checksum-verify your VSS backups, it is possible that a damaged page could remain undetected in the database and eventually become present in all existing backups. The only way to recover from this circumstance is to repair the database. Database repair will require extensive downtime and will result in at least some data loss (at least the loss of the data that was on the damaged pages).

However, if your last VSS backup has been verified to contain all good pages, you can purge damaged pages from the database by restoring the verified backup and rolling it forward with transaction logs created since the backup was taken. The downtime required to do this will typically be much shorter than for a database repair, and this method of recovery can correct the database problems with zero data loss.

Thus, you should not consider a VSS backup to be good until all the files in it have been checksum-verified.

You should follow the two rules below for verifying backup integrity:
 * For database files: You must always keep an integrity-verified copy of the database files available. An integrity-verified backup is one in which page checksum verification has completed on the database file(s) in the backup set.

A recently backed up database that has not yet passed checksum verification cannot be considered a valid backup. You must not discard a previous verified backup until you have verified the integrity of the current backup.
 * For transaction log files: All transaction log files required for recovery of the most recent integrity-verified database backup must also be backed up and their checksum-level integrity also verified.

These transaction logs include, at minimum, the inclusive range of log files listed in the Log Required field in the header of each database in the last verified backup. If these log files are not available, the database will not be mountable after restoration.

Important This requirement applies to the last integrity-verified backup, not to the most recently performed backup. Until the most recent backup has passed checksum verification, it is not considered a valid backup.

Optionally, you may also preserve the additional logs required to completely roll the database forward after restoration of a database backup. These are all the transaction logs in unbroken sequence starting with the lowest Log Required file up to the most recently created transaction log that has been purged from the Exchange server. Detailed examples and explanations of what this means are provided below.

Preserving transaction logs beyond those listed in the Log Required range(s) is optional, in the sense that doing so is not strictly required in order to successfully restore and mount a backed up database. However, if you do not preserve all these logs, then restoring from backup will cause you to lose all changes in the database past the point of backup. Microsoft strongly recommends that you not only preserve the transaction logs required to restore and mount a backed up database, but also all subsequent transaction logs needed to roll the database forward without data loss.

Determining which transaction log files are required
If an Exchange database is backed up while it is online, at least one transaction log file will always be backed up with it. This is regardless of whether you use the streaming backup API or the VSS backup API.

After restoration of an online backup, information from the transaction logs must be applied to the database (&quot;replayed&quot;) before the database will be mountable again. The Log Required field of each database header records the sequence (generation) numbers of the range of transaction log files that must be replayed into the database.

If the Log Required field reads 0-0, this means that the database is mountable without having to replay any additional transaction log data. The only time the Log Required value will be 0-0 is after a database is brought into a clean-shutdown state. While a database is running, the Log Required field always records the range of transaction logs that have not yet been applied to the database. This range is updated continually.

A database backed up online will always have a non-zero Log Required range, and these logs must be backed up along with the database. If, after restoration, these logs are not available, the database will not be mountable. (You can repair the database if the necessary logs cannot be found, but there is no guarantee that repair will be successful, and repair will almost always result in some level of data loss, even if only the data in the missing logs.)

If you use the Exchange streaming backup API or the VSS backup API contained in the Exchange VSS writer, then required log files needed to mount a database will be backed up automatically with the database. If you replay only the Log Required files, this will result in the database being restored to the point in time at which the backup finished. If you wish to roll the database forward past that point, you must also play the log files generated after the backup was done.

In order to completely roll the database forward from any particular backup, you must preserve all the log files in unbroken sequence starting from the lowest log in the Log Required range up to the most recently generated log file in the database's storage group. If any log in this series is missing or damaged, you will only be able to roll forward up to the point of the last good log before the missing or damaged file.

Therefore, if you wish to recover from backup with no data loss, it is essential that you maintain good copies of all transaction log files going forward from your last verified good database backup.

Transaction log pruning
If transaction logs are not removed from an Exchange server, they will continue to accumulate until they fill all available disk space. Therefore, both the streaming and VSS backup APIs support &quot;pruning&quot; of transaction log files after completion of a Normal or Incremental backup. Log files older than those needed to recover the most recent backup are automatically deleted from the server after the backup applications signals Exchange that backup has completed successfully.

With the streaming API, checksum verification of the database is done during the backup process. By the time a backup completes, the entire database and the required log files have been checked for physical integrity. With the VSS API, checksum verification cannot be done as part of the actual backup process. The vendor must verify the physical integrity of the database independently of the backup process. This can be done with Eseutil either before or after signaling Exchange that the backup has completed.

If checksum verification is done before backup completes, and a problem is found in the backup set, then Exchange can be informed that the backup was unsuccessful. Doing this will prevent Exchange from pruning log files from the server.

If checksum verification is postponed until after signaling backup completion, then Exchange will delete older log files from the server. Some of these log files may have been needed for rolling forward from a previous good backup. If you have not already made copies of these logs, then you will not be able to roll forward completely.

Microsoft therefore recommends, but does not require, that checksum verification be performed on a VSS backup before the backup application signals backup completion to Exchange. If checksum verification is postponed until after the backup has already completed, then the backup application must ensure that copies are made of all transaction log files that were pruned from the server, unless rolling forward completely is not important to you.

In most cases, all transaction logs needed to roll forward a VSS backup will be available in the set of log files saved with the previous backup along with those saved with the current backup. However, customers should verify that this is the case when considering a particular vendor.

Restoring unverified backups
There may be cases where a disaster requiring restoration occurs before checksum verification has completed on a recent backup. In such cases, Microsoft recommends that you restore a previous verified backup and roll that backup forward rather than relying on an unverified backup.

However, you may have service level agreements that require you to restore data more quickly than can be done from the previous backup. In these cases, restoration of the unverified backup may be a better option, as long as you still maintain a previous verified backup and all the log files needed to roll forward completely from it. If you meet these requirements, then you will still be able to roll forward from a known good backup in case the last backup is discovered to be bad.

How to check snapshot consistency
VSS Requestor must check snapshot consistency by running Eseutil.exe against the database and log files using the appropriate options as in the following table. VSS Requestor must verify that all the exit ERRORLEVELs that are returned are non-negative. To see the ERRORLEVEL at the command line, type echo %errorlevel% after Eseutil.exe finishes running. A negative ERRORLEVEL means that there is corruption in the files. Before VSS Requestor calls BackupComplete, VSS Requestor must make sure that the backup component's status in the Backup Component Document reflects the result of the consistency check. That is, the backup component's status should be FALSE if any corruption is found, and it should be TRUE if no corruption is found. Verification of snapshot consistency is a mandatory requirement for the solution to be supported by the Exchange team.

The following table shows the combination of integrity checks for each backup type.

Due to the snap nature of VSS backups, JET does not get the opportunity to touch all the pages to do the necessary consistency checks. Therefore, it is VSS Requestor's responsibility to ensure snapshot consistency.

<pre class="fixed_text">Checkpoint: (0x20,9D,187) where 0x20 is the log generation number of the checkpoint log file.
 * All the log files with log file generation number equal to or greater than the checkpoint log file are required to recover a snapshot database. If exists the current log file (Enn.log) is also required for database recovery. If any of the required log files fail the consistency check, Requester must ensure that the status of the backup component is set to FALSE prior to calling BackupComplete. To determine the checkpoint log file, run eseutil.exe against the snapshot checkpoint file and parse the output for &quot;Checkpoint:&quot; For example, &quot;c:\eseutil.exe /mk E01.chk&quot; shows the following:

In this example, any log files, including E0100020.log and above, must be non-corrupt to recover the snapshot database even if the database itself had already passed the physical consistency check.


 * All log files in an Incremental or Differential backup set are required for database recovery. Consistency of a whole log sequence can be checked by running eseutil against the log file prefix. For example, “eseutil /k E01” will run consistency checks against all files of the form E01xxxxx.log on a given path.

Keywords: kbtshoot KB822896

-

[mailto:TECHNET@MICROSOFT.COM Send feedback to Microsoft]

© Microsoft Corporation. All rights reserved.