Knowledge Base

Lingering objects may remain after you bring an out-of-date global catalog server back online

Article ID: 314282

Article Last Modified on 3/29/2007

APPLIES TO

Microsoft Windows 2000 Service Pack 1
Microsoft Windows 2000 Service Pack 2
Microsoft Windows 2000 Service Pack 3
Microsoft Windows 2000 Service Pack 4
Microsoft Windows 2000 Advanced Server
Microsoft Windows 2000 Advanced Server
Microsoft Windows 2000 Advanced Server
Microsoft Windows 2000 Advanced Server
Microsoft Windows 2000 Professional Edition

This article was previously published under Q314282

SYMPTOMS

After you bring back online a domain controller or global catalog server that has been offline for a long time, any of the following problems may occur:

E-mail messages are not delivered to a user whose user object was moved between domains. After you bring the outdated domain controller or global catalog server back online, both instances of the user object appear in the global catalog. Both objects have the same e-mail address, so e-mail messages cannot be delivered.
A user account that no longer exists still appears in the global address list.
A universal group that no longer exists still appears in a user's access token.

These problems may occur if the domain controller or global catalog server has been offline for longer than the value of the Tombstone Lifetime setting.

CAUSE

A domain controller (which may also be a global catalog server) that was offline for longer than the value of the Tombstone Lifetime setting (the default value is 60 days) may contain objects that have been deleted on other domain controllers or global catalog servers. Additionally, tombstones for these objects may no longer exist. When you bring the outdated domain controller back online, it cannot be notified of the object deletions. If any of the objects are modified, they are reactivated in the rest of the domain.

For lingering objects that replicate into read/write naming contexts, the standard behavior (Loose Replication Consistency) is for the receiving domain controller to re-create the objects that are not already present in the local database (DIT). These objects are then replicated back to the originating domain controller, effectively re-creating the deleted objects. If the object should not exist in Active Directory at all (for example, if the object was reintroduced by an outdated domain controller), you can delete the objects with the standard tools (such as ADSIEdit or the Active Directory Users and Computers snap-in).

It is easy to remove lingering objects for read/write naming contexts. This article describes how to remove lingering objects that have already appeared in global catalog (and therefore read-only) naming contexts. For more information about tombstone issues, click the following article number to view the article in the Microsoft Knowledge Base:

216993 Useful shelf life of a system-state backup of Active Directory

For more information about lingering objects in read/write copies of naming contexts, click the following article number to view the article in the Microsoft Knowledge Base:

317097 Lingering objects prevent Active Directory replication from occurring

RESOLUTION

Service pack information

To resolve this problem, obtain the latest service pack for Windows 2000. For more information, click the following article number to view the article in the Microsoft Knowledge Base:

260910 How to obtain the latest Windows 2000 service pack

Hotfix information

Note Before you install this hotfix, read the entire "More Information" section in this article. The "More Information" section contains important information about how to install and use this hotfix.

The English version of this hotfix has the file attributes (or later file attributes) that are listed in the following table. The dates and times for these files are listed in Coordinated Universal Time (UTC). When you view the file information, it is converted to local time. To find the difference between UTC and local time, use the Time Zone tab in the Date and Time item in Control Panel.

   Date         Time   Version            Size    File name
   ------------------------------------------------------------
   16-Jan-2002  22:07  5.0.2195.4685     123,664  Adsldp.dll
   16-Jan-2002  22:07  5.0.2195.4762     130,320  Adsldpc.dll
   16-Jan-2002  22:07  5.0.2195.4016      62,736  Adsmsext.dll
   16-Jan-2002  22:07  5.0.2195.4797     356,112  Advapi32.dll
   16-Jan-2002  22:07  5.0.2195.4797      41,744  Basesrv.dll
   11-Dec-2001  03:33  5.0.2195.4571      82,704  Cmnquery.dll
   16-Jan-2002  22:07  5.0.2195.4141     133,904  Dnsapi.dll
   16-Jan-2002  22:07  5.0.2195.4379      91,408  Dnsrslvr.dll
   11-Dec-2001  03:33  5.0.2195.4534      41,744  Dsfolder.dll
   11-Dec-2001  03:33  5.0.2195.4534     156,944  Dsquery.dll
   11-Dec-2001  03:33  5.0.2195.4574     110,352  Dsuiext.dll
   16-Jan-2002  22:16  5.0.2195.4814     521,488  Instlsa5.dll
   16-Jan-2002  22:07  5.0.2195.4630     145,680  Kdcsvc.dll

   27-Nov-2001  01:33  5.0.2195.4680     199,440  Kerberos.dll
   16-Jan-2002  22:07  5.0.2195.4829     708,880  Kernel32.dll
   04-Sep-2001  17:32  5.0.2195.4276      71,024  Ksecdd.sys
   09-Jan-2002  19:50  5.0.2195.4814     503,568  Lsasrv.dll
   09-Jan-2002  19:50  5.0.2195.4814      33,552  Lsass.exe
   08-Dec-2001  01:05  5.0.2195.4745     107,280  Msv1_0.dll
   16-Jan-2002  22:07  5.0.2195.4594     306,960  Netapi32.dll
   16-Jan-2002  22:07  5.0.2195.4686     359,184  Netlogon.dll
   16-Jan-2002  22:07  5.0.2195.4797     476,432  Ntdll.dll
   16-Jan-2002  22:07  5.0.2195.4827     916,240  Ntdsa.dll
   15-Jan-2002  09:34  5.0.2195.4839   1,688,192  Ntkrnlmp.exe
   15-Jan-2002  09:36  5.0.2195.4839   1,687,744  Ntkrnlpa.exe
   15-Jan-2002  09:36  5.0.2195.4839   1,708,480  Ntkrpamp.exe
   15-Jan-2002  09:34  5.0.2195.4839   1,665,856  Ntoskrnl.exe
   16-Jan-2002  22:07  5.0.2195.4827     388,368  Samsrv.dll
   16-Jan-2002  22:07  5.0.2195.4583     128,784  Scecli.dll
   16-Jan-2002  22:07  5.0.2195.4600     299,792  Scesrv.dll
   16-Jan-2002  22:07  5.0.2195.4600      48,400  W32time.dll
   06-Nov-2001  20:43  5.0.2195.4600      56,592  W32tm.exe
   16-Jan-2002  22:07  5.0.2195.4827     125,712  Wldap32.dll

Note This hotfix has been replaced by a rollup fix. For more information, click the following article number to view the article in the Microsoft Knowledge Base:

326797 Some Windows 2000 Active Directory hotfixes may cause a conflict with S326797 for Windows 2000

STATUS

Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section. This problem was first corrected in Windows 2000 Service Pack 3.

MORE INFORMATION

This hotfix adds support for removing lingering objects. This procedure requires the objectGUID of a domain controller that has a read/write copy of the object, and the objectGUID of the object itself. If you must remove more than one object, determine whether any of the objects are in a parent/child relationship (you can determine this from the objects' distinguished names). If this is the case, order the deletions so that all of the child objects are deleted before their parent objects.

The best way to identify in which domain an object is located (and from that to determine the name of a domain controller that has a read/write copy of the object) is to establish the distinguished name of the object. You can do this by searching for the name (or parts of the name) of the duplicate user, group, or distribution list by using the Ldp.exe tool from the Support Tools:

Start Ldp.exe.
On the Connection menu, click Connect.
Type the name of a global catalog. Type 3268 as the port to which to connect. Click OK.
On the Connection menu, click Bind. Type valid credentials if your current credentials are not sufficient to query all of the global catalog contents. Click OK.
On the View menu, click Tree. Type the distinguished name of the forest root. Click OK.
Right-click the forest root in the tree list, and then click Search.
Create a filter of the following form:

([attribute]=[value])

Substitute appropriate data for [attribute] and [value]. For example, to create a filter to return results where the sAMAccountName attribute has a value that is set to a user account named "testuser", type (sAMAccountName=testuser) in the Filter box. The cn, the userPrincipalName, the sAMAccountName, the name, the mail, and the sn attributes are good candidates for finding a user object. For group objects, use cn, sAMAccountName, or name. Note that you can use asterisks (*) in the [value] field if required.

For more information on Lightweight Directory Access Protocol (LDAP) filter syntax, visit the following Microsoft Web site:

http://msdn2.microsoft.com/en-us/library/aa746475.aspx
Click Subtree as the search scope.
Click Options. In the Search Options dialog box, move to the end of the Attributes control.
Append objectGUID; to the list. Click OK.
Click Run to run the query.
View the results. You must identify which of the displayed objects should be removed from the global catalog. One indication that you have found a bad object is that the object does not exist on a read/write copy of the naming context.
If it is required, rephrase the query and run it again.
If you have identified the lingering object, note its distinguished name and objectGUID.

After you obtain the distinguished name of the object, identify the domain in which it was located by looking at the "dc=" part of the distinguished name. For example, the domain of cn=FirstName LastName,cn=Users,dc=name1,dc=name2,dc=com is name1.name2.com. Next, locate a domain controller for the domain (it can also be a global catalog server).

Run the repadmin /showreps dc-name command (where dc-name is the name of the domain controller you located). Repadmin.exe is included with the Support Tools. From the output, note the domain controller's objectGuid:

C:\>repadmin /showreps some-DC
Your-Site\some-DC
DSA Options : (none)
objectGuid : d1fa2207-ae85-466f-88fd-908f1c623ea7

Install the hotfix that is described in this article on all of the global catalog servers that have lingering objects. The hotfix is not required on domain controllers that you identify as containing read/write copies of the lingering objects, unless they are also global catalog servers that contain lingering objects. Each global catalog server on which you intend to run the delete operation must have network connectivity to the domain controller that you identified.

For few objects

If you have only a few objects and global catalogs, follow these steps to delete the objects by using Ldp.exe:

Log on to each global catalog server that has the hotfix installed (and that contains a copy of the lingering object) by using Enterprise Administrator credentials.
Start Ldp.exe and connect to port 389 on the local domain controller (leave the Server box empty).
On the Connection menu, click Bind. Leave all of the boxes empty (you are already logged on as an Enterprise Administrator).
On the Browse menu, click Modify.
Leave the Dn box empty.
In the Attribute box, type RemoveLingeringObject.
Type <GUID= as the value.
Append the GUID of the domain controller that you obtained from the command repadmin /showreps dcname earlier.

Note In this example, dcname is a domain controller that hosts the writable naming context of the lingering object.
Append > : <GUID=. Do not omit the spaces.
Append the GUID of the lingering object.
Append >.
The complete value should look similar to:

<GUID=85dd0fee-de1b-461c-b9c0-27e9e8249484> : <GUID=eeeb70e5-4501-4895-a572-94a87e8f8ac7>
Click the Replace operation, and then click Enter on the interface. Now the command appears in the Entry list.
Click Run to run the request. The right side of the Ldp.exe window contains the result of the request. It should look similar to this:

***Call Modify...
ldap_modify_s(ld, '(null)',[1] attrs);
Modified "".

For many objects

If you have many objects to delete and many global catalog servers, it may be easier to use the following scripts:

Paste the following text below into a new file named Walkservers.cmd in a new folder:

for /f %%j in (server-list.txt) do walkobjects %%j
Paste the following text into a file named Walkobjects.cmd:

for /f "delims=@" %%i in (object-list.txt) do cscript //NoLogo MODIFYROOTDSE.VBS %1 "%%i" >>update-%1.log

Note This is a single command line. Line breaks are inserted here for readability.

Paste the following text into a file named Modifyrootdse.vbs:

'********************************************************************
'*
'* File:        MODIFYROOTDSE.VBS
'* Created:     January 2002
'* Version:     1.0
'*
'* Main Function: Writes Active Directory information to clean up 
'* objects as per: Q314282.
'* Usage: Modifyrootdse.vbs <TargetServer> <GUID PAIR>
'* Parameter are fed into the script using a pair of batch files.
'*
'* Copyright (C) 2002 Microsoft Corporation
'*
'********************************************************************

OPTION EXPLICIT
ON ERROR RESUME NEXT

Dim objDomain
Dim ObjValue, strServerName, adsLdapPath 
Dim i

'Get the command-line arguments
    if Wscript.arguments.count <> 2 Then
     Print "Invalid Number of Parameters. Use with WalkServers.CMD and WalkObjects.CMD"
    WScript.quit
End If

    strServerName = Wscript.arguments.item(0)
    ObjValue = Wscript.arguments.item(1)

    adsLdapPath = "LDAP://" & strServerName & "/RootDSE"

    Set objDomain = GetObject(adsLdapPath)
    If Err.Number <> 0 Then
        WScript.Echo "Error opening ROOTDSE. Error number is: " & Err.Number & ". Error description is: " & Err.Description & "."
    Set objDomain = Nothing
        WScript.quit
    End If

    objDomain.Put "RemoveLingeringObject", ObjValue
    objDomain.Setinfo

    If Err.Number = 0 Then
       WScript.Echo "Object " & ObjValue & " was removed."
Else
       WScript.Echo "Object " & ObjValue & " could not be removed. Error number is: " & Err.Number & ". Error description is: " & Err.Description & "."
    End If
WScript.Quit

NOTE: If you start Modifyrootdse.vbs manually, make sure to enclose in quotation marks any parameters that contain spaces.

Create a list of all of the global catalog servers that contain the lingering objects. Place the server names in a Server-list.txt file in the same folder. Use the fully qualified domain names to avoid DNS suffix searches.
Add the GUID pairs that you obtained earlier in this procedure to an Object-list.txt file. Add one pair per line. Use the following syntax:

<GUID = DC GUID> : <GUID = object GUID>

A sample entry looks resembles the following:

<GUID=85dd0fee-de1b-461c-b9c0-27e9e8249484> : <GUID=eeeb70e5-4501-4895-a572-94a87e8f8ac7>

Here, the first value is the GUID of the writable domain controller that is used to confirm that the original object no longer exists. The second value is the GUID of the lingering object to be removed.
Run the Walk-servers.cmd file. The scripts generate a log file that is named Update-server-name.log for each global catalog server that is listed in the Server-list.txt file. The log files contain a line for each object that is to be deleted.

Note that errors in the log files do not necessarily indicate a problem because the lingering objects may not exist on all global catalog servers. However, error messages of the form "operation refused" or "operation error" indicate a problem with the GUIDs or with the syntax of the value. If these errors occur, verify these items:

Make sure that the domain controller GUIDs are the correct GUIDs for domain controllers that contain a writable copy of the domain that contains the object.
Make sure that the object GUIDs identify lingering objects in global catalog (read-only) naming contexts.
Verify that the hotfix is installed on all of the domain controllers and global catalog servers that you use in this procedure. Verify that you restarted the servers after you installed the hotfix.

Error message when running Walkservers.cmd to modify many lingering objects in the environment

Object <GUID=ae856ce5-839a-4e44-b2fb-f37082ca2555> : <GUID=514f7510-451a-4297-8129-9b4c8ab79axx> could not be removed. Error number is: -2147016672. Error description is: .

Cause

This error occurs because the script is run against the GUID of a domain controller that does not contain a writeable partition that contains the lingering object. Verify the location of lingering object by the Ldp.exe tool.

Example

In the following example, the lingering object that causes the error message to be removed is located in the corp.company.local domain. However, the <GUID=ae856ce5-839a-4e44-b2fb-f37082ca2555> from the objects-list.txt file is associated with a domain controller in the company.local domain that does not have a writeable partition for corp.company.local.

ldap_search_s(ld, "DC=company,DC=local", 2, "(cn=User*)", attrList,  0, &msg)
Result <0>: (null)
Matched DNs: 
Getting 4 entries:
>> Dn: CN=User\, Joe,OU=Exec,OU=Corporate Users,DC=corp,DC=company,DC=local
    1> canonicalName: corp.company.local/Corporate Users/Exec/User, Joe; 
    1> cn: User, Joe; 
    1> description: CEO; 
    1> displayName: User, Joe; 
    1> distinguishedName: CN=User\, Joe,OU=Exec,OU=Corporate Users,DC=corp,DC=company,DC=local; 
    4> objectClass: top; person; organizationalPerson; user; 
    1> objectGUID: 814226ed-3414-4193-b96d-3a5ea4bf9351; 
    1> name: User, Joe; 
>> Dn: CN=User\, Joe,OU=Migration,DC=corp,DC=company,DC=local
    1> canonicalName: corp.company.local/Migration/User, Joe; 
    1> cn: User, Joe; 
    1> description: Disabled Account; 
    1> displayName: User, Joe; 
    1> distinguishedName: CN=User\, Joe,OU=Migration,DC=corp,DC=company,DC=local; 
    4> objectClass: top; person; organizationalPerson; user; 
    1> objectGUID: 514f7510-451a-4297-8129-9b4c8ab79axx; 
    1> name: User, Joe;

Obtain the GUID of a server in the corp.company.local domain by running the following command:

repadmin /showreps DC-name

In this command, DC-name is a placeholder for the name of a domain controller in the corp.company.local domain. Change the GUID in the Objects-list.txt file to match the GUID of the domain controller in the corp.company.local domain. In this example, the Objects-list.txt file will appear as:

<GUID=c4fd9c30-b433-40a1-a862-9fdf1f804dc8> : <GUID=514f7510-451a-4297-8129-9b4c8ab79a7c>

The first GUID is the GUID of the domain controller in the corp.company.local domain. The second GUID is the GUID of the lingering object from the Lightweight Directory Access Protocol (LDAP) search.

When you run Walk-servers.cmd, the command will now complete successfully without the -2147016672 error.

If you cannot resolve the errors in the log files by using these methods, you may be experiencing a different problem. Contact Microsoft Product Support Services for additional assistance.

For more information about how to obtain a hotfix for Windows 2000 Datacenter Server, click the following article number to view the article in the Microsoft Knowledge Base:

265173 The Datacenter Program and Windows 2000 Datacenter Server product

For more information about how to install multiple hotfixes while restarting only once, click the following article number to view the article in the Microsoft Knowledge Base:

296861 How to install multiple Windows updates or hotfixes with only one reboot

For more information about how to install Windows 2000 and Windows 2000 hotfixes at the same time, click the following article number to view the article in the Microsoft Knowledge Base:

249149 Installing Microsoft Windows 2000 and Windows 2000 hotfixes

Additional query words: kbDirServices

Keywords: kbbug kbfix kbwin2000presp3fix kbqfe kbwin2000sp3fix kbdirservices kbhotfixserver KB314282

Microsoft KB Archive/314282

Contents