Microsoft KB Archive/271640

= Cannot Crawl a Web Site That Is Secured with Site Server Membership =

Article ID: 271640

Article Last Modified on 10/23/2000

-

APPLIES TO


 * Microsoft Site Server 3.0 Standard Edition

-



This article was previously published under Q271640



SYMPTOMS
Third party crawlers, such as those used by Yahoo, AltaVista and Lycos, cannot index a Microsoft Internet Information Server (IIS) Web site if you use Site Server Membership to secure it.

This problem also applies to Site Server Search. For additional information on this problem in Site Server Search, click the article number below to view the article in the Microsoft Knowledge Base:

258779 Search Cannot Crawl Web Site with Forms or Cookie Authentication



CAUSE
Site Server Membership is an authentication process with various degrees of security: forms, automatic cookie and Distributed Password Authentication (DPA). To browse a Web site that is secured with Site Server Membership, the browser or crawler must be able to provide a suitable level of authentication. If the browser or crawler does not provide this authentication, even if it is as simple as accepting a cookie, Site Server does not allow the browser to view the site.

When you use forms or cookie authentication, the browser must accept a cookie to browse the site. This allows Site Server to identify the user. Because most crawlers do not support cookies, they cannot access the site. In addition, you cannot enable anonymous access to resolve this issue because the browser must still accept a cookie to allow Site Server to identify the user as anonymous.

DPA authentication requires that the browser enter a valid user name and password; therefore, crawlers are not able to gain access unless they provide the correct credentials. Most crawlers do not support DPA.



WORKAROUND
If a browser fails to accept a cookie three times, the browser is forwarded to _mem_bin/nocookie.asp. You can modify this page so that it includes the suitable meta-tags or details about the site. This enables the crawler to record information about the site for its catalog.

Keywords: kbprb KB271640

-

[mailto:TECHNET@MICROSOFT.COM Send feedback to Microsoft]

© Microsoft Corporation. All rights reserved.