Microsoft KB Archive/314892

= INF: How to Allow Search Crawlers to Index the MSIB Web Site =

Article ID: 314892

Article Last Modified on 2/23/2007

-

APPLIES TO


 * Microsoft Content Management Server 2001 Enterprise Edition
 * Microsoft Commerce Server 2000 Standard Edition

-



This article was previously published under Q314892



SUMMARY
Microsoft Solution for Internet Business (MSIB)-based Web sites are programmatically configured to prohibit external search engines from crawling (searching) the site, because a given site may contain sensitive documents. If you want to allow external search engines to crawl your MSIB-based site, you must index your site and then change the settings for the Content Management Server content on your site.



MORE INFORMATION
Use either of the following methods to configure your MSIB-based Web site:  Configure the site on a posting-by-posting basis by using the Site Builder program.

You can do this manually in the Site Builder Channels program. Right-click each posting, and then select Properties. On the Publishing tab, click to select the Web Robots Can Crawl Links and Web Robots Can Index This Posting check boxes. Note that when you configure settings with this method, the settings are not inherited by new postings, and therefore it is necessary to repeat this process for new postings.

-or-  Use code to configure the entire site.

Properties set by this script are inherited by all existing postings in the channel, as well as by any new postings. Save the script in a template of your choice, and then run the script to change properties for all channels and postings in a single Web site, as follows: <%

On Error Resume Next

Dim strThisURL, pChannels

' Check if user has right to edit and approve the site. If Not Autosession.UserCanModifySite Or Not Autosession.UserCanApprove Then Call Autosession.ManualLogin(&quot;Please log-in as administrator.&quot;, &quot;/ChannelAllowRobot.asp&quot;) End If

strThisURL = Request.ServerVariables(&quot;URL&quot;) ' Check that we're in update mode If Not Autosession.isModeUpdate then ' someone has called this ASP page directly. Redirect them back to this page in update mode. ' NOTE: you cannot switch modes mid-page, therefore we must redirect back to this page. Response.Redirect( strThisURL & &quot;?&quot; & autosession.rootchannel.queryStringModeUpdate) End If

Set pChannels = Autosession.RootChannel.Channels

Response.Write &quot;Changing all channels and postings to allow robot following and indexing... &quot;

Call SetChannelsToAllowRobot(pChannels)

If Err.Number = 0 Then Call Autosession.CommitAll Response.Write &quot;All Changes are committed successfully.&quot; Else Call Autosession.RollbackAll Response.Write &quot;Error &quot; & Err & &quot;: &quot; & Err.Description End If

Set pChannels = Nothing

'================================================ Sub SetChannelsToAllowRobot(ByRef pChannels)

Dim pChannel On Error Resume Next

For Each pChannel In pChannels

pChannel.IsRobotFollowable = true pChannel.IsRobotIndexable = true Response.Write &quot;Channel &quot;&quot;&quot; & pChannel.Name & &quot;&quot;&quot;&quot; Response.Write &quot; IsRobotFollowable=&quot; & pChannel.IsRobotFollowable Response.Write &quot; IsRobotIndexable=&quot; & pChannel.IsRobotIndexable Response.Write &quot; &quot; If pChannel.Postings.Count > 0 Then Call SetPostingsToAllowRobot(pChannel.Postings) End If       If pChannel.Channels.Count > 0 Then Call SetChannelsToAllowRobot(pChannel.Channels) End If

Next

End Sub

'================================================ Sub SetPostingsToAllowRobot(ByRef pPostings) Dim pPosting

On Error Resume Next

For Each pPosting In pPostings

pPosting.IsRobotFollowable = true pPosting.IsRobotIndexable = true pPosting.Approve Response.Write &quot;&nbsp &nbsp Posting &quot;&quot;&quot; & pPosting.Name & &quot;&quot;&quot;&quot; Response.Write &quot; IsRobotFollowable=&quot; & pPosting.IsRobotFollowable Response.Write &quot; IsRobotIndexable=&quot; & pPosting.IsRobotIndexable Response.Write &quot; &quot; Next

End Sub

%>                   

Additional query words: MSO

Keywords: kbhowto kbinfo KB314892

-

[mailto:TECHNET@MICROSOFT.COM Send feedback to Microsoft]

© Microsoft Corporation. All rights reserved.