#EXAMPLE METHOD #1.  EXCLUDE BY FILE NAME.
#
#If viewing in Notepad please turn word wrap off for best results
#This is an example robots.txt file
#A robots.txt file should always be placed directly under your server's
#root directory.  Placing it anywhere else is useless.
#
#Tip:  If you use this file as your template, remove all the comments
#      before uploading it.
#
#Tip #2:  To avoid potential discrimination, avoid naming your files 
#or directories where they include the name or initials of the 
#search engine.  Instead, use generic names like "Product1" or
#"Product2", etc. 
#
#
#This subsection will show how to disallow all pages from AltaVista,
#except for MyAltaVistaPage.html which is not included in the list of
#disallowed pages below.  AltaVista's agent name is called "Scooter".
#The idea below is to exclude all pages that do not apply to AltaVista.
#You would of course replace these pages with your own page names.  Your
#list of pages will probably be longer if you have more than one page
#designed for each search engine.  The example below only shows 
#one page per engine. Therefore, you may want to consider 
#excluding files organized by directory to avoid this file getting too
#large.
#Although we only list MyInktomiPage.html below, you could have 
#different pages for every site that uses the Inktomi engine to some 
#extent like MSN, Google, Iwon, etc.  
User-agent: Scooter
Disallow: /MyInfoseekPage.html
Disallow: /MyInktomiPage.html
Disallow: /MyExcitePage.html
Disallow: /MyGooglePage.html
Disallow: /MyLycosPage.html
Disallow: /MyFastWebPage.html
Disallow: /MyDirectHitPage.html

#This subsection will show how to disallow all pages from Infoseek/Go,
#except for the MyInfoseekPage.html which is not included in the list of
#disallowed pages below.
User-agent: Infoseek
Disallow: /MyAltaVistaPage.html
Disallow: /MyInktomiPage.html
Disallow: /MyExcitePage.html
Disallow: /MyGooglePage.html
Disallow: /MyLycosPage.html
Disallow: /MyFastWebPage.html
Disallow: /MyDirectHitPage.html

#This subsection will show how to disallow the Inktomi spider from the 
#non-Inktomi pages below.
#Therefore Hotbot, MSN, Goto, Snap, Looksmart, AOL Search and Iwon will 
#all have access to your Inktomi pages since they are Inktomi based
#engines. 
#We are not aware of a way to handle each Inktomi engine differently
#since they all depend on the same spider.  However, many times  
#a handful of well designed pages can get you ranked well in all 
#the Inktomi engines.
User-agent: Slurp
Disallow: /MyAltaVistaPage.html
Disallow: /MyInfoseekPage.html
Disallow: /MyExcitePage.html
Disallow: /MyGooglePage.html
Disallow: /MyLycosPage.html
Disallow: /MyFastWebPage.html
Disallow: /MyDirectHitPage.html

#This subsection will show how to disallow the Excite spider from the 
#non-Excite/WebCrawler pages below.
#Therefore Excite and WebCrawler will both
#have access to your Excite & WebCrawler pages since they are both
#Excite-based engines.
#We are not aware of a way to handle Excite & WebCrawler differently
#since they both depend on the same spider.  
User-agent: ArchitextSpider
Disallow: /MyAltaVistaPage.html
Disallow: /MyInfoseekPage.html
Disallow: /MyInktomiPage.html
Disallow: /MyGooglePage.html
Disallow: /MyLycosPage.html
Disallow: /MyFastWebPage.html
Disallow: /MyDirectHitPage.html

#This subsection will show how to disallow the Google spider from all 
#non-Google pages.
#Therefore Google, Netscape, and Yahoo Web Pages will 
#all have access to your Google/Netscape/Yahoo files since they are
#Google based engines.
#We are not aware of a way to handle each Google engine differently
#since they all depend on the same spider.  However, many times  
#a handful of well designed pages can get you ranked well in all the 
#Google engines.
User-agent: Googlebot
Disallow: /MyAltaVistaPage.html
Disallow: /MyInfoseekPage.html
Disallow: /MyInktomiPage.html
Disallow: /MyExcitePage.html
Disallow: /MyLycosPage.html
Disallow: /MyFastWebPage.html
Disallow: /MyDirectHitPage.html

#This subsection will show how to disallow all pages from Lycos, except
#for MyLycosPage.html which is not included in the list of disallowed
#pages below.
User-agent: Lycos
Disallow: /MyAltaVistaPage.html
Disallow: /MyInfoseekPage.html
Disallow: /MyInktomiPage.html
Disallow: /MyExcitePage.html
Disallow: /MyGooglePage.html
Disallow: /MyFastWebPage.html
Disallow: /MyDirectHitPage.html

#This subsection will show how to disallow all pages from Fast, except
#for MyFastWebPage.html which is not included in the list of disallowed
#pages below
User-agent: Fast
Disallow: /MyAltaVistaPage.html
Disallow: /MyInfoseekPage.html
Disallow: /MyInktomiPage.html
Disallow: /MyExcitePage.html
Disallow: /MyGooglePage.html
Disallow: /MyLycosPage.html
Disallow: /MyDirectHitPage.html

#This subsection will show how to disallow all pages from DirectHit
#except for MyDirectHitPage.html which is not included in the list of #disallowed pages below
User-agent: Grabber
Disallow: /MyAltaVistaPage.html
Disallow: /MyInfoseekPage.html
Disallow: /MyInktomiPage.html
Disallow: /MyExcitePage.html
Disallow: /MyGooglePage.html
Disallow: /MyLycosPage.html
Disallow: /MyFastPage.html

            **Use your "Back" button**