# robots.txt file for http://dietchoices.com/ # Allowed: User-agent: Googlebot Disallow: /about/ Disallow: /agreement/ Disallow: /privacy/ User-agent: Mediapartners-Google* Disallow: User-agent: Mediapartners-Google Disallow: User-agent: Googlebot-Image Disallow: /about/ Disallow: /agreement/ Disallow: /privacy/ User-agent: Slurp Disallow: /about/ Disallow: /agreement/ Disallow: /privacy/ User-agent: Yahoo-MMCrawler Disallow: User-agent: YahooSeeker Disallow: User-agent: msnbot Disallow: /about/ Disallow: /agreement/ Disallow: /privacy/ User-agent: MSRBOT Disallow: /about/ Disallow: /agreement/ Disallow: /privacy/ User-agent: Teoma Disallow: /about/ Disallow: /agreement/ Disallow: /privacy/ User-agent: lycos Disallow: /about/ Disallow: /agreement/ Disallow: /privacy/ User-agent: Snappy Disallow: /about/ Disallow: /agreement/ Disallow: /privacy/ User-agent: Gigabot Disallow: /about/ Disallow: /agreement/ Disallow: /privacy/ User-agent: CJNetworkQuality Disallow: User-agent: Browsershots Disallow: # Not allowed (partial listing): User-agent: ia_archiver Disallow: / User-agent: AboutUsBot Disallow: / User-agent: SurveyBot Disallow: / User-agent: OpenISearch Disallow: / User-agent: MSIECrawler Disallow: / # And everybody else: User-agent: * Disallow: /