# robots.txt file to block robots from traversing various areas of www.corningfcu.org # reference site - http://www.robotstxt.org # --------------------------------------------------------------------------------- # examples # --------------------------------------------------------------------------------- # this entry allows the webcrawler robot to go anywhere; nothing is disallowed # User-agent: webcrawler # Disallow: # this entry allows the lycra robot go nowhere; all directory traversing is disallowed # User-agent: lycra # Disallow: / # --------------------------------------------------------------------------------- # end examples # --------------------------------------------------------------------------------- # all robots, disallowed crawling the /newsletter/ directory User-agent: * Disallow: /newsletter/ Disallow: /ebranchdemo/ Disallow: /disclosures/ebranch-mobile.htm