# This script specifies the files and folders the website that shouldn't # be indexed by search engine spiders. #User-agent field specifies the robot, for example googlebot. In this case all robots User-agent: * #Disallow prevents the robot from indexing/downloading the specified folders Disallow: /_notes/ Disallow: /assets/ Disallow: /csss/ Disallow: /assets/ Disallow: /emediacms/ Disallow: /emediacms/3rdParty/ Disallow: /emediacms/3rdParty/jqueryPlugins/ Disallow: /emediacms/3rdParty/jquery/ Disallow: /emediacms/data/ Disallow: /emediacms/lib/ Disallow: /emediacms/lib/images/ Disallow: /emediacms/lib/languages/ Disallow: /emediacms/plugins/ Disallow: /emediacms/uploads/ Disallow: /includes/ #Disallow prevents the robot from indexing any dynamic urls Disallow: /? Sitemap: http://www.spokes-bicycles.com/sitemap.xml