# robots.txt - crawler configuration file User-agent: * Disallow: /_includes/ Disallow: /_private/ Disallow: /aspnet_client/ Disallow: /admin/ Disallow: /bin/ Disallow: /cpanel/ Disallow: /css/ Disallow: /database/ Disallow: /images/ Disallow: /js/ Disallow: /webmail/ Sitemap: http://www.site-works.com.au/sitemap.xml