# All robots will spider the domain User-agent: * Disallow: # Disallow directory /webcalendar/ User-agent: * Disallow: /webcalendar/ # Disallow directory /adserver/ User-agent: * Disallow: /adserver/ # Disallow directory /traffic/ User-agent: * Disallow: /traffic/ # Disallow directory /webalizer/ User-agent: * Disallow: /webalizer/