# All robots will spider the domain User-agent: * Disallow: contact.html Disallow: /collection/ Disallow: /devedge/ Disallow: /temp/ Disallow: /avatars/