# /robots.txt file for http://webcrawler.com/ # mail contact@budgetharrisburg.com for constructive criticism User-agent: webcrawler Disallow: User-agent: * Disallow: /Admin Disallow: /admin Disallow: /DB Disallow: /db Disallow: /db2 Disallow: /Images Disallow: /images Disallow: /Include Disallow: /include Disallow: /JS Disallow: /js Disallow: /Old Disallow: /old Disallow: /logs Disallow: /Styles Disallow: /styles