Does anyone know of a link crawler tool that will check for broken links given a domain? I’ve been asked to look for a tool that can access our public facing site as well as our private site (where the link crawler would have to login to the application).
A publicly available tool is : http://validator.w3.org/checklink/
I’m looking for something that we’d host internally and point to our internal and external domains. Anyone know of a good link crawler?
An update: I'm trying Linkcheck, http://search.cpan.org/dist/W3C-LinkChecker/bin/checklink.pod . Its a perl module that so far seems to do the job perfectly! A real nicety is its ability to output the test result into an html document. This will come in handy when sending the test results daily in an email format.