Using our vulnerability assessment tool (NetClarity) and our penetration tool (Core Impact) we were unable to compromisethe Gatekeeper or the computer behind it.Well duh.
- SC Magazine, April 2007, Pg 63
Both the tools listed are only as strong as their signatures, exploits and platform shellcode. That statement is like running Core Impact against a copy of OpenVMS and saying IMPENETRABLE! when you're done. Technically it's valid but it's no measure of strength.
Maybe these statements are made because of a contractual obligation. "Say our product name five times and we'll give you free copies" sort of thing. Unfortunately there will be InfoSec managers and the like who will listen and wonder if maybe they should use these tools in lieu of hiring security professionals who actually know something.
Maybe I'm just being too overly critical and hypersensitive about this. I don't think I am as I've looked at a number of Web Application Security tools on the market and none of them have been able to find the more serious vulnerabilities vs. a team of two or three highly skilled testers have. We still need good QA but attack Frameworks like CORE Impact, Canvas and Metasploit aren't automated tools. Don't treat them as such.