When to conduct penetration tests vs. vulnerability assessments
FireOak’s recommendations for Clients
“Penetration tests” and “vulnerability assessments” are phrases that are often used interchangeably, but they are different things. They both have a place, but for most of our clients, there’s a clear sequence — and not all clients benefit from all of these types of evaluations.
At FireOak Strategies, we almost always recommend that our clients start with a vulnerability assessment, remediate all of the vulnerabilities uncovered in the vulnerability assessment, and then, as an optional next step if there is a specific need, hire a different company to conduct a penetration test.
Here’s Why
Vulnerability assessments come in two main flavors: automated (black box) scans and human-powered (white box) assessments.
Automated Vulnerability Scans
Automated vulnerability scans rely on an automated tool such as Nessus or Qualys to scan a website from an outside perspective. Sometimes this type of vulnerability test is referred to as a “black box” assessment because it does not require any type of special access to a site. Results from this type of assessment are often quite limited in nature and are typically in the form of a hundred-page automated report produced by the scanning tool.
Organizations that run complex sites, have the internal capacity to run these tools on a regular basis, and are in a position to decipher and act upon the results should consider incorporating running a vulnerability scan on their sites on a monthly or quarterly basis.
Human-Powered Vulnerability Assessments
For most of our clients, FireOak conducts human-powered “white box” assessments. Instead of relying exclusively on an outside view of the site, we fully examine all of the components of your website from the inside out. From this point of view, we can get a complete view of the entire ecosystem related to your site, including how other systems might be impacting your site’s security.
For example, if your site is hosted on a server along with other sites, and one of those sites is compromised, it could potentially impact your site. This is the type of vulnerability that an automated scan can’t detect.
For these sites, we’ll use some automated scanning tools, but these are complemented by having humans conduct a series of hands-on testing of various components of your site in our lab, in such a way that it won’t impact your site but gives us the ability to identify relevant vulnerabilities, explain the risks associated with each finding, and recommend appropriate ways to address each finding. We’ll also carefully comb through your site to surface findings that automated tools simply can’t detect. We’ll bring these to light and explain the ramifications of each finding from a security perspective.
This approach allows us to give you the most comprehensive assessment of your site’s security.
A vulnerability assessment is only as good as what happens next. After the assessment, your organization needs to take action to address the vulnerabilities that have been identified.
After all of the identified vulnerabilities have been remediated should you consider hiring a firm to conduct a penetration test.
Penetration Tests
A penetration test is designed to break into your site via whatever means necessary; it is an intentional attack on your systems in order to identify vulnerabilities that an attacker could leverage, so these vulnerabilities can be remediated in a proactive way.
As a result, penetration tests can be destructive in nature. They usually have limited rules, so pentesting can involve breaking in through social engineering attacks, phishing attacks, physical security, or by exploiting a weakness in any other system in place in your organization that can be used as a jumping off point to access a website. By their very nature, all systems are in scope for a penetration test, since the purpose is to try to break into an organization. Pentests can be extremely costly and often include traveling on-site to the client’s location. The exercise can be disruptive to staff, and the outcomes can be overwhelming, expensive, and difficult to try to implement all at once.
FireOak’s Recommended Approach for Penetration Tests vs. Vulnerability Assessments
Instead of starting with a pentest, in nearly all cases, we recommend conducting a hands-on, human-powered vulnerability assessment first, which examines all of the technical components in scope for a particular assessment. If your initial focus is on your organization’s website, focus on that. Once the website vulnerability assessment is complete, get the list of recommendations and remediate them.
Then conduct a broader information security assessment for your organization — conduct a deep dive into how Microsoft 365, Google, your Salesforce org, etc. have been configured — and address those issues. Or decide that the time is right to look at your organization’s approach to security in a holistic manner and examine the security of the organization’s full technical stack, information/data governance policies, and security procedures/operations.
Only after you’ve gone through these types of non-destructive external assessments, have remediated all known vulnerabilities, and have a mature information security program in place should an organization consider conducting a penetration test.
As for who should conduct a pentest: it absolutely should not be the same organization as the one who conducted a vulnerability assessment. If we conduct the vulnerability assessment, we know exactly what vulnerabilities are in place and would hone in on those issues. Instead, have another firm conduct the pentest, one with zero prior knowledge of your site, organization, policies, procedures, and security operations.
Our goal is to help organizations strengthen their approach to security, and we want organizations to be successful. As a result, in most cases, a penetration test isn’t the right place to start — it can create unintended consequences for an organization that is in the early stages of strengthening its overall approach to security. So when it comes to penetration tests vs. vulnerability assessments, start with a vulnerability assessment.
Read more:
