This article is the first in a series from the BlackBerry Incident Response (IR) team examining critical behaviors that either stop incidents from happening or greatly reduce their impact. Due to the recent trend of high-profile critical vulnerabilities (at times on externally facing devices), these lessons are becoming increasingly important. The following examples have kept defenders particularly busy patching and hopefully checking for signs of compromise:
- Citrix - CVE-2019-19781
- F5 - CVE-2020-5902
- Palo Alto - CVE-2020-2021
- Cisco - CVE-2020-3330, CVE-2020-3323, CVE-2020-3144, CVE-2020-3331, & CVE-2020-3140
- Microsoft - CVE-2020-1350
As IR consultants, we usually become involved in situations that have escalated to the point that in-house teams need additional help — as a result, we see plenty of extreme conditions. Having the ability to investigate the root cause of so many breaches allows us to see key inflection points of incidents that end up being a defining moment, for better or for worse.
Obviously, the best place to stop any attack is before it happens, since this is by far the cheapest and least disruptive option. However, this is not always possible. The primary cause of breaches seen by our incident response team usually stems from one (or more) of three issues: human weakness (phishing), weak external access mechanisms, or unpatched external vulnerabilities. Even if you have a very mature and layered security posture, it is likely that a motivated and well-funded threat actor will eventually gain initial access. All is not lost though, as there are some key mitigations that help stop the majority of attacks or at least limit the damage.
The first mitigation is as old as the first computer bug, but time and time again this is something overlooked by even some large and experienced organizations: Vulnerability management. Vulnerability management is often considered to be “just” patch management, but the complexity and risk can be enormous. After all, the defender needs to patch everything, and the attacker can benefit from finding just one unpatched vulnerability — if it is the right one.
In fact, in our recent increased trend of IR investigations resulting from the Citrix Vulnerability assigned CVE-2019-19781, we noticed that threat actors (TA's) were front loading their exploit efforts. In essence, taking advantage of the publicized vulnerability in an accelerated time frame, not with the intention of carrying out their objectives at the time (due to their own bandwidth limitations), but to install backdoors and revisit at a later time.
It is a race to gain a foothold into the environment before externally facing systems can be patched. Once the backdoor is in place, patching the original vulnerability is not enough to remove the attacker’s access to the environment.
In the case of the most recent F5 (CVE-2020-5902) vulnerability, NCC saw scans on their honeypot less than three days after the vendor disclosed the vulnerability, initial exploitation in under four days, publicly released exploits (including Metasploit) available after only five days, and increased exploit activity after only six days. The timing of this vulnerability was also problematic with the situation rapidly evolving just before and during the July 4th holiday weekend, which may have slowed down the responsiveness of security teams.
There are various recommendations that provide rough timeframes for applying and verifying patches based on vulnerability risk. However, some organizations still operate on the assumption that they are safe until an exploit has been publicly released or systems are being exploited in the wild. We saw this with Citrix CVE-2019-19781, where nearly a month and half after the vulnerability was announced it was estimated that one in five organizations had still not patched: