It’s been an unusual few weeks. Since the massive Sunburst supply chain compromise attacks, which exploited a backdoor in organizations’ SolarWinds Orion network management software, my team’s day-to-day activities have changed: we’ve spent a lot of time doing vulnerability and compromise assessments for companies alongside our usual work of remediating actual breaches and cyber incidents. Naturally, organizations using SolarWinds are concerned that their networks may have been vulnerable or breached.
So we’ve spent a lot of time on calls with companies, walking them through the appropriate steps to find out if they were using the vulnerable versions of the SolarWinds Orion suite and, if they were, helping them to assess if their systems had been compromised and guiding them through the process of removing the backdoor and updating their plans. The good news is that most of our assessments found no breaches.
Then, when this Sunburst-related work was starting to tail off, news of the Hafnium exploits of Microsoft Exchange vulnerabilities broke, launching my team into another round of compromise assessments and helping companies to patch and update their systems. It reminded me of the situation in cyber security five to 10 years ago when web shells were standard.
Back then, good security practice involved finding out which web servers were exposed to the internet, and mitigating risks by regular patching and updates against vulnerabilities, deploying a demilitarised zone (DMZ) between web-facing servers and internal networks, closing ports that were not used, and deploying two-factor authentication (2FA) for admin access to servers. The SolarWinds and Exchange vulnerabilities highlight how relevant those security fundamentals are today.
Journey to the DarkSide
After many compromise assessment calls with companies, you can find yourself thinking that it would be nice to have a cyber incident that you can get your teeth into. Well, be careful what you wish for…
A call comes in from a large organization that’s been hit by ransomware. We find that it’s the relatively new and aggressive DarkSide ransomware, which we see more and more of.
Initially, the attack seemed different from other ransomware variants – the attackers find a way onto the target network, exfiltrate data, deploy the ransomware from a domain controller, and leave instructions for the victim to contact them to negotiate the ransom. But it turned out to be far from a routine ransomware incident.
We spent days working with the customer, trying, again and again, to find any trace of the root cause of the attack while the customer’s IT team recovered its systems and data. But the group behind the attack has anticipated our actions and created a group policy object that makes a scheduled task on all machines to delete event logs every 12 hours.
This means any evidence we could use to trace the attack disappears. The company’s firewall logs don’t last long, either. They are not exported to a SIEM system, so by the time we’ve got to the records, nothing covers the time of the ransomware deployment, let alone the time before the deployment when the attackers were exploring the network.
So we deploy scanning technology to see what we can find. We see many infected machines, PowerShell leftovers, and multiple remote admin tool leftovers – but, unfortunately, these are not clued about what has happened; it’s more like examining the debris after a bomb explosion.
We still have no idea how the attackers got in, where they’ve been on the network, or what they’ve used, let alone anything we can attempt to block, mitigate, or contain.
Finding the enemy within
A couple of days in, we get an urgent phone call from the customer late in the day: they’ve just received a message from the attacker sent via their internal network. S**t!
The attacker can cover their tracks and is either still inside the network or still has remote access. We’re on the phone with the customer until 2:30 am, trawling through logs and firewall alerts to decide what, who, and where to block.
Then, I discovered something new, which gave us a breakthrough. In Microsoft Office365 logs, a DeviceID and the IP address can be searched in the Azure Active Directory to provide a specific machine’s name.
While the IP address is useless as it was of the customer’s datacentre from which the attacker came in, identifying the machine from which the attacker sent the message was the vital clue we needed to enable us to start resolving the incident.
Several days later, we’re still speaking with the customer daily as they find something else in their environment concerning them. This is quite common after an organization has been breached – their IT and security teams are naturally worried that they may have found signs of a new attack, so things can appear suspicious even when they are not.
We suggest the company sets more aggressive firewall rules to block most outbound traffic and only allow what’s necessary for the business. We’ve also recommended they work with a partner organization that delivers a managed security information and event management (SIEM) service to help identify other compromise indicators. Case closed, hopefully – and all because I learned a new trick.
A specialist in incident response (IR) is on the front lines of the ongoing battle against malicious cybercriminals, ransomware, and other threats. The Secret IR Insider works at cyber security services, and solutions supplier Check Point. Their true identity is a mystery.
