By Ron Murch, I.S.P., ITCP – Haskayne School of Business, University of Calgary.
On December 13, FireEye published a report about how it had detected cyberattack activity at multiple entities worldwide. They traced the attack to a vulnerability in the digitally-signed component of the Orion software framework made by SolarWinds Corporation. They reported the attack and contacted SolarWinds. The resulting investigation has determined that the vulnerability had been inserted into the Orion software several months earlier – around March 2020 – and resulted in compromised administrator credentials on systems using the infected software. Other cybersecurity firms have also commented – an excellent article was published on December 15 by Remediant.
This was clearly not the work of amateurs. Early news reports indicated that the US Treasury and the departments of homeland security, state, defence, and commerce were attacked along with a variety of private companies. SolarWinds reported that 18,000 customers downloaded a compromised patch from its website which, if applied, could make the customers’ Orion installation vulnerable. This attack demonstrates just how vulnerable any organization is to the software supply chain it relies on. It was especially concerning because the Orion software is used by SolarWinds’ customers in their supply chain processes and would have ramifications for other parties in that supply chain.
On December 16, a number of Canada’s top Chief Information Security Officers gathered on-line to discuss how they were affected by this revelation and to share their knowledge and the steps they were taking to address it.
The session started with an experienced cybersecurity professional summarizing what was known at this point and how further investigation was proceeding. (Several good web references are listed below.) The compromise was described as a “classic espionage” initiative. It was sophisticated and far reaching. It appeared to be a state-sponsored initiative that demonstrated highly competent and technical operational security to disguise the attack and was deliberately designed to use SolarWinds’ Orion platform as a vector to attack other specifically-targeted organizations. The attackers’ tools were made to cover or hide its activities and make attribution difficult. The attack was carried out over a long period of time before detection. The malware was designed to high-grade targeted networks based on characteristics like industry and enterprise size, and was a targeted, not a blanket attack. It was clear that the “bad actors” still had access to at least some of their compromised targets as companies reported malicious activity in December.
In the true spirit of “Never waste a good crisis”, the CISOs used this discussion session to deal with questions of “How prepared were we?” and “How did we respond and let our organizations and partner organizations know?”. In addition to addressing this situation responsibly, they all were focused on learning from each other and improving for the next situation.
The underlying question of attribution has not yet been clearly determined. However, from the tools and techniques used, and from speculating on the possible motivations, it appears to be either hackers-for-hire or a deliberate, state-sponsored attack.
The CISOs explored several key questions:
1. Is there any feedback from Interpol? It’s still early and no one had seen anything from Interpol or other law enforcement agencies yet. They are probably working with other policing and government cybersecurity organizations. In Canada, the Canadian Centre for Cyber Security is most likely the lead organization and is working collaboratively with the RCMP.
2. Has the SolarWinds supply chain been resecured? Do we still have to rely on faith or trust, or can confidence be clearly demonstrated? SolarWinds has acted quite openly and responsibly and they are a strong, competent company. That said, due diligence by their customers continues to be important. Open communication is also clearly important – at all levels of the organization, it has to be “all hands on deck” for clear communication.
3. Can we confirm that the malicious software was injected during the software patching processes? This is an excellent example of a compromised tool that is generally used with implicit trust. It was compromised by subverting the update certificates upon which customers placed trust so that the updates appeared to be genuine. This type of threat – that a security tool can itself be compromised – is not often in an organization’s risk registry. Risk introduced by reliance upon 3rd party suppliers should always be considered, and this also applies to cloud services and platforms. As Canadians, we necessarily are concerned with data residency and sovereignty; we also need to consider this type of supply-chain risk. It is also important to consider your own organization’s full supply chain as well as your role in your clients’ or customers’ supply chains. You may represent a risk to them. This was illustrated by discussion from one CISO who indicated that he is working closely with two of his company’s suppliers to ensure appropriate safeguards are in place in their tightly-integrated operations. A large professional services firm indicated that they were deliberately analyzing their role with their clients as they are part of their clients’ supply chains in a variety of ways. Responsible disclosure is important to maintaining a healthy professional relationship.
4. Did this trigger anyone’s Incident Response Plans? There was broad agreement that initially this disclosure by FireEye and SolarWinds should be treated as a full-on incident and should trigger an organization’s Incident Response Plan. One firm is using their approach to help to mature their incident response processes. Another activated their “war room” as a readiness precaution. A third indicated they were exploring how best to respond to future incidents – designate an incident commander, a scribe, formal debrief processes, and deliberately challenge the learning from their response policies and activities. Several participants commented that the triage of this incident is not as straightforward as incidents involving system shutdowns, ransomware, or exfiltration of information. This one was sustained, and not all infected systems were victims of exfiltration. Most of the CISOs were still reviewing logs and systems to determine how deeply the malware had penetrated, and whether any exfiltration or other malware installation vectors were invoked. The main approach after day 4 (after notice from SolarWinds) was “threat hunting” and log-reviewing.
5. Can we confirm that March 8 is the start date? It is clear that March 8 was the date that the first “poisoned patch” for Orion was released to SolarWinds’ customers, and 18,000 customers downloaded it. However, it is also clear that there was a lot of preparatory work that the hackers would need to do to reach that point. Perhaps the most exposed threat will be to Active Directory services in customer environments.
6. Is it possible to propagate beyond the 18,000 organizations initially infected? Yes. In this case the digital signature keys were also compromised. It is impossible to state that the current SolarWinds attack cannot propagate from victim to victim, although that has not been detected. The techniques and browser plug-ins incorporating some of the attack’s ideas can be sold to other hackers and those can then be used to infect a previously-trusted plug-in. Thus the need for customer due diligence was further emphasized.
As the discussion concluded, one final piece of caution was offered. Normally, for a threat to be effective, it has to attack at least two or three of the “pillars” – people, process, technology, infrastructure, and partners. This attack appears to have successfully done that!
For more information contact:
Chief Information Security Officer
403 209 4879
Follow-up articles re this incident:
(Click on the title to follow the link)
- December 13 – Highly Evasive Attacker Leverages SolarWinds Supply Chain to Compromise Multiple Global Victims With SUNBURST Backdoor
- December 14 – Microsoft, FireEye confirm SolarWinds supply chain attack
- December 14 – US government agencies compromised by foreign nation-state
- December 14 – FireEye Discovered SolarWinds Breach While Probing Own Hack
- December 14 – SolarWinds Orion and UNC2452 – Summary and Recommendations
- December 15 – The Role of Admin Credentials in the SolarWinds Attack
- December 15 – SolarWinds Orion: More US government agencies hacked
- December 16 – Microsoft AV now blocks applications with malicious SolarWinds binaries
- December 17 – SolarWinds FAQ: Security Advisory
- December 17 – Senate Intelligence Committee Member On Latest Hacks In Government Agencies
- December 21 – A second hacking group has targeted SolarWinds systems
- December 21 – Partial lists of organizations infected with Sunburst malware released online
- December 22 – Infosec pros warned of second SolarWinds Orion vulnerability
- December 23 – Patching can close holes that could be exploited by stolen FireEye tools, according to new report.