Financial Crisis Exercise at RSA 2016

This year, at the RSA Conference, held in San Francisco from February 29 – March 4, Fox-IT was asked to host a financial cyber crisis table top exercise for the Learning Labs portion of the conference.

This was a great opportunity for us to showcase some of what Fox-IT does for companies:  training and aiding companies in incident response. Our exercise provided an opportunity to address a cyber threat scenario in an interactive and collaborative tabletop exercise.

Goliath National Bank

RSA_2The exercise took two hours to complete, tracking down discrepancies in the balance sheets of one Goliath National Bank (GNB), a prominent (fictitious) retail bank in the (fictitious) European country of Ramul. The exercise was designed to:

  • elicit constructive discussion as participants examine and resolve problems
  • identify where existing approaches need to be refined
  • establish relationships and share information with other organizations & partners
  • raise the awareness of the security community about challenges when dealing with a cyber crisis

The exercise was designed as a paper-based exercise with a facilitated discussion of a scripted scenario, where planners and players sit together in one room for the exercise execution.

Overall the session was very popular. There were a lot more people queuing up for the session than there was place for during the session. The attendees that did make it in, were very engaged.

Real life role playing

Each team distributed roles such as CIO, CISO, HR Director, PR Director, General Counsel or IT director. Reflecting the wide variety of attendees at RSA, we were delighted to discover that many of the roles were represented by players who had these roles in real life.

RSA_3Teams played in rounds where new information about an incident was revealed in every turn. The attendees had to pick their next steps and the closer they were with the ideal scenario, the more points they scored.

As crisis teams work through serious events, there is often partial information and there are unclear causes of events and unclear future effects. Therefore, war gaming and cyber crisis table top sessions are required on a regular basis for the crisis management team to gain experience in this field of expertise.

Restoring operations

The most difficult phase of the Learning Lab (as well as in a real life incidents) is the moment a crisis team receives the details about how the incident took place. From that moment in time the team has to switch from focusing on ‘identifying the root cause’ to ‘restoring operations’. They must find a healthy balance wherein the investigation continues, but the ‘restore operations’ priority becomes the most important. We can call this moment between investigation and mitigation an ‘impasse moment’. In order to make the right call, the crisis management team should be able to look at the incident from a helicopter view and come to a clear decision with regards to the next steps, by taking into account the investigation findings, business interests and potential future consequences related to the incident.

Fox-IT’s cyber crises exercises

Fox-IT regularly hosts cyber crisis exercises. Ranging from high-level tabletop sessions where an organizations’ crisis team is involved down to detailed, multi-day, technical challenges for computer emergency response teams and other IT personnel that is involved in a crisis. Whether you want a first introduction into crisis management, or want to train your crisis team periodically, our seasoned experts are able to help.

Would you like to know how we can help you to improve your organization’s resiliency? Please contact Rombert Anjema from FoxAcademy, tel. +31 (0)15 284 79 99, e-mail

Kevin Jonkers, Manager Forensics & Incident Response at Fox-IT, Sarah Brown, Principal Cyber Security Expert at Fox-IT and Krijn de Mik, Principal Cyber Security Expert at Fox-IT

RSA 2016: A Long Road Ahead for Security

We recently attended the RSA Conference, held in San Francisco from February 29 – March 4, to speak with our European clients. Does that surprise you? Far more Europeans visit this conference than you might think. The RSA Conference is the largest trade show for security in the world, yet its main attraction lies not so much in what can be seen on the main floor. Of even more interest are the meetings with security officers from a wide array of organizations that coincide with the trade show. This is where the real action takes place.

Importance of Integrating Solutions

Every year, I’m struck by the enormous gap that exists between the claims made out on the trade show floor and what is found to work in the field. Conversations at the trade show are the only way to find out whether the claims are true, and whether there is even a need for them in practice. As you navigate the trade stands, you encounter a myriad of solutions that are perfectly capable of fending off a specific type of attack — or at least claim to do so. Given that there are all manner of attacks, one should therefore deploy multiple solutions for effective, high-level security. Well, naturally CISOs are not going to go for that. Rather, for them, the challenge lies in integration: using the security solutions you already have in place and processing their results in such a way that you enable your team to efficiently and effectively counter a broad spectrum of attacks.

Costs Rising for Hackers

This focus on integration is evident in, for example, the growing attention paid to the economics of hacking. There are even CISOs who are held accountable for how successful they are at raising the costs for a hacker to mount an attack, and who therefore go so far as set a specified amount as a personal target. This means that CISOs need resources that allow them to block the paths that are the simplest — and therefore the cheapest — for hackers.

Intelligence Hype

The concept of threat intelligence has been on the rise for a long time. If you know who is targeting you and how they operate, you can find them more readily in your network or systems. Meanwhile, the number of suppliers that furnish intel feeds, often consisting of no more than lists of ‘bad’ IP addresses or file hashes, has proliferated. This all leads to market hype. CISOs are drowning in information; one feed after another issues a torrent of information that is too cumbersome for them to work with. What I’m hearing from CISOs is that they would prefer to receive far less technical information, and would instead like more context: Who are the attackers? What are their motives? What are they targeting?

First Aid for Panic

Another increasingly common need is to receive help once a data breach has been discovered. Panic usually breaks out first, especially in the Netherlands now that Meldplicht datalekken, a Dutch law concerning the mandatory notification of data breaches, is in force. To fulfil this notification obligation, everything has to be set in motion immediately: Involved parties (such as users and patients) must be informed, the Dutch Data Protection Authority must receive the necessary information, and numerous technical measures must be taken. Owing to the panic and time pressures, there is a pressing need for expertise and tools that can support an organization. Examples include ensuring that the breached company can demonstrate its conduction of a thorough investigation of the incident and helping it to supply the required information on time. This overarching need is consistent with the idea that it is not possible to set up 100% foolproof security. Even when things go wrong, organizations need to be able to mount a swift and adequate response.

Major Cloud Providers Serious About Data Storage Location

Rules and regulations are also an issue in other areas. Following the issue around Safe Harbor, several major cloud providers now take seriously the fact that Europe has a different stance on privacy than the United States. These cloud providers realize that they need to be able to deliver hard guarantees about where data is stored. This clearly goes against the ‘cloud mindset’ that data location no longer matters. But the awareness is slowly sinking in that there is no way to avoid European regulations.

No Silver Bullet

The RSA trade show floor teems with promises of, if you will, security ‘silver bullets’, whether these involve machine learning, threat intelligence, APT defense, or some other term that’s trendy at the moment. Despite this, I have yet to find at RSA the panacea that would give security professionals a distinct advantage over their attackers. This is perhaps unsurprising, given that RSA is ultimately a marketing extravaganza. It’s an excellent place to get a feel for what’s ‘hot’, but it’s not likely to provide the most reliable information for your next security purchase. Fortunately, the security community surrounding the trade fair appears to understand this. No one doubts the importance of technology, but it is only useful insofar as it supports your security operations team and the professional management of your security.

Jeremy_ButcherJeremy Butcher, Fox-IT Director of Operations

RSA 2016: security heeft nog een lange weg te gaan

Wij stonden op de RSA conferentie in San Francisco van 29 februari tot 4 maart om te praten met onze Europese klanten. Jawel, er komen veel meer bezoekers uit Europa dan je zou denken. De RSA is dan ook de belangrijkste securitybeurs ter wereld. Toch gaat het niet zozeer om wat er op de beursvloer allemaal te zien is. De meetings daaromheen met securityverantwoordelijken van de meest uiteenlopende organisaties zijn nog interessanter. Dáár gebeurt het.

Integratie van oplossingen belangrijk

Wat elk jaar opvalt is de enorme kloof tussen wat er op de beursvloer allemaal wordt geclaimd en wat daarvan in de praktijk werkt. Pas in gesprekken rondom de beurs kom je te weten of dat ook zo is en waar in de praktijk echt behoefte aan is. Zo kom je op de beursvloer talloze oplossingen tegen die perfect in staat zijn om één specifiek soort aanval tegen te houden (of in elk geval claimen dat te zijn). Maar aangezien er talloze soorten aanvallen bestaan, zou je dus ook meerdere van die oplossingen moeten inzetten voor een goede beveiliging. Tja, daar gaan CISO’s natuurlijk niet voor. Voor hen is de uitdaging eerder integratie: de securityoplossingen die je wél hebt op zo’n manier inzetten, en de resultaten ervan op zo’n manier bij elkaar brengen, dat je team efficiënt en effectief een breed spectrum aan aanvallen kan tegengaan.

Kosten voor de hacker omhoog

Die integrale focus uit zich bijvoorbeeld in toenemende aandacht voor de economische aspecten van het hacken. Er zijn zelfs CISO’s die zich laten afrekenen op de mate waarin zij erin slagen de kosten van een inbraak voor een hacker omhoog te brengen en die zelfs zo ver gaan dat ze daar een concreet bedrag als persoonlijk target aan verbinden. CISO’s hebben daarom belangstelling voor middelen om de voor hackers simpelste – en dus goedkoopste – paden te blokkeren.


Al een tijd is threat intelligence in opkomst: als je weet wie het op je gemunt heeft en hoe ze te werk gaan, dan zal je ze gemakkelijker vinden op je netwerk of in je systemen. Inmiddels is het aantal aanbieders van intel feeds – die in veel gevallen uit niet meer bestaan dan lijsten ‘foute’ IP adressen of file hashes – niet meer te tellen. Het gevolg is een hype-markt: de CISO verdrinkt in de inlichtingen, de ene feed na de ander zorgt voor een stortvloed waar niet meer mee te werken valt. Wat ik van de CISO’s hoor, is dat zij vooral veel minder technische informatie willen, maar juist méér context: wie zijn de mogelijke aanvallers, wat zijn hun drijfveren en waar hebben zij het op gemunt.

Eerste hulp bij paniek

Nog een behoefte die steeds meer partijen hebben, is hulp nadat een datalek is ontdekt. Meestal breekt eerst paniek uit, helemaal nu in Nederland de Meldplicht datalekken van kracht is. Er moet direct van alles in gang worden gezet: volgens de meldplicht moeten betrokkenen (bijvoorbeeld gebruikers of patiënten) worden ingelicht, de Autoriteit Persoonsgegevens moet de nodige informatie krijgen en ook technisch moet er van alles gebeuren. Door de paniek en de tijdsdruk is er grote behoefte aan expertise, maar ook aan tools die een organisatie hierbij ondersteunen, bijvoorbeeld door te zorgen dat zij kunnen aantonen dat het onderzoek van het incident volledig is en door te helpen alle informatie op tijd te verstrekken. Deze behoefte sluit ook aan op de gedachte dat 100% beveiliging niet mogelijk is. Ook als het mis gaat, moet de organisatie snel en adequaat kunnen reageren.

Grote cloudproviders serieus over datalocatie

Ook op andere vlakken is wet- en regelgeving een thema: na Safe Harbor neemt een aantal grote cloudproviders het nu serieus dat in Europa anders tegen privacy wordt aangekeken dan in de VS. Zij realiseren zich dat zij harde garanties moeten kunnen bieden over de locatie van de opgeslagen data. Nu gaat dit duidelijk in tegen de cloudgedachte dat locatie er niet meer toe doet, maar het besef daalt langzaam in dat er niet onder de Europese regelgeving is uit te komen.

Geen silver bullet

De RSA-beursvloer loopt over van beloftes van security ‘silver bullets’, of het nu machine learning, threat intelligence, APT defense of een andere hippe term is. Maar ik heb op deze RSA niet het wondermiddel gevonden waarmee beveiligers een grote voorsprong kunnen nemen op de aanvallers. Misschien niet verwonderlijk, want het is uiteindelijk een marketingfestijn: een uitstekende plek om de vibe mee te krijgen wat er ‘hot’ is, maar misschien niet direct de beste informatie voor je volgende security-aankoop. Gelukkig blijkt de securitycommunity rondom de beurs dat goed te begrijpen: natuurlijk is de techniek belangrijk, maar vooral voor zover die ondersteunend is voor je security operations team en professioneel management van je security.

Jeremy_ButcherJeremy Butcher, Director of Operations bij Fox-IT

RSA-512 Certificates abused in the wild

During recent weeks we have observed several interesting publications which have a direct relation to an investigation we worked on recently. On one hand there was a Certificate Authority being revoked by Mozilla, Microsoft and Google (Chrome), on the other hand there was the disclosure of a malware attack by Mikko Hypponen (FSecure) using a government issued certificate signed by the same Certificate Authority. That case however is not self-contained and a whole range of malicious software had been signed with valid certificates. The malicious software involved was used in targeted attacks focused on governments, political organizations and the defense industry. The big question is of course, what happened, and how did the attackers obtain access to these certificates? We will explain here in detail how the attackers have used known techniques to bypass the Microsoft Windows code signing security model.

Recently Mikko Hypponen wrote a blog on the F-Secure weblog detailing the discovery of a certificate used to sign in the wild malware. Specifically this malware was embedded in a PDF exploit and shipped in August 2011. Initially Mikko also believed the certificate was stolen, as that is very common in these days, with a large amount of malware families having support, or optional support, for stealing certificates from the infected system. Apparently someone Mikko spoke to mentioned something along the lines that it had been stolen a long time ago. During the symposium Mikko mentioned the certificate again, but now he mentioned that according to the people involved with investigating the case in Malaysia it likely wasn’t stolen.

The reason why Mikko looked at this specific sample and this certificate was likely the recent revocation of Digisign Server ID (Enrich) by Microsoft and earlier by Mozilla. The interesting part in those articles is that Microsoft does not mention anything about the code signing abilities of certificates while Mozilla does. Microsoft does mention that the certificates were not fraudulently issued but were duplicated due to cryptographically weak keys. The option of stolen certificates is left completely in the middle here.

The whole commotion around Digisign was actually caused by an investigation completed by Fox-IT in mid-October, in which we recovered a number of signed executables embedded in exploits and downloaded additionally by any of the executables. While our investigation did not focus on the signing of those executables, the report was shared in the relevant community, and if you looked at the 4 certificates initially found, it was easy to determine that all were 512bit RSA and used on HTTPS websites, which were still up at the time of writing. Later during our investigation we encountered 5 more certificates which also were used to successfully sign malware throughout 2011 by the same attacker, all 512 bit RSA.

So it is rather obvious what happened, all related RSA-512 keys had been factored and also abused to sign malicious software for the purpose of infiltrating high value targets. You might ask how difficult it is to execute an attack against RSA-512, well, over 12 years ago the RSA-512 challenge was successfully factored. Also we still encounter RSA-512 in protection systems deployed even today, with relatively modern hardware in a small cluster, relatively inexpensive, it takes a couple of weeks. With the lifetime of these certificates being a couple of years, the attackers had plenty of time to do the factoring.

So the reason why Digisign Server ID (Enrich)/DigiCert Sdn. Bhd, was revoked was because their certificates had no CRL in the certificate which allowed to easily revoke the certificate. Also all those certificates were issued without a purpose, in which case the certificates can be used for anything. The certificates we found to be used in the wild recently are:

  • (Digicert Sdn. Bhd.)
  • (Digicert Sdn. Bhd.)
  • (Digicert Sdn. Bhd.)
  • (Cybertrust)
  • (Cybertrust)
  • (Cybertrust)
  • (GlobalSign Inc)
  • (TAIWAN-CA.COM Inc.)
  • (Anthem Inc)

Additionally an external party found several other samples which contained 512 bit RSA certificates signed by Digicert Sdn. Bhd:

One of those samples was found in August 2010, and possibly used back in March 2010, indicating how long this issue has been going on without any clear action from the industry. Microsoft whose platform has been targeted by this is the victim of this, and I think that Microsoft should not have relied on weak security properties for a security solution that can apparently be bypassed by parties far outside of the control of Microsoft. Microsoft could simply deny verification of executables which have been signed with a 512bit RSA key after a certain date, as 512 bit RSA has been considered weak for a long time. From the article at TechNet it is clear that Microsoft understand the problem and that they have acted on this accordingly, but the question is if it was not a bit late. Also interesting is that none of the samples have an actual timestamp, we think this is another design decision made that makes these executables pass verification, but it might cause the executables to no longer pass verification after the certificate has expired, we were unable to test this however.

Also one certificate that was used,, did not have the “Digital Signature” property in “Key Usage”, thus it should not have passed verification. But we wonder if that indeed is true, as why would the attackers go through great lengths of factoring the RSA key and using it to sign their executables, if it did not pass verification? Either the attackers overlooked something here, or the digital signature verification system in Windows is at fault. We are however unable to verify this as the relevant certificate has expired in April 2011.

So the problem will solve itself eventually with CAs no longer signing 512 bit and more attention is given on the subject. The model of code signing certificates is however not very good as even the expensive code signing certificates can be stolen, and this can be done by simple off-the-shelf malware, such as ZeuS and SpyEye. But let’s focus on the issue at hand, how could we go about finding other certificates that might have been abused or could be abused, but that we do not have the executables from to prove it? Well someone already did all the work for us, that would be Peter Eckersly (EFF), Jesse Burns (iSec Partners) and Chris Palmer (EFF) who have worked in 2010 on EFF SSL Observatory that has indexed certificates used on port 443 (HTTPS). They have done presentations on this various times and we want to explicitly thank these guys for their hard work.

So we have used the database from mid-2010 which might be more close to the data the attackers had. So lets see, if we check the 9 certificates we have found being abused in the wild:

DigitalSignature Key Usage    Ext. Key Usage    RSA bits   Common Name
+                             -                 512
+                             -                 512
-                             -                 512
+                             -                 512
+                             -                 512
+                             -                 512
+                             -                 512
+                             -                 512
+                             -                 512
+                             -                 512
+                             -                 512

Bingo, they are all there, this is a good indication the people who found these certificates used a similar method to find these certificates, scanning port 443 (HTTPS) for valid 512 bit RSA certificates with no Extended Key Usage property defined and being usable. Note again that the has no Digital Signature Key Usage property.

Okay, so now let’s see what other certificates there are in the database from mid-2010 which match similar search criteria, that were valid according the the certificates from Microsoft at the time and have not expired yet.

Key Usage    Ext. Key Usage    RSA bits   Common Name                    Issuer
+            -                 512         GlobalSign nv-sa
+            -                 512               DigiNotar B.V.
+            -                 512  DigiNotar B.V.
+            -                 512           DigiNotar B.V.
+            -                 512          DigiNotar B.V.
+            -                 512  DigiNotar B.V.
+            -                 512           DigiNotar B.V.
+            -                 512          DigiNotar B.V.
+            -                 512             GlobalSign nv-sa
+            -                 512         GlobalSign nv-sa
+            -                 512          Digicert Sdn. Bhd.
+            -                 512           Digicert Sdn. Bhd.
+            -                 512    Cybertrust

Okay, we can find two certificates in there which we know that have been abused, specifically and, those have 512 bit RSA keys that have been factored. All other certificates seem to have been either revoked directly or indirectly, these are the ones related to DigiNotar and Digicert Sdn. Bhd/Digisign Server ID (Enrich). Others have been individually revoked after they have been replaced in the past. So, it looks like indeed the problem has been solved by revoking trust in Digicert Sdn. Bhd. and DigiNotar B.V. (unrelated) and revoking those specific certificates. We have not observed in the wild usage of other certificates such as the ones signed by DigiNotar, possibly there are other constraints which make it unusable for Code Signing. We will leave it as an exercise to the reader to inspect the other databases which the EFF SSL Observatory has created to find other certificates.

Concluding we can say that definitely there has been a lot of attention on Certificate Authorities and their procedures and also with several incidents such as the forging of data to match an md5 signature and the more recent DigiNotar incident. But this specific issue, while factoring is widely known, had not been addressed and has been used over a year for targeted break-ins of high value targets. While it has not been used for signing of drivers as far as we know, as was done with the stolen certificates used in the Stuxnet and Duqu attacks, it did play a part in the attacks we have observed and as such we think that letting the issue unaddressed for such a long time might have helped the attackers. It was trivial to find these certificates using published data by EFF and luckily it appears that all known certificates have now been revoked, but the question is if there are more certificates which have not been recorded in a database and have been, for example, used on other ports than HTTPS. The fact remains that the code signing mechanism is relying on trust in many parties which are not necessarily trusted, as human error and intentional or unintentional bypassing of procedures can break the entire security model.

Download the print version here.

Michael Sandee,
Principal Security Expert at Fox-IT