Setting up a national DDoS clearing house

Setting up a national DDoS clearing house

Increasing the Netherlands’ DDoS resilience together, part II of III

Last week, we spoke about the Dutch Anti-DDoS Coalition, how it came about and its goal of realising a cooperative DDoS mitigation strategy by using a combination of five instruments. This week, we’ll take a closer look at one of them: the DDoS clearing house, a technical system that enables members of the coalition to measure and share the properties of DDoS attacks in real time. We’ll also discuss the status of our work, with the cliff-hanger being our lessons learned and outlook, which we’ll discuss next week in our third and final blog in the current series.

DDoS clearing house

One of the pillars of the Dutch Anti-DDoS Coalition is our so-called DDoS clearing house, a system that enables coalition members to measure the properties of DDoS attacks they handle and share the information within the coalition. The coalition members include ISPs, banks, internet exchanges and government agencies.

Figure 2 illustrates how the clearing house works using service providers SP1 through SP3 as an example. SP2 is the victim of a DDoS attack (A in Figure 2 ), who shares the properties of A with SP1 and SP3. Those properties include information such as A’s source addresses, packet sizes, duration and protocol types. SP2 shares the information in the form of a “DDoS fingerprint” (FP in Figure 2 ), for instance using packet captures (PCAPs), NetFlow [Conrads19], or IPFIX. SP2’s operations team decides which fingerprints to share with the other service providers and could potentially generate a fingerprint manually if A causes their systems to fail extensively. The team may also include the filtering rules they used to mitigate the attack in the fingerprint, for instance in the form of snort rules.

The operations teams of SP1 and SP3 use the fingerprint they receive from SP2 to prepare their infrastructures in case A hits them next, for instance by installing packet filtering rules on their routers (e.g. rules R1 and R3 in Figure 2 ). Without SP2’s fingerprint, they would have to write such rules while an attack is already in progress, when they are typically under intense pressure. For example, if A were to hit SP1, then their operations team would have to inspect A’s traffic, write a packet filtering rule (R1) for the different types of equipment in their network and push it into their network, all perhaps while the availability of SP1’s services were starting to degrade. Having A’s fingerprint beforehand gives them more time to implement R1, which increases the probability that the rule will be effective in mitigating the attack, because they have a head start. In addition, SP1 can use the fingerprint to detect the attack and then hand off the traffic to a (third party) scrubbing service.

Figure 2. Exchanging DDoS fingerprints.

While the sharing of DDoS fingerprints gives service providers such as SP1 a proactive edge, service providers like SP2 still need to have facilities in place to handle attack traffic. The DDoS clearing house is thus complementary to scrubbing services such as those provided by NBIP-Nawas or commercial equivalents, rather than a replacement for such services.

Service providers can also use the fingerprints for other purposes, such as working with law enforcement agencies to track down and prosecute perpetrators, and analysing the evolution of DDoS attacks. Such uses are outside the scope of this blog, however.

An attacker may also DDoS several service providers simultaneously, which will then each share their fingerprint of the attack traffic they receive, just like SP2.

Our approach: a centralised national clearing house

We’ve chosen to develop a national DDoS clearing house for all service providers in the Anti-DDoS Coalition to use, regardless of the sector they’re in. However, the concept could be applied in other forms as well. You could have a clearing house for a specific sector (e.g. financial services), for example, or for the various divisions of a large corporation. The model might also be applied across EU member states: there could be an EU-wide clearing house for financial service sector, for example, or for NRENs or mobile operators.

We also opted for a logically centralised DDoS clearing house at this stage to test the basic premise that sharing the fingerprints of attacks is useful for mitigate DDoS attacks more proactively. The H2020 CONCORDIA project will also explore other deployment models (e.g. fully distributed using blockchain technology [BloSS19]), for instance to explore means to further increase the technical scalability of the clearing house.

Another aspect of our approach is the reuse of existing components to develop the clearing house, specifically the DDoS dissector and the DDoS-DB [DDoSDB], which are prototype systems developed by the University of Twente (more on that below).

Key challenge: nationwide usage

The key challenge we’re facing is to operationalise the clearing house on a national scale in the Netherlands. That’s a challenge because we need to bring the clearing house from a lab environment to a “pre-production” level (around TRL 5-7) in a series of steps, while also matching non-technical constructs for each step (e.g. legal agreements and governance models) and keeping a wide range of service providers aligned. To keep the complexity of the work manageable, we started with a relatively small group of ten service providers: the Dutch Payments Association, KPN, NBIP-Nawas, NCSC-NL, NL-ix, SIDN, SURF, THTC, the University of Twente and Vodafone-Ziggo.

Where are we now?

In the last year and a bit, we’ve been focusing on setting up a first iteration of a technical pilot, and in parallel we’ve started work streams to develop the non-technical “glue”, such as a data sharing agreement and an organisational structure. Going forward, we expect to develop several new versions of all of those components in order to get to the pre-production maturity level we need.

Pilot iteration #1: One of our key results so far is that we’ve set up the first iteration of a technical pilot with the DDoS clearing house, which involved setting up a central instance of DDoS-DB [DDoSDB] (see Figure 3) at SIDN Labs and several partners experimenting with generating fingerprints (e.g. KPN and THTC). For simplicity, the DDoS fingerprints we’re sharing in this first iteration of the pilot only contain DDoS metadata and no packet captures (PCAPs).

Data sharing agreement: Our second key result is that we’ve developed a simple data sharing agreement for the pilot, covering basic legal aspects like objectives, liability, security, personal identifiable information (PII) and governance. The data sharing agreement is valid for a fixed but extensible duration of six months and is currently being signed by the pilot partners. The development of the data sharing agreement had a long lead time, partly because the legal and tech experts had to understand each other’s problem spaces and agree on a common approach, as is often the case.

System architecture: Our third result is a high-level architecture for the clearing house (Figure 3), which serves as the foundation for future iterations of the pilot. The architecture revolves around three key components: the dissector [Conrads19] (generates fingerprints from DDoS traffic), DDoS-DB [DDoSDB] (distributes fingerprints and provides a searchable fingerprint history) and a converter (maps fingerprints to traffic filtering rules). Figure 3 shows those components superimposed on the example of Figure 2.

Figure 3. DDoS clearing house architecture.

System requirements: we’ve also developed the technical requirements and use cases for improving the clearing house’s components (dissector, DDoS-DB and converter) and extending their functionality. The requirements include, for example, that the dissector must not include any sensitive information about the victim of a DDoS attack in a fingerprint (e.g. destination IP or MAC addresses) and that the DDoS-DB must allow an authenticated user to perform searches on the index of fingerprints and download them. The development of the requirements was jointly funded by the Dutch Payments Association, the Netherlands’ National Cyber Security Centre (NCSC-NL), NBIP and SURF.

Next week

The last part of this blog series will be about what we learned in attaining the above results and our way forward for 2020.

Acknowledgements

SIDN, SURF and the University of Twente were partly funded by the European Union’s Horizon 2020 Research and Innovation program under Grant Agreement No 830927. Project website: https://www.concordia-h2020.eu/

References

[DDoSDB] DDoS-DB homepage, https://github.com/ddos-clearing-house

[Conrads19] J. Conrads, “DDoS Attack Fingerprint Extraction Tool: Making a Flow-based Approach as Precise as a Packet-based”, M.Sc. Thesis, University of Twente, the Netherlands, Aug 2019

[BloSS19] Bruno Rodrigues and Burkhard Stiller, “Cooperative Signaling of DDoS Attacks in a Blockchain-based Network”, SIGCOMM Posters and Demos ’19: Proceedings of the ACM SIGCOMM 2019 Conference Posters and Demos August 2019, https://doi.org/10.1145/3342280.3342300

(By Cristian Hesselman (SIDN and University of Twente), Remco Poortinga-van Wijnen (SURF), Gerald Schaapman (NBIP) and Remco Ruiter (Dutch Payments Association))