This post is dedicated to Phúc Toàn, a friendly and joyful guy from Ho Chi Minh City, Vietnam. He was one of the participants in IPv4/IPv6 Routing Workshop at APRICOT 2017. He created his own lab to test RTBH using blackhole community but for some reason it wasn’t working and he showed it to me after the class. We could find the problem, and finally it worked as expected. Later, he took me to a cafe and we had some seasonal fresh fruit juice along with one of his colleagues and then he gave me a ride on his bike around the Saigon river, it was really a beautiful view of HCM City in the evening. Thanks to Phúc Toàn, I wish to meet him someday again.
What is RTBH?
Remotely Triggered Black Hole (RTBH) is a technique where configuration performed in your network triggers an action to be executed automatically in a remote network to black hole traffic that are intended to reach to the destination in your network. RTBH can be used (in fact widely used to blackhole DDoS traffic).
Why and when you may need it?
If you want to filter out unwanted incoming traffic (towards a certain IP or IP block in your network or in your customer network) closer to the source or as far away as possible from your network, RTBH can do that for you. A very popular example would be filtering Distribute Denial of Service (DDoS) traffic. You can filter out the incoming packet with other methods like using inbound ACL in your gateway router. But, the traffic actually already reached your gateway router (before getting dropped) using your uplink circuit. Which means, your valuable uplink bandwidth is already used by those unwanted packet and your gateway router would process the packet before discarding. If those traffic is of large amount enough to chock your uplink bandwidth or to utilize router’s full process, the whole network and services will be affected. All the clients would start complaining. If those unwanted packets could have been filtered out far away from the network, those consequences can be avoided.
But the problem is…
While the unwanted incoming packets are filtered out by triggering remotely, it will discard the valid packets as well. Because, the remote networks are only triggered to filter out traffics towards a certain IP address or a certain network, there is now way to instruct them to allow valid packets based on valid source addresses.
How RTBH can help during a DDoS attack?
Suppose one of your servers is under a DDoS attack. The attacking devices are residing all over the Internet and participating in the attack from different locations. The torrents of incoming traffic can be blocked in your gateway router using an inbound ACL but that would not solve the problem. As I explained before, the huge volume of traffic is already congesting your uplink bandwidth and may have consumed router’s whole process. All your other clients and services must suffer in this case. Otherwise, what you can do is, you can blackhole those traffic in your network destined to that affected server’s IP address and ask your upstream routers to do so. BGP blackhole community is used to trigger this instruction. Then, your upstream would blackhole traffic towards your server’s IP and ask their upstreams/peers to do so. They would ask their upstreams/peers and so on. This way you can trigger and propagate blackhole instruction as far away as possible from your network. However, as said earlier, all the valid traffic would also get backholed due to this action. Which means, eventually the attackers win as your services running on that server would temporarily remain unavailable until you restore/resume it using another IP address.
Here is our lab scenario where we have an ISP (or enterprise), a national or regional transit provider (depicting a Tier-2 ISP or so) and a large ISP or Global Transit Provider (depicting a Tier-1 ISP). The reason why this topology is chosen is to show the probable configuration of BGP blackhole community at each level and how it works throughout the whole Internet. The target is to blackhole the incoming traffic towards Victim Server (192.168.0.2) using blackhole community.
- Private IP address blocks are used for this lab instead of public IP addresses.
- BGP is configured between Autonomous Systems and community sharing is permitted (send-community is configured between neighbors)
- Regional-Transit-1 and Regional-Transit-2 do not filter customer prefixes automatically based on RPKI/ROA/RADB. Rather, they use route map, prefix list and AS filter list to do so.
- Global-Transit-1 and Global-Transit-2 filters customer prefixes automatically based on RPKI/ROA/RADB.
- XYZ-ISP knows the blackhole community of all its upstream (i.e 65502:222 for Regional-Transit-1 and 65503:333 for Regional Transit-2).
- Regional-Transit-1 also knows the blackhole community of all its upstream. In our lab, the ultimate upstream are Global-Transit-1 (blackhole community is 65504:444) and Global-Transit-2 (blackhole community is 65505:555).
Before configuring the RTBH, the basic BGP and interface configurations are done in each router. For example, the configuration of XYZ-ISP may look like this:
Other routers are also configured accordingly.
RTBH Specific Configuration:
At first, XYZ-ISP need to prepare a route-map to set all its upstream’s blackhole community there.
Configure Regional-Transit-1 (similar configuration is applied to Regional-Transit-2):
The transit providers of XYZ-ISP need to perform two tasks. Task# 1 is, they need to ripple the trigger of RTBH towards their upstream. Which means if XYZ-ISP advertises a prefix attaching the blackhole community of Regional-Transit-1 and/or Regional-Transit-2, the transit routers will match their community and attach their upstream’s blackhole community just to move the trigger a bit further away from the XYZ-ISP. And Task# 2 is, Regional-Transit-1 and Regional-Transit-2 must be able to blackhole locally originated traffic or its customer traffic that are intended to reach the Victim-Server.
For Task# 1, XYZ-ISP’s transit provider must know their upstream’s blackhole community. Here in our example, Regional-Transit-1 would ask its upstream Global-Transit-1 and Global-Transit-2 to get their blackhole community 65504:444 and 65505:555. Similar tasks for Regional-Transit-2. If they have multiple upstream, they need to know the blackhole community of each of their upstream. Now, in order to set upstream’s blackhole community for any prefix coming from the downstream, Regional-Transit-1 needs to add Global-Transit-1’s blackhole community to that prefix. Here’s how to do that:
Configure a loopback interface to nullify blackhole traffic that may comes to Regional-Transit-1 network (Task# 2 mentioned above). This traffic may be generated by Regional-Transit-1’s local network or by its clients. (BTW, why I used loopback? See the Note at the bottom for the answer).
For Task# 1, configure a community list permitting its blackhole community:
Now as we considered that the regional transit network is filtering clients’ prefixes manually, we need to configure AS filter list, prefix list and route map as shown below:
You may be thinking, why we need two prefix lists to permit the same client prefix. Prefixes with subnet length grater than /24 is not allowed in global BGP table. However, during DDoS, you may need to advertise a single IP address (/32) with blackhole community. For that reason, two prefix lists for the same client prefix are created, prefix-list CLIENT1-LE24 permits up to /24 prefixes only and prefix-list CLIENT1-LE32 permits up to /32 prefixes of the client IP block.
route-map CLIENT1-IN permit 10 matches XYZ-ISP’s valid prefix of any size (192.168.0.0/16 up to /32), if and only if the prefix originates from AS 65501 (XYZ-ISP) and have the blackhole community tag of Regional-Transit-1. Once there is a match, the route-map sets that incoming prefix with blackhole community of all the upstream of Regional-Transit-1 and also it forces to set the next-hop of the prefix to loopback 0 (100.100.100.100) so that traffic towards that traffic gets blackholed.
However, under normal condition when no DDoS is affecting XYZ-ISP network, it won’t send any prefix with the blackhole community. Hence, under normal condition route-map CLIENT1-IN permit 10 won’t be used. Instead, route-map CLIENT1-IN permit 20 will be used where it simply permits XYZ-ISP’s valid prefixes with a maximum subnet length of /24.
Now, apply the route-map to inbound bgp peering with XYZ-ISP:
Regional-Transit-1 and Regional-Transit-2 need to do similar configurations for each of its clients.
Configuration in Global-Transit-1 network (similar configuration is applied to Global-Transit-2):
The configuration of Global-Transit-1 is quite similar to the Regional-Transit-1. First, configure a loopback interface to nullify blackhole traffic.
Configure a community list permitting its blackhole community:
Configure a route-map to match blackhole community sent by clients (i.e. Regional-Transit-1) and force set the next-hop to the loopback 0.
Now, apply the route-map to inbound bgp peering with all the clients:
Global-Transit-1 and Global-Transit-2 need to do similar configurations for each of its clients.
Before experiencing any DDoS:
Prefix 192.168.0.0/24 is originated and announced by XYZ-ISP which is visible in global routing table.
The server (192.168.0.2) is reachable from remote end (across the Internet).
During the DDoS attack to Victim-Server (192.168.0.2):
Once the XYZ-ISP realised that one of their servers (192.168.0.2) is experiencing DDoS attack, all they need is to announce that single IP address using BGP blackhole community that was prepared beforehand.
Now, two prefixes are visible in global table (one is the /24 prefix and other one is DDoS affected /32 IP address). The next hope of that /32 prefix is the loopback in the router which means that it will blackhole any traffic targeting 192.168.0.2.
Hence, any traffic towards the Victim-Server will be dropped far away from the XYZ-ISP network.
All the references in the Internet about RTBH has a static route towards null interface to blackhole the traffic. I tried to use it but it didn’t work. As you can see below, 100.100.100.100 pointing towards null0 show inaccessible hence the router won’t install the /32 route in the routing table. So, packets towards the Victim-Server are not actually blackholed. Hence, I used the configuration with loopback interface instead of null interface. I understood, it may increase the process a little bit compared to using null0 interface, but I am okay with that as the process is not that much increased.