Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

Quick Links


Categories

In this Discussion

› › Hetzner (and other) traffic passing Cogent rerouted over Moscow
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Hetzner (and other) traffic passing Cogent rerouted over Moscow

WilliamWilliam Member
edited May 2017 in Providers

It seems Hetzner networks are partly hijacked currently, our traffic flows now on Cogent routes to Moscow, to St. Petersburg, back to Stockholm on Telia and then to Hetzner:

See from hop 12 on here:

 7. AS174    be2988.ccr21.vie01.atlas.cogentco.com (154.54.59.86)               0.0%    0    33    33  35.4  28.4  36.6  45.0   4.8  36.2  2.4  5.8 13.7 86.9
 8. AS174    be2975.ccr22.muc03.atlas.cogentco.com (154.54.58.13)               0.0%    0    33    33  43.5  33.8  42.8  55.5   4.7  42.6  4.2  5.7 16.0 69.4
 9. AS174    be2960.ccr42.fra03.atlas.cogentco.com (154.54.36.253)              0.0%    0    33    33  54.9  42.0  47.5  54.9   3.3  47.4  3.0  3.6  9.9 57.6
10. AS174    be2846.rcr22.fra06.atlas.cogentco.com (154.54.37.30)               0.0%    0    33    33  54.9  42.0  52.8  74.7   8.0  52.3  1.7  7.0 23.0 102.
11. AS174    154.25.9.46                                                        0.0%    0    33    33  48.1  43.0  49.1  66.4   5.9  48.8  1.4  5.7 19.9 77.6
12. AS174    149.14.69.218                                                     36.4%   12    21    33  95.6  43.4  70.5  97.8  19.9  67.7  2.7  9.8 45.1 97.1
13. AS12714  212.1.243.163                                                     93.9%   31     2    33 398.0  83.9 240.9 398.0 222.1 182.7 314. 157. 314. 314.
14. AS12714  89.20.140.2                                                       93.9%   31     2    33 225.1 108.4 166.8 225.1  82.5 156.2 116. 58.3 116. 116.
15. AS1299   s-b2-link.telia.net (213.248.93.109)                              50.0%   16    16    33 283.4 183.8 283.9 332.7  37.7 281.3 49.3 14.9 56.0 157.
16. AS1299   s-bb4-link.telia.net (62.115.119.114)                             46.9%   15    17    33 284.5 216.9 284.3 326.3  33.8 282.3 38.8 10.3 38.8 121.
17. AS1299   s-b10-link.telia.net (62.115.114.160)                             40.6%   13    19    33 280.3 163.4 278.7 330.3  42.0 275.2 50.0 16.5 89.3 171.
18. AS1299   ae0-1299.sto10.core-backbone.com (213.248.77.134)                 46.9%   15    17    33 280.3 188.3 280.3 328.4  43.0 276.9 42.3 11.5 42.3 124.
19. AS201011 ae10-2021.fra20.core-backbone.com (80.255.14.6)                   46.9%   15    17    33 296.2 226.0 301.7 429.1  44.5 298.8 35.2 23.0 203. 207.
20. AS201011 core-backbone-100g-fra.hetzner.de (80.255.15.122)                 53.1%   17    15    33 284.2 233.8 294.2 380.5  37.7 292.0 36.0 21.1 146. 191.
21. AS24940  core4.fra.hetzner.com (213.239.245.2)                             46.9%   15    17    33 281.1 197.5 290.0 339.2  36.3 287.6 44.0 18.1 92.0 177.
22. AS24940  core24.fsn1.hetzner.com (213.239.203.150)                          0.0%    0    19    20 284.1 214.3 322.9 920.1 149.0 305.6 40.0 48.0 705. 371.
23. AS24940  ex9k2.rz16.hetzner.de (213.239.245.150)                            5.3%    1    18    19 293.0 211.9 291.6 344.0  37.2 289.2 32.9 13.0 33.0 146.

MTR.SH traces, all Cogent using ISPs cross Russia:

https://scr.meo.ws/paste/1495485047657816552.txt

Try yourself:

traceroute 5.9.181.167

This also affects other things, our Ukrainian colo is now also rerouted over the same link, passing Moscow, SPB, and back to EU, then on to UA...

Thanked by 1vimalware

Comments

  • GenjiSwitchPlsGenjiSwitchPls Member

    Any official word from Hetzner/upstreams?

  • maldoviamaldovia Member

    dis is a new feature

    Thanked by 1jixun
  • WilliamWilliam Member

    Traffic returns to normal for most routes now:

      2. AS174   be4690.rcr21.b002695-3.lax01.atlas.cogentco.com (38.140.154.89)   0.0%    0    10    10   0.7   0.6   0.9   2.4   0.5   0.8  1.7  0.4  1.7  3.9
  3. AS174   be2918.ccr22.lax01.atlas.cogentco.com (154.54.41.193)             0.0%    0    10    10   0.7   0.5   0.7   1.1   0.0   0.7  0.1  0.2  0.5  1.8
  4. AS174   be2932.ccr22.phx02.atlas.cogentco.com (154.54.45.161)             0.0%    0    10    10  12.2  12.2  12.4  13.1   0.0  12.4  0.3  0.2  0.7  1.5
  5. AS174   be2930.ccr21.elp01.atlas.cogentco.com (154.54.42.78)              0.0%    0    10    10  20.3  20.0  20.2  20.4   0.0  20.2  0.1  0.1  0.3  0.9
  6. AS174   be2928.ccr42.iah01.atlas.cogentco.com (154.54.30.161)             0.0%    0    10    10  36.1  35.7  36.0  36.2   0.0  36.0  0.0  0.2  0.4  1.5
  7. AS174   be2690.ccr42.atl01.atlas.cogentco.com (154.54.28.129)             0.0%    0    10    10  50.2  50.1  50.2  50.5   0.0  50.2  0.1  0.1  0.2  0.7
  8. AS174   be2113.ccr42.dca01.atlas.cogentco.com (154.54.24.221)             0.0%    0    10    10  60.9  60.8  61.0  61.3   0.0  61.0  0.2  0.2  0.4  1.6
  9. AS174   be2807.ccr42.jfk02.atlas.cogentco.com (154.54.40.109)             0.0%    0    10    10  67.3  67.0  67.3  67.7   0.0  67.3  0.2  0.2  0.5  1.2
 10. AS174   be2490.ccr42.lon13.atlas.cogentco.com (154.54.42.86)             10.0%    1     9    10 140.5 140.2 140.4 140.7   0.0 140.4  0.1  0.2  0.4  1.2
 11. AS174   be12488.ccr42.ams03.atlas.cogentco.com (130.117.51.42)           10.0%    1     9    10 145.0 144.8 145.1 145.3   0.0 145.1  0.2  0.2  0.5  1.5
 12. AS174   be2814.ccr42.fra03.atlas.cogentco.com (130.117.0.142)            10.0%    1     9    10 151.8 151.3 151.5 151.8   0.0 151.5  0.3  0.1  0.3  0.8
 13. AS174   be2960.ccr22.muc03.atlas.cogentco.com (154.54.36.254)            10.0%    1     9    10 156.2 156.0 156.3 156.5   0.0 156.3  0.0  0.1  0.5  0.8
 14. AS174   be2270.rcr21.nue01.atlas.cogentco.com (154.54.37.217)            10.0%    1     9    10 159.3 159.3 159.4 159.5   0.0 159.4  0.0  0.1  0.2  0.5
 15. AS174   te0-0-1-0.nr11.b040138-0.nue01.atlas.cogentco.com (154.25.0.10)  10.0%    1     9    10 162.4 162.3 162.5 162.6   0.0 162.5  0.1  0.1  0.2  0.8
 16. AS174   149.6.158.6                                                      10.0%    1     9    10 162.2 159.2 159.5 162.2   0.9 159.5  3.0  0.4  3.0  3.3
 17. AS24940 core12.nbg1.hetzner.com (213.239.229.165)                        11.1%    1     8     9 159.2 159.1 159.3 159.7   0.0 159.3  0.2  0.2  0.5  1.4
 18. AS24940 core24.fsn1.hetzner.com (213.239.245.30)                         11.1%    1     8     9 168.1 168.1 169.9 181.7   4.8 169.8  0.1  3.4 13.6 25.0
 19. AS24940 ex9k2.rz16.hetzner.de (213.239.245.150)                          11.1%    1     8     9 165.2 164.4 164.8 165.8   0.4 164.8  0.7  0.4  1.3  2.8
 20. AS24940 signull.abuse.li (5.9.85.139)                                    22.2%    2     7     9 161.8 161.7 161.8 161.9   0.0 161.8  0.0  0.1  0.1  0.3
 21. AS24940 static.167.181.9.5.clients.your-server.de (5.9.181.167)          11.1%    1     8     9 162.1 162.0 162.1 162.1   0.0 162.1  0.0  0.0  0.1  0.2
  • jiggawattjiggawatt Member

    This isn't the first time: https://arstechnica.com/security/2017/04/russian-controlled-telecom-hijacks-financial-services-internet-traffic/

  • WilliamWilliam Member

    That seems unlikely though, they essentially killed their entire network in RU as visible on the 93-98% packetloss - i assume, for now, unintentional hijack and the dumb Cogent open sessions allowing passing of spoofed AS paths

  • jiggawattjiggawatt Member

    It wasn't meant to be seen as an intentional hijack, or even mop up anything useful, it was just meant to be seen...

    Thanked by 1vimalware
  • WilliamWilliam Member

    The Rostelecom one did get traffic though, and not kill their network at all, plus very specific networks (mostly financial as noted there and at Dyn) - this one just replicated half the internet.

    ASN doing that now was https://bgpview.io/asn/AS12714#info http://www.netbynet.ru/ which says 500Gbit+ capacity, if we assume 250G+ available they just got a LOT of useless packets routed their way... from that view, a route leak most likely.

  • jiggawattjiggawatt Member
    edited May 2017

    Whether they got useful or useless packets isn't the point - this is dialogue. A new form of communication, the likes of which the world has never seen. This is RU speaking to US, EU, UA...

    Thanked by 1vimalware
  • WilliamWilliam Member

    As this is not a gov ISP, unlike Rostelecom, but a private owned one i still doubt that - Russia, as in gov, would as with the financial then and other things before, leverage Rostelecom which can carry the traffic fine and has no legal issues to fear.

    This on the other hand is now a major cost for them, and the value of 5minutes of 300Gbit+ trash traffic is questionable at least...

  • jiggawattjiggawatt Member

    In RU, things can be privately owned, but if they don't follow the government's orders, the owner might wake up one day and own nothing, if he wakes up at all...

    Thanked by 2vimalware elwebmaster
  • WilliamWilliam Member

    uh, we have similar laws in the West as well ("national security order"), this is not really anything different ultimately.

    Taking down their network, absolutely obvious to anyone, is just a dumb idea especially if you did hijack before with great success.

  • dragonballz2kdragonballz2k Member

    @William said:
    uh, we have similar laws in the West as well ("national security order"), this is not really anything different ultimately.

    Taking down their network, absolutely obvious to anyone, is just a dumb idea especially if you did hijack before with great success.

    is it possible to prevent issues like this by having verification or something to make something likes doesn't keep happening?

  • WilliamWilliam Member

    dragonballz2k said: is it possible to prevent issues like this by having verification or something to make something likes doesn't keep happening?

    As path filtering, which any decent provider does - open sessions should also not be handed out. In this case, blame is mostly on Cogent... as so often before.

  • LeviLevi Member

    Does the Hetzner explained this traffic anomaly?

  • jiggawattjiggawatt Member
    edited May 2017

    William said: uh, we have similar laws in the West as well ("national security order"), this is not really anything different ultimately.

    There is a huge difference. Apples to oranges.

    dragonballz2k said: is it possible to prevent issues like this by having verification or something to make something likes doesn't keep happening?

    RPKI: https://www.arin.net/resources/rpki/

    It's not widely adopted yet due to some technical limitations.

    LTniger said: Does the Hetzner explained this traffic anomaly?

    BGP hijacks are common and usually innocuous. It's not standard industry practice to publicly explain anything about them.

    Only recently have observers been publicizing hijackings that cross into RU. The Mainstream Media isn't technically adept enough to sensationalize it yet, and the experts generally write it off as innocuous, so that's why you don't hear much about it.

  • FuslFusl Member
    edited May 2017

    jiggawattz said: RPKI: https://www.arin.net/resources/rpki/

    It's not widely adopted yet due to some technical limitations.

    Doesn't help in this case as the origin AS was still the same (Hetzner), the difference was just that the hops in between have changed (route leaks). RPKI/ROA only validates the originating AS# but not the upstreams or other BGP path hops in the middle.

    RPKI/ROA only helps preventing accidental hijacks, it does not help at accidental route leaks, nor does it prevent a full hijacking of prefixes with AS spoofing. I could as well just say I'm AS15169 and my prefix is 8.8.8.0/24, my upstream has an open session without much or any filtering in place, I have an open session with them and voila, I have the (almost) perfect BGP hijacking.

    It's just a mess, BGP is broken but it's IMHO so far the best available solution to connect autonomous systems together.

    Thanked by 1jiggawatt
  • WilliamWilliam Member
    edited May 2017

    jiggawattz said: There is a huge difference. Apples to oranges.

    Not really, it is the same ultimately:

    • not allowed to disclose (jail)
    • no way to go against it (no monetary loss as paid by gov, thus no legal recourse)
    • forced to comply (jail)
    • control taken over by gov if not trusted or not complied in past (and... jail)

    This is, literally, the same as in Russia - only difference is that the law is, mostly, not misused for now (however as we have seen on the BND scandal and similar it has been before.)

Sign In or Register to comment.