Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    DDoS pfSense dies on XSYN and OVH scripts.

    Scheduled Pinned Locked Moved Firewalling
    93 Posts 11 Posters 27.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • K
      kejianshi
      last edited by

      This is exactly why I never joke about farting in a crowded elevator to my GF.
      I'll mention it in jest.  Someone will actually do it and suddenly….  I'm the bad guy...  :-\

      1 Reply Last reply Reply Quote 0
      • ?
        Guest
        last edited by

        @Supermule:

        Dude….pls. READ what it says....

        IT takes the forum and the store offline. IT didnt say I did it.... but suggests that YOU are vulnerable as well ....

        So I inform you and then its my fault and me beeing behind it??

        I am only reporting whats found among people I talk to IRL frequently. They test like mad people at the moment to come up with whats wrong with the software and I only report it when and IF we find something....

        Reporting issues with the software is fine.  Reporting issues with the software in such a way that someone can reproduce them is even better (so your 'script' is actually useful).

        Attacking other people's infrastructure (which you reported having done) is not fine.

        And, frankly, you DID NOT INFORM ME.

        Having a private conversation with Chris (and at this point I don't care if it was you, or Lowprofile , or someone else) and having a discussion in the forum where you report that "no response from the pfsense guys"

        @Supermule:

        We have had ZERO response from the pfSense guys. This is quite disturbing since we can take down any site protected by pfSense as it is.

        Right now its better to run without pf at all and rely on windows Firewall on VM's and let pf handle the routing. Only way to survive the attacks as it is.

        Thinking og getting my old ISA2006 online again to test and see how it behaves.

        is not responsible, or friendly, or even … professional.

        1 Reply Last reply Reply Quote 0
        • ?
          Guest
          last edited by

          @kejianshi:

          This is exactly why I never joke about farting in a crowded elevator to my GF.
          I'll mention it in jest.  Someone will actually do it and suddenly….  I'm the bad guy...  :-\

          So your assertion is that Supermule was joking when he reported:

          @Supermule:

          It didnt help. It takes this forum and store.netgate.com down as well easily.

          1 Reply Last reply Reply Quote 0
          • K
            kejianshi
            last edited by

            I'd have though he was just assuming.

            1 Reply Last reply Reply Quote 0
            • S
              Supermule Banned
              last edited by

              Did the guy reporting that WTC towers collapsed fly the plane?

              Did Charlie Hebdo hire those who almost wiped out the magazine??

              Because I report it in here, then I am suddenly responsible for any attack that takes either this site or any other site offline?

              Is that professional to accuse me of doing that or could we somehow get to the bottom of this as fast as possible so we can be ahead of those who wants us of the interwebs??

              Guess what I prefer….. but hey. I reported it, so I must be the one....

              I DDoS the shit out of myself and my own sites at the moment since I am trying to combat this with what I have and what I can tweak in the code.

              We have gotten a long way but are not there yet...

              1 Reply Last reply Reply Quote 0
              • ?
                Guest
                last edited by

                So this thing is a fairly straight-forward SYN flood.

                It does a bunch of work to support random generating source addresses.

                rand_cmwc(void) generates values that can easily be turned into IP addresses

                so, for instance, assuming that you correct rand_cmwc() and init_rand(), then making a small test driver:

                int main()
                {
                        int i;
                        struct in_addr a;

                init_rand(time(NULL));
                        for (i = 0; i < 9; i++) {
                          a.s_addr = rand_cmwc();
                          printf("Random address: %s\n", inet_ntoa(a));
                        }
                }

                will yield (as an example, your IP addresses will be different)

                [jim@qattest0 ~]$ ./a.out
                Random IP addressr: 8.25.10.176
                Random IP addressr: 48.232.250.197
                Random IP addressr: 240.99.230.219
                Random IP addressr: 93.5.177.255
                Random IP addressr: 52.97.250.155
                Random IP addressr: 97.95.88.207
                Random IP addressr: 60.97.173.24
                Random IP addressr: 188.23.14.176
                Random IP addressr: 194.233.252.197

                The main part of the code implements a function named "flood()", which is run by one or more threads.
                Flood runs in a loop, using a random source address, IP id, and TCP source port and TCP sequence number.

                1 Reply Last reply Reply Quote 0
                • S
                  Supermule Banned
                  last edited by

                  Yes go on??

                  1 Reply Last reply Reply Quote 0
                  • ?
                    Guest
                    last edited by

                    @Supermule:

                    Did the guy reporting that WTC towers collapsed fly the plane?

                    Did Charlie Hebdo hire those who almost wiped out the magazine??

                    Meh.

                    @Supermule:

                    Because I report it in here, then I am suddenly responsible for any attack that takes either this site or any other site offline?

                    No, because you report to have used this to attack "this site" (the forum) and "store.negate.com" makes you, by your own admission, someone who used this to attack our infrastructure.

                    @Supermule:

                    Is that professional to accuse me of doing that or could we somehow get to the bottom of this as fast as possible so we can be ahead of those who wants us of the interwebs??

                    What?  You reported that you took down store.netgate.com and the forum.

                    @Supermule:

                    I DDoS the shit out of myself and my own sites at the moment since I am trying to combat this with what I have and what I can tweak in the code.

                    We have gotten a long way but are not there yet…

                    I don't care what you do to your on infrastructure.

                    1 Reply Last reply Reply Quote 0
                    • S
                      Supermule Banned
                      last edited by

                      Great…

                      1 Reply Last reply Reply Quote 0
                      • ?
                        Guest
                        last edited by

                        @kejianshi:

                        I'd have though he was just assuming.

                        He made a definitive statement.  "It takes this forum and store.netgate.com down as well easily."

                        He did not say, "It would probably take this forum and store.netgate.com down as well easily."

                        He did not say, "I assume it takes this forum and store.netgate.com down as well easily."

                        Even if he didn't attack these sites, (and, if you read carefully, I have not accused, only reported back what he said),
                        He complained about a "lack of response" from the pfsense team, but:

                        Neither he or lowprofile contacted coreteam@
                        Neither he or low profile contacted security@

                        All I have is:

                        • reports that Chris was involved, somehow.

                        • knowledge that the forum was taken offline by what appears to be a similar attack on Saturday

                        • people running around here reporting that this is broken in pfSense but not in FreeBSD, when, given inspection of the "script" (code) posted, there is no reason to believe that such a statement is true.

                        1 Reply Last reply Reply Quote 0
                        • ?
                          Guest
                          last edited by

                          @Supermule:

                          Yes go on??

                          Go on?  First we seek to understand what is happening, and now we'll go test it, and if there is something we can do about it (it's poorly named, I don't see how it generates a SYN at all, this may be a flaw we can advantage), we will develop a fix and either document it (if it's something that can be done from the GUI or by, i.e. plugging tunables into loader.conf.local) or develop a patch and get it into a release.

                          1 Reply Last reply Reply Quote 0
                          • ?
                            Guest
                            last edited by

                            Does anyone have a copy of whatever the "OVH script" might be?

                            1 Reply Last reply Reply Quote 0
                            • S
                              Supermule Banned
                              last edited by

                              Thats great to hear.

                              @gonzopancho:

                              @Supermule:

                              Yes go on??

                              Go on?  First we seek to understand what is happening, and now we'll go test it, and if there is something we can do about it (it's poorly named, I don't see how it generates a SYN at all, this may be a flaw we can advantage), we will develop a fix and either document it (if it's something that can be done from the GUI or by, i.e. plugging tunables into loader.conf.local) or develop a patch and get it into a release.

                              1 Reply Last reply Reply Quote 0
                              • H
                                Harvy66
                                last edited by

                                It sounds like there is still some doubt about what this actually was.

                                After looking at the C code, it looked to me that it is a SSYN(Spoofed SYN) attack. Is this not correct? I know SYN cookies help against "SYN" attacks, but does it primarily only help when the source IP is not spoofed, but coming from a single IP or select few IPs?

                                I guess my question is, does PFSense only handle a SYN DOS but not a SYN DDOS?

                                edit: As pointed out, I forgot that this attack still works even if no states are created(SYN blocked), but it is exasperated if states are created.

                                1 Reply Last reply Reply Quote 0
                                • S
                                  Supermule Banned
                                  last edited by

                                  Limiting the creation of states help a great deal but despite low total traffic and almost no states created, the firewall chokes somehow.

                                  We dont know why yet and actually dont have a clue where to look.

                                  Using the syn cookie feature doesnt help.

                                  So you could be right about the SYN DDoS difference. Thats why we are looking at the FW's response to spoofed packages thats not really traffic…

                                  1 Reply Last reply Reply Quote 0
                                  • S
                                    Supermule Banned
                                    last edited by

                                    We have even added some things under system -> tunables to see if it will combat the flooding, but it doesnt.

                                    We cannot get higher maxsockbuf then the FW crashes constantly. It could be an issue on 10GbE connections since they state that this needs to be increased because of bigger load and throughput…

                                    system_tunables.PNG
                                    system_tunables.PNG_thumb

                                    1 Reply Last reply Reply Quote 0
                                    • S
                                      Supermule Banned
                                      last edited by

                                      I think we have to look at the kernel route cache and see if that could be tuned somehow.

                                      As stated it seems to choke somewhere when flooded and it doesnt take that big amount of traffic if the attack is done right…

                                      1 Reply Last reply Reply Quote 0
                                      • H
                                        Harvy66
                                        last edited by

                                        hmmm…

                                        Does this affect UDP? If a bunch of traffic from many source IPs and ports in the form of UDP packets hit, would the same issues occur? If not, then what is different about SYN packets...

                                        If this was a routing issue, then I would think it'd affect both protocols equally, unless TCP was somehow integrated into the routing. In an ideal, but not always practical world, layer 3 and 4 would not leak into each other.

                                        1 Reply Last reply Reply Quote 0
                                        • S
                                          Supermule Banned
                                          last edited by

                                          It can be made to hit UDP as well

                                          21:42:35.519506 4c:00:82:ee:cf:d9 > 00:0c:29:d4:ef:2c, ethertype IPv4 (0x0800), length 67: (tos 0x0, ttl 70, id 28520, offset 0, flags [none], proto UDP (17), length 53)
                                              38.236.77.218.27015 > 80.197.144.99.80: [no cksum] UDP, length 25
                                          21:42:35.519522 4c:00:82:ee:cf:d9 > 00:0c:29:d4:ef:2c, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 69, id 45833, offset 0, flags [none], proto UDP (17), length 48)
                                              110.201.38.197.27015 > 80.197.144.99.80: [no cksum] UDP, length 20
                                          21:42:35.519528 4c:00:82:ee:cf:d9 > 00:0c:29:d4:ef:2c, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 38, id 61664, offset 0, flags [none], proto UDP (17), length 48)
                                              168.105.75.13.27015 > 80.197.144.99.80: [no cksum] UDP, length 20
                                          21:42:35.519536 4c:00:82:ee:cf:d9 > 00:0c:29:d4:ef:2c, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 77, id 18528, offset 0, flags [none], proto UDP (17), length 37)

                                          [modded by gonzopancho.  This is what gists are for.]

                                          54.77.185.24.27015 > 80.197.144.99.80: [no cksum] UDP, length 25
                                          21:42:35.532664 4c:00:82:ee:cf:d9 > 00:0c:29:d4:ef:2c, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 6, id 2771, offset 0, flags [none], proto UDP (17), length 48)
                                              121.176.97.110.27015 > 80.197.144.99.80: [no cksum] UDP, length 20
                                          21:42:35.532670 4c:00:82:ee:cf

                                          1 Reply Last reply Reply Quote 0
                                          • ?
                                            Guest
                                            last edited by

                                            @Harvy66:

                                            It sounds like there is still some doubt about what this actually was.

                                            After looking at the C code, it looked to me that it is a SSYN(Spoofed SYN) attack. Is this not correct? I know SYN cookies help against "SYN" attacks, but does it primarily only help when the source IP is not spoofed, but coming from a single IP or select few IPs?

                                            I guess my question is, does PFSense only handle a SYN DOS but not a SYN DDOS?

                                            edit: As pointed out, I forgot that this attack still works even if no states are created(SYN blocked), but it is exasperated if states are created.

                                            As it turns out, linux (the code only runs on linux) creates a new 'connection' as a side-effect.  So there are SYNs in each packet.

                                            The question "does PFSense only handle a SYN DOS but not a SYN DDOS" is nonsense.

                                            this is an attack on anything that has a state table.  Point in fact, if you turn off "keep state" on the affected rule, pfSense happily runs quite well.
                                            (we're testing on/through 10G interfaces here.)

                                            So, assuming you have "keep state" in the affected rule(s), this will ALSO occur on FreeBSD.  (So the report that it didn't was false, or didn't properly test the condition..)

                                            We're investigating synproxy as a defense, but, if you want your firewall to stay up right now, some combination of turning off state for the affected rule(s), or keeping state table creation bounded on the affected rules, is your best bet for now.

                                            More as it occurs.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.