Lots of snort alerts after pfsense update to 2.1.2



  • Since pfSense Update to 2.1.2 and the snort to v. 2.9.6.0 Package 3.0.6 on mainsite FW there have been lots of (http_inspect) alerts…
    In particular the followings

    (http_inspect) NO CONTENT-LENGTH OR TRANSFER-ENCODING IN HTTP RESPONSE

    suppress gen_id 120, sig_id 3

    (http_inspect) HTTP RESPONSE GZIP DECOMPRESSION FAILED

    suppress gen_id 120, sig_id 6

    (http_inspect) INVALID CONTENT-LENGTH OR CHUNK SIZE

    suppress gen_id 120, sig_id 8

    I had to disable those rules to get stable connectivity.

    It seems like situations described here
    https://forum.pfsense.org/index.php?topic=50367.15 (decoding issue)
    or
    https://forum.pfsense.org/index.php?topic=70695.msg386540#msg386540 (false positive)

    Respect to those threads the difference is that I got 4 upgraded pfSense FW (same HW, same snort version and conf), but just 1 (mainsite) produces (tons of) alerts... and hence blocks!
    Just to understand where the issue is... I tried to install a fresh "test fw" (i.e. a fresh 2.1.2 pfsense install (same hw), + snort on WAN interface)
    and the alerts arise as well.

    At this point, the big question is: why do alerts arise only on mainsite?
    Could be a mainsite WAN connection issue (DSL? ROUTER? PROVIDER?) ?



  • @vielfede:

    Since pfSense Update to 2.1.2 and the snort to v. 2.9.6.0 Package 3.0.6 on mainsite FW there have been lots of (http_inspect) alerts…
    In particular the followings

    (http_inspect) NO CONTENT-LENGTH OR TRANSFER-ENCODING IN HTTP RESPONSE

    suppress gen_id 120, sig_id 3

    (http_inspect) HTTP RESPONSE GZIP DECOMPRESSION FAILED

    suppress gen_id 120, sig_id 6

    (http_inspect) INVALID CONTENT-LENGTH OR CHUNK SIZE

    suppress gen_id 120, sig_id 8

    I had to disable those rules to get stable connectivity.

    It seems like situations described here
    https://forum.pfsense.org/index.php?topic=50367.15 (decoding issue)
    or
    https://forum.pfsense.org/index.php?topic=70695.msg386540#msg386540 (false positive)

    Respect to those threads the difference is that I got 4 upgraded pfSense FW (same HW, same snort version and conf), but just 1 (mainsite) produces (tons of) alerts... and hence blocks!
    Just to understand where the issue is... I tried to install a fresh "test fw" (i.e. a fresh 2.1.2 pfsense install (same hw), + snort on WAN interface)
    and the alerts arise as well.

    At this point, the big question is: why do alerts arise only on mainsite?
    Could be a mainsite WAN connection issue (DSL? ROUTER? PROVIDER?) ?

    vielfede:

    The key variable to check is the SRC or DST IP address of the sites getting blocked.  The fact some other firewalls running Snort are not experiencing the same event could very well be due to machines behind those firewalls not contacting the same Internet IP.  Another thing to check is that all the firewalls have the same Suppress List settings.  Perhaps the alerts are suppressed on the other firewalls?

    As a test, try to browse to the same sites that are blocked on the Main Site WAN from one of the other firewalls.  I am assuming here that you have different sites and each site has its own Internet connection.

    Bill



  • @bmeeks:

    The key variable to check is the SRC or DST IP address of the sites getting blocked.  The fact some other firewalls running Snort are not experiencing the same event could very well be due to machines behind those firewalls not contacting the same Internet IP.  Another thing to check is that all the firewalls have the same Suppress List settings.  Perhaps the alerts are suppressed on the other firewalls?

    As a test, try to browse to the same sites that are blocked on the Main Site WAN from one of the other firewalls.  I am assuming here that you have different sites and each site has its own Internet connection.

    Bill

    First of all thank you!
    Second … I have already done the test you suggested: From site 2 (for example) if I contact the same site, generating alert sid 120:3 on mainsite,  no alert is generated on site2 fw.
    Suppress list is different, but sid 120:3 is not in!
    Indeed on site2 suppresslist contains just sid 120:8...
    All 4 firewalls block SRC IP.

    Just to clarify my post: before upgrade no "strange" alerts were present on mainsite fw logs and hence something is changed upgrading snort 2.9.4.6=>2.9.6.0.



  • @vielfede:

    @bmeeks:

    The key variable to check is the SRC or DST IP address of the sites getting blocked.  The fact some other firewalls running Snort are not experiencing the same event could very well be due to machines behind those firewalls not contacting the same Internet IP.  Another thing to check is that all the firewalls have the same Suppress List settings.  Perhaps the alerts are suppressed on the other firewalls?

    As a test, try to browse to the same sites that are blocked on the Main Site WAN from one of the other firewalls.  I am assuming here that you have different sites and each site has its own Internet connection.

    Bill

    First of all thank you!
    Second … I have already done the test you suggested: From site 2 (for example) if I contact the same site, generating alert sid 120:3 on mainsite,  no alert is generated on site2 fw.
    Suppress list is different, but sid 120:3 is not in!
    Indeed on site2 suppresslist contains just sid 120:8...
    All 4 firewalls block SRC IP.

    Just to clarify my post: before upgrade no "strange" alerts were present on mainsite fw logs and hence something is changed upgrading snort 2.9.4.6=>2.9.6.0.

    It may be time for a packet capture and compare to see what is happening.  It could be that something with the provider or connection time is causing some issues.  The issues may have always been there, but the older Snort may have ignored or not detected them.  That's just a guess.  It's also possible the new Snort binary is false triggering on something with HTTP_INSPECT.  There are some tunable settings for HTTP_INSPECT that could help.  Those are on the PREPROCESSORS tab.

    Bill



  • @bmeeks:

    It may be time for a packet capture and compare to see what is happening.  It could be that something with the provider or connection time is causing some issues.  The issues may have always been there, but the older Snort may have ignored or not detected them.  That's just a guess.  It's also possible the new Snort binary is false triggering on something with HTTP_INSPECT.  There are some tunable settings for HTTP_INSPECT that could help.  Those are on the PREPROCESSORS tab.

    Bill

    Thanks again…
    This morning I have done some tests... packet capture, preprocs option check (1by1) etc.
    Meanwhile I notice the following: if I  try to connect to http://www.somesite.com/index.php/Sp%C3%A9cial:Suivi_des_liens/page (a DOUBLE_DECODE link trigger) to produce a 119:2 alarm on demand and...

    • from mainsite=>alert is triggered on mainsite fw

    • from site2=>alert is not trigghered on site2 fw

    Now consider picture below: wan ip is in the pass list!.
    Am I wrong or the correct behaviour is the firt one? alert but no block (sorry, I read doc and modifyed)

    If yes… Why doesn't site2 produce alarm 119:2?

    I looked everywhere without success.
    Thank you in advance.




  • @vielfede:

    @bmeeks:

    It may be time for a packet capture and compare to see what is happening.  It could be that something with the provider or connection time is causing some issues.  The issues may have always been there, but the older Snort may have ignored or not detected them.  That's just a guess.  It's also possible the new Snort binary is false triggering on something with HTTP_INSPECT.  There are some tunable settings for HTTP_INSPECT that could help.  Those are on the PREPROCESSORS tab.

    Bill

    Thanks again…
    This morning I have done some tests... packet capture, preprocs option check (1by1) etc.
    Meanwhile I notice the following: if I  try to connect to http://www.somesite.com/index.php/Sp%C3%A9cial:Suivi_des_liens/page (a DOUBLE_DECODE link trigger) to produce a 119:2 alarm on demand and...

    • from mainsite=>alert is triggered on mainsite fw

    • from site2=>alert is not trigghered on site2 fw

    Now consider picture below: wan ip is in the pass list!.
    Am I wrong or the correct behaviour is the firt one? alert but no block (sorry, I read doc and modifyed)

    If yes… Why doesn't site2 produce alarm 119:2?

    I looked everywhere without success.
    Thank you in advance.

    If you have BOTH enabled for which IP address to block, then it should not block the WAN IP but it will block the destination.  On the other hand, if you change the which IP to block to SRC, then it should not insert a block for the WAN (and would not for the DST since it is not selected).

    As for why one FW does and one does not, are you sure that both are configured exactly the same (in particular the "which IP to block" is set to BOTH on both of them)?  Are the Suppress Lists identical for both firewalls, and is the correct (and same) Suppress List enabled for both firewalls?  This is configured on the INTERFACE SETTINGS tab.

    Bill



  • @bmeeks:

    If you have BOTH enabled for which IP address to block, then it should not block the WAN IP but it will block the destination.  On the other hand, if you change the which IP to block to SRC, then it should not insert a block for the WAN (and would not for the DST since it is not selected).

    As for why one FW does and one does not, are you sure that both are configured exactly the same (in particular the "which IP to block" is set to BOTH on both of them)?  Are the Suppress Lists identical for both firewalls, and is the correct (and same) Suppress List enabled for both firewalls?  This is configured on the INTERFACE SETTINGS tab.

    Bill

    I block just SRC on both FW. Suppress lists are not identical but 119:2 is not in both suppress lists.
    Just one thing is different mainsite fw is an appliance whereas site2 fw is a VM (VMWare). Could be that?



  • @vielfede:

    @bmeeks:

    If you have BOTH enabled for which IP address to block, then it should not block the WAN IP but it will block the destination.  On the other hand, if you change the which IP to block to SRC, then it should not insert a block for the WAN (and would not for the DST since it is not selected).

    As for why one FW does and one does not, are you sure that both are configured exactly the same (in particular the "which IP to block" is set to BOTH on both of them)?  Are the Suppress Lists identical for both firewalls, and is the correct (and same) Suppress List enabled for both firewalls?  This is configured on the INTERFACE SETTINGS tab.

    Bill

    I block just SRC on both FW. Suppress lists are not identical but 119:2 is not in both suppress lists.
    Just one thing is different mainsite fw is an appliance whereas site2 fw is a VM (VMWare). Could be that?

    Yes, it's possible that VMware is masking whatever is triggering the false positive from that particular web site.  This alert is triggered by little idiosyncrasies in the structure of the packet bits.  It seems to be easily tripped up.  I don't know if that is the fault of the code within the Snort binary, or a problem with web sites not precisely following all the RFC standards.  I tend to think there are issues in the binary.

    Instead of worrying about it too much, I would simply add the Suppress List entry for the 119:2 signature and forget about it.  You can add it to both firewalls, or only the one giving you a problem.  In my humble opinion, there are many issues with the HTTP_INSPECT preprocessor when it runs in the "real world".  I know the Snort developers might take issue with my assessment, but it seems to generate a ton of false positives (by far the most of any of the preprocessors).  For that reason, I don't obsess too much when I put these on a Suppress List.  You can also entirely disable that particular rule if you desire.

    Bill



  • @bmeeks:

    Instead of worrying about it too much, I would simply add the Suppress List entry for the 119:2 signature and forget about it.  You can add it to both firewalls, or only the one giving you a problem.  In my humble opinion, there are many issues with the HTTP_INSPECT preprocessor when it runs in the "real world".  I know the Snort developers might take issue with my assessment, but it seems to generate a ton of false positives (by far the most of any of the preprocessors).  For that reason, I don't obsess too much when I put these on a Suppress List.  You can also entirely disable that particular rule if you desire.

    Bill

    I agree with you. My concern was about different behaviour.

    I have to point out just one more thing: +or- every preprocs alert come from
    SRC=WAN IP
    DST= ANY
    i.e. http request generate the alert, and not the web response.
    And I stated on my first post This is a standard behaviour also on a fresh pfsense 2.1.2 + snort install.
    Didn't you notice / try / test that?



  • @vielfede:

    I have to point out just one more thing: +or- every preprocs alert come from
    SRC=WAN IP
    DST= ANY
    i.e. http request generate the alert, and not the web response.
    And I stated on my first post This is a standard behaviour also on a fresh pfsense 2.1.2 + snort install.
    Didn't you notice / try / test that?

    Not sure I'm completely understanding your question.  If you are using NAT and running Snort on the WAN interface, then every packet leaving your network will appear to come only from the WAN IP (hence the SRC will be the WAN IP) – and the same for all incoming packets:  the destination will appear to be the WAN.  So any HTTP_INSPECT issues would appear to always be only from the firewall.

    I have not tested sourcing an HTTP request to random web sites directly from the firewall.  Generally you would not want to do that anyway (visit HTTP destinations directly from the firewall's interface) except for known good sites like the pfSense package repositories, for example.

    Bill



  • @bmeeks:

    Not sure I'm completely understanding your question.  If you are using NAT and running Snort on the WAN interface, then every packet leaving your network will appear to come only from the WAN IP (hence the SRC will be the WAN IP) – and the same for all incoming packets:  the destination will appear to be the WAN.  So any HTTP_INSPECT issues would appear to always be only from the firewall.

    That's my conf!  ;)

    @bmeeks:

    I have not tested sourcing an HTTP request to random web sites directly from the firewall.  Generally you would not want to do that anyway (visit HTTP destinations directly from the firewall's interface) except for known good sites like the pfSense package repositories, for example.

    Neither did I! Every request comes from LAN clients (using NAT)

    P.S. Sorry to answer late.



  • @vielfede:

    Neither did I! Every request comes from LAN clients (using NAT)

    P.S. Sorry to answer late.

    Then I am baffled as to why one firewall has the issue while another (supposedly configured the same way) does not.  There would be only two logical explanations:  (1) either they are, in fact, not actually configured the same although they are supposed to be, or (2) you said one was VMware-based, and it is conceivable that VMware's virtual networking could play into the mix with that HTTP_INSPECT alert.

    Bill


  • Moderator

    Then I am baffled as to why one firewall has the issue while another (supposedly configured the same way) does not.  There would be only two logical explanations:  (1) either they are, in fact, not actually configured the same although they are supposed to be, or (2) you said one was VMware-based, and it is conceivable that VMware's virtual networking could play into the mix with that HTTP_INSPECT alert.

    Maybe setting up a standalone pfSense install to see if VMWare is the culprit is the best solution to rule out the Virtual machine idiosyncrasys. or a VM machine for the Main Site.