HTTP keep-alive problem after upgrading from 2.2.4 to 2.3.4



  • Hi all

    Just upgraded our pfsense from 2.2.4 to 2.3.4, and something strange happened.
    We have a web server behind the firewall, running IIS 7 on Windows 2008 R2.
    This server has some php pages that query local files but via the server's public url. i.e., we have things like getimagesize('https://www.ourserver.com/image.jpg').
    So the server si querying itself, but the query is routed via the pfSense firewall since it uses the public URL of the server, which resolves to one of the WAN ips of the firewall.

    This worked fine, but suddenly stopped working after this upgrade. The url worked just fine when executed manually on the server (wget, curl, or even in a browser, everything worked), but not when queried via a php script.

    After some research, it appeared that the PHP function where waiting for the connection to be closed by the server, i.e. to receive a "Connection: close" header in the response.

    What could have changed between 2.2.4 and 2.3.4 ? What can I do to solve this problem ?
    Thanks a lot !



  • I wouldn't be surprised if the state isn't fully established and the only reason it seems to work is the request is smaller than the transmission window. Check the states that are created, see if they get fully established.



  • @Harvy66:

    I wouldn't be surprised if the state isn't fully established and the only reason it seems to work is the request is smaller than the transmission window. Check the states that are created, see if they get fully established.

    Hi, thanks for your help.

    I think this is the offending state (apparently pfsense is smart enough to route this connection via its LAN interface) :
    LAN tcp 192.168.1.253:23655 -> 192.168.1.40:443 ESTABLISHED:FIN_WAIT_2 110 / 105 6 KiB / 148 KiB

    What should I do to fix this ? Is the problem on my server ?



  • FIN indicates the states has been signaled to be closed. One of the sides sent a FIN packet. Are the packets even getting sent to the web server? Is the NAT still working?



  • On the server, the "server" side connection is in a FIN_WAIT_2 state :

    Proto  Local Address          Foreign Address        State
    TCP    192.168.2.40:80        192.168.2.253:22360    FIN_WAIT_2
    

    but the "client" side connection is in an ESTABLISHED state though :

    Proto  Local Address          Foreign Address        State
    TCP    192.168.2.40:1985      <mywanip>:80             ESTABLISHED</mywanip>
    

    And the corresponding pfSense States :

    LAN 	tcp 	192.168.2.40:2059 -> 127.0.0.1:19000 (<mywanip>:80) 	ESTABLISHED:ESTABLISHED 	32 / 58 	1 KiB / 81 KiB 	
    LAN 	tcp 	192.168.2.253:1916 -> 192.168.2.40:80 	ESTABLISHED:FIN_WAIT_2 	60 / 59 	3 KiB / 81 KiB</mywanip>
    


  • I'm going to have to leave this to someone with more knowledge or experience on the subject. Seeing "127.0.0.1" in the states is very strange to me.

    It actually looks like an asymmetric route. 192.168.2.40 is connected directly to 192.168.2.253, but 192.168.2.40 thinks it's connected to <mywanip>. I think you have the wrong NAT type being used, but like my first sentence, I'm not familiar with this exact situation.</mywanip>



  • @Harvy66:

    I'm going to have to leave this to someone with more knowledge or experience on the subject. Seeing "127.0.0.1" in the states is very strange to me.

    It actually looks like an asymmetric route. 192.168.2.40 is connected directly to 192.168.2.253, but 192.168.2.40 thinks it's connected to <mywanip>. I think you have the wrong NAT type being used, but like my first sentence, I'm not familiar with this exact situation.</mywanip>

    Well, thanks for your insights !
    Your were correct : I was using the wrong type of NAT !
    This NAT rule was using NAT+Proxy (from what I understand from the docs, this was the only option in earlier pfSense versions), and switching to pure NAT fixes this NAT Reflection problem.

    Thanks a lot !



  • OK, my bad, it did not work after all (I made a mistake with my test script, NAT Reflection did not happen).
    I resorted to configure a split DNS for this specific server.