Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Policy routing squid3 tcp marked packets to VPN using firewall rules

    Scheduled Pinned Locked Moved Routing and Multi WAN
    12 Posts 4 Posters 4.0k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • H
      heper
      last edited by

      have you tried setting the outgoing address to 127.0.0.1 & using floating rules (quick/out)?

      this might work, testing required

      1 Reply Last reply Reply Quote 0
      • kesawiK
        kesawi
        last edited by

        If I set the outgoing address to 127.0.0.1 I'm assuming it will end up on the lo interface. How would I assign a floating rule as there's no localhost option in the interface when creating a rule?

        1 Reply Last reply Reply Quote 0
        • H
          heper
          last edited by

          localhost will try to go out the default gateway.

          if you can catch it before it goes out "default_wan" and force out the vpn_gateway, you might get away with it. (its a pain to get it working)

          1 Reply Last reply Reply Quote 0
          • kesawiK
            kesawi
            last edited by

            How would I configure the firewall rule? I don't want to create a specific rule for each destination address, as I want to route squid based on the dstdomain acl type, and maintaining an alias list would be an administrative overhead. Would the source address be 127.0.0.1 or the WAN interface IP? If I use either of these addresses without a destination, will that have other unintended consequences (even if I just limit the destination ports to 80 and 443)?

            1 Reply Last reply Reply Quote 0
            • H
              heper
              last edited by

              its been too long since i've bothered with squid & have no intention of going that way again.

              -start by not specifying an interface or source, direction out, dest_port 80, turn on logging.
              -see if you can get it todo what you want with policy routing
              -trigger the rule with squid and see what the source address is & add it to the fw rule to avoid unintended behaviour.

              it won't work the first time, it probably won't work the second time … if you keep going at it for long enough, there is a small chance of getting it right ;)

              1 Reply Last reply Reply Quote 0
              • kesawiK
                kesawi
                last edited by

                I've come up with a concept of how I could possible get squid to direct traffic via the VPN with a dynamic IP but I'm not sure how to implement it in pfSense:

                • Create an additional localhost interface (say lo1 with IP 127.0.0.2)

                • Redirect traffic from the new lo1 out through my VPN interface ovpnc1

                • Change the squid tcp_outgoing_address configuration directive to point to the new local host address 127.0.0.2

                In theory this should resolve my issue with a dynamic IP address for the VPN interface as squid will only have the  static IP of the new localhost interface to contend with. How do I undertake steps 1 & 2?

                1 Reply Last reply Reply Quote 0
                • S
                  sjen5
                  last edited by

                  Did you get this to work.  I have been trying to do that same thing.

                  The following works fine;

                  • set tcp_outgoing_address to the dynamic IP assigned by your VPN provider using acls on certain domains

                  acl TEST dstdomain .ifconfig.co;
                  acl TEST dstdomain .whatismyipaddress.com;
                  tcp_outgoing_address 10.x.x.x   TEST;

                  The above works, however would stop working every time i get a new IP address from the VPN provider 10.x.x.x.  So I tried the following based on various posts around the internet, however I cannot get it to work.

                  • add a VIP to localhost of 127.0.0.2
                  • set tcp_outgoing_address to the VIP using acls on certain domains

                  acl TEST dstdomain .ifconfig.co;
                  acl TEST dstdomain .whatismyipaddress.com;
                  tcp_outgoing_address 127.0.0.2   TEST;

                  At this point in my configuration neither of the above domains works, I believe because there is no route to the internet for 127.0.0.2 to the internet.  As you can see there is a VIP for 127.0.0.2

                  [2.2.2-RELEASE][…..................]/home/…...............: ping 127.0.0.2
                  PING 127.0.0.2 (127.0.0.2): 56 data bytes
                  64 bytes from 127.0.0.2: icmp_seq=0 ttl=64 time=0.592 ms
                  64 bytes from 127.0.0.2: icmp_seq=1 ttl=64 time=0.442 ms

                  So I thought it may be because there is no outgoing nat rule in my case for 127.0.0.2 port 80.  So I add a nat rule like below, at this point routing just to WAN.

                  WAN 127.0.0.2/32 * * * WAN address * NO 127.0.0.2 to WAN

                  Now whatismyipaddress.com works fine, it's routed out of my WAN.  I want it to go out the VPN, so I modify the nat above to the following;

                  VPN_WAN  127.0.0.2/32 * * * VPN_WAN address * NO 127.0.0.2 to VPN

                  Again, page wont load, squid error (49) Can't assign requested address

                  So now I thought that perhaps its because the traffic isnt actually being routed to this gateway, so i created a floating firewall rule;  This should forward all TCP packets for source 127.0.0.2 out the VPN_WAN

                  IPv4 TCP 127.0.0.2 * * * VPN_WAN none 127.0.0.2

                  Unfortunately at this point rather than squid returning an error, the page load just times out, it seems that the packet forwarding inside pfsense is getting a bit lost :(  There are no obvious entries in the firewall log that would indicate an issue.  There is an entry however which has source 127.0.0.2 as expected and a destination of 104.71.204.14:80 (http://whatismyipaddress.com/) as expected, however the interface is still WAN.  It would appear that the floating rule is not forwarding out the VPN_WAN

                  any ideas?

                  simon

                  1 Reply Last reply Reply Quote 0
                  • kesawiK
                    kesawi
                    last edited by

                    I was able to create the additional loopback interface using the method described in /index.php?topic=63653.msg344553#msg344553 and then make it persistent at each boot with the Shellcmd package, but haven't been able to work out how to do the traffic redirection.

                    1 Reply Last reply Reply Quote 0
                    • kesawiK
                      kesawi
                      last edited by

                      I ended up solving my problem by creating a script which I use cron to run every 5 minutes. The script checks the current VPN interface address against the one provided in the squid.conf file and if it's different, replaces it and forces squid to reload the configuration file:

                      #!/bin/sh
                      
                      # Variables
                      VPN_IFACE=ovpnc2
                      SQUID_CONFIG_FILE=/usr/local/etc/squid/squid.conf
                      
                      # Get current IP address of VPN interface
                      VPN_IFACE_IP=$(ifconfig $VPN_IFACE | awk '{print $2}' | egrep -o '([0-9]+\.){3}[0-9]+')
                      
                      # Check if VPN interface is up and exit if it isn't
                      if [ -z "$VPN_IFACE_IP" ]
                      then
                              exit 0;
                      fi
                      
                      # Check current IP for VPN interface in squid.conf file
                      VPN_CONFIG_IP=$(grep -m 1 "tcp_outgoing_address" $SQUID_CONFIG_FILE | awk '{print $2}' | egrep -o '([0-9]+\.){3}[0-9]+')
                      
                      # Check if the config file matches the current VPN interface IP, and if so exit script
                      if [ "$VPN_IFACE_IP" == "$VPN_CONFIG_IP" ]
                      then
                              exit 0;
                      fi
                      
                      # Replace the previous IP address in the squid.conf file with the current VPN interface address
                      sed -ie 's/'"$VPN_CONFIG_IP"'/'"$VPN_IFACE_IP"'/' $SQUID_CONFIG_FILE
                      
                      # Force reload of the new squid.conf file
                      /usr/local/sbin/squid -k reconfigure
                      
                      
                      kesawiK 1 Reply Last reply Reply Quote 0
                      • R
                        rob_kae
                        last edited by

                        Just wanted to give kesawi some kudos for this script, it really works and to me is the best way to control the outbound IP of SQUID when you want to use a VPN which can have a dynamic IP, and you dont want to set the default gateway on your PFsense box as a VPN connection (which for me caused problems)

                        The script doesnt change the IP address in the web interface (that probably needs the webinterface to be restarted) but if you download the CRON pfsense package, and save the script that Kesawi wrote to a file on the PFsense box (I called mine /usr/bin/IP.sh) the script really works.

                        If you continually run " grep -m 1 "tcp_outgoing_address" /usr/local/etc/squid/squid.conf | awk '{print $2}' | egrep -o '([0-9]+.){3}[0-9]+' " in a terminal session on the Pfsense box, you can see if the CRON job is changing the value.

                        Nice work!

                        Rob_kae

                        1 Reply Last reply Reply Quote 1
                        • kesawiK
                          kesawi @kesawi
                          last edited by

                          I've since updated this script to handle failover to a second VPN where required.

                          #!/bin/sh
                          
                          # Variables
                          # VPN_IFACE1 is the primary VPN interface, VPN_IFACE2 is the backup VPN interface
                          VPN_IFACE1=ovpnc1
                          VPN_IFACE2=ovpnc2
                          SQUID_CONFIG_FILE=/usr/local/etc/squid/squid.conf
                          
                          # Check whether VPN interfaces are connected and assign connected interface to VPN_IFACE. Exit if both are down
                          VPN_IFACE1_STAUS=$(ifconfig $VPN_IFACE1 | awk '{print $2}' | egrep -o UP)
                          VPN_IFACE2_STAUS=$(ifconfig $VPN_IFACE2 | awk '{print $2}' | egrep -o UP)
                          
                          if [ -z "VPN_IFACE1_STATUS" ]
                          then
                          	VPN_IFACE=$VPN_IFACE1
                          elif [ -z "VPN_IFACE2_STATUS" ]
                          then
                          	VPN_IFACE=$VPN_IFACE2
                          else
                          	echo "Both VPN interfaces down"
                          	exit 1;
                          fi
                          
                          # Get current IP address of VPN interface
                          VPN_IFACE_IP=$(ifconfig $VPN_IFACE | awk '{print $2}' | egrep -o '([0-9]+\.){3}[0-9]+')
                          
                          # Check current IP for VPN interface in squid.conf file
                          VPN_CONFIG_IP=$(grep -m 1 "tcp_outgoing_address" $SQUID_CONFIG_FILE | awk '{print $2}' | egrep -o '([0-9]+\.){3}[0-9]+')
                          
                          # Check if the config file matches the current VPN interface IP, and if so exit script
                          if [ "$VPN_IFACE_IP" == "$VPN_CONFIG_IP" ]
                          then
                                  exit 0;
                          fi
                          
                          # Replace the previous IP address in the squid.conf file with the current VPN interface address
                          sed -ie 's/'"$VPN_CONFIG_IP"'/'"$VPN_IFACE_IP"'/' $SQUID_CONFIG_FILE
                          
                          # Force reload of the new squid.conf file
                          /usr/local/sbin/squid -k reconfigure
                          
                          1 Reply Last reply Reply Quote 0
                          • First post
                            Last post
                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.