Pfsense 2.0.3 and external (commercial) tproxy solution



  • Hello!

    This is my first post here, glad to be part of pfSense comunity.

    I work to an ISP and we have a service of tproxy caching which runs on top of FreeBSD (from the very gentle crew of http://www.freebsdbrasil.com.br/). We can only put it to work when manualy removing the two default blocking rules from pf, by exporting its configs, removing it, and importing again (pfctl -nr | sed [remove the lines] > out.pf && pfctl -f out.pf or the like), and put a script to load ipfw related things.

    They are sad, too, to say that their product does not work with PF in any sense (sorry for the pun, it was accidental), so they crafted a script that does the job with ipfw under pfSense. What a mess, dude!

    Well, these are the steps for tproxy to work:

    From internel network:

    • Packet arrive in to pfSense from internal network

    • If destination port is 80, just forward the packet to tproxy

    • else, move it on to the Internet lands!

    From Internet:

    • Packet arrive into pfSense from the Internet

    • If source port is 80, just forward it to tproxy

    • else, move it on to the destination inside the ISP network!

    Inside tproxy (and very generaly speaking):

    • If the requested content is in cache, made a packet simulating a response from the original destination and send the cahced content back to the source.

    • else, forward the request without changing the packet to its real destination.

    From tproxy:

    • Packet arrive in to pfSense from tproxy

    • If destination port is 80, just forward the packet to the Internet

    • else, if source port is 80, forward to destination

    • else, just do the job of all times.

    My testing pfSense setup is something like this:

    • Two WANs in a MULTIWAN gateway, and their respective gateways, WAN1 and WAN2

    • A gateway called TPROXY

    • A gateway called INT_NET (which is the way to internal network from our pfSense)

    • A rule that fwd (all tcp traffic from internal network to port 80) to TPROXY

    • A catch all rule from int net to MULTIWAN

    • A rule on each WAN that fwd (all tcp traffic from Internet port 80 to internal network) to TPROXY

    • A rule that fwd (all tcp traffic from tproxy interface to port 80) to MULTIWAN

    • A rule that fwd (all tcp traffic from port 80 to internal network) to INT_NET

    My production config is just passing all traffic from INT_NET to MULTIWAN and allowing everything that comes in from WAN{1,2} and RIPv2 announcing the routes to INT_NET valid IPs via both WANs. There is no NAT (which is made by other server in INT_NET), there is no other complex setup. Just a little border gateway ;).

    Here is a picture of the ideal workflow: http://devio.us/~raitech/pfsenseexttproxy.png (attached too). My thinkering was about using route-to (policy routing under pfSense, right?), but it do not work.

    Many thanks for your atention and help!



  • Ok…

    do not know why, but it kinda work now. With some packets being dropped by the default block rule.

    From this thread http://forum.pfsense.org/index.php?topic=35400.0 I may think that this issue have something to do with multiwan and symetric routing enforcement.

    Filtering the firewall logs with the tproxy interface, I have a clue on that: all blocked packets have TCP:FA or TCP:RA. I am stucked on that now...

    Thank you in advance!



  • Problem solved with a change to OpenBSD 5.3… :/