Transparent proxy not working after upgrade to 2.2



  • although it's not a squid issue, I've tried to use netcat to listen to the same port as squid and I don't receive any traffic

    this is what I get with pfctl while trying to connect to google.com from a client(192.168.1.9)

    rdr on fxp0 inet proto tcp from any to ! (fxp0) port = http -> 127.0.0.1 port 3128
    pass in quick on fxp0 proto tcp from any to ! (fxp0) port = 3128 flags S/SA keep state
    fxp0 tcp 127.0.0.1:3128 (173.194.40.3:80) <- 192.168.1.9:55613       CLOSED:SYN_SENT
    
    


  • What squid version are you using? Did you tried squid3?



  • @marcelloc:

    What squid version are you using? Did you tried squid3?

    it's squid3



  • Any clues on log files? I've tested squid3 on 2.2



  • @marcelloc:

    Any clues on log files? I've tested squid3 on 2.2

    no but as I said I don't think it's a squid issue, if I kill squid and run nc -l 3128 I still get nothing
    I'm not too familiar with freebsd but I think that
    fxp0 tcp 127.0.0.1:3128 (173.194.40.3:80) <- 192.168.1.9:55613      CLOSED:SYN_SENT
    should mean that the auto created firewall rule is kinda working but maybe something else blocks it



  • so I tried to create a similar nat rule to redirect outgoing connection to 8080 to the pfsense ssh port

    but I have the same problem

    nat on fxp0 inet proto tcp from 192.168.1.0/24 to 127.0.0.1 port = 8080 -> (fxp0) round-robin
    rdr on fxp0 inet proto tcp from any to any port = 8080 -> 127.0.0.1 port 22
    fxp0 tcp 127.0.0.1:22 (8.8.8.8:8080) <- 192.168.1.9:60274       CLOSED:SYN_SENT
    
    

    every rules redirecting to pfsense itself don't work, is there a way to debug this? I don't see anything being blocked in the firewall logs



  • looks like a fw issue …
    try creating a LAN firewall rule that allows lan devices to connect to 127.0.0.1:3128

    ![2015-01-25 17_23_49-Action center.jpg_thumb](/public/imported_attachments/1/2015-01-25 17_23_49-Action center.jpg_thumb)
    ![2015-01-25 17_23_49-Action center.jpg](/public/imported_attachments/1/2015-01-25 17_23_49-Action center.jpg)



  • @TieT:

    looks like a fw issue …
    try creating a LAN firewall rule that allows lan devices to connect to 127.0.0.1:3128

    still not working



  • I've tried to disable all non essential rules and interfaces and this is the content of /tmp/rules.debug  http://pastebin.com/M8D21nC1
    I've compared it to a fresh installation and it looks kinda the same



  • I've did once again a clean install, check steps I did to get it working without hacks.

    https://forum.pfsense.org/index.php?topic=87424.msg480232#msg480232



  • @marcelloc:

    I've did once again a clean install, check steps I did to get it working without hacks.

    https://forum.pfsense.org/index.php?topic=87424.msg480232#msg480232

    unfortunately a reinstall is not an option for me since it's 300km away



  • Can you try removing all packages  and reapply 2.2 firmware update?



  • @marcelloc:

    Can you try removing all packages  and reapply 2.2 firmware update?

    I'll try tonight, should I also remove the packages configuration from config.xml?



  • I don't think so.  As you posted, it doesn't looks like a package issue.



  • I tried to reapply the update but it's still not working



  • for what it's worth:
    I'm using squid on 2.2 in transparent mode and its working.
    But I'm not caching anything.

    Can you output the contents of the squid.conf



  • @TieT:

    for what it's worth:
    I'm using squid on 2.2 in transparent mode and its working.
    But I'm not caching anything.

    Can you output the contents of the squid.conf

    well it's not a squid issue but here it is http://pastebin.com/2htk2jg1



  • Here is mine

    # Do not edit manually !
    http_port 192.168.1.254:3128
    http_port 127.0.0.1:3128 transparent
    icp_port 0
    
    pid_filename /var/run/squid.pid
    cache_effective_user proxy
    cache_effective_group proxy
    error_directory /usr/pbi/squid-i386/etc/squid/errors/English
    icon_directory /usr/pbi/squid-i386/etc/squid/icons
    visible_hostname fw1.home
    cache_mgr tielens.t@gmail.com
    access_log /var/squid/logs/access.log
    cache_log /var/squid/logs/cache.log
    cache_store_log none
    logfile_rotate 7
    shutdown_lifetime 3 seconds
    # Allow local network(s) on interface(s)
    acl localnet src  192.168.1.0/255.255.255.0
    uri_whitespace strip
    
    cache_mem 10 MB
    maximum_object_size_in_memory 32 KB
    memory_replacement_policy heap GDSF
    cache_replacement_policy heap LFUDA
    cache_dir diskd /var/squid/cache 500 16 256
    minimum_object_size 0 KB
    maximum_object_size 5 KB
    offline_mode off
    cache_swap_low 90
    cache_swap_high 95
    
    # No redirector configured
    
    squid.conf...skipping...
    # Do not edit manually !
    http_port 192.168.1.254:3128
    http_port 127.0.0.1:3128 transparent
    icp_port 0
    
    pid_filename /var/run/squid.pid
    cache_effective_user proxy
    cache_effective_group proxy
    error_directory /usr/pbi/squid-i386/etc/squid/errors/English
    icon_directory /usr/pbi/squid-i386/etc/squid/icons
    visible_hostname fw1.home
    cache_mgr TieT
    access_log /var/squid/logs/access.log
    cache_log /var/squid/logs/cache.log
    cache_store_log none
    logfile_rotate 7
    shutdown_lifetime 3 seconds
    # Allow local network(s) on interface(s)
    acl localnet src  192.168.1.0/255.255.255.0
    uri_whitespace strip
    
    cache_mem 10 MB
    maximum_object_size_in_memory 32 KB
    memory_replacement_policy heap GDSF
    cache_replacement_policy heap LFUDA
    cache_dir diskd /var/squid/cache 500 16 256
    minimum_object_size 0 KB
    maximum_object_size 5 KB
    offline_mode off
    cache_swap_low 90
    cache_swap_high 95
    
    # No redirector configured
    
    # Setup some default acls
    acl all src 0.0.0.0/0.0.0.0
    acl localhost src 127.0.0.1/255.255.255.255
    acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901  3128 1025-65535
    acl sslports port 443 563
    acl manager proto cache_object
    acl purge method PURGE
    
    


  • Just to be sure, Are you selecting interfaces on squid general tab?

    I'm asking because other users had this "issue".



  • @marcelloc:

    Just to be sure, Are you selecting interfaces on squid general tab?

    I'm asking because other users had this "issue".

    yeah I had that issue as well after upgrading the other day but I fixed it soon after



  • I tried to load a config.xml that looks as much as possible like a fresh installation and also did a checksum of all files and compared them but I don't think there was anything wrong with them

    the only clue I have is the CLOSED:SYN_SENT state but it's not much



  • That means, you have send a sync request but no one answers (connection is closed)

    Did you set a virtual IP on the WAN or LAN ?



  • @TieT:

    That means, you have send a sync request but no one answers (connection is closed)

    Did you set a virtual IP on the WAN or LAN ?

    no



  • I have this issue as well.

    Foud this in the log:
    php-fpm[83033]: /pkg_edit.php: The command '/usr/pbi/squid-i386/sbin/squid -k shutdown' returned exit code '1', the output was 'Shared object "libz.so.5" not found, required by "libmd5.so.1"'



  • @Koenig:

    Foud this in the log:
    php-fpm[83033]: /pkg_edit.php: The command '/usr/pbi/squid-i386/sbin/squid -k shutdown' returned exit code '1', the output was 'Shared object "libz.so.5" not found, required by "libmd5.so.1"'

    Is this setup a migration from 2.1 or any 2.2RC version?

    Not getting it on amd64. :-\


  • Banned

    @Koenig:

    Foud this in the log:
    php-fpm[83033]: /pkg_edit.php: The command '/usr/pbi/squid-i386/sbin/squid -k shutdown' returned exit code '1', the output was 'Shared object "libz.so.5" not found, required by "libmd5.so.1"'

    That thing gets removed. https://github.com/pfsense/pfsense/blob/master/etc/pfSense.obsoletedfiles

    
    $ ls -l /usr/lib/libz.so*
    lrwxr-xr-x  1 root  wheel  14 Jan 24 06:00 /usr/lib/libz.so -> /lib/libz.so.5
    
    $ ls -l /lib/libz.so.5
    ls: /lib/libz.so.5: No such file or directory
    
    $ ls -l /lib/libz.so.6
    -r--r--r--  1 root  wheel  79180 Jan 22 22:07 /lib/libz.so.6
    
    

    This clearly is not done properly on FULL installs, as seen from the dangling dead symlinks.

    Cf. that with nanobsd:

    
    $ ls -l /usr/lib/libz.so*
    lrwxr-xr-x  1 root  wheel  14 Jan 22 22:07 /usr/lib/libz.so -> /lib/libz.so.6
    
    $ ls -l /lib/libz.so.6
    -r--r--r--  1 root  wheel  79180 Jan 22 22:07 /lib/libz.so.6
    
    

    @marcelloc:

    Is this setup a migration from 2.1 or any 2.2RC version?
    Not getting it on amd64. :-\

    Apparently an upgrade process bug on full installs, rendering the updated library pretty much useless as it cannot be found. Just a few examples:

    
    /usr/lib/libcrypto.so -> /lib/libcrypto.so.6
    $ ls -l /lib/libcrypto.so.6
    ls: /lib/libcrypto.so.6: No such file or directory
    
    /usr/lib/libmd.so -> /lib/libmd.so.5
    $ ls -l  /lib/libmd.so.5
    ls: /lib/libmd.so.5: No such file or directory
    
    $ ls -l /usr/lib/libusb.so*
    lrwxr-xr-x  1 root  wheel     11 Jan 24 06:00 /usr/lib/libusb.so -> libusb.so.2
    -r--r--r--  1 root  wheel  65068 Jan 22 22:07 /usr/lib/libusb.so.3
    
    /usr/lib/libufs.so -> /lib/libufs.so.5
    $ ls -l /lib/libufs.so.5
    ls: /lib/libufs.so.5: No such file or directory
    
    

    Badly, badly broken.  >:( => https://redmine.pfsense.org/issues/4328



  • I think this can be resolved by creating a link between those libs.
    and restarting squid.

    ln -s /lib/libz.so.5 /usr/lib/libz.so.6
    

    (if the file is in /lib of course)
    You can do a search to make sure

    
    su
    find / -name libz.so.*
    

    I had the same problem with squid and snort a while back
    let me know what happens  ;)


  • Banned

    @TieT:

    I think this can be resolved by creating a link between those libs.
    and restarting squid.

    ln /lib/libz.so.6 /lib/libz.so.5
    

    Uh no, this is completely wrong "fix".



  • Uh no, this is completely wrong "fix".

    So you're saying it won't work untill 2.2.1 is rolled out ?


  • Banned

    You are fixing the wrong thing! That .5 vs .6 actually has a meaning - the new version of the library does not have a backward-compatible ABI! If you want to fix something, then fix the dead /usr/lib/*.so symlinks so that they point to proper file. Don't create "compatibility" symlinks pretending a deliberately removed outdated ABI library is still there.



  • Renato is going to rebuild squid pbi under  i386, this may fix most issues.



  • Yes you're right, but do you expect users to have a non functional box for god knows how long.

    like i said, its a temp fix that works for me.
    when squid is rebuild and functioning again, I will undertake the needed steps to make it point to the right libs.

    On the other hand, disable squid and wait for an update is also a solution…
    But where is the fun in that  :P

    And yes, I know the .5 - .6 has a meaning, thank you for being so condescending.
    also i made a typo, it should be```
    ln -s /lib/libz.so.6 /usr/lib/libz.so

    and i'm guessing squid points to /usr/lib/ to find the files ?


  • @TieT:

    Yes you're right, but do you expect users to have a non functional box for god knows how long.

    Is this question for me? ???



  • Is this question for me? ???

    No  ;)
    Its was an answer to doktornotor



  • Hello,

    for me after the 2.1.5 to 2.2 version, the captive portal not worked.

    I added an new Rules in Firewall to accept 8002 port and it's works.



  • @john880:

    for me after the 2.1.5 to 2.2 version, the captive portal not worked.

    Captive portal itself(better start a new thread) or Squid integration with captive portal?



  • Hi,
    My name is fred and I am sorry if I write in your post but but I did not find how to start a discussion
    I have a problem with pfsense installation script was ok but now I cannot access to the box anymore. What can I do? Is it possible to reset the box?

    You can write me directly fredbovyeirl@gmail.com

    The in advance and sorry again
    Fred



  • @Brandhor:

    although it's not a squid issue, I've tried to use netcat to listen to the same port as squid and I don't receive any traffic

    this is what I get with pfctl while trying to connect to google.com from a client(192.168.1.9)

    rdr on fxp0 inet proto tcp from any to ! (fxp0) port = http -> 127.0.0.1 port 3128
    pass in quick on fxp0 proto tcp from any to ! (fxp0) port = 3128 flags S/SA keep state
    fxp0 tcp 127.0.0.1:3128 (173.194.40.3:80) <- 192.168.1.9:55613       CLOSED:SYN_SENT
    
    

    Hi Brandhor,

    Have you got anywhere with this?

    I started a similar thread pfSense 2.2: Squid 3.4.10_2 pkg 0.2.6 redirection not working in transparent mode before spotting you had the same issue.

    You'll see below that I did exactly the same as you (great minds think alike) and came to the same conclusion.

    @Steve:

    To prove the point that the redirected transparent connection is not getting through to squid, I stopped the squid process and then ran the following on the pfSense console.

    : nc -l 3128
    

    I then telnetted to pfsense from my laptop and a connection was established and characters passed.

    $ telnet 10.5.1.1 3128
    Trying 10.5.1.1...
    Connected to pfsense.scevans.com.
    Escape character is '^]'.
    hello
    
    : nc -Dl 3128
    hello
    

    I repeated trying to telnet to port 80 on news.bbc.co.uk which should have been directed by the redirect, and no connection was made.

    Steve

    It was later in this thread that Marcelloc commented on the compile settings I'd got from squid.

    @marcelloc:

    @Steve:

    Now here's an oddity.

    This is one of pbi behaviors. same binary, different folder, different results. (Imagine get all it working together :))

    And here is the confirmation that pbi build on i386 is outdated

    '--disable-ipf-transparent' '--disable-ipfw-transparent'
    

    Go to amd64 and it will work  :)

    Thanks for your feedback

    Seeing that there was a firewall related compile time option I decided to wait for the squid update, but alas when that came the behaviour was unchanged.

    Trying to understand what was going on I started another thread Order of rule application for redirects to which there has been no response.

    Keen to hear where you are with this issue.

    Thanks,

    Steve



  • I haven't tried again since I got an mpls line upgrade and now I route all the traffic through another pfsense via openvpn



  • Hi Steve / all

    Although it's my first post I can confirm that I have the same issue with the transparent proxy not processing requests.  This was a clean 64 bit 2.2 install on an APU4 appliance.

    Chris


Log in to reply