Re: Squid with identd lookups - SOLVED!



  • Hi all,

    UPDATE - Look at about the fifth reply down for my solution to this problem.
    I saw this topic, but it didn't seem to cover my particular situation. And I've googled for results, hit my head against a wall on it for a little while, and searched the forums, so hopefully that covers the RTFM due diligence tests.  :)

    Here's what I'm trying to do:

    Last week I replaced a customer's existing firewall solution with pfSense 2.0.

    At the time I installed Squid and lightSquid as additional add-on packages, and configured it to work with their existing environment.

    They have two locations, and I'm making the slave location go through the proxy at the master location where the pfSense firewall is running.

    So, I now have a healthy log of activity in lightSquid. But I don't have usernames, just IP addresses.

    Under the previous firewall solution I was using Squid with Identd lookups enabled. In Squidalyzer it would show me the usernames.

    If I tail -f access.log in an SSH console window and watch it, I can see the website traffic appearing there, but where the username should be, I see a dash (-). I researched this on the almighty internet and came up with these directives that needed to be in place…

    acl {Location1} src {Location1InternalNetworkSubnet}/24;
    acl {Location2} src {Location2InternalNetworkSubnet}/24;
    ident_lookup_access allow {Location1};
    ident_lookup_access allow {Location2};
    ident_lookup_access deny all;
    ident_timeout 3 seconds;

    (Note that anything in braces has a real, common sense value to it, but I've obscured it to maintain my customers' privacy)  :)

    So, I put these lines in the Custom Options section of my Squid configuration. I save and apply. I check my access.log on the new traffic - still dashes.

    I check the workstations from the firewall shell prompt…:

    [2.0-RELEASE][admin@{FWHostname}.{InternalDomain}/root(13): [color=red]telnet {Workstation IP} 113
    Trying {Workstation IP}…
    Connected to {Workstation IP}.
    Escape character is '^]'.

    So, the ident service is running on the workstations, the workstation firewalls are off, and I can connect to ident on port 113 on a workstation from the firewall. So it's not a connectivity issue.

    Then I thought "what if Squid is throwing an error and I just can't see it because I'm using the GUI. So at the shell prompt, I tried this:

    [2.0-RELEASE][admin@{FWHostname}.{InternalDomain}]/root(14): squid -k reconfigure
    2011/10/25 17:27:49| aclParseAclLine: WARNING: empty ACL: acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
    [2.0-RELEASE][admin@{FWHostname}.{InternalDomain}]/root(15):

    … so other than an empty acl, I'm doing fine on my configuration.

    Any ideas what's wrong? Could it be that the package admin for squid didn't configure it with --enable-ident-lookups or whatever in the configuration?

    I care because my customer looks at the lightsquid logs and wants to know the username that is attached to http://www.bigboobs.net and not just the IP address. Anyone could be on the workstation with that IP address, but using someone else's username on that workstation is a tad more difficult.

    Thanks, in advance!



  • … I'm sort of glad to see I stumped the panel. Makes me feel a little less like an idiot!

    I'm still plugging away at this issue. I have another firewall distribution where this is working. I compared the ./config compile parameters used on both the original firewall distribution and pfSense 2.0, and they appear very similar. My next step is to look for a way to get squid to dump out the current running configuration parameters and compare those to see what similarities and differences I can find.

    Again, any feedback would be appreciated. Especially if you know the answer!  ;)



  • any luck with this?

    Im trying to do a similar thing with the pfSense captive portal which I'm trying to get user names logged to the squid logs instead of ip address

    any help would be appreciated



  • Maybe this can help you: http://lightsquid.sourceforge.net/How It Work.html
    "_…if you want to collect info using user nick, not ip, you should use the following syntax
    set $ip2name="list" in lightsquid.cfg

    edit ip2name/ip2name.list and edit path to ip2name list_"

    The path for lightsquid.cfg is /usr/local/etc/lightsquid



  • @johnnybe:

    Maybe this can help you: http://lightsquid.sourceforge.net/How It Work.html
    "_…if you want to collect info using user nick, not ip, you should use the following syntax
    set $ip2name="list" in lightsquid.cfg

    edit ip2name/ip2name.list and edit path to ip2name list_"

    The path for lightsquid.cfg is /usr/local/etc/lightsquid

    Thanks, JohnnyBe. I'll be happy to try it, but I don't think this is a LightSquid problem, and more a Squid problem.

    LightSquid just parses the access.log file and outputs some pretty html files. It only uses what it gets from access.log.

    When I look at access.log, where the username/authname from ident should be appearing, I see a dash (-). This means Squid either isn't querying identd for an authname, or it's querying and it's not outputting what it receives back to access.log. I've been changing settings in Squid while running a tail -f on access.log to watch for that magic configuration change that does it. So far, no luck. I even tried taking a working squid.conf and replicating the configuration on pfSense. The result was a crashed squid daemon that wouldn't start. I didn't do too much digging when this happened, as I had employees yelling because the internet was "down" all of a sudden. But it's something I'd like to investigate more.



  • After working on this problem for quite awhile, I think I have it resolved. Here's what I found that works.

    First, please keep in mind that I'm no squid expert, so some of my settings may not be optimal, and I welcome others who might jump in and have suggestions on optimizing them. As always, YMMV (your mileage may vary) on this solution and I offer NO WARRANTY and NO SUPPORT whatsoever, but I've tried it on two pfSense 2.0 firewalls and it appears to work swimmingly on both of them.

    The issue originates with two settings on the General config tab of the Squid Proxy Server package. The Allow users on interface and Transparent proxy settings are just too damned good at their jobs. As a result if you have these settings checked, ident lookups do not occur.

    So, here it is, step by step:

    (Please note that from the time you begin this process to the time you end it, your users may not have internet access. In other words, do it before or after a workday!!!)

    BACK UP YOUR CONFIGURATION UNDER DIAGNOSTICS -> BACKUP/RESTORE, and make sure you leave the packages option set to back up package configuration!

    General Tab:

    • Leave all settings as default, except for the following changes…

    • Allow users on interface = Uncheck this

    • Transparent proxy = Uncheck this

    • Log store directory = /var/squid/log (this is because LightSquid likes it to live there)

    • Custom Options = http_port {LAN IP}:8080 transparent; ident_lookup_access allow all; ident_timeout 3 seconds; Note: it doesn't matter what port you pick here, as long as it's not 3128 and not one you're already using for something else (like the web-admin for pfSense!). I use port 81 for firewall admin, so 8080 is free. But if you choose some port other than 8080, modify it later in this walk through.

    Click Save Settings.

    Upstream Proxy: Leave all settings to default.

    Cache Management: Leave all settings to default.

    Access Control:

    • Allowed subnets = {LAN Network}/{CIDR Bitmask} (ex: 192.168.0.0/24, which equals 192.168.0.0/255.255.255.0)

    • OPTIONAL - Blacklist: {A list of domains or partial domains, one per line that you don't want people on}, ex: facebook.com

    Click Save Settings.

    Traffic Mgmt: Leave all settings to default.

    Auth Settings: Authentication Method = None

    Local Users: No users required

    Now go to Firewall -> Aliases
    Click the New Alias button

    • Name = A name. For my locations, I use {Cityname}Internal, ie: GrandRapidsInternal

    • Type = Network(s)

    • Network(s) = Click Add Network Button.

    • Network = Your internal subnet, ie: 192.168.0.0

    • CIDR = Your subnet bitmask. Most commonly this is 24.

    Click Save Button.

    Now go to Firewall -> NAT
    Click the New NAT rule button (bottom of existing NAT rules, right).

    • Interface: LAN

    • Source = Click Advanced Button

    • Source Type = Single host or alias

    • Source Address = Alias name from above (ie: GrandRapidsInternal)

    • Destination = any

    • Destination Port Range From = 80

    • Destination Port Range To = Leave Blank

    • Redirect Target IP = {Internal Firewall IP} (ie: 192.168.0.1)

    • Redirect Target Port = 8080 (if you used a different port above in Squid General Custom Options, put it here!)

    • Description (Optional) = Squid Redirect

    Click Save Button
    Apply Configuration Changes
    Close Config Change Status message

    Under Firewall -> Rules -> LAN tab, you should now see a rule that matches your NAT rule. Optional: move it to the top of the rule set, just below the one that is in gray and cannot be modified or moved.

    You should now have internet on your workstations again.

    If you have an ident client installed on the workstations (I prefer rndware's Windows Ident Server, installed as a service so it doesn't have to run as an app, with the workstation firewall turned OFF), you should now start to see usernames appearing in the squid access.log file. In Lightsquid, you'll see a username in place of the IP address.

    I always have one or two users that have a legitimate need to bypass the proxy server. For example, I block facebook per management's request and then the marketing person needs access to keep the company facebook site up to date. For this you can put in the following rule…

    • Either set a static IP address on the workstation, or set up a DHCP reservation for their MAC address on the DHCP server. I use Windows Servers in most environments, and the Windows server (not the router!!) is the DHCP server. This is exceedingly simple in the Windows environment, but setting up a reservation is outside the scope of this post. Google it if you need help.

    • Make sure they get that IP address from DHCP. Again, google it if you're stuck.

    • Set up an alias for that workstation in Firewall -> Aliases

    • Set up a NAT rule for that workstation in Firewall -NAT. The settings are:

      • Interface: LAN

      • No RDR (NOT): Checked

      • Source Type: Single host or Alias

      • Source Address: Alias you defined

      • Source Port Range: any to any

      • Destination Type: any

      • Destination Range From: 80

      • Destination Range To: {Leave Blank}

      • Description: Give yourself a good description here. In a year you won't remember why you did this!

      • Click Save Button

      • Checkmark the new rule at the bottom

      • Scroll up to the redirect rule that redirects LAN traffic from 80 to 8080 that we created above

      • Click the Move Selected Rules before this rule icon that looks like a hand and an arrow

      • Click Reload Configuration Button

      • Click Close

    Again, NO WARRANTIES, NO SUPPORT, and YMMV! Feel free to ask a question, but don't get angry with me if it takes me a month to respond!

    Enjoy! :-)



  • Just a question:

    With this Custom Options = http_port {LAN IP}:8080 transparent; I suppose that Squid is working as a transparent proxy, isn't it?
    If so, I'll follow you with those settings.



  • @johnnybe:

    Just a question:

    With this Custom Options = http_port {LAN IP}:8080 transparent; I suppose that Squid is working as a transparent proxy, isn't it?
    If so, I'll follow you with those settings.

    Yep, it's completely transparent. That's one of my stipulations for it. I don't want it querying the user for a username and password, and I don't want it doing LDAP/AD lookups for usernames either. It simply asks the workstation who is signed in. If nothing responds, it times out within 3 seconds and puts a - in the log. If something responds with a username, it logs the response in access.log, which lightsquid then picks up.



  • @anomaly0617:

    @johnnybe:

    Just a question:

    With this Custom Options = http_port {LAN IP}:8080 transparent; I suppose that Squid is working as a transparent proxy, isn't it?
    If so, I'll follow you with those settings.

    Yep, it's completely transparent. That's one of my stipulations for it. I don't want it querying the user for a username and password, and I don't want it doing LDAP/AD lookups for usernames either. It simply asks the workstation who is signed in. If nothing responds, it times out within 3 seconds and puts a - in the log. If something responds with a username, it logs the response in access.log, which lightsquid then picks up.

    Well well well… it's time to follow your settings...  ;D
    And check if it works for my purposes. I'm almost sure it will work for what I'm looking for.



  • Just keep in mind that ident lookup can be easily spoofed/forged.

    For example, pfSense has widentd package that "RFC1413 auth/identd daemon with fixed fake reply".

    One time I've tried to transparent authenticate web users with ident lookups but to get some security, I needed to write my own server/client ident daemon to check if client's answer are reliable.

    Nowadays, many paid softwares do AD queries to see who is using ip xxx.yyy.ddd.zzz and log current user. It's nice if you are not using TS application servers.

    Anyway, thanks for sharing tutorial to all of us.  ;)



  • @marcelloc:

    Just keep in mind that ident lookup can be easily spoofed/forged.

    For example, pfSense has widentd package that "RFC1413 auth/identd daemon with fixed fake reply".

    One time I've tried to transparent authenticate web users with ident lookups but to get some security, I needed to write my own server/client ident daemon to check if client's answer are reliable.

    Nowadays, many paid softwares do AD queries to see who is using ip xxx.yyy.ddd.zzz and log current user. It's nice if you are not using TS application servers.

    Anyway, thanks for sharing tutorial to all of us.  ;)

    Absolutely true, however in this context, we're not using ident to identify users that are external to the network, and rather using it to identify the users that are accessing the firewall from internal (usually corporate) workstations. And while those users could spoof their ident, it isn't likely they would. Or, more accurately, if they're spoofing their ident, I need to employ them in the IT world instead of having them be a marketing or engineering drone. :)

    This all came about because for management it wasn't enough for me to say "this site was accessed at this time by this workstation." They (rightfully so) wanted to know who was on that workstation, not just what workstation it came from.

    With this configuration running, I can say "this site was accessed at this time by this user at this workstation" and now it's a little more concrete. I have the who, what, and where in the equation.  :)



  • hello,

    i got same problem when i tried to see a proxy report without authentication, it not show real name, i see this ? … so i follow your instructions to modified a file lightsquid.cfg "set $ip2name="list" in lightsquid.cfg" but i don't understand when you say " edit ip2name/ip2name.list and edit path to ip2name list" where i find this file to edit, because i can't find it in lightsquid.cfg directory so try another directory /usr/local/libexec/lightsquid, i find file ip2name.list but i still don't where to modified it... please help me..

    att
    Ailton varela



  • @avarela:

    i got same problem when i tried to see a proxy report without authentication, it not show real name, i see this ? … so i follow your instructions to modified a file lightsquid.cfg "set $ip2name="list" in lightsquid.cfg" but i don't understand when you say " edit ip2name/ip2name.list and edit path to ip2name list" where i find this file to edit, because i can't find it in lightsquid.cfg directory so try another directory /usr/local/libexec/lightsquid, i find file ip2name.list but i still don't where to modified it... please help me..

    My guess is that your workstations are not issuing valid identd output. Try this:

    1. On your computer, install PuTTY if you haven't already
    2. Open PuTTY
    3. select Telnet as the protocol under connection type
    4. At the bottom under Close Window on Exit, select Never.
    5. Now pick a random machine on your network and type its IP Address into the Host Name or IP Address field.
    6. Set the port to 113
    7. click Open at the bottom
    8. Can you connect? If not, either the firewall is enabled on the workstation (turn it off), the firewall is enabled and there is no exception for port 113 or your subnet (turn off the firewall!) or you don't have identd software (install some… see below).
    9. If you did connect without errors, Press enter. If IdentD is working correctly, you should see four fields appear, seperated by colons [:]. One of the fields should say the username of the user logged into the workstation.

    If it does not, something is wrong with your identd installation on the workstation. There are multiple identd servers out there for free. Google something like "windows identd service", find one, install it on a workstation, test it, and then install it on all your workstations once you know it works.



  • By this setting, whether https traffic goes through squid?

    I was little confused with my setup. I am having squid and squidguard with transparent mode. When I want to block internal Ip from accessing the internet, if I put that Ip in firewall they will not get https pages, but they can browse other http pages. Then I had to put the IP number in squid blacklist.



  • @chowtamah:

    By this setting, whether https traffic goes through squid?

    In transparent mode, never.


Locked