Еще раз о ipcad+squid+lightsquid

dr.gopher

Доброго времени суток!
Поставил ipcad + lightsquid без squid

В отчетах фигурируют только IP адреса, а хотелось бы видеть посещаемые хосты.

Может кто подскажет, как подправить?

deutsche

скорей всего pfsense не может резолвить имена - не указан dns.

rubic

Конечный URL (например имя скачанного файла) без squid не будет видно все равно. Да и сайты тоже толком не покажет, ибо их куча может быть на одном IP.

Eugene

@rubic:

забацал

красиво забацал, спасибо. Поставь себя автором пожалуйста.

dr.gopher

@rubic:

Да и сайты тоже толком не покажет, ибо их куча может быть на одном IP.

В этом и загвоздка.
Может можно чем-то другим "поиметь" статистику посещаемых ресурсов без проксирования?

rubic

@Evgeny:

Поставь себя автором пожалуйста.

Не пойму как?

Eugene

Da prosto otredaktiruy svoyu statyu I postav' pervoj strokoj:
Avtor: …

forestman99

просьба не кидаться тапками, переделал как умел скрипт, в результате через dig -x пытаемся отресолвить адреса куда ходили, если не ресолвиться - остается ip,
плюс к этому хочу заметить, что идущий в поставке 1.2.3 lightsquid 1.7.1 косячит когда данные о клиентах только адреса и куда ходили - тоже адреса - путает адресата и адрес куда ходили. решил ручной заменой lightsquid на 1.8

tolog.txt

dr.gopher

При запуске скрипта:

./tolog.sh: line 13: syntax error near unexpected token }' ./tolog.sh: line 13: }'

forestman99

не знаю что Вам сказать, скрипт с рабочего сервера, вот что в итоге получается

dr.gopher

::)
Я его не на PfSense тестил

forestman99

тогда смотрите на свой bash и пилите под себя

dr.gopher

А можете скрипт в .tar запаковать?
Может я криво .тхт "копипостю"
Потому как, bash "просит" некоторые директивы заключить в ковычки.

Eugene

  { 
# echo
  }

Поставь тут хоть чё-нидь типа

  { 
# echo
   ;
  }

Ilyuha

Для установки ipcad на 2.0 beta 5 нужно ставить compat6x-i386 ?

dr.gopher

Добрый день!
PF 1.2.3

Чет парюсь второй день… Не хочет через веб интерфейс ставиться lightsquid
Зависает на этом месте (картинка) и ничего больше не пишет.
http://thin.kiev.ua/images/stories/pfsense/antivirus/lite.jpg

До вчера, всё ставилось на ура.

Видел, что пути к портам изменились.
Может подскажет кто, как и откуда, ручками установить lightsquid?

Упс.
Увидел ошибку...

Warning: main(squid.inc): failed to open stream: No such file or directory in /usr/local/pkg/lightsquid.inc on line 36 Fatal error: main(): Failed opening required 'squid.inc' (include_path='.:/etc/inc:/usr/local/www:/usr/local/captiveportal:/usr/local/pkg') in /usr/local/pkg/lightsquid.inc on line 36

dvserg

Установите squid. Там требуется его файл.

dr.gopher

Спасибо. Уже догадался… Поставил, все ок. Странно, что без сквида никак... Или поставить, а потом сквид удалить?

dvserg

@dr.gopher:

Спасибо. Уже догадался… Поставил, все ок. Странно, что без сквида никак... Или поставит, а потом сквид удалить?

Нет, там реконфигурация сквида происходит. Надо будет проверку наличия файла добавить.. Ну кто-же знал, что лайт еще как-то использоваться будет.

Ilyuha

Подскажите пожалуйста что я не правильно сделал в конфигах так как ipcad не пишет ничего в лог.
Процесс в памяти висит.

$ ifconfig
em0: flags=8a43 <up,broadcast,running,allmulti,simplex,multicast>metric 0 mtu 1500
	options=9b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:0c:29:5c:ed:fe
	inet 192.168.1.249 netmask 0xffffff00 broadcast 192.168.1.255
	inet6 fe80::20c:29ff:fe5c:edfe%em0 prefixlen 64 scopeid 0x1 
	nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
em1: flags=8a43 <up,broadcast,running,allmulti,simplex,multicast>metric 0 mtu 1500
	options=9b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:0c:29:5c:ed:08
	inet 192.168.132.2 netmask 0xffffff00 broadcast 192.168.132.255
	inet6 fe80::20c:29ff:fe5c:ed08%em1 prefixlen 64 scopeid 0x2 
	nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
em2: flags=8a43 <up,broadcast,running,allmulti,simplex,multicast>metric 0 mtu 1500
	options=9b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:0c:29:5c:ed:12
	inet 192.168.232.2 netmask 0xffffff00 broadcast 192.168.232.255
	inet6 fe80::20c:29ff:fe5c:ed12%em2 prefixlen 64 scopeid 0x3 
	nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
em3: flags=8a43 <up,broadcast,running,allmulti,simplex,multicast>metric 0 mtu 1500
	options=9b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:0c:29:5c:ed:1c
	inet 192.168.32.2 netmask 0xffffff00 broadcast 192.168.32.255
	inet6 fe80::20c:29ff:fe5c:ed1c%em3 prefixlen 64 scopeid 0x4 
	nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
plip0: flags=8810 <pointopoint,simplex,multicast>metric 0 mtu 1500
lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
	options=3 <rxcsum,txcsum>inet 127.0.0.1 netmask 0xff000000 
	inet6 ::1 prefixlen 128 
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6 
	nd6 options=3 <performnud,accept_rtadv>pfsync0: flags=0<> metric 0 mtu 1460
	syncpeer: 224.0.0.240 maxupd: 128
pflog0: flags=100 <promisc>metric 0 mtu 33200
enc0: flags=0<> metric 0 mtu 1536</promisc></performnud,accept_rtadv></rxcsum,txcsum></up,loopback,running,multicast></pointopoint,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,allmulti,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,allmulti,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,allmulti,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,allmulti,simplex,multicast>

#!/bin/sh
net1="192.168.132"
net2="192.168.232"
net3="192.168.32"
ttime=`/usr/bin/rsh localhost sh ip acco | grep 'Accounting data saved' | awk '{print ($4)}'`
rsh localhost clear ip accounting
rsh localhost show ip accounting checkpoint | grep $net1 | awk -v vtime=$ttime '{if ($5 != 0) print (vtime".000",1,$2,"TCP_MISS/200",$4,"CONNECT",$1":"$5,"-","DIRECT/"$1,"-")}' >> /var/squid/log/access.log
rsh localhost show ip accounting checkpoint | grep $net2 | awk -v vtime=$ttime '{if ($5 != 0) print (vtime".000",1,$2,"TCP_MISS/200",$4,"CONNECT",$1":"$5,"-","DIRECT/"$1,"-")}' >> /var/squid/log/access.log
rsh localhost show ip accounting checkpoint | grep $net3 | awk -v vtime=$ttime '{if ($5 != 0) print (vtime".000",1,$2,"TCP_MISS/200",$4,"CONNECT",$1":"$5,"-","DIRECT/"$1,"-")}' >> /var/squid/log/access.log

#
# Configuration file for ipcad - Cisco IP accounting simulator daemon.
# Copyright (c) 2001, 2002, 2003, 2004, 2005
# 	Lev Walkin <vlm@lionet.info>.
#
# Please see ipcad.conf(5) for additional explanations.
# Please contact me if you have troubles configuring ipcad. My goal is to make
# initial configuration easier for new users, so your input is valuable.
#

##################
# GLOBAL OPTIONS #
##################

#
# Enable or disable capturing UDP and TCP port numbers, IP protocol and
# ICMP types for RSH output.
#
#     capture-ports {enable|disable} ;
#
# Enabling this will BREAK Cisco RSH output format compatibility,
# increase memory requirements and may slow down traffic processing.
# This option takes effect IMMEDIATELY, that is, it can be specified
# multiple times, even between interfaces configuration.
# This option has NO effect on NetFlow operation (NetFlow always captures
# port information).
#

capture-ports enable;

#
# Buffers to be used for transferring the data from the kernel,
# if applicable (BPF, ULOG).
# Using larger buffers may increase the performance but will
# affect responsiveness.
#
# buffers = <number>[{k|m}] ;
#
# Reasonable defaults are used if this parameter is not set.
#

## buffers = 64k;

#####################
# INTERFACE OPTIONS #
#####################

#
# interface <iface>[ promisc ] [ input-only ]
#			[ netflow-disable ] [ filter "<pcap_filter>" ] ;
# OR
# interface ulog group <group>[, group <group>...]
#			[ netflow-disabled ];
# OR
# interface ipq [ netflow-disabled ];	# man libipq(3)
# OR
# interface {divert|tee} port <divert-port># man divert(4)
#			[ input-only ] [ netflow-disabled ];
# OR
# interface file <tcpdump-output.pcap>[ netflow-disabled ];
#
# Options meaning:
#
# promisc:
# 	Put interface into promiscuous mode.
# 	This enables listening for the packets which are not destined for
# 	this host and thus ipcad will count and display all the traffic within
# 	the local network. Note that the interface might be in promiscuous mode
# 	for some other reason.
#
# input-only: 
# 	Use kernel feature of counting only incoming packets.
#
# netflow-sampled: 	(DO NOT ENABLE THIS OPTION, unless you have to!)
# 	If the NetFlow export mechanism is used, this option instructs
#	the interface to supply only one out of N packets to the NetFlow
#	accounting code, thus lowering the CPU requirements. The value of N
#	is configured explicitly in a NetFlow configuration section.
#	NOTE: This option is NOT used to enable NetFlow on the interface,
#	it just modifies the NetFlow behavior on this interface.
#	DO NOT ENABLE THIS OPTION, UNLESS YOU HAVE TO!
#
# netflow-disable: 
#	By default, all interfaces are included into NetFlow accounting.
#	This option is used to disable NetFlow on a particular interface.
#
# filter: 
# 	Install a custom filter on packets instead of basic
#	IP protocol filter. Requires libpcap (even if BPF is being used).
#	May be employed to eliminate CPU overhead on passing unnecessary
#	data between the kernel and user space (by installing the filter
#	directly into the kernel).
#
# NOTES:
#  * "input-only" directive must be supported by kernel.
#    Probably, you were noticed about it during the compilation process
#    if it was not supported.
#    FreeBSD 3.x and elder kernels do not support this feature.
#  * ULOG packet source (interface ulog) is supported under
#    Linux >= 2.4.18-pre8.
#    You should configure iptables to dump the packet stream
#    into the appropriate group, i.e.:
#        iptables -A OUTPUT -j ULOG --ulog-nlgroup <group>#    Given ULOG groups will be OR'ed together.
#  * A wildcard (*) may be specified as part of an interface name.
#

interface em1 filter "ip and dst net 192.168.132.0/24 and not src net 192.168.132.0/24 and not src port 80";
interface em2 filter "ip and dst net 192.168.232.0/24 and not src net 192.168.232.0/24 and not src port 80";
interface em3 filter "ip and dst net 192.168.32.0/24 and not src net 192.168.32.0/24 and not src port 80";

#
# aggregate <ip>/ <masklen>strip <maskbits>;
#
# Aggregate addresses from the specified network (<ip>/<masklen>),
# by AND'ing with specified mask (<maskbits>).
#
#

aggregate 192.168.132.0/24 strip 32; /* Don't aggregate internal range */
aggregate 192.168.232.0/24 strip 32; /* Don't aggregate internal range */
aggregate 192.168.32.0/24 strip 32; /* Don't aggregate internal range */
aggregate 0.0.0.0/0 strip 32;	/* Aggregate external networks */

#
# aggregate <port_range_start>[-<port_range_end>] into <port>;
#
# Aggregate port numbers. Meaningful only if capture-ports is enabled.
#

#aggregate 1-19 into 65535;
#aggregate 20-21 into 21;
#aggregate 22-23 into 22;
#aggregate 25 into 25;
#aggregate 24 into 65535;
#aggregate 26-79 into 65535;
aggregate 80 into 0;
#aggregate 81-109 into 65535;
#aggregate 110 into 110;
#aggregate 111-442 into 65535;
#aggregate 443 into 443;
#aggregate 444-3127 into 65535;
#aggregate 3128 into 3128;
#aggregate 3129-65535 into 65535;

##########################
# NetFlow EXPORT OPTIONS #
##########################

#
# Enable Cisco NetFlow export method.
# NetFlow uses UDP to feed flow information to the receiver.
# If the destination is not specified, NetFlow is disabled.
#

# netflow export destination 127.0.0.1 9996;
netflow export version 5;	# NetFlow export format version {1|5}
netflow timeout active 30;	# Timeout when flow is active, in minutes
netflow timeout inactive 15;	# Flow inactivity timeout, in seconds
netflow engine-type 73;		# v5 engine_type; 73='I' for "IPCAD"
netflow engine-id 1;		# Useful to differentiate multiple ipcads.

# The following option is enabled by the "netflow-sampled" interface flag.
#netflow sampling-mode packet-interval 10;   # 1 out of 10 packets accounted
# DO NOT ENABLE THIS UNLESS YOU KNOW WHAT ARE YOU DOING.

#
# NetFlow protocol exports an SNMP id instead of the interface name
# (i.e., "eth0", "ppp32"). The following statements options define
# mapping between the interface names and a set of "SNMP identifiers".
#
netflow ifclass eth mapto 0-99;		# i.e., "eth1"->1, "eth3"->3
netflow ifclass fxp mapto 0-99;		# i.e., "fxp4"->4, "fxp0"->0
netflow ifclass ppp mapto 100-199;	# i.e., "ppp32"->532, "ppp7"->507
netflow ifclass gre mapto 200-299;
netflow ifclass tun mapto 300-399;	# i.e., "tun0"->300

######################
# RSH SERVER OPTIONS #
######################

#
# Enable RSH Server:
#
#   rsh {enable|yes|on|disable|no|off} [at <listen_ip>];
#
# If "at <listen_ip>" omitted, rsh server listens on IP address 0.0.0.0,
# which may be undesirable.
#

rsh enable at 127.0.0.1;

#
# RSH access rules:
#
# rsh [<user>@] <host_addr>{admin|backup|[default]|view-only|deny} ;
#

rsh root@127.0.0.1 admin;	/* Can shutdown ipcad */
rsh root@127.0.0.1 backup;	/* Can dump/restore/import accounting table */
rsh root@127.0.0.1;		/* Can view and modify accounting tables */
/* Note the order! */
#rsh luser@127.0.0.1 deny;	/* Deny this user from even viewing tables */
rsh 127.0.0.1 view-only;	/* Other users can view current tables */

# Keep IP packet time to live reasonably low to avoid remote attacks.
# (The rsh client must reside no more than three hops away from the
# router running ipcad.)
rsh ttl = 3;

# Set rsh timeout for the same purpose.
rsh timeout = 30;

#
# Dump active IP accounting table to this file on exit and read on startup.
# (read about -s and -r options in ipcad(8) manual page)
# NOTE: This setting has no effect on NetFlow operation. The flow cache
#       contents are flushed to the collector upon ipcad termination.
#

dumpfile = ipcad.dump;	# The file is inside chroot(), see below...

#################
# OTHER OPTIONS #
#################

#
# Chroot to this directory before processing.
#
# Of course, you could disable chroot()'ing by commenting it out,
# but it is not recommended, so I left this confusing default
# to encourage you to change it.
#

chroot = /var/log/ipcad;

#
# File to keep getpid() in it. ipcad will also hold a lock.
#
# WARNING: Pidfile is created AFTER chroot()'ing, so if you're using
# chroot statement above, make sure the path to the pidfile exists
# inside chrooted environment.
#

pidfile = ipcad.pid;

#
# UID/GID privileges dropping
# Please note: RSH service will be UNAVAILABLE when uid is not zero.
# Use it only when you know what are you doing (i.e., NetFlow without RSH).
#
# uid = 65534;
# gid = 65534;

#
# Few useful settings.
#

#
# Memory usage limit for storing per-stream entries.
# 
# memory_limit = <number>[{k|m|e}] ;
# Where k, m and g are for kilobytes, megabytes or table "entries".
#

memory_limit = 10m;</number></host_addr></user></listen_ip></listen_ip></port></port_range_end></port_range_start></maskbits></masklen></ip></maskbits></masklen></ip></group></tcpdump-output.pcap></divert-port></group></group></pcap_filter></iface></number></vlm@lionet.info>