[tcpdump] Rotating pcaps vs port-mirror switch/appliance

IOerror

Hi All,

So I'm wanting to capture all traffic between my modem and my pfSense box and save it to my file server to analysis later. I'm working on setting up a dedicated pcap appliance with a port mirror in the near future, but until then I'd like to try this. Disclaimer: This is a private home network, not a business. I own all hosts behind the firewall.

Here's what I plan on doing:

1.) Mount share at boot, /media/pcaps (fstab)
2.) Setup a cron job to start tcpdump (or restart if it stops running)
3.) Setup a netflow probe to monitor for large spikes in traffic and review that time frame later

So here's my setup:

[Internet]–--[Modem]–--[em0-WAN:pfSense:LAN-em1]–--[Switch/WiFi AP]–-[FreeNAS, hosts, NetFlow/Syslog server]

Hardware:
Intel D2500CCE Mini-ITX (Atom D2500 1.86GHz dual core)
–Dual Intel 82574L NICs
4GB DDR3-1333 RAM
Intel 64GB SSD (overkill, I know, was just laying around collecting dust)
Intel PRO/1000 PCI Gigabit NIC (soon, in the mail)

Goals:
Capture all traffic on WAN interface and OpenVPN clients (dunno if this is doable)
Rotate 50*10MB pcaps, with 500MB in total (for now). Goal is to try and have 30 days worth of data.
Write a script to copy data from timestamp x until timestamp Y into a seperate file (huge spike happened, grab it, analyze it)

Concerns:
1.) Is this a bad idea? Should I just wait until I have a dedicated box with a port-mirror capable switch/hub?
2.) Will this eat up CPU cycles? If so, can I force tcpdump to run on one core to minimize impact?
3.) Will this capture all traffic before rules are applied or after?
4.) Should I adjust the tcpdump buffer?
5.) What are the best tcpdump options to use?
-- Here's what I'm planning right now: 'tcpdump -i em0 -s 0 -B 1024 -KSfnNvvvXX -C 10 -W 50 -w /media/pcaps/em0.pcap'
6.) Any suggestions for a good free netflow monitor that'll run on a Raspberry Pi?

Thanks in advance!

Derelict

I would size the pcap files to something your wireshark can comfortably load. Then keep as many of those as you need for the buffer size you want.

You can filter larger files then load into wireshark but that's a pain.

Would I do this at all on my firewall? Probably not. I'd build the host and use a mirror port on the outside switch.

IOerror

I was planning on doing 50 captures at 10MB a piece. Should be small enough for Wireshark to handle.

BBcan177

http://blog.securityonion.net/p/securityonion.html
‎
‎https://github.com/Security-Onion-Solutions/security-onion/wiki/IntroductionToSecurityOnion
‎
‎https://groups.google.com/forum/#!forum/security-onion
‎
‎You can also disable the IDS/HIDS and just use it for full packet capture.

Derelict

@IOerror:

I was planning on doing 50 captures at 10MB a piece. Should be small enough for Wireshark to handle.

I would test it and see how big you can make them, not guess at it.

johnpoz

There use to be a perfect tool for this.. http://www.colasoft.com/nchronos/ had a FREE version was really sweet.. But they got rid of the free version, and the cost makes it prohibitive for home use..

I would be interested if there was some sort of RNA appliance or packetvault appliance that could be setup.. I have been looking for an alternative to nchronos off an on since it stopped being free and have not found a good solution.

BBcan177

I would be interested if there was some sort of RNA appliance or packetvault appliance that could be setup.. I have been looking for an alternative to nchronos off an on since it stopped being free and have not found a good solution.

Have you tried S.O. It has all those features.

johnpoz

Security Onion? Yeah so those links haven't played with in quite some time have to look into that option.

IOerror

All,

Thanks for the quick replies! As to the suggestions for Security Onion, I was already planning on this once I find a suitable bit of kit. Frankly though, learning an entire new system is a bit much for something I'm trying to learn step by step. I did get the message though, thank you.

While researching this, I still haven't found a way to capture decrypted OpenVPN traffic. Is there a way to forward all VPN traffic from pfSense to my monitor (second NIC presumably)? If I had a dedicated VPN device, this would be trivial.

Regards,
IOerror

johnpoz

If you capture the traffic on the lan side of pfsense, than any vpn traffic would already be decrypted.. Any traffic inside a ssl/tls connection is going to be encrypted as well.

IOerror

So I tried what you suggested and was not able to capture any OpenVPN traffic across the LAN interface. All I could find was State mappings for my WAN interface to my ovpns1 interface. The only traffic I could see hit my LAN interface was when the destination was a LAN host. So it seems that if I want to capture decrypted OpenVPN traffic, I would need to monitor the ovpns1 interface which is a virtual interface.

This is as it should be. OpenVPN is a seperate collision domain, thus I would not see any of that traffic on my LAN. So this leads me back to the same problem, how do I capture OpenVPN traffic with a dedicated packet capture device? Theoretically, I could create a bridge between OPT1 and ovpn1s, then monitor there. I'm not even sure if this is possible (or advisable). If it is, I imagine it would be an ugly hack.

Regards,
IOerror

NOYB

If you have the private key, Wireshark should be able to decrypt the traffic, I think. I know it can for https, have never done with OpenVPN though. Maybe someone can confirm yea or nay.

johnpoz

When I meant to capture decrypted traffic to your lan I figured you would be sniffing on a span port somewhere. But if you sniff on pfsense lan you should see all traffic in and out of that interface.