I am confusion...IDS inline on single WAN running an OpenVPN server which is LAN



  • DIsclaimer: I'm not a pro, I'm a hobbyist, I am just starting to learn pfsense but I am a linux guy, this is running on a lab network just for learning purposes, I am reading the pfsense book as I go and most of my questions do find an answer there, but here I'm having issues understanding

    I am having some troubles wrapping my head around what I'm actually doing and where the packets are moving in the stack in this setup.

    1 WAN interface, this is the only physical interface (vtnet0)
    1 LAN interface, this is the tun0 interface created by the OpenVPN server (ovps0)

    Everything is working as anticipated, basically the only firewall rule i had to add was allowing 1194UDP from WAN to This Firewall and with automatic NAT rules the VPN does get internet access (this is running on a VPS and will be used as a VPN aggregator basically)

    I've run suricata in inline mode both on WAN and LAN, it works, I was able to inspect packets and have alerts/blocks but if I try to wrap my head around where the packet go in the stacks I'm kinda lost.

    I mean inline means that as soon as the packet arrives on the NIC at some point in the stack they get intercepted by the IDS and analyzed, this happens where exactly? before or after the firewall rules?

    but that's also, more or less, what OpenVPN is doing, OpenVPN does it with a tun device that works as a filter on the packets in the stack.

    Am I doing this twice when I don't actually need to?
    I think I do because I'm running the IDS on the packets coming in from the WAN, all of them, including the OpenVPN encrypted packets, and then I'm running it again inside (? this is the confusing part) the OpenVPN stack? is there such a thing as the OpenVPN stack since there's a device?

    I am confusion...
    can anybody point me out to good resources on this, specific to pfsense I mean.


  • LAYER 8 Global Moderator

    What exactly do you hope to accomplish with such a setup?



  • learning and having fun, some people make puzzles as a hobby, I dig through documentation to make complex setups and then look after my servers :)
    I have other VPS with other services (nextcloud, etc.) and I only use them privately, I also have a NAS at home so just to have fun instead of having a VPN server on some VPS and just connect everything there as a client I dedicated one to do this, just for the fun of it really, it doesn't really need to do anything beside entertaining me.

    I mean specifically since I'm only allowing UDP packets on 1194 in on WAN do I really need to run an IDS there? there isn't much point in doing it? is it? does the firewall comes before or after the IDS in the stack? even theoretically speaking it doens't make any difference, does it?

    as I said, it is working, if I set the proper rules in the proper direction I get alerts/blocks, and they happened on the LAN interface, where the packets are going through decrypted with source and dest being VPN IP.

    Edit: I probably need to run it on WAN too but only checking that the actual decryted packets LEAVING WAN do not break rules about bad hosts and stuff like that, don't I?



  • Suricata and Snort both work outside of the firewall, but whether you call it "before" or "after" the firewall depends on which netflow direction you are talking about. For the WAN, Suricata and Snort see packets directly off the NIC driver. So for inbound packets (from Internet to WAN port) the IDS is seeing the packets before any firewall rules (including NAT) have been applied. For outbound traffic (from the WAN port to the Internet), the IDS is seeing the traffic after any firewall rules and after NAT has been applied. The same situation exists when the IDS is on the LAN except you would swap in the phrase "WAN" in place of Internet for most setups with just a WAN and LAN interface.

    A VPN protects you from nothing in terms of becoming a malware or virus victim. There is some misunderstanding of that out there I believe. We frequently have folks post here saying they use a VPN to improve security when all they are really using it for is to hide their Internet destinations from their ISP. You can still have your machine compromised by malware even when using a VPN. So when you use a VPN an IDS/IPS is still a good tool for examining the packets to look for malware and other threats. Obviously with a VPN arrangement the IDS/IPS needs to be at a place in the chain where it can see the unencrypted packets, so putting it on the LAN is the best choice.



  • @bmeeks said in I am confusion...IDS inline on single WAN running an OpenVPN server which is LAN:

    Suricata and Snort both work outside of the firewall, but whether you call it "before" or "after" the firewall depends on which netflow direction you are talking about. For the WAN, Suricata and Snort see packets directly off the NIC driver.

    so literally the order for a packet incoming on WAN is:

    Internet => WAN NIC => NIC Cache => IDS (rules are applied here, including drops) => Firewall (other drops might happen here) => rest of the stack.
    does this behaviour changes if I use it in legacy-mode? pcap makes it's own copy of the packets in a different cache and basically they move alongside independently until (if) suricata calls a drop on a packet, correct?

    in my case "rest of the stack" means (eventually) the OpenVPN server process, at least on linux I know that a whole new stack is actually being created there, with it's own cache and all, and encryption/decryption capabilities. Those packets are coming in and going out in two different ways tho.

    At the OS level once a packet that came in on UDP 1194 has traversed the stack and all of the packet field have been processed (layer 1 to 4 I'd say, IP level) the actual data that was found in the packet is being copied from the same cache where the NIC put it at he beginning to a different place in memory so the stack is free to keep going with other packets, the appropriate process is called upon and told that there's this new data available.

    when that happens with an OpenVPN packet the data that's being handed over to it it's encrypted, the first thing the OpenVPN server has to do is decrypt it.

    Once decrypted what's found is basically a whole new packet, with it's own internal VPN IP addressing, TCP fields and all that you'd find in a standard packet.
    in most cases this packets needs to be routed to an OpenVPN client which is physically reachable on the initial stack, and in order to do that first OpenVPN creates such a packet (with internal vpn ip addresses) then encrypts it and creates another UDP packet that encapsulates that (in an ecrypted form) and has it's own fields (a different origin:dest than in the packet encapsulated) and injects it back in the OS stack through the virtual device it created, right before the NIC sends it out, but after outbound firewall rules are being applied (at least this is on linux).

    I also know that this actually gets a bit more complicated than this, the OpenVPN server also has to manage DHCP/ICMP packets and different protocols (this is where the difference between tun and tap device really is) and all that and this is a whole field of expertise on it's own.

    when exactly does the IDS get's called in in this stack? same rules apply or something different happens at the level of firewall rules/IDS that I should be aware of?

    I am now in the process of reading the rules, this is the first time I actually play with an IDS, and on this pfsense instance I actually run a number of OpenVPN servers that connect both to my home/laptop and a couple of services, since I'm basically doing all the routing on this box it makes sense to have both a firewall and an IDS running on it; I felt accomplished when I could stop nmap scans from my home network going outside, meaning with the appropriate inverse rule they would be stopped the other way around (and they are, I tried) and I received an email alert.

    I now have lots of work to do in setting up these babies :)



  • @nuclearstrength said in I am confusion...IDS inline on single WAN running an OpenVPN server which is LAN:

    @bmeeks said in I am confusion...IDS inline on single WAN running an OpenVPN server which is LAN:

    Suricata and Snort both work outside of the firewall, but whether you call it "before" or "after" the firewall depends on which netflow direction you are talking about. For the WAN, Suricata and Snort see packets directly off the NIC driver.
    

    so literally the order for a packet incoming on WAN is:

    Internet => WAN NIC => NIC Cache => IDS (rules are applied here, including drops) => Firewall (other drops might happen here) => rest of the stack.
    does this behaviour changes if I use it in legacy-mode? pcap makes it's own copy of the packets in a different cache and basically they move alongside independently until (if) suricata calls a drop on a packet, correct?

    Yes, this is correct. Using Legacy Mode does not change the order other than now you two parallel paths (the original packet proceeds to the pfSense network stack and firewall while the copy proceeds to the IDS).

    in my case "rest of the stack" means (eventually) the OpenVPN server process, at least on linux I know that a whole new stack is actually being created there, with it's own cache and all, and encryption/decryption capabilities. Those packets are coming in and going out in two different ways tho.

    At the OS level once a packet that came in on UDP 1194 has traversed the stack and all of the packet field have been processed (layer 1 to 4 I'd say, IP level) the actual data that was found in the packet is being copied from the same cache where the NIC put it at he beginning to a different place in memory so the stack is free to keep going with other packets, the appropriate process is called upon and told that there's this new data available.

    when that happens with an OpenVPN packet the data that's being handed over to it it's encrypted, the first thing the OpenVPN server has to do is decrypt it.

    Once decrypted what's found is basically a whole new packet, with it's own internal VPN IP addressing, TCP fields and all that you'd find in a standard packet.
    in most cases this packets needs to be routed to an OpenVPN client which is physically reachable on the initial stack, and in order to do that first OpenVPN creates such a packet (with internal vpn ip addresses) then encrypts it and creates another UDP packet that encapsulates that (in an ecrypted form) and has it's own fields (a different origin:dest than in the packet encapsulated) and injects it back in the OS stack through the virtual device it created, right before the NIC sends it out, but after outbound firewall rules are being applied (at least this is on linux).

    I also know that this actually gets a bit more complicated than this, the OpenVPN server also has to manage DHCP/ICMP packets and different protocols (this is where the difference between tun and tap device really is) and all that and this is a whole field of expertise on it's own.

    when exactly does the IDS get's called in in this stack? same rules apply or something different happens at the level of firewall rules/IDS that I should be aware of?

    The IDS interacts soley with the physical NIC driver. If using Inline IPS Mode, that means the netmap device is activated and used. The netmap device creates a pipe between the physical NIC driver and OS kernel stack. So again, for WAN this means IDS rules are after firewall rules for outbound traffic, and IDS rules are before firewall rules for inbound traffic. For the LAN, IDS rules are before the firewall for inbound traffic (traffic entering the LAN interface and destined for other interfaces) and the IDS rules are after the firewall rules for outbound traffic (traffic that is leaving the LAN physical port heading to the switch and/or devices on the LAN).

    I now have lots of work to do in setting up these babies :)

    Have fun. Learning to administer an IDS/IPS is a fun challenge. It will require quite a bit of Google research as you learn and try to figure out which rules you actually need in your network and which are more prone to false positives and might need to be disabled. You also only need to apply rules that protect against vulnerabilities you have. My favorite two examples are public-facing mail and web servers. If you don't have a public facing mail or web server, then there is no need to activate the rule categories for those threats.



  • @bmeeks said in I am confusion...IDS inline on single WAN running an OpenVPN server which is LAN:

    The IDS interacts soley with the physical NIC driver. If using Inline IPS Mode, that means the netmap device is activated and used. The netmap device creates a pipe between the physical NIC driver and OS kernel stack.

    hold on, I am confusion again (less than before tho)

    I mean having activated the IDS on my LAN device, which is an ovpns device, so a virtual one, the IDS is tapping in somewhere on this virtual device stack, where?

    If you're saying the IDS is interacting only with the physical NIC driver this throws me off a bit, what's creating the pipes associated with the ovpns0 netmap device?

    how is this working then? because it is, I see packets being analyzed by the IPS on the ovpns0 device with their internal VPN IP mapping, the physical NIC sees only the UDP encrypted packets coming in on WAN and the unecrypted packets NATted through the VPN with their IP mapped outside the VPN leaving WAN



  • @nuclearstrength said in I am confusion...IDS inline on single WAN running an OpenVPN server which is LAN:

    @bmeeks said in I am confusion...IDS inline on single WAN running an OpenVPN server which is LAN:

    The IDS interacts soley with the physical NIC driver. If using Inline IPS Mode, that means the netmap device is activated and used. The netmap device creates a pipe between the physical NIC driver and OS kernel stack.

    hold on, I am confusion again (less than before tho)

    I mean having activated the IDS on my LAN device, which is an ovpns device, so a virtual one, the IDS is tapping in somewhere on this virtual device stack, where?

    If you're saying the IDS is interacting only with the physical NIC driver this throws me off a bit, what's creating the pipes associated with the ovpns0 netmap device?

    how is this working then? because it is, I see packets being analyzed by the IPS on the ovpns0 device with their internal VPN IP mapping, the physical NIC sees only the UDP encrypted packets coming in on WAN and the unecrypted packets NATted through the VPN with their IP mapped outside the VPN leaving WAN

    Go Googe "FreeBSD netmap device" and that should begin to answer your questions. You can also research in detail how OpenVPN creates its hooks into the FreeBSD networking kernel stack.

    pfSense is your "origination" and "endpoint" for VPN connections to/from your LAN. Traffic flowing in and out of the physical LAN interface is unencrypted, so the IDS can inspect it.





  • @bmeeks said in I am confusion...IDS inline on single WAN running an OpenVPN server which is LAN:
    Go Googe "FreeBSD netmap device" and that should begin to answer your questions. You can also research in detail how OpenVPN creates its hooks into the FreeBSD networking kernel stack.

    pfSense is your "origination" and "endpoint" for VPN connections to/from your LAN. Traffic flowing in and out of the physical LAN interface is unencrypted, so the IDS can inspect it.

    @NollipfSense said in I am confusion...IDS inline on single WAN running an OpenVPN server which is LAN:
    https://www.unix.com/man-page/freebsd/4/netmap/

    thank you guys so much for pointing me towards the right direction, netmap is what I needed to dig a bit into to really understand this.
    I did not understand what it was doing in the system and you guys sent me through a very very interesting rabbit hole.

    let me add a couple of links in this thread (which is already popping in my google bubble for the search "freebsd netmap device openvpn"), this is the original paper from the guy who wrote netmap, Luigi Rizzo, an Italian IT professor, it goes into details about how it works and how it does it's pipes and answered a lot of my questions.

    netmap: a novel framework for fast packet I/O

    At a very high level, when a program requests to put an interface in netmap mode, the NIC is partially dis-connected (see Figure 3) from the host protocol stack.The program gains the ability to exchange packets withthe NIC and (separately) with the host stack, through circular queues of buffers (netmap rings) implemented in shared memory.

    figure 3
    alt text

    so specifically about suricata it does support netmap devices. So out of the box in inline mode, if your network device has netmap capabilities packets are gonna get from the NIC to suricata via the netmap magic (TX and RX rings, operating in shared memory).
    Am I getting this right?

    Specifically about OpenVPN another chapter should be opened because it comes with it's own way to implement things, it should be noted here that OpenVPN does stuff in user-mode and has its own hooks to get the encrypted packets coming in from the stack, authenticate/decrypt/(de)compress and then give em back to the stack, I haven't dug deep into it (yet), so thank you for clearing the air for me and allowing me to go further, much appreciated.

    If we were to consider the other main VPN implementation, IPsec/IKE then IPsec is already happening in kernel and only IKE is happening in user mode, so it's got different piping to encrypt/decrypt the packets while talking with the stack/netmap, so the two probably will end up having the packets follow very different routes in a netmap system.

    I'm also a linux guy and the tools to observe your own system are slightly different so doing this on BSD is extremely difficult for me, I've got an added learning curve, but exactly the learning experience I want because I'm really liking pfsense.

    plus, my VPS pfsese box deployed in the wild is working like a charm, the IDS rules are easier to tackle than I expected, given my setup I only really block scans and known bad hosts since all my potentially vulnerable services are accessible only via VPN, so that's probably why, monitoring an actual active webserver is probably a very different task.


Log in to reply