Inter-Vlan Routing Accross VPN
-
Hello,
I have one pfsense box setup in a test lab (to replace our cisco 2800s) we have 4 branches offices (1 of which is consider our main office as far as IT goes)
The test box I have configured right now which works great is for Branch 1 (site with the server farm)
It will have the following VLANs (I currently have the vlans setup with intervlans routing working fine)VLAN 10 - Branch 1 staff 10.10.100.0/24
VLAN 20 - Branch 1 VoIP (both Phones and contoller at this site) 10.20.100.0/24
VLAN 30 - Branch 1 Public Access Computers 10.30.100.0/24
VLAN 40 DMZ 10.40.100.0/24Currently I have this config working. VLAN 10 can get to VLAN 20, 30 & 40. VLAN 30 Can get to VLAN 40 (since 1 to 1 Nat's don't work internally)
I need 3 additional boxes over VPN with the corresponding VLANs only allowed to talk to there corspeonding vlan at the other site but the STAFF vlan needs to have access to all of them (from any site).
I know this needs to be done with ACLs/firewall rules somehow since it's Layer 3, but How, I know how to do it locally but doing it over a VPN tunnel is confusing me. and would this be better done with OpenVPN or IPSec (all Fiber connections are from the same ISP)
Here is a list of all the vlans
Branch 2
VLAN 11 - Branch 2 Staff 10.10.110.0/24
VLAN 21 - Branch 2 Voip (phones only) 10.20.110.0/24
VLAN 31 - Branch 2 Public Access Computers 10.30.110.0/24Branch 3
VLAN 12 - Branch 3 Staff 10.10.120.0/24
VLAN 22 - Branch 2 Voip (phones only) 10.20.120.0/24
VLAN 32 - Branch 2 Public Access Computers 10.30.120.0/24Branch 4
VLAN 13 - Branch 4 Staff 10.10.130.0/24
VLAN 23 - Branch 4 Voip (phones only) 10.20.130.0/24
VLAN 33 - Branch 4 Public Access Computers 10.30.130.0/24Any Help?? Please.
Thanks in Advance!
Jason
-
What sort of bandwidth will you be using via the VPN between the main office and the other offices?
Openvpn is better in almost every case, but it has to go back and forth between user-space and kernel space. This makes its upper limit of possible throughput lower than the theoretical upper limit of IPsec, which operates in kernel space.However, I say "theoretical" because depending on your hardware you might never reach that limit, in which case I like Openvpn much better.
-
It will be the same as our current VPN is on with the CISCO 2800's at each location (though there are no vlans in the current config) which is a 20MB Fiber connection (but I believe the 20MB may be a shared limit of all the links combined)
I guess I should have noted that normal internet bound traffic should go out the WAN connection at each site directly and not over the VPN (I guess that's assumed though)
-
OpenVPN really starts to measure up favourably performance wise against IPsec as the numbers of clients increases for each. At 20MB, especially shared between many client offices, OpenVPN is going to perform very well. If it were 100MB to a single client, it might be a different story.
Thats my feeling on that. Either will work, but one is definitely less fuss than the other.As far as what subnet can communicate with what subnet or not using VLANs, I'd think that would be an easy set of rules to establish using firewall floating rules to block or pass communication between subnets on each pfsense box. (Long maybe - but easy). I know a couple of the HERO guys are VLAN gurus. Maybe one will suggest something better.
If Staff, VoiP and Public each got separate VPN tunnels back to main office, this would all suddenly become easy to manage I think. (so says the amateur)
-
Do you think a P4 3.4ghz Box with 2GB RAM at the main office (old Checkpoint C6P-CP UTM-1 2050)
Main Site has
5 Servers (Domain/Webserver (internal mostly)/File Server/IT-MGMT (spiceworks/Antivirus console)/Terminal service for 3 users)
12 Staff Computers
18 Public Computers
Public Wifiand P4 1.5ghz Mobile Box with 1GB at each remote site could handle this (old Checkpoint UTM-1 450 C2P-CP)
Remote Sites Have
1 Domain Controller Each (no more Servers)
~10 Staff Machines
~14 Public Computers
Public Wifi -
I think the real question is can each of those handle 20MB/3 continuous over VPN, since thats the only CPU intensive thing you have mentioned thus far. So figure 20MB peak for any one and about 6MB throughput each on average. Yes. Piece of cake. Easy. If you can figure out the VLAN problem. (I'm assuming this is firewall/routing and not a bunch of snort intrusion detection)
Hint on the VLAN issue. I think "jimp" could answer to the VLAN question definitively.
-
Yes, I plan on doing just Firewall/Routing and maybe the Captive portal with no authentication on the Public Vlans, But I don't think that would really use much resources. Definitely no snort, I've always had issues with it.
I will see if Jimp can help
-
Yeah - The 1.5GHZ processors will do that no problems at all. I manage 5MB links even with a 300MHZ linksys E1000 as client to my servers and those have very weak processors. Hardware won't be an issue for you. Figuring out the VLAN layout should be your only worry. When you get this worked out, can you post the VLAN over Openvpn solution? It would be nice to know.
-
I guess I'm on my own on this one. He [Jimp] told me not to contact him. He want's you to buy commercial support. (which we being a public library could not do)
-
Sorry about that - Thats my mistake.
Referring this or that person by user-name here is probably a Faux pas.
In the event you can't get VLAN tagging to work accross the VPN:
I would probably handle this by having 3 Openvpn server threads running on that main pfsense in main office. One per function.
Then I'd probably have every computer in at every office be a client (24/7) to one of those VPN server threads and control their access to each other that way.
No VLANS required for that to work.
Are these computers windows? If they are, that makes my alternate solution ridiculously easy to implement. -
I think I found a solution, but it will require me to use IPSEC
http://doc.pfsense.org/index.php/IPsec_with_Multiple_Subnets
I can map the routes the additional subnets (vlans)
-
Hey - Thats neat. Not exactly what you were looking for but if that works as advertised, might be fine. Let me know how that turns out and how stable IPsec is for you. I've never needed to do this before, but for this scenario looks like, as far as I can tell, IPsec is better. Thats very cool if it works. I had read that using the TAP interface rather than the TUN in (maybe in bridged mode) might accomplish what you desire but if IPsec works for you, no need experiment.
-
For what its worth I don't think VLANs would work for what you were trying to do. Remember that Vlans are a layer 2 way of breaking up broadcast domains. Once you cross a layer 3 device your layer 2 Vlan tag will be lost. Remember that when the router moves the packet from one interface to another it will change the Ethernet header information which contains the source and destination MACs. Now if there were a way to keep your layer 2 information to persist over the routed connection then what you wanted to do could work.
-
https://forum.openwrt.org/viewtopic.php?id=33678
-
For what its worth I don't think VLANs would work for what you were trying to do. Remember that Vlans are a layer 2 way of breaking up broadcast domains. Once you cross a layer 3 device your layer 2 Vlan tag will be lost. Remember that when the router moves the packet from one interface to another it will change the Ethernet header information which contains the source and destination MACs. Now if there were a way to keep your layer 2 information to persist over the routed connection then what you wanted to do could work.
Yes, I know Lan is at layer 2. I don't plan on having the VLAN tags go across the VPN. The Layer 2 Vlans match up to Layer 3 Subnets (see my OP all of them are serperate subnets/vlan which would require routing, the vlans wouldn't match up for just tagging to work) anyway. All I need is to get all the subnets to be able to route across the VPN and use ACLs at each point to keep the correct subnets where they are suppose to be.
Also with my current config on my Cisco Routers I have A Multipoint VPN (I think it's technically called a Dynamic multi-point VPN) is there any feature like this in Pfsense, meaning that I don't have just one site being the server and the rest being clients (hub a spoke design) but all sites interconnect?
-
You mean full-mesh? TINC. Hmmm. Not in the packages for my 2.03 though.
-
yes
-
I think the open-source full mesh vpn solution is TINC. I know its been talked to go into pfsense but not sure if its in the 2.1
I know it can have NAT issues, but people like you don't have NAT issues. I'm sorta surprised if its not already a package in 2.1 -
Yes, I know Lan is at layer 2. I don't plan on having the VLAN tags go across the VPN. The Layer 2 Vlans match up to Layer 3 Subnets (see my OP all of them are serperate subnets/vlan which would require routing, the vlans wouldn't match up for just tagging to work) anyway. All I need is to get all the subnets to be able to route across the VPN and use ACLs at each point to keep the correct subnets where they are suppose to be.
Also with my current config on my Cisco Routers I have A Multipoint VPN (I think it's technically called a Dynamic multi-point VPN) is there any feature like this in Pfsense, meaning that I don't have just one site being the server and the rest being clients (hub a spoke design) but all sites interconnect?
I'm sorry I misunderstood your post, I thought you were trying to get your vlans to persist across the VPN connection. I did see that you are using different vlans. I was thinking you wanted routing across all sites but just wanted to be sure. I think you would probably need a point to point at each site, it sounds kind of ugly but it would accomplish your task. This has me interested now though, it should be possible with out all the extra configs so I will make a mock setup and report back. If you need it down quickly I would do it the ugly way and then work on the routing through the main site. Might be better to make the mesh setup because that way you don't lose connection to the other sites if the main site goes down and also there is less un-needed processing on the router at your main site.
https://forum.openwrt.org/viewtopic.php?id=33678
Neat trick but I don't think this would work if you wanted to have multiple vlans go across a VPN Connection.
-
I'm sure soon someone will figure a way to build VLAN support smoothly into VPN of some flavour or another, but I'm not seeing it being easy yet.
-
Yes, I know Lan is at layer 2. I don't plan on having the VLAN tags go across the VPN. The Layer 2 Vlans match up to Layer 3 Subnets (see my OP all of them are serperate subnets/vlan which would require routing, the vlans wouldn't match up for just tagging to work) anyway. All I need is to get all the subnets to be able to route across the VPN and use ACLs at each point to keep the correct subnets where they are suppose to be.
Also with my current config on my Cisco Routers I have A Multipoint VPN (I think it's technically called a Dynamic multi-point VPN) is there any feature like this in Pfsense, meaning that I don't have just one site being the server and the rest being clients (hub a spoke design) but all sites interconnect?
I'm sorry I misunderstood your post, I thought you were trying to get your vlans to persist across the VPN connection. I did see that you are using different vlans. I was thinking you wanted routing across all sites but just wanted to be sure. I think you would probably need a point to point at each site, it sounds kind of ugly but it would accomplish your task. This has me interested now though, it should be possible with out all the extra configs so I will make a mock setup and report back. If you need it down quickly I would do it the ugly way and then work on the routing through the main site. Might be better to make the mesh setup because that way you don't lose connection to the other sites if the main site goes down and also there is less un-needed processing on the router at your main site.
https://forum.openwrt.org/viewtopic.php?id=33678
Neat trick but I don't think this would work if you wanted to have multiple vlans go across a VPN Connection.
I upgraded one of my boxes to 2.1RC0 and installed TINC (which I've never heard of before, granted I'm more of Cisco guy than an open source guy). I haven't tried it in practice yet, but It looks like it will pass all the subnets based on this anyway.
and TINC has firewall rules so you can allow subnets only to go to specific subnets.
Let's hope this works.
and then since TINC has firewall rules.
-
I'm sure soon someone will figure a way to build VLAN support smoothly into VPN of some flavour or another, but I'm not seeing it being easy yet.
L2VPN does this exactly, it provides no security itself though. and I don't believe pfsense does IPSEC l2vpn as of yet.
I never looked into it much, but I believe l2vpn would be similar to router-bridging. So it would mess with your broadcast domains and cause more than necessary traffic
-
I think that if you own a nice static public address at every site and don't hit NAT issues (you shouldn't) a full mesh network is good. It even has the added benifit of not laying all the bandwidth burden on one central server. In theory, should make things work alot faster and offer greater resiliency because nodes can go up and down without taking out the entire network. I've yet to install it, so please do let me know how it works for you.
-
Looking at your rule you are making there… Will you only be passing TCP? Because TCP is whats selected there.
I also don't know how automatic any rule creation is on the WAN when you use TINC in pfsense but I do know that there are some ports that have to be opened, either automatically or manually. 655 UDP and TCP for sure. -
Looking at your rule you are making there… Will you only be passing TCP? Because TCP is whats selected there.
Its blocking rule. and It was for example only. no port was configured either for that matter.
-
Yes. I see the block at the top now. Almost chopped, but not quite.
-
I'm not having any lucky with it yet. It installed easy though.
I have both boxes WAN port plugged into our current lan.
One Box set to 10.10.100.52
Second One 10.10.100.60Both Get internet traffic fine. But they can't ping each other which I assume is the problem. I did setup a rule on the WAN interface of both to allow ICMP from ANY to ANY.
This is the TINC log either one only shows itself right now. NAME changed to protect the innocent ;)
Statistics for Generic BSD tun device /dev/tun0:
total bytes in: 620
total bytes out: 900
Nodes:
NAME at MYSELF cipher 0 digest 0 maclength 0 compression 0 options c status 0018 nexthop NAME via NAME pmtu 1518 (min 0 max 1518)
End of nodes.
Edges:
End of edges.
Subnet list:
192.168.1.0/24#10 owner NAME
End of subnet list. -
You have your public IP NATed > pfsense boxes?
TINC doesn't like NAT. I assumed you would be setting this right against the public IP as the primary router/firewall so TINC would not be behind any NAT.
I'm not a TINC Expert or even TINC novice for that matter. I know a few people do chat about using it.
The guys at the last DEFCON were saying they use it for their Chaos Network. Maybe some of their grey hats would be willing to set you up.
(Kidding) - I hope you get it worked out. I might later find an excuse to use it, but not so far. -
They will each have Public IPs in practice.
however I need to do labs with them before I deploy them.
Too bad Pfsense doesn't have WIC cards like Cisco Router (hehe)
I've tried a cross-over cable between both boxes, with static WAN IPs (and even tried put the opposite one as the others Gateway)
Tried them on the same switch with etc.Nothing seems to work to make them talk over a fake WAN locally. This happens with any of the three VPN technologies currently. Any Ideas how to make them talk? I need to do some labs with time to make sure they will configure correctly before I just deploy them.
-
If you want to pretend they are in a Public IP environment, with no NAT screwing with them, try this.
Use a cheap off the shelf old router (like a linksys or belkin or whatever). Use DHCP.
Plug the WAN of each of your PFsense boxes into LAN ports on that router.
Now, they should each get a IP and they shouln't be behind NAT.
At this point they should be able to do whatever it is you are trying to make them do.
However, this assumes TINC is working correctly and your settings are correct.
I'm not sure what your LAN is like, but I know that a cheap dumb router should let you accomplish this.
(Disclaimer - I've never set up TINC, so no idea if the package works. My fingers are crossed) -
Just an Update, I could never get TINC working it try to connect and does for a few mintine or so, and then fails..
IPSEC works fine though.
To bad pfsense doesn't have this: http://sourceforge.net/projects/opennhrp/
-
I'm glad its working…
"NHRP, GRE and IPsec. It aims to be Cisco DMVPN compatible."
I've had many many bad experiences with GRE and I avoid it like the plague, but I'll take a look it this.
Are you still on pfsense then?
Other than simply "IPSEC" what other issues did you work out?
-
you know one other thing I didn't think about with TINC is the firewall may need to be opened on wan for port 655. all the other (Ipsec and OpenVPN) automatically do that, without creating rules but since tinc is not an official package it may not.. just a though. I'll check it again.
-
"the firewall may need to be opened on wan for port 655"
haha… I said that early on, but maybe it was lost in the clutter and frustration.
It happens.
-
I feel really dumb now.. The firewall rules was the only Issue with it not connecting. I'm going to play with the multiple subnets this weekend but it's looking promising. Seems to have much less over head than IPsec does too
-
To error is human… And a little funny when its someone else erroring :D
I feel your pain. I've been there.
For what its worth, you sound wicked smart and fast learner.
-
FYI, this is working great It's been in production for a while now. Great throughput even over TINC VPN
now the only thing I wish I could figure out is how to get Pfsense to do local dns lookup for dhcp client that don't specify a domain, just a hostname.
-
Services > DNS forwarder
Options there don't get you what you want?
-
It works if I do an NSlookup/ping for Computer.localdomain but just computer does not work. if I do the nslookup or ping from pfsense itself it works with just the computer hostname.
-
I wonder if a ubuntu machine with Samba 4.0 set up as a WINs server would help?