Two gateways, two subnets, one internet, subnet connectivity issue
-
If I were to go with the route of connecting the two networks via LAN to LAN, could I simply reassign the WAN NIC to a LAN and not have a WAN port on the pfSense Machine and do what you have described?
Good question, I suppose if you remove the gateway from the "WAN" interface and just rename it… it would then become a LAN interface... sure.
For now, I'd like to properly setup a port forward in pfSense to allow an IP from LAN1 to access LAN2. The IP I'd like to give access is 192.168.1.115. pfSense port forward options under firewall-nat-port forward
Interface: WAN
Protocol: TCP
Source: What type of source? Wan IP, LanIP, Network etc…?
Source Port Range: ANY
Destination: LAN net: 192.168.5.0 /24 I think this is right, but let me know
Destination Port Range: Type/Number???
Redirect Target IP: ?
Redirect Target Port: ?Port forwards are for redirecting external traffic to a specific internal resource on specific ports, so that's not going to work. For example, if you wanted to only allow 192.168.1.115 access to a web server on 192.168.5.100, you would enter this:
Interface: WAN
Protocol: TCP
Source: "Single host or alias" "192.168.1.115/32"
Source Port Range: ANY
Destination: "WAN address"
Destination Port Range: 80
Redirect Target IP: 192.168.5.100
Redirect Target Port: 80If you want to allow anyone to the web server, you would change your source back to "any"
Then, @ LAN 1, to access the web server @ LAN 2, you would enter the WAN address of LAN 2 in your browser…i.e. http://192.168.1.112 and the traffic will be redirected to 192.168.5.100 on port 80.
-
So the port forward doesn't really seem like the best way to go about it then.
I'm not really familiar with the site-site tunnel at all or IPSEC.
Seems like trying to change the WAN on the pfsense box to a LAN connection and attempt the following:
You have a design issue that needs to be addressed. If you're connecting two sites via a direct connection, you want to connect the sites via LAN interfaces (not WAN). So, if you were going to keep both edge devices where they are, I would:
Add a 3rd NIC to PFsense on LAN 2 and assign it a static IP in the 192.168.1.1/24 range (e.g. 192.168.1.254)
@ LAN 2, patch your wireless bridge to the 3rd NIC
@ LAN 2, add an any/any rule to the 3rd NIC interface
@ LAN 1, add an any/any rule to the LAN interface (you can refine it later if needed)
@ LAN 1, add a static route to 192.168.5.0/24 with a gateway of 192.168.1.254
@ LAN 2, PFsense already knows how to get to 192.168.1.0/24 because of the locally connected interface, so no static route needed here
And you're done. -
Good question, I suppose if you remove the gateway from the "WAN" interface and just rename it… it would then become a LAN interface... sure.
I'm not seeing where I can remove the gateway on the WAN interface.
-
So the port forward doesn't really seem like the best way to go about it then.
Correct. Because you have to create a port forward for every different connection you want to make
I'm not really familiar with the site-site tunnel at all or IPSEC.
I would actually use OpenVPN here, the setup is easier, but either way, there's no reason to add the encryption overhead if it isn't necessary.
Seems like trying to change the WAN on the pfsense box to a LAN connection and attempt the following:
From a design perspective, this is your best option, yes. In an ideal world, you would configure a separate interface on DDWRT and create an isolated transit network, but that's another conversation and I'm not familiar with creating and assigning interfaces on DDWRT.
I'm not seeing where I can remove the gateway on the WAN interface.
In the "General Configuration" section, Change the IPv4 Configuration Type to "static" and then in the "Static IPv4 Configuration" section, leave the IPv4 Upstream gateway option as "None"
-
Thanks, reading over your post now. Someone posted up a picture of a hand written setup. I was looking over that when I refreshed and it was taken down.
-
Thanks, reading over your post now. Someone posted up a picture of a hand written setup. I was looking over that when I refreshed and it was taken down.
Yes, he had the right idea about creating a transit network on separate interfaces on both sides, but some of the networking was incorrect and it wouldn't have worked.
-
@ LAN 2, add an any/any rule to the 3rd NIC interface
For this, you are referring to the firewall rules, correct? If I named newly resigned WAN to NIC3 since that's what you were referring to adding and to lessen confusion between the LANS then would the rule be like this:
This rule is being created ON LAN2 (pfsense)
Interface: NIC3
TCP/IP: IPv4
Protocol: TCP
Source: any
Destination: any
Destination Port Range: Leave blank or? -
I've created the static route on LAN1.
Destination: 192.168.5.0
Subnet MASK: 255.255.255.0
Gateway: 192.168.1.254Then when I go to change the WAN to static and assign it an IP of 192.168.1.254, I get the following error in pfsense: This IPv4 address conflicts with a Static Route.
edit: by the way, thank you very much for your assistance. It is greatly appreciated. Same goes to everyone else.
-
I kind of figured you might run into that error. I'm thinking something like this might work:
http://i.imgur.com/95ouWv4.jpg
I accidentally deleted my post with it originally.
-
The local and remote site are backwards in respect to the hardware being used (pfsense vs ddwrt).
Are you suggesting to still hook the wireless bridge up on the remote site to a WAN port or to a LAN port?
-
I have not used DDWRT in a long time so I'm unfamiliar with the options. Do you have the ability to create an OPT type port on the DDWRT side?
-
I can unbridge one of the physical ports and I believe assign it a new IP.
Can I load pfsense on this DDWRT router? or is there a better option. It is a WRT320N (V1) router.
-
Yea that's probably what you're looking for.
I still think an IPSEC tunnel between the sites where the phase two protocol set to AH is your best option. That way you avoid the NAT issue and port forwarding issues.
I'm pretty sure DDWRT supports IPSEC but I couldn't tell you how to set it up. Optimally, it'd be best if you could put a pfSense device on the remote site. All you'd need is a spare PC with two NIC ports.
-
That'd be nice. I'm fresh out of NIC ports and extra PCs. I've utilized just about every piece of hardware I have owned over the last 15 years in the current networks on both sides.
I am unfamiliar with IPSEC. I want to run OpenVPN with a VPN service on the pfsense side. Can I do the IPSEC and have the OpenVPN/VPN service simultaneously?
-
I don't see why you couldn't. There's no overlapping ports.
You'd just have to make sure the pfSense box is strong enough to do the encryption/decryption for both IPSEC and OpenVPN at the same time. Though if you setup the IPSEC phase two entries with just AH, the CPU impact would be far less.
-
Seems like there should be a simpler solution than IPSEC for LAN communication. I'm just baffled, I figured connecting two subnets would have been much simpler.
-
It really is simple. You don't have to use IPSEC, I was only suggesting it because I thought that would be the easiest solution in this case.
In reality, the only difficult part of this is the DDWRT setup because I'm unfamiliar with it.
If you could figure out how to not use the WAN port (to avoid NAT) on the DDWRT side, you'd just set it up like how I have in the picture and then add the rules to deny/allow the types of traffic you want pass between the two networks.
Edit:
This might be even better: http://www.dd-wrt.com/phpBB2/viewtopic.php?t=78029
-
Thanks, I'm reading the link now.
The WAN port from the PFsense machine is connected to the LAN on the DDWRT. Just in case there was confusion. I do have a third router with DDRWT on it as well if that router could simply handle the routing between the two subnets, but it seems like that would be redundant.
-
I kind of figured you might run into that error. I'm thinking something like this might work:
http://i.imgur.com/95ouWv4.jpg
I accidentally deleted my post with it originally.
jamesonp, the design is sound, but the interfaces on the transit network have to be on the same network. i.e. the OPT interfaces would need to be 172.16.0.1/30 and 172.16.0.2/30 then adjust the static routes accordingly.
Then when I go to change the WAN to static and assign it an IP of 192.168.1.254, I get the following error in pfsense: This IPv4 address conflicts with a Static Route.
I'm guessing you forgot to remove your old route sending 192.168.1.0/24 to 192.168.1.1? That would be the reason for the conflict.
Also, after doing some more research, if you use my suggestion, the inter-connectivity between the sites would work, but we would still need to use policy routing to get LAN 2 to the internet. Which is fine, but it's an extra step. In your specific case, all we really needed to do from the beginning is disable outbound NAT (Firewall -> NAT -> Outbound -> check Disable Outbound NAT) which turns PFsense into a routing only firewall and put an any/any rule on the interface connected to the bridge.
Another option is to disable both NAT and the firewall which turns PFsense into a routing only platform:
System -> Advanced -> Firewall & NAT -> check Disable all packet filteringI apologize for missing these steps. This could've been a much shorter thread :)
-
Thanks, I'll give it a shot today. Possibly this morning if I can find time before work. I'll post back.
-
I tried the following as it seemed like the simplest test.
Another option is to disable both NAT and the firewall which turns PFsense into a routing only platform:
System -> Advanced -> Firewall & NAT -> check Disable all packet filteringI have now deleted an firewall rules that I've created and same went for routing tables on the pfSense box. When I did the above, I lost internet connectivity AND access to the router on the other side of the wireless bridge. I still have a route on the DDWRT router to the pfsense gateway and 192.168.5.0 subnet. What am I missing as that seems like it should do the trick. I still had local access to the 192.168.5.0 clients though, that was good.
-
You now need a default route @ LAN 2 pointing back to LAN 1 (192.168.1.1).
You also need to verify that the static route @ LAN 1 is pointed at the current IP configured @ LAN 2. (i.e. if the LAN 2 interface is 192.168.1.254, then the route should be destination 192.168.5.0/24 gateway 192.168.1.254)
-
Ok, so I did this:
System -> Advanced -> Firewall & NAT -> check Disable all packet filtering
WAN was set to 192.168.1.254
Default Gateway on wan is set to: 192.168.1.1Still nothing. I am on the 192.168.5.0 subnet doing all this and once I make those changes I can't access the internet. On LAN1 I have confirmed that I DO have the static route pointing at the WAN IP provided above.
Is there something that needs to be set on LAN2 pfsense box to tell the LAN on that box to communicate with the WAN since all NAT/Filtering is disabled?
-
Check your routing table, what is the gateway of the default route? Diagnostics -> Routes. Post a screen shot if you can.
-
I've been using .112 instead of .254 :) My static routes on LAN1 match this.
![route table.png](/public/imported_attachments/1/route table.png)
![route table.png_thumb](/public/imported_attachments/1/route table.png_thumb) -
Can you ping 192.168.1.1? Can you ping 8.8.8.8?
-
I doubt it, but I'll change the configuration back right now and let you know.
-
Ping on both was 100% failure.
-
Can you post a network map of how things are currently connected?
Also, what default gateway are your clients using on the PC's?
What happens when you ping 192.168.1.1 from PFsense itself?
-
What happens when you ping 192.168.1.1 from PFsense itself?
I'm checking now.
Please excuse my terrible drawing, but it is all there. Let me know if you need clarification.
-
Where can I input commands in pfSense?
Either from the shell via ssh or by going to Diagnostics -> Ping
-
Ping Results directly from pfSense. Looks good. That means something is still prohibiting a connection between the LAN and WAN on the pfsense box as I'm unable to ping from a client connected to the pfSense Subnet (192.168.5.0)
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=0 ttl=58 time=19.789 ms
64 bytes from 8.8.8.8: icmp_seq=1 ttl=58 time=19.992 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=58 time=18.517 ms–- 8.8.8.8 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 18.517/19.433/19.992/0.653 msPING 192.168.1.1 (192.168.1.1): 56 data bytes
64 bytes from 192.168.1.1: icmp_seq=0 ttl=64 time=1.897 ms
64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=3.034 ms
64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=2.398 ms--- 192.168.1.1 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 1.897/2.443/3.034/0.465 ms -
Run a tracert on your laptop to 8.8.8.8 and post the results.
-
Traceroute while I have internet access:
Traceroute has started…
traceroute to 8.8.8.8 (8.8.8.8), 64 hops max, 72 byte packets
1 192.168.5.1 (192.168.5.1) 2.032 ms 1.566 ms 3.057 ms
2 192.168.1.1 (192.168.1.1) 4.491 ms 2.755 ms 3.445 ms
3 * * *
4 WANISP.ptr.us.xo.net (WANISP) 13.697 ms 11.825 ms 15.092 ms
5 ge-0-0-2-spk-cr1.cet.com (198.202.27.65) 14.796 ms 12.241 ms 12.642 ms
6 te-1-1-sea-cr2.cet.com (198.202.26.2) 18.589 ms 20.696 ms 20.326 ms
7 six.sea01.google.com (206.81.80.17) 20.489 ms 20.139 ms 19.410 ms
8 108.170.245.113 (108.170.245.113) 20.336 ms 20.854 ms 20.953 ms
9 209.85.250.19 (209.85.250.19) 20.182 ms 20.219 ms 21.039 ms
10 google-public-dns-a.google.com (8.8.8.8) 19.162 ms 20.637 ms 20.258 msLet me run one when I turn NAT/Firewall off
traceroute to 8.8.8.8 (8.8.8.8), 64 hops max, 72 byte packets
1 192.168.5.1 (192.168.5.1) 1.249 ms 3.578 ms 1.490 ms
2 * * *
3 * * *
4 * * *
5 * * *
6 * * *
7 * * *
8 * * *
9 * * *
10 * * *
11 * * *
12 * * *
13 * * * -
So, that traceroute shows you hitting the LAN 2 internal interface, then routed to LAN 1 and out to the internet.
So, what do you mean when you say "traceroute while I have internet access"?
ahh… I think I know what you mean now, so once you turn off the firewall and NATing.... you can't get past the local adapter. Which tells me there's an internal routing issue.
-
Yes, that is correct. It hits the LAN interface on LAN2 and then dies on the spot.
-
Do I need a static route between the LAN interface and the WAN?
-
I can ping everything on LAN1 (192.168.1.0), however when I try to ping something on LAN2 I get something like this:
92 bytes from dd-wrt (192.168.1.1): Destination Host Unreachable
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 5400 4fc9 0 0000 3f 01 a41b 192.168.1.115 192.168.5.192 bytes from dd-wrt (192.168.1.1): Redirect Host(New addr: 192.168.1.254)
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 0054 f9f5 0 0000 3f 01 f9ee 192.168.1.115 192.168.5.1I'm not sure what the .254 address is.
-
I found where the .254 was coming from. DDWRT was holding on to an old route that I set up previously in this thread. A reboot and it let go. Makes most of what I've been doing irrelevant and must go back now. Waiting for DHCP on the other end so I can connect to a PC over VNC for testing. I have not idea why VNC works over this network, but I'm glad that it is working.
-
I have it working! Finally. I have a several routes/rules/NATs in place and not 100% certain which allowed it to work or is doing nothing and haven't had the time to strip each one away at a time to see. I have access from both directions and internet on both sides. One strange thing is that I can not ping devices on LAN2 from LAN1, but I can connect to their web interfaces etc… I must still be missing one small key here.
On the DDWRT Router (192.168.1.1) that is hooked to the modem, I have a static route setup. Operating mode is gateway
Lan Dest: 192.168.5.0
Subnet Mask: 255.255.255.0
Gateway: 192.168.1.112Then on pfSense (192.168.5.1)
WAN of pfsense is essentially hooked to the LAN of DDWRT
WANIP: 192.168.1.112
Gateway: 192.168.1.1
LANIP: 192.168.5.1Static Route WAN
Destination Network: 192.168.5.0
Gateway: 192.168.1.1Firewall/NAT are still on essentiallly. I pretty much bypassed them with the local network rules I have set.
NAT --- Outbound
Hybrid Outbound NAT
I added the following rule
Interface WAN
Source 192.168.1.0/24
Source Port *
Destination 192.168.5.0
Destination Port: *
Nat Address: WAN Address
Nat Port *
Static Port NOBasically on the firewall rules I added any/any to WAN and LAN.
I'm sure this looks a bit ridiculous to someone with much more networking knowledge and I bet there are some major flaws as well, but it works and it is still behind the head router's firewall.