Trying to connect from main network into subnet.
-
As far as accessing your WAN interface from the same segment, i.e., 192.168.1.0 - I suggest you open up your firewall rule just to test. See attached. I was not able to ping my WAN interface until I did that.
-
Well that was a huge help, thank you :)
I still don't want to leave it completely open (I know I'm still behind my main router's firewall, but this is to test how secure I can be while still having access myself.)
Opening it up worked, but then I altered it slightly, and then came up with a second rule to access the other subnet directly (10.0.0.0/24)
Rather than leaving the destination completely open (*), I set it to WAN Net, which still allows me to ping my 192.168.1.201 (pfSense's WAN address)
I also set up this little alias so my source is literally all 3 of my networks, so everything I have can connect through it (my 172.16.x.x needs to be reinstalled, but just knowing I can ping and access pfSense through web interface on 192.168.x.x was a major accomplishment for me, thank you.)
The other rule to 10.0.0.0/24 allows my iMac to ping and have web interface pfSense access on the 10.0.0.1 LAN interface as well.
The only thing I still can't seem to do is ping my Windows 10 VM at 10.0.0.3…
$ ping -c 4 10.0.0.3 PING 10.0.0.3 (10.0.0.3): 56 data bytes Request timeout for icmp_seq 0 92 bytes from netgear (192.168.1.1): Redirect Host(New addr: 192.168.1.201) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 3fcc 0 0000 3f 01 702d 192.168.1.5 10.0.0.3 Request timeout for icmp_seq 1 92 bytes from netgear (192.168.1.1): Redirect Host(New addr: 192.168.1.201) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 2e50 0 0000 3f 01 81a9 192.168.1.5 10.0.0.3 Request timeout for icmp_seq 2 92 bytes from netgear (192.168.1.1): Redirect Host(New addr: 192.168.1.201) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 3a70 0 0000 3f 01 7589 192.168.1.5 10.0.0.3 --- 10.0.0.3 ping statistics --- 4 packets transmitted, 0 packets received, 100.0% packet lossOnce I can do that, I think I'll have all the rules in place to be able to access ANY of my other machines on 10.0.0.0/24, and use these same rules
to set up my 172.16.1.0/24 network.Any ideas why the ping to Windows doesn't work?
Thanks again, you've been an awesome help, great birthday present for me today :)
-Bryan
 -
Quick update - Still not sure what's going on with the ping stuff, but I tried booting up my Linux server VM, and with the rules I put in place earlier, it looks like I DO have access into the 10.0.0.0 network from 192.168.1.0 – I was able to successfully ssh into my Linux box from the iMac, so I guess those may be the only firewall rules I need.
That ping is still bugging me, although I know it's obviously not the only method of checking connectivity...
-
First, you have an asymmetric routing problem.
The default gateway for the mac is the netgear so all traffic for 10.0.0.0/24 gets sent there first. That is why you are seeing the ICMP redirects. That is lousy design.
A better way would be to make another router interface on the netgear to connect only to the pfSense WAN interface on another subnet. See the attached image. It is not identical to your situation but it demonstrates the problem.
You should probably disable outbound NAT on pfSense too. Set advanced/manual mode there then disable all of the rules.
Is the windows firewall enabled on host 10.0.0.3?
-
I don't see a way to directly add another router interface, but these 2 things came to mind…
Attach #1: Maybe set each subnet as its own VLAN (based on the port it's plugged into on the Netgear)
or
Attach #2: Set up a static route, something like what I typed in here, so the Netgear can get directly to that subnet.
Would one of these fix the asymmetric problem?
And yeah, of course I forgot about the Windows firewall...turning that off allowed the ping requests through.
Would it honestly be worth leaving the local firewall on, since all my VMs are already behind pfSense, and then behind the Netgear router as well?
Thanks for the ideas so far,
-Bryan
 -
Sorry I don't know anything about that router. I'd be running pfSense there too.
You really need to be able to route like that second design in order for this stuff to work reliably and without ICMP redirects flying all over the place.
If there's a way to coerce that router to be two inside interfaces, then you would make a static route to the 10.0.0.0/24 to that. The destination gateway would be 172.16.212.26. in the diagram I posted.
-
It doesn't look like it's possible to add a second interface like that…I do have that Cisco router, but we're about to start learning how to program those, and I'd rather not lose internet access while trying to figure all that out :)
I tried adding a static route from Netgear to 10.0.0.0/24, but that didn't help, so just took it back out.
I had also disabled outbound NAT like you suggested, but that killed my internal Internet access pretty much immediately, so turned it back on.
For right now, it does do the basics of what I need it to do - be able to communicate between all my machines in some form or other (whether just ping, or ssh...I'll be setting up a tftp server on my Linux box to send my Cisco config files to once we start doing stuff like that in class.) Yes, it still has the redirects, but class starts in like 2 days, so I really wanted to at least get the communications up and running :)
Maybe once I know more about programming Cisco devices, I'll replace my Netgear with my 1841 Router plugged into the 48 port switch and do proper VLANs for each subnet, and turn the Netgear into just an access point (because we still need wifi access in the house.)
I do have some questions about funky stuff pfSense was doing with its DHCP, but I'll post that over on the proper forum.
Thanks,
-Bryan
-
Yeah you could put the 1841 between both subnets but then your lab router would be in production.
Eliminating outbound NAT required a good route from the outside router to the inside addresses.
You can use it like it is if you NAT everything on the pfSense WAN, including port forwards to the inside addresses. That means you would not be able to directly access 10.0.0.3 but you could port forward, say 192.168.1.201:8443 to 10.0.0.3:8443
You can use as many inside addresses as you like so you could carve out a /26 or something and 1:1 NAT for, say, 192.168.1.192/26 to 10.0.0.192/26 and use 10.0.0.192 through 10.0.0.254 on the VMs.
You could 1:1 192.168.1.192/26 to 10.0.0.0/26 but in order to hold on to your sanity you probably want to leave the last octet the same.
In either of those cases all communication with pfSense and anything inside pfSense would be from and to 192.168.1.201 (or 192.168.1.X) and would be same-subnet to everything on 192.168.1.0/24 so the routing table would not be used.
-
Ok, here's the last network I'm adding (it's basically just another server + pfSense and 2 additional VMs.) (Attach 1)
Nearly every machine is talking to every other one exactly as I had hoped, with one weird issue.
As tmack8080 advised, I came up with these 2 rules so that my iMac could talk with everyone in the 10.0.0.0/24 network: (Attach 2)
So, seeing as the new network was basically the same, I did the same rules, but the 2nd rule's destination would be the local subnet (172.16.1.0/24): (Attach 3)
To avoid having to go back to previous posts, Attach 4 is the alias I made that basically says that the source location is ANY of my networks.
So what's the problem? Specifically, I can't ping either the internal LAN of the new pfSense (172.16.1.1) or any computers inside (Windows: 172.16.1.3 – and yes, this time the Windows firewall is completely OFF.)
The first rule in my attachments allows me to access the 192.168.1.x address of either pfSense, so that one works great on both ends. The 2nd rule with the destination 10.0.0.0/24 network works great, both my iMac and my entire 172.16.1.0 network can access everything in the 10.0.0.0 network.
But for some reason, the 2nd rule in my 3rd attachment doesn't want to work the same way -- It won't let my iMac or my 10.0.0.0 network access anything in the 172.16.1.0 network. Any clues? The firewall rules are otherwise exactly the same...am I missing another setting somewhere?
Oh, and this is specifically what I'm seeing either on my iMac or my 10.0.0.0 attempts to ping:
$ ping -c 4 172.16.1.1 PING 172.16.1.1 (172.16.1.1): 56 data bytes Request timeout for icmp_seq 0 Request timeout for icmp_seq 1 Request timeout for icmp_seq 2 --- 172.16.1.1 ping statistics --- 4 packets transmitted, 0 packets received, 100.0% packet lossAny help on this last bit would be greatly appreciated :)
-Bryan
 -
Well, I got the last bit I wanted to work - I can now get into my 172.16.1.0/24 network :)
How? I found this blog post: https://networkguy.de/?p=409
I based a static route on my Netgear router (Attach 1) on his 2nd picture with the "route -p" command listed at the bottom of the
picture, mapping his numbers to approximately what I have in place on my network.Basically: I made a static route to the destination network (172.16.1.0/24), through the WAN IP of that pfSense router (192.168.1.101).
The asymmetric routing is still there, but only in specific connections:
The pfSense router (172.16.1.1) Ping Redirects the router and any computers in 10.0.0.0/24, but pings the entire 192.168.1.0/24 normally.
VMs behind that router ping everything normally, including the 10.0.0.0/24.My iMac (192.168.1.5) has a Redirect Host to both subnets (10.0.0.0/24 and 172.16.1.0/24)
My other pfSense router (10.0.0.1) Ping Redirects anything in 172.16.1.0/24 network.
It also Ping Redirects any computers in 192.168.1.0/24, BUT it pings the router (192.168.1.1) normally.
Any machines behind this router ping both of the other networks (172.16.1.0/24 and 192.168.1.0/24) normally.Again, my current router has no option for an additional interface (off the shelf model), but even with redirects, I managed to get everything to communicate,
so that's definitely something to be happy about - just in time for class to start tomorrow night as well, so I'll be able to do plenty of network testing.Any thoughts about the weird redirects couldn't hurt - how can your router/gateway ping redirect to an entire network (first example), but all the machines behind it can ping that same network normally?
Weird.
Anyway, hope this can help someone, and thanks to everyone who helped me along to finally getting my stuff working (if not 100% cleanly.)
-Bryan
