Lan 1 to Lan 2 Connection Fail
-
A question for the group:
We have a pfsense machine with 3 interfaces, let’s call them wan, homenet, labnet. Snort is running on the wan interface and all unsolicited wan incoming traffic is denied, no rules. Both homenet and labnet have a single rule each to allow outgoing access to wan / anything for normal internet connectivity. Labnet has a sun sparc server running nfs, mysql vnc etc, which I would like to be able to access from a couple of housenet machines.
Started by writing a nat rule to do this, but after trying just about every combination possible from the setup screens, just can’t seem to make it work. However, repatching the housenet machine (DL140, Suse) directly to the server and I can telnet in.to the server with no problems. Also use a second pfsense box to access a webserver, wan > webnet and there the nat works perfectly and never even blinks. Both pfsense’s have an identical hardware config, using old sff desktop machines.
Wireshark is running on the housenet machine and tcpdump on the server. tcpdump is also running on the pfsense text console. If I try to telnet to the server, the packet can be seen on it’s way out on wireshark, as can the retries, but nothing comes back. At the server end, the packet sequence is as follows:
$ tcpdump -q host 172.16.100.205
tcpdump: verbose output suppressed,
listening on bge019:00:27.909689 IP 172.16.100.205.53942 > firelight.telnet: tcp 0
19:00:27.909804 IP firelight.telnet > 172.16.100.205.53942: tcp 0
19:00:30.914297 IP 172.16.100.205.53942 > firelight.telnet: tcp 0
19:00:30.914362 IP firelight.telnet > 172.16.100.205.53942: tcp 0
19:00:31.290075 IP firelight.telnet > 172.16.100.205.53942: tcp 0
19:00:36.922216 IP 172.16.100.205.53942 > firelight.telnet: tcp 0
19:00:36.922286 IP firelight.telnet > 172.16.100.205.53942: tcp 0
19:00:38.060093 IP firelight.telnet > 172.16.100.205.53942: tcp 0
19:00:42.930063 IP firelight.telnet > 172.16.100.205.57243: tcp 0This shows the telnet request and server response, together with several retries. Pfsense console tcpdump also shows the outgoing request to the server on the interface. However, it never sees the server response, even though it’s clearly on the wire on that interface. Looks like it’s being blocked on the way back in, but why ?.
I get the same response even deleting the nat rule altogether, which might (?) be due to the single outgoing “access anything” rule on housenet and labnet, but no amount of tweaking seems to get the response back to the requesting machine. I spent a nearly a whole day on this yesterday and i’m probably missing something obvious. Any help or suggestions would be appreciated to put out of misery :-)….
Regards,
Chris
-
"Started by writing a nat rule to do this,"
Why would you nat from lan to lan that are both rfc1918?
Lets just be clear here – what are you to lan networks? For example I have 3 lan segments
192.168.1.0/24 lan
192.168.2.0/24 wlan
192.168.3.0/24 dmzNow I allow lan to talk to ANYHING it wants this is simple, wlan can talk to my ntp server on my lan, and then anything else that is ! my lan net (192.168.1.0/24)
DMZ can talk to anything it wants that are not my locals - this is an alias that has lan, wlan and my openvpn network segments.
There should be really no reason to nat between local segments. There should be NO gateways on the interfaces on local segments.. The only thing that you need to setup is the correct filrewall rules on each segment to work.
So why don't you let us know what your lan network segments are - like I did above.. and post your rules like I did below.
If you do not see the response from the server your talking too, and you see the packets leave pfsense -- then tells me the server has firewall and is not answering.. Or maybe has wrong gatewaysetup /mask and thinks to talk to your first segment it needs to send traffic to some other IP(gateway).
-
Hi John,
Thanks for the reply. Like I said, there are 3 different subnets:
Wan: 10.x.x.x
Housent: 172.x.x.x
Labnet: 192.x.x.xSorry if I did’t make that clear.
It seems reasonable that we need a nat rule to talk from housenet to labnet. If not, perhaps you could tell me what I’m doing wrong ?.
The 192.x.x.x network is historical and dates back to sun 3 days, while the 172.x.x.x network is not assigned and not routable, fwir. Besides, it makes the overall system more secure by having separate numerical subnets for each. It wouldn’t be easy to change either of these and in any case, pfsense nat should be able to handle it. I have no trouble with nat between wan -> webnet on another pfsense box, so why between 2 x internal lan ports ?.
Oh yes, system reboot today and without the nat rule, telnet returns “no route to host”, rather than just timing out as it does with the nat rule included, so it is doing something. It’s just not returning packets from the target, as is shown by tcpdump. I wonder why ?…
Regards,
Chris
-
John,
Also, I did show the output from tcpdump at the server, firelight:
19:00:27.909689 IP 172.16.100.205.53942 > firelight.telnet: tcp 0
19:00:27.909804 IP firelight.telnet > 172.16.100.205.53942: tcp 0
19:00:30.914297 IP 172.16.100.205.53942 > firelight.telnet: tcp 0
19:00:30.914362 IP firelight.telnet > 172.16.100.205.53942: tcp 0
19:00:31.290075 IP firelight.telnet > 172.16.100.205.53942: tcp 0
19:00:36.922216 IP 172.16.100.205.53942 > firelight.telnet: tcp 0
19:00:36.922286 IP firelight.telnet > 172.16.100.205.53942: tcp 0
19:00:38.060093 IP firelight.telnet > 172.16.100.205.53942: tcp 0
19:00:42.930063 IP firelight.telnet > 172.16.100.205.57243: tcp 0Which shows an incoming packet from the 172 telnet request, line 1, the server response, line 2 then several similar retry attempts from the telnet request host.
The telnet request packet can also be seen running tcpdump on the pfsense hardware console,, but the reply not there at all, even though it’s definitely on the wire, so it’s 1) being ignored or 2) blocked at the input to the labnet pfsense interface.
If you have any idea of what the normal procedure is to nat across dissimilar local networks, would be interested to know…
Regards,
Chris
-
"It seems reasonable that we need a nat rule to talk from housenet to labnet. If not, perhaps you could tell me what I’m doing wrong ?."
Why do you think you would need to nat?? NO you do not need a Nat to talk local network to local network.. If you have created some your going to have ISSUES most likely.
Why are you hiding the lasts parts of your network - they are RFC1918, ie private - they are NOT routable on the public net.. Everyone has the same networks on their local networks. If you tell them - it tells me NOTHING about your location, not can I talk to your pfsense box using a rfc1918 address. 172.16-31.x.x, 10.x.x.x, 192.168.x.x
Once you give pfsense an IP address in that range it will know to use that interface to talk - route to that network.
So I suggest you put your nat back in Auto.. And give out some into so we can help you. When adding a network network, your labnetwork - all that you need to go is setup your firewall rules, since unlike the default lan pfsense will not create any firewall rules on opt interfaces. And there should be NO gateway on LAN interfaces – or pfsense will think it is a WAN interface..
Here is my routing table.. notice the 3 network segments and the route out the interfaces in those segments 192.168.x.253 are all my interfaces in my 3 lan segments. Those 10.x networks are my openvpn segments.
if you are trying to NAT there is no reason.. You just need to have simple firewall rules. And boxes in each segment need to alk the pfsense address in that network segment.
So for exmaple my pfsense IPs are
192.168.1.253 lan
192.168.2.253 wlan
192.168.3.253 dmzSo machines in wlan have 192.168.2.253 as their gateway off their network, and dmz devices point to 192.168.3.253 as their gateway.
You only need to NAT when you go from private address space rfc1918 to public addresses, see the 24.13.x.x address in my route table that is my internet connection. If your pfsense has a Wan address of 10.x.x.x then your behind a DOUBLE nat already -- you have another router in front of pfsense if your wan is 10.something.. There is normally NO reason to do this unless you can put your ISP device in bridge mode.
edit:
"If you have any idea of what the normal procedure is to nat across dissimilar local networks,"Again YOU DO NOT NEED to NAT between your own local networks!!! Put your nat in auto, and delete any rules you have might of put in there.. This is what your outbound nat rules should look like - see attached.
And there is not need for any port forwards between your networks.. I could not tell much from your tcpdump with that firelight.telnet vs the address of firelight..
If you show your box answering, but you don't see it on pfsense..
19:00:27.909689 IP 172.16.100.205.53942 > firelight.telnet: tcp 0
19:00:27.909804 IP firelight.telnet > 172.16.100.205.53942: tcp 0Then your box is setup wrong for its gateway or its network mask and thinks that IP address is local to its network.. Again its hard to tell where you talking with firelight vs an IP address.
-
NO you do not need a Nat to talk local network to local network
What?!?
If you have a client device behind the router it will send anything outside its subnet (assume 192.168.10.0/24) towards the router.
How will the router know what to do with it without NAT? (network address translation)
If your using autonat then the router is doing it for you automatically.
-
You clearly don't understand NAT do you..
How will it know what to do with traffic to say 192.168.10.42 for example – is it will look in its routing table.. See the one I attached! And since it has an interface directly attached to that 192.168.10.0/24 network -- its send the packets out that interface..
Why do you think it would need to NAT that in anyway?? Maybe you need to look up what NAT and or NAPT is and when its needed. You do not need to NAT anything if you can route between the networks.
The reason you NAT to the public internet, is the public internet has no clue how to route rfc1918 address space - since its PRIVATE and designed to be used in your location. Now if you want to have a device talk to say google which is on the public internet, and has a IP address that is routable on the internet - then you need to have an address google can talk to. So your router that has a IP in public space changes your private space to its public ONE so google can talk back to you.
It does that really with NAPT, so that more than one private address can share the 1 public IP you have. This does not have to be done when your router knows how to talk to both networks, since it has an interface in both networks.
With your wan address there on pfsense of 10.. Your doing NAPT twice when you talk to google.. Since google can not talk to a 10.x address
So you have this
pc 192.168.1.a --- 192.168.1.b pfsense 10.x.x.A -- 10.x.x.B router publicIP - internet -- google
So your Natting twice in your setup with that private address on pfsense WAN.. What you normally have is this
192.168.1.14 -- 192.168.1.1 pfsense publicIP - internet - google
but for pfsense to route traffic between 2 network it has interfaces in say
192.168.1.14 -- 192.168.1.1 pfsense 172.16.1.1 -- 172.16.1.3
there is NO reason to NAT or NAPT that.. And it will cause you all kinds of problems doing so that there is NO reason to do.
edit pfsense AUTO nat only nats to the WAN from its lan interfaces - so here is my NAT table.. You can see this by just changing to manual
Notice there are NAT entries for my local networks to my WAN address ONLY, you don't see any NAT entries between the local networks..
-
Obviously we had different' teachers…
from the "Firewall: Nat: Outbound:" page...
If automatic outbound NAT selected, a mapping is automatically created for each interface's subnet
-
Hi John,
This is in answer to your oringinel reply, 2 posts back
And there should be NO gateway on LAN interfaces – or pfsense will think it is a WAN interface..
No gateways are defined for either housenet or labnet, but one is defined for the wan 10.x.x.x network as you would expect.
_if you are trying to NAT there is no reason.. You just need to have simple firewall rules. And boxes in each segment need to alk the pfsense address in that network segment.
You only need to NAT when you go from private address space rfc1918 to public addresses_
Ok, but the housenet interface at 172.16.x.x is in private address space, while the server labnet is public, so to follow your argument, I do need nat to connect the two ?.
Also, what you really seem to be saying here is that the wan interface is a special case in nat terms, where I would have expected nat to be applicable to any interface. Is this in fact the case ?.
I will try a few simple rules to see if that can be made to work. I did a full restore from backup today to get the baseline back, but will try some firewall rules later to see if that can work.
There is normally NO reason to do this unless you can put your ISP device in bridge mode.
I have been using double layer hardware firewalls for years, with pfsense or other as the inner layer. I do have good tech reasons as well, but it’s probably not relevant to this discussion…
Regards,
Chris
-
"Whatever… Obviously we had different' teachers..."
Your TEACH told you that you need to NAT to talk from say 192.168.1.0/24 to 192.168.2.0/24 ??? No I don't think so -- if they did, they sure and the F should not be teaching anything to do with networking.
If automatic outbound NAT selected, a mapping is automatically created for each interface's subnet
Yeah I agree with you - from the local private lan segments to your WAN segment, be that wan public or private does not matter pfsense will auto nat from lan to wan.. But it DOES not auto nat from lan to lan segments.. Se my attached rules from my automatic NAT. All that nat rules nat from the local or openvpn segments to the WAN IPs.. there are no rules that nat between the local or openvpn segments because there is NO reason too, since pfsense has interfaces in all of those network and can talk to them.. While stuff on the WAN side only know how to talk to pfsense wan IP, not some rfc1918 space on one of its lan segments.
-
"while the server labnet is public"
Why would you be using PUBLIC address space on your local network, behind a NAT router??
"I have been using double layer hardware firewalls for years"
There is HUGE difference between firewalls between public network and having say another firewall between your DMZ and your local network.. But you sure and the hell do not need to double NAT to accomplish more than 1 firewall.. And that is behind the scope of this discussion I would agree yes. But NORMALLY there is NO point to double natting.. But that is outside the current issue - what IP space that is on your wan currently other than the fact that is 10, so private is besides the point.
I don't know why you would be running public space behind a NAT which you are if pfsense wan is 10.. I have to assume your just natting that outbound to pfsense 10 address - so what is the point of the public space?
That being said - you don't have to NAT it to talk to another segment directly connected to pfsense.
-
Hi John,
The 192.x.x.x network is historical, dating back to the earliest networked systems here in the lab. The server hosts file still has most of them listed as well, even if many of the machines were retired years ago. For me, ongoing context is important and that's the way it is, that's why :-).
Is there a solution using the network as it is, or do I have to dig out an old smc barricade mini ethernet router to port forward the subnets and get the job done while I try to work out what's wrong ?…
Regards,
Chris
-
What??
So your saying you have public address space behind nat 10.x – for what possible freaking use??
But again its besides the POINT.. you do NOT need to nat from 2 networks directly connected to pfsense..
Does not matter what I use for the network segments on pfsense they could be public, they could be private does not matter.. between lan I do NOT have to NAT.. I only need to nat to wan if devices connected to wan and beyond don't know how to route to the lan segments of pfsense.. Like the internet not knowing how to talk to 192.168.1.0/24
If the devices connected beyond pfsense wan will know how to talk to 192.16.1.0/24 then I don't have to nat at pfsense.. I can nat when my networks talk to some network, where they wont know how to talk to my private networks -- ie the internet.
Let say you have this - see attached.
Does not matter what the lan networks are, be it they 192.168.x.x, 172.16-31.x.x or 42.15.0.0/23
inetnum: 42.8.0.0 - 42.15.255.255
netname: SAMSUNGSDS-KR
descr: SamsungSDS Inc.Since I just pulled that network out of my A_S ;)
Since pfsense has a interface in that network -- lets say 42.15.0.1/23 and has an IP in say another lan segment 192.168.1.1/24 there is NO need to NAT between 192.168.x.0/24 and 42.15.0.0/23
because all the devices talking to pfsense as their gateway -- pfsense knows how to route to those 2 networks.
Now when either of those 2 networks need to go out pfsense wan (that is connected to the internet) then Yes pfsense would need to NAT that.. since IPs beyond pfsense wan IP don't know how to get to 192.168.x.0 network connected to pfsense.. And if you want to use public IP space -- you wouldn't need to nat it -- if that public network is ROUTED to pfsense WAN address.. But with your 10.x.x.x on your wan I find that HIGHLY Unlikely!
This is networking 101 - how is it your not understanding this?? When you state you always use double hardware firewall, ets. etc..
-
Hi John,
Looking at your network, the reason why you don't need nat may be that all the subnets are on the same private block and pfsense may recognise that internally. It's difficult to say if this is the case without wading through the sources, since there appears to be no docs that describe pfsense internals, or perhaps there are ?.
Anyway, all my subnets are not on the same private block and need to get this working. A nat rule partially works, but doesn't return packets, so it looks like the smc box bridging / port forwarding the subnets while I try to figure out why nat is broken may be the best short term solution…
Regards,
Chris
-
pfsense knows how to route to those 2 networks.
This is what Im (obviously incorrectly) heaping into the NAT arena…
Seems to me I had to build NAT rules (since I was using manual NAT at the time) when I added my second LAN subnet a year ago.
-
"Anyway, all my subnets are not on the same private block and need to get this working. "
Dude what part are you NOT understanding about 2 lan networks directly connect to pfsense NOT needing nat.
If these 2 networks are directly connect to pfsense - then NO NAT IS NEEDED and is only going to cause you problems - because now if your trying to nat between LAN networks you going to have to do that manual, and your also going to have to create port forwards for the traffic you want to create to IPs behind the NAT, etc..
Again - I am going to say this yet again.. THERE is NO NAT between LAN network segments directly connected to a pfsense..
Do I really need to change my DMZ segment to some other network to show you that?? Really??
-
ok LOOK – changed my DMZ network to 172.15.0.0/24 -- gave pfsense 172.15.0.1/24 address.. See attached. Did not change my dmz rules.. It can talk to anything it wants other than my local networks.. But my local networks can create connections to it..
That is a PUBLIC network space.. Not rfc1918.. But notice my pfsense has route to it.. And I can ping a host I brought up on 172.15.0.42/24 with gateway pointing to pfsense 172.15.0.1 address from my 192.168.1.0/24 network
That took me all of what 2 minutes to setup??
There is NO freaking NATS needed between 2 locally connected networks to pfsense.. I assure you there are NO nats between those networks!!
What traffic I allow between 192.168 lan and 172.15 dmz would be my firewall rules ONLY - there are NO port forwards required for these 2 local network to talk to each other - no matter what IP space I use on them.
-
Well- Thanks John! I learned something new today.
http://www.zytrax.com/tech/protocols/ip-classes.html#nat
A well written NAT system also acts as a 'poor mans' firewall since it has the additional advantage that Internal IP addresses are not visible from outside the organisation
Obviously this isn't something that LAN to LAN would want or need.
Also- verified here as well and turned off all NAT and was still able to move around throughout the various LANS here. Including one of the VPNs to my office network.
-
:-[
May i just one….... "what if ?"
What if of the "private" networks isn't using the pfSense as its GW ?
Let's say that the 172.15.0.42 host is using as GW an IP other than your pfSense 172.15.0.1.... It would you be able to ping it ?
:-[
-
"Let's say that the 172.15.0.42 host is using as GW an IP other than your pfSense 172.15.0.1."
When I have asked already multiple times in this thread..
Or maybe has wrong gatewaysetup /mask and thinks to talk to your first segment it needs to send traffic to some other IP(gateway).
If you show your box answering, but you don't see it on pfsense..
19:00:27.909689 IP 172.16.100.205.53942 > firelight.telnet: tcp 0
19:00:27.909804 IP firelight.telnet > 172.16.100.205.53942: tcp 0Then your box is setup wrong for its gateway or its network mask and thinks that IP address is local to its network..
But WHY would my 172.15.0.42 NOT use pfsense as its default gateway?? But yeah if that was the case, then sure you "could" nat so that my 172.5.0.42 saw the traffic as coming from its local network, and would not talk to a gateway to talk to it..
If that is the case why has the OP not stated this - I have brought up that scenario a couple times already.. Real early in the thread even when he stated pfsense was not seeing a response.
Its really simple when you need nat - you need nat when the dest would not know how to talk to the source IP, so you need to change the source IP to a IP that the dest can talk too.. Or you need to create routes so it does know..
if you had like what your saying.
192.168.1.0/24 pfsense 172.15.0.1 –- 172.15.0.42 -- 172.15.0.254 router otherIP -- other networks.
Where .254 was say the default gateway for .42 you have 2 options.. You could nat traffic coming from 192.168.1.0/24 so it LOOKs like its really from 172.15.0.1 -- so .42 thinks its just local. Not my first choice since be it you use NAT or NAPT you made it more complicated than simple route and firewall rules (if even firewall between and not just router)
OR!! Simple option –> You put a route on .42 that says hey when you want to talk 192.168.1.0/24 use 172.15.0.1 as your gateway to that network and don't send it out your "default" gateway..
This is like the pfsense routing table.. Since pfsense has routes to the networks involved, it doesn't send the traffic out its "default" gateway.. It sends the traffic out the interface connected to that network.
