Two subnets, one LAN (yet another one) [SOLVED]
-
Ok, the first question I've seen people ask when this is brought up is… why?
I have been granted 5 static IP addresses from my ISP, and one of them must reside on my firewall for it to route properly. Other machines in my network will have an IP in both my internal subnet and in my public subnet.
The configuration is a bit... odd. I'll focus on IPv4 at the moment to simplify.
Firewall config:
WAN: 192.168.1.11/24 (gateway 192.168.1.254) - note that this is not a public IP - the modem only communicates in the 192.168.1.0/24 subnet.
LAN:
1. 10.4.12.10/24 (internal network)
2. 107.x.y.89/29 (public IP for the firewall)HOST A:
LAN:
1. 10.4.12.19/24 (gateway 10.4.12.10)
2. 107.x.y.90/29
3. 107.x.y.91/29HOST B:
LAN: 10.4.12.36/24 (gateway 10.4.12.10)Now, recently, I've been noticing that HOST B can initiate contact with HOST A (EDIT)107.x.y.90(/EDIT), but then traffic becomes interrupted and times out… and then becomes available again. I see a bunch of entries in my firewall log that look like this:
Block Interface LAN FROM 10.4.12.36 PORT (something random) TO 107.x.y.90 PORT 443 PROTO (TCP:A, TCP:FA, TCP:PA, TCP:FPA)
My rules allow all traffic from the LAN interface to anywhere, and all traffic from 107.x.y.88/29 to anywhere.
I've read this, which makes sense to me in that HOST B initiates contact with HOST A via the firewall (B has no route to A, so it uses the default route), then B responds to A… but notices the address B is communicating from and notes that it has an address that can route directly, so the response apparently goes directly to HOST B... then HOST B sends further traffic to HOST A, but this time it's blocked by the firewall as it's getting a SYN-ACK without a SYN. This is all speculation until I actually get my hands dirty and capture some real packets, but it makes sense to me. A short-circuited route could wreak havoc with a stateful firewall.
The user who saw this behavior posted a solution he thought would work, but he was unable to test it: System: Advanced: Firewall and NAT, Static route filtering -> Bypass firewall rules for traffic on the same interface. Well… I've tried that and it doesn't work.... but perhaps that's because it doesn't consider my second subnet to be the same interface?
I'll admit, I'm new to pfsense... I saw this behavior on FreeBSD 10 (with pf), was utterly stumped by what I was seeing and by the appearance that my firewall was ignoring my rules to pass the packets, and decided to try pfsense (been meaning to migrate for a few years anyways, this just gave me an excuse). At least now I know that whatever's going on is not constrained to FreeBSD 10.
Any other ideas on how I can attempt to resolve this? I have a VLAN-capable managed switch, but I'm not sure it would really help in this case (and I'm utterly unfamiliar with VLANs at the moment). The end goal is to be able to reach my public IPs from my internal network (they are already reachable from the internet in this configuration) without having to buy a separate switch and 5 new ethernet cards to go with it. The firewall and HOST A are incapable of additional physical interfaces - they're both mini-ITX machines, HOST A has its one PCI-e port used, and the firewall is a 1U box without expansion slot capabilities. I suppose I could add a USB ethernet dongle, but that just sounds horrifying.
-
You are stumped by what you are seeing because, frankly, it is a nonsensical design.
Deleted - read wrong.
So convoluted it took me a bit.
WAN: 192.168.1.11/24 (gateway 192.168.1.254) - note that this is not a public IP - the modem only communicates in the 192.168.1.0/24 subnet.
Huh? So your ISP routes 107.x.y.90/29 to 192.168.1.11?
It looks like you want to create a VLAN, assign it to an interface, number it with the /29, and put your servers there.
-
Close, my ISP routes 107.x.y.88/29 to 192.168.1.11 (or, rather, their modem is configured to route that to my router/firewall). Not the way I'd design it, but it's what they offer.
So, sounds like I need to do some reading on how to configure VLANs on my switch, enable the appropriate features on all machines requiring multiple VLANs (or do I need VLAN support on all machines period, including phones, tablets, etc? Guess some reading will answer that), and do some reading on what options to tweak on pfsense to properly support VLANs. Sounds like I've got my weekend planned…
-
Jusdt plug the hosts into an untagged port on the VLAN they need to be on and they won't know the difference.
-
"will have an IP in both my internal subnet and in my public subnet."
Why would you do that???? And I doubt they route that network, if they did there would be a transit network… So that they did is hang that /29 off their network and let you use the IPs in while pointing to one of them as the gateway to get off the network.
So you put the public IPs you want to use on pfsense WAN, and then port forward the traffic you want to host on internal machines to the IP those services are hosted on.
But since you mention your isp device is using 192.168.1.0/24 sounds like your behind a NAT anyway.. Put that in bridge mode, contact your isp so that a device you plug into their device (pfsense in this case) can get/use a public IP...
What this has to do with vlans have no idea. Vlans would depend on how many internal network segments you want, and or if you don't have the physical hardware to put the networks on their own layer 2s.. IF you only have 1 switch an you want more than 1 network segment then yeah that sure points to use of vlans. If you only have 1 nic in pfsense for use on your network side and you want more than 1 network than yeah that points to vlan use as well.
But as Derelict mentions already your "client" don't have to understand what vlan they are on at all.
-
I was hoping to not have those public IPs behind a NAT at all, but as you say, it sounds like my ISP is not giving me a choice given the firewall is on an RFC1918 address. The less mangling of internal packets the better - I'm OK with blocking/rejecting (by default) on my public IPs, but for general usage I'd really rather not do any redirecting.
I did successfully implement a VLAN, putting the firewall and the one machine I currently use on a separate VLAN for the public subnet (both tagged, as both also have private subnets). It took me a while to work out all the firewall gotchas attached to it (outbound NAT had me scratching my head for a while), but I got it working. The good news is, the firewall is no longer blocking traffic from my private net to my public net, so mission accomplished.
Working with the switch was… well... not an easy task. It's an old Dell PowerConnect 6024, and I was able to gain access to it with the RS-232 port. I was eventually able to get the web configuration service up and running... only to find that it was only compatible with Internet Exploder 6 (chrome would not work, even with an agent masker). So... yeah... RS-232 (and ssh) all the way...
I'm also getting the impression that I need to start migrating the remainder of my clients onto a VLAN that isn't VLAN 1, and make the to-be-created VLAN untagged.
Also, on a somewhat related note, does pfsense support any form of dynamic VLAN (like GVRP or similar)? I didn't see any options to tick.
At any rate, I think I'll mark the topic as [SOLVED] (provided the forum supports modifying topic names - haven't tried yet).
-
Working with the switch was… well... not an easy task. It's an old Dell PowerConnect 6024, and I was able to gain access to it with the RS-232 port. I was eventually able to get the web configuration service up and running... only to find that it was only compatible with Internet Exploder 6 (chrome would not work, even with an agent masker). So... yeah... RS-232 (and ssh) all the way...
I know you might not ever see this message but I'll post anyway.
If the Powerconnect 6xxx series is anything like the 5xxx you need to update the firmware and then it will work with all modern browsers. I also read somewhere that you can click the compatibility mode in higher versions of Internet explorer to use the powerconnect web admin prior to updating the firmware.
