splitting a subnet, moving from LAN to WAN

sgw

I have a pfsense at a customer where we split a given /24 VLAN into two segments:

pfSense NICs

WAN : 10.96.20.10/25 (its upstream gateway is 10.96.20.1)
LAN : 10.96.20.129/25

The LAN interface is plugged into a separate hw switch, and 2 physical servers as well, on these servers I run QEMU-VMs with IPs in that upper /25

example

IP 10.96.20.180/25
default gateway $LAN_OF_pfsense = 10.96.20.129

Now we have to move the VMs away from these servers ... and keep their IPs if possible.

So we cloned a VM and moved it to the new vsphere-node, where the same VLAN is available.

If I give the VM an IP in "the range of the lower /25", I can reach it from "outside":

10.96.20.99/24, gw 10.96.20.1

But not if I give it 10.96.20.180/24, gw 10.96.20.1

An admin mentioned proxy-arp, is that an issue here? If I google that in terms of pfsense, I only find Virtual IPs of that type (checked that, none set here).

I know: it would be easier to modify the DNS-Records of (services in) the VMs ... point to IPs in the lower range, done. Trying to get that, their enterprise level IT isn't very fast ...

My goal: I need to maintain access to servers in that upper /25 while I have to move the cloned VMs out of that range. Is it clear what I try to describe here?

Other words:

the cloned/new VMs should run in the WAN-part of the /24
the physical servers in the LAN-/25 (for example .140) should be reachable even after that
sadly the physical servers and the pfSense will be history in a few weeks anyway. So it could be some kind of hack also for that short time

thanks for any help here

viragomann

@sgw said in splitting a subnet, moving from LAN to WAN:

If I give the VM an IP in "the range of the lower /25", I can reach it from "outside":

What do you consider as "outside" here?

10.96.20.99/24, gw 10.96.20.1

But not if I give it 10.96.20.180/24, gw 10.96.20.1

Why don't you configure the correct network mask?

sgw

@viragomann

See my wonderful diagram attached ;-)

As I understand it "vm_new" is placed in 10.96.20.0/24 ... there is no more /25 in the setup with the esxi.

"outside" is "my_laptop" up in the left: I have a site to site VPN into 10.96.20.0/24.

thanks for checking my thoughts

just to add: "vm_old" is turned OFF already

sgw

Just for fun I set 10.96.25.180/25 on vm_new: doesn't work.

I have (ip a in SLES):

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:50:56:ad:68:9e brd ff:ff:ff:ff:ff:ff
    altname enp11s0
    altname ens192
    inet 10.96.20.99/24 brd 10.96.20.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet 10.96.20.180/24 scope global secondary eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::250:56ff:fead:689e/64 scope link 
       valid_lft forever preferred_lft forever

.99 is reachable, .180 not

And I need .180 as it is the IP in DNS etc

I am right now doublechecking how they set up that VLAN in ESXi. Maybe some mismatch on that level.

viragomann

@sgw
And your laptop is connected to the VLAN123 and has an IP out of 10.96.20.0/24?

sgw

@viragomann said in splitting a subnet, moving from LAN to WAN:

@sgw
And your laptop is connected to the VLAN123 and has an IP out of 10.96.20.0/24?

No. My laptop is in my LAN behind my pfsense, and that runs a site-to-site-IPSEC-VPN to their main VPN gateway ... from there they route it to that VLAN.

Works for years.

What's new is that the VLAN is now bridged to VMs in their VMware cluster.

So I am not yet sure if they correctly route the whole /24 (this might be an explanation why I don't reach vm_new with .180, right?). I just asked my contact there to check that VLAN.

Setting 10.96.20.180/24 should be OK, you agree?

As long as I don't get their feedback I could play with other IPs to see if my theory holds water.

EDIT: .127 works, .137 not ...

viragomann

@sgw said in splitting a subnet, moving from LAN to WAN:

So I am not yet sure if they correctly route the whole /24 (this might be an explanation why I don't reach vm_new with .180, right?)

Yes, if they statically route the lower /25 to pfSense, that could be a reason. But why should they do that?

pfSense has just an IP in this subnet from their view and it would rather make sense to route the upper /25 to it, because this is behind of pfSense.

sgw

@viragomann said in splitting a subnet, moving from LAN to WAN:

Yes, if they statically route the lower /25 to pfSense, that could be a reason. But why should they do that?

pfSense has just an IP in this subnet from their view and it would rather make sense to route the upper /25 to it, because this is behind of pfSense.

They might have an error in setting up the VLAN in vSphere.

It is routed correctly to the network port where the pfSense is attached, otherwise all my stuff wouldn't have worked for years.

But it's new and untested in the VMware environment. And it might be wrong. At least I don't have another idea right now.

I am looking forward to their reply.

As far as I understand things, even when the LAN interface in pfSense is configured in the upper /25, it won't hijack traffic going to an IP in /24. pfSense itself has a route to that upper /25 then, but this ~~should~~ must not affect routing in the larger /24 VLAN "above" ...

I even tried that yesterday: triggered a reboot of pfsense and pinged vm_new .180 all the time ... just to see if somehow things change while the pfsense is offline. It would have surprised me, but anyway ... I can only search in my range for now.

Maybe I overlook something, but so far @viragomann hasn't spotted a real misunderstanding, right? thx ...

sgw

When I try to ping an IP that has not been used before and in the LAN-part (10.96.25.128/25) from "upstream"/"outside", it gets logged on the pfsense as blocked by the firewall (which is OK in terms of fw-rules).

So the pfsense seems to take over traffic pointed to these IPs.

Could I modify NAT-rules maybe? Is that related to Outbound NAT? Could I somehow exclude

I requested a DNS-change from their network admins. If the new vms are located in the lower /25 things should work and I can go on with my actual work ;-)

(but I'd be happy to learn about the reasons for this behavior anyway)

EDIT: DNS-change requested, so I don't have to actually do something on the pfsense right now. Still interested, though. thanks all.

viragomann

@sgw said in splitting a subnet, moving from LAN to WAN:

When I try to ping an IP that has not been used before and in the LAN-part (10.96.25.128/25) from "upstream"/"outside", it gets logged on the pfsense as blocked by the firewall (which is OK in terms of fw-rules).

So the pfsense seems to take over traffic pointed to these IPs.

Could I modify NAT-rules maybe? Is that related to Outbound NAT? Could I somehow exclude

It seems that the lower IP range ist routed to pfSense WAN address. If so, you can do nothing. You cannot use the same IP outside pfSense, because pfSense was not able to route the traffic to the VM, since the subnet is defined on the LAN.

You only option would be to use an IP of the upper /25 on the VM and forward it on pfSense. But note that doing this also requires an outbound NAT rule (masquerading) for the forwarded traffic.