Hetzner Root Server > ESXi > PFSense > /29 Subnet
-
I have been battling with Hetzner now for a few months.. We got the dreaded 'Unallowed Mac address detected' email. I noticed my mistake and corrected it on the routing VM.
However, I want to use PFSense as a firewall from within ESXi as a VM. The problem we have is that everytime I have tried this I get the MAC Address email.
The current config is this
ESXi Host > Ubuntu Router VM assigned a public ip and /29 IP > /29 Subnet VM's with Ubuntu VM as gateway with ipv4 forwarding enabled.
What I want to achieve is ESXi > PFSense VM > /29 Subnet, last time I tried this I had a WAN vSwitch connected to the NIC configured in ESXi for PFSense, I then configured a new vSwitch in ESXi with no network interface attached called LAN, I then connected all the VM's on the /29 subnet to that internal vSwitch. Setup routing within PFSense and NAT to utilise the /29 IP's. However, Hetzner still emailed saying there was Unallowed MAC Address's
Does anyone have any experience as to how we can use PFSense and ESXi with an additional subnet without getting these MAC Address emails?
Thanks in advance
-
What actually doesn't work when you do that?
What MAC are they objecting to? The pfSense WAN MAC?
You can just spoof it yo be the same as the Ubuntu VM you were using.Steve
-
Thanks for the reply.
It's not the fact it doesnt work.. it does, however, Hetzner see the VM MAC Address as unallowed (they are ESXi created so random MAC's not assigned by their portal)
What I need to do is prevent the VM MAC been passed over the WAN Adapter into their network.. this is where the problem is, as soon as they see an unauth'd MAC on their network they send an email advising they will lock the server if not resolved.
The PFSense WAN MAC and ESXi are authorised MAC's, it's more the VM MAC's on the /29 subnet are getting passed through the WAN NIC.
Even if I create a LAN Network within PFSense with DHCP/Static on say a 10.0.0.0/24 address with PFSense as the gateway it seems to be still be passing the VM MAC Address over the to the WAN Adapter so Hetzner then see the unauth'd mac. Is there a way to spoof the MAC of all traffic out of PFSense WAN Adapter to a certain MAC?
Apologies if that seems jumbled, lack of sleep and covid giving me brain fog!
Thanks Steve
-
@ashton324 said in Hetzner Root Server > ESXi > PFSense > /29 Subnet:
it's more the VM MAC's on the /29 subnet are getting passed through the WAN NIC.
That doesn't happen in a routed network. Anything on the WAN side of pfSense would only see the WAN MAC address. The only way the LAN MACs would be passed is if it's bridged. So that would be either deliberately bridged in pfSense or somehow accidentally bridged in ESXi.
Do they give you the MAC they have seen?
Steve
-
So ive decided to start from scratch.. I most likely did something in the old config whilst trying to figure a way around it.
So the current Ubuntu routing VM has x2 NIC's one linked to the Hetzner Auth'd MAC with a public IP, it then has a 2nd NIC which has an IP of the /29 subnet, ipv4 forwarding is then enabled to pass traffic from the /29 to the WAN NIC. This works as expected, Hetzner require the VM's on the /29 to use the 2nd NIC of the routing VM as their gateway. For instance the routing VM has the following as an example
WAN NIC: 123.123.123.123 > ESXi vSwitch with a physical NIC attached
/29 NIC: 124.124.124.225 > ESXi vSwitch with no Physical NIC (doesnt touch Hetzners network)
The VM's then have their NIC connected to the same vSwitch for the subnet, use the 124.124.124.225 as their gw then use an ip from that /29 subnet as their IP.
Routing VM then passes traffic from /29 to WAN as expected.I need to achieve pretty much the same setup within PFSense, the only way I see it (correct me if im wrong) is to have a new GW and NIC defined in PFSense for the /29 subnet, use that as the GW on the VM's and pass the traffic over to the WAN Interface.
Would that be viable or is there a simpler way? The whole problem relates to Hetzner and their MAC Address limiting, if that didnt exist then it would be alot easier!
Thanks
-
Sorry forgot to mention, yes they do provide the MAC Address's they are seeing, all the MAC's relate to VM's on that /29 subnet.
-
@ashton324 said in Hetzner Root Server > ESXi > PFSense > /29 Subnet:
they do provide the MAC Address's they are seeing, all the MAC's relate to VM's on that /29 subnet.
Then it must be bridged somehow and not routed.
Doing this in pfSense should be very straight forward. The only thing you have to do is disable outbound NAT for the routed /29. But even if you didn't, and pfSense NAT'd everything to the WAN IP, the internal MACs would not be visible to the WAN gateway.
You should not have a LAN side gateway defined in pfSense. The internal NIC has an IP from the /29 and other VMs there use that as their gateway.
https://docs.netgate.com/pfsense/en/latest/recipes/route-public-ip-addresses.html
Steve
-
Thanks for the info.. the good news is that after following what you mentioned, the OPT1 INT (/29) is working as expected, passing traffic to the WAN with correct IP's and vice versa. Hetzner have also not reported any Unauth'd MAC's.
I have another issue now, I have some IP's that dont fall into that /29 range, these have assigned MAC's already from Hetzner. I just need to place them behind PFSense. So my plan was to use the LAN INT, assign a 10.0.0.0/24 IP to them machines and then use Virtual IP's to utilise the public IP's.
However, for some reason I can ping from the 10.0.0.0/24 to the PFSense gateway of 10.0.0.254, however, I cannot seem to get internet access. Pinging 8.8.8.8 from a VM drops all pings and tracert stops at the PFSense box. I disabled the firewall and still had the same problem, I have checked and have f/wall rules in for LAN to WAN and WAN to LAN (for testing).
Also, as outbound NAT is disabled, would my plan of using Virtual IP's work to utilise the public IP's?
Thanks
-
Ive realised the problem..
With NAT disabled ofcourse the traffic doesnt know where to go.. dumb moment.
Enabling NAT fixed the issue, If I switch to hybrid NAT and create manual rules the MAC of the /29 would not pass through the WAN gateway correct?
-
You can use hybrid outbound NAT mode and add a 'do not NAT' rule for the /29.
Or you can use manual outbound NAT mode and just remove all the rules except those for the LAN subnet.
Either will work. Using hybrid mode means that if you add another internal private subnet at any time it will automatically work still.
Steve
-
@ashton324 I have the same problem, I have no notification for unauthorized use of mac, only /29 can you help me how to introduce it to the pfsense side? I'm glad your issue was resolved.
-
Hey,
Firstly, are you using ESXi?
If you are then I can tell you how I got it to work, it's a bit of a pain but Hetzner are happy with the setup and it works fine for me.
Thanks
-
@ashton324 Yesss, Esxi 6.7 I installed :) I bought 1 additional IP, I wanted the /29 subnet to be forwarded to One additional IP address, apart from the main server IP address. I can give internet to virtual machines with one additional IP address. I don't have a MAC problem, but I couldn't introduce my /29 subnet to the pfSense side.
One additional IP -> pfSense
-
Ahh I did this on ESXi 7.0 but it should still work.
So just to get this straight you have the following setup?
Main IP > ESXi Host
Additonal IP > PFSense VMAnd then a /29 subnet that is routed through the additional IP on Hetzners side?
-
-
S stephenw10 referenced this topic on
