Force traffic through a gateway with specific mac address

lzdfhuni

Hello,

My pfSense is version 2.5.0. I have a static wan address, the gateway of my ISP is eg. 1.2.3.4, but in the pfSense system log I see in a few minutes:

Mar 24 19:30:01 kernel arp: 1.2.3.4 moved from ca:e1:6f:db:1a:01 to 0d:6d:5b:16:2f:b1 on ue0
Mar 24 20:00:18 kernel arp: 1.2.3.4 moved from 0d:6d:5b:16:2f:b1 to ca:e1:6f:db:1a:01 on ue0
Mar 24 20:15:18 kernel arp: 1.2.3.4 moved from ca:e1:6f:db:1a:01 to 0d:6d:5b:16:2f:b1 on ue0

etc.

Even sometimes I see same data several times one after another:

Mar 24 21:00:21 kernel arp: 1.2.3.4 moved from ca:e1:6f:db:1a:01 to 0d:6d:5b:16:2f:b1 on ue0
Mar 24 21:15:19 kernel arp: 1.2.3.4 moved from ca:e1:6f:db:1a:01 to 0d:6d:5b:16:2f:b1 on ue0
.....
I assumed probably there are two ISP gateways with same IPs but different macs in the ISP network. They told me from ISP that they had only one gateway machine on the net but did not want to tell which mac is the correct one. However, may be I found the real mac address 0d:6d:5b:16:2f:b1 with arp ping command, for example:

arping -v -c 3 -i ue0 somethingdotcom output:

This box: Interface: ue0 IP: (my masked IP) MAC address: (my masked mac address)
ARPING 74.6.143.26
64 bytes from 0d:6d:5b:16:2f:b1 (some internet ip): index=0 time=565.651 usec
64 bytes from 0d:6d:5b:16:2f:b1 (some internet ip): index=1 time=764.632 usec
64 bytes from 0d:6d:5b:16:2f:b1 (some internet ip): index=2 time=848.233 usec

--- some internet IP statistics ---
3 packets transmitted, 3 packets received, 0% unanswered (0 extra)
rtt min/avg/max/std-dev = 0.566/0.726/0.848/0.119 ms

But maybe sometimes my traffic passes through the fake gateway, I am not sure in that.

My question is how can I block mac address ca:e1:6f:db:1a:01 of the fake gateway and is there a way to specify in pf settings only the gateway IP with mac address 0d:6d:5b:16:2f:b1?

Thank you so much
(sfmbe)

johnpoz

What I would assume is happening is the vip of their gateway, I would have to assume its some sort of HA setup is moving between boxes.

So I wouldn't suggest you block a specific mac..

But if you wanted to - the easy way to do it would be set a static arp for the one you want to use as your gateway IP.

lzdfhuni

This is a brilliant idea, thanks a lot!

I'll try later:

arp -S 1.2.3.4 0d:6d:5b:16:2f:b1

and inform you.

And I very much hope pfSense to add option to block mac address in the firewall web settings.

Finally, I'd want to share something very interesting. See the output of the arpinging the ISP's gateway IP:
arping -v -c 3 -i ue0 1.2.3.4 output:

This box: Interface: ue0 IP: (my IP) MAC address: (my mac)
ARPING 1.2.3.4
64 bytes from 0d:6d:5b:16:2f:b1 (1.2.3.4): index=0 time=1.265 msec
64 bytes from ca:e1:6f:db:1a:01 (1.2.3.4): index=1 time=2.539 msec
64 bytes from 0d:6d:5b:16:2f:b1 (1.2.3.4): index=2 time=446.850 usec
64 bytes from ca:e1:6f:db:1a:01 (1.2.3.4): index=3 time=728.453 usec
64 bytes from 0d:6d:5b:16:2f:b1 (1.2.3.4): index=4 time=1.224 msec
64 bytes from ca:e1:6f:db:1a:01 (1.2.3.4): index=5 time=1.274 msec

--- 1.2.3.4 statistics ---
3 packets transmitted, 6 packets received, 0% unanswered (3 extra)
rtt min/avg/max/std-dev = 0.447/1.246/2.539/0.656 ms

johnpoz

@lzdfhuni said in Force traffic trough a getaway with specific mac address:

0d

Maybe I need more coffee - but that is a multicast mac address..

A multicast mac is used normally in an active active HA setup.. So that both nodes (or all nodes if more than 2) of the cluster will see the traffic.

lzdfhuni

May be, because in this topic I changed the real mac addresses with fictive ones.

johnpoz

Dude you can't do that ;) if you want actual help..

Not sure what you think someone could do if they new the mac address of your Isp gateway?

There are some settings that would/could prevent a multicast mac from being allowed into the arp table, etc.

RFC 1812 I believe ;)

A router MUST not believe any ARP reply that claims that the Link
   Layer address of another host or router is a broadcast or multicast
   address.

Edit - the only time I could see fudging the mac address would be if it was the mac address of your wireless router. Since this mac could in theory be looked up in a wifi war driving db.. Like https://wigle.net/

If so then really all you need to do is fudge the last 3 numbers... Ie the device ID, the block ID or OUI the 1st 3 numbers could be left alone, this only identifies the vendor that made the device. Not the actual device.

Do you really care if someone knew that your isp gateway was an intel vs realtek nic for example ;)

The mac addresses could be help for example if they were off by 1 or only a couple and were flapping could point to say device with multiple interfaces in the same nic. That were used in a lagg or something, etc. That vip was flapping between.

All of that being said - if your having issues with a mac coming back for your gateway that is not working for you. Setting static arp on pfsense for the IP to the mac that works should prevent you from using the other mac - but depending on how your isp is actually setup for I have to assume HA device.. This could cause you pain down the road, etc.

lzdfhuni

Thank you.

So, the problem is solved and the kernel panic no longer appears! I installed Shellcmd (the Shellcmd utility is used to manage commands on system startup) in pfSense from System / Package Manager / Available Packages. Then I added In Services / Shellcmd / Command:

arp -S 1.2.3.4 0d:6d:5b:16:2f:b1

where 1.2.3.4 was the IP address of the ISP's gateway and 0d:6d:5b:16:2f:b1 - the mac address on the LAN-card of the ISP for static public addresses. At last, the ISP gave me the mac address explaining that 0d:6d:5b:16:2f:b1 was used for static addresses and ca:e1:6f:db:1a:01 was used for their DHCP clients, and both of the LAN cards were of the same IP. It did not turn out to be an IP Spoof as I was thinking at first. :)

PS. Please, correct "tHrough" in the title.

lzdfhuni

@lzdfhuni said in Force traffic trough a getaway with specific mac address:

PS. Please, correct "tHrough" in the title.

Also do it for "getaway'. Then I've been drunk apparently

lzdfhuni

@johnpoz said in Force traffic trough a getaway with specific mac address:

If so then really all you need to do is fudge the last 3 numbers... Ie the device ID, the block ID or OUI the 1st 3 numbers could be left alone, this only identifies the vendor that made the device. Not the actual device.

I am very pleased with this model USB-to-LAN. I have previously tried up to 9-10 USB2LAN adapters, pfSense (and probably FreeBSD) had no drivers for some or others had large load losses. But only this model surprisingly endured tests with high loads on the net without loss.