Nice. Good result!

It's an honor to hear this from you, thanks ;-)

Yes, the internal switch config on Appliance was missing the configs for the additional WAN-interfaces.

The MAC-spoofing might not be needed at all, a test with a random laptop came online on that upstream fiber gateway without problems (with some random MAC).

Now with the correct VLANs etc on the internal switch the plugging works perfectly (we can plug between the 2 appliances and connectivity is up immediately).

thanks!

Maybe on appliance 2 the assigned order of the interfaces got mixed up?

I am not 100% sure if that is possible, the config.xml was restored so that should be identical.

What I think of: the local admin plugs the cable for WAN2 into ETH2 on App1 and things work (I just make up an example).

So we'd assume ETH2 on Appliance2 should also be assigned to WAN2 (and use its configured static IP etc). The admin just plugs the cable FROM App1:ETH2 TO App2:ETH2 ...

If that's not the case and for some reason WAN2 points to ETH3 on Appliance2 (because of whatever ... timings at bootup?) this would result in the logged behavior: ETH2 plugged in would show the spoofed MAC, but no configured IP.

The theory maybe lacks a bit .. but could that be the case?

You talk of the internal switch?
There is no switch between the 7100 and the fiber-appliance.

Thanks for clarifying. It was just a wild guess while I was listing the possible differences between plugging in the 2 7100-appliances.

The LAGG and VLAN shouldn't make any difference. None of that layer 2 stuff exists beyond the switch.

After that the displayed MAC was the spoofed one. Looked OK.

BUT the fiber-router (or whatever that is) didn't "see" the pfSense interface connected.
We had the admin of the provider on the phone, he ran diagnostics, we rebooted and replugged that router without getting it online.

We had electrical link, but he didn't see the MAC arp-wise (I assume).

Plugging back to the original appliance (without spoofing) worked immediately.

The provider-admin asks upstream .. he is only the tech for the reseller, or so.

I wonder: could the fact that it's a LAGG be a problem? I think of stuff like LACP, STP or whatever is different from a plain access interface.

I am just wondering what's the difference from the view of that fiber-box.

I'll keep you posted. Maybe I hear something tomorrow.

Hmm, so there is no LAGG interface on the standby device?

That one is offline right now. It's the same config as the online device ->

lagg0 is in the default config for the 7100. It can be removed to create other configs, multiple WAN MACs for example.
It is not assigned as an interface by default though and you need to do that to get the MAC spoof field.

But, yes, adding or removing that means re-configuring the switch and it's VERY easy to loose connectivity if that's how you are connected.

It is visible as "Available network port", but not yet assigned on the active device (in interfaces_assign.php).

I won't try that now, I could try it on the 2nd device in a maintenance window (plus the spoofing part).

I wouldn't remove the Interface LAGG0, I would only add an "Interface Assignment" to then be able to spoof the MAC there. Right?

I think I know my way, I have to make that appointment with the local admin guy first.

thanks

lagg0 is in the default config for the 7100. It can be removed to create other configs, multiple WAN MACs for example.
It is not assigned as an interface by default though and you need to do that to get the MAC spoof field.

But, yes, adding or removing that means re-configuring the switch and it's VERY easy to loose connectivity if that's how you are connected.

But there is no field for assigning a MAC in the GUI.

Recreating sounds dangerous: wouldn't I lose connectivity then? I very likely have to do that from remote, so I'd need a safe procedure.

Bonus question: the resulting config would also work unchanged on the appliance with "physical MAC = spoofed MAC" ?

thanks!

lagg0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
	description: OPT4
	options=4e138bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
	ether 00:11:22:33:44:55
	hwaddr 00:00:00:00:00:00
	inet6 fe80::208:a2ff:fe0e:a593%lagg0 prefixlen 64 scopeid 0x17
	laggproto loadbalance lagghash l2,l3,l4
	laggport: ix2 flags=4<ACTIVE>
	laggport: ix3 flags=4<ACTIVE>
	groups: lagg
	media: Ethernet autoselect
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lagg0.4091: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
	options=4600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6,MEXTPG>
	ether 00:11:22:33:44:55
	inet 192.168.127.1 netmask 0xffffff00 broadcast 192.168.127.255
	inet6 fe80::211:22ff:fe33:4455%lagg0.4091 prefixlen 64 scopeid 0x18
	groups: vlan
	vlan: 4091 vlanproto: 802.1q vlanpcp: 0 parent interface: lagg0
	media: Ethernet autoselect
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lagg0.4090: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
	options=4000000<MEXTPG>
	ether 00:11:22:33:44:55
	inet 172.21.16.219 netmask 0xffffff00 broadcast 172.21.16.255
	inet6 fe80::211:22ff:fe33:4455%lagg0.4090 prefixlen 64 scopeid 0x19
	groups: vlan
	vlan: 4090 vlanproto: 802.1q vlanpcp: 0 parent interface: lagg0
	media: Ethernet autoselect
	status: active
	nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>

You will need to reboot or recreate the LAGG to see that change.

So no need to do anything with the switch if that is sufficient.