Can't get an IP from ISP behind switch

JKnott

@johnpoz said in Can't get an IP from ISP behind switch:

@JKnott I think he did that already when he says

"Pfsense box to the same switch port as the primary"

Must have missed that. I haven't had my morning beer yet.

johnpoz

@JKnott yes its quite possible that your isp might give you more than 1 IP, etc.

But I doubt he is trying to do it at the same time, because he clearly stated

"When I disable WAN on the primary box, and enable it on the secondary"

JKnott

@johnpoz said in Can't get an IP from ISP behind switch:

LAYER 8
GLOBAL MODERATOR
@JKnott
4 minutes ago

@JKnott yes its quite possible that your isp might give you more than 1 IP, etc.

But I doubt he is trying to do it at the same time, because he clearly stated

"When I disable WAN on the primary box, and enable it on the secondary"

As you mentioned, he should do some packet captures to see what's actually happening. It's a point I've often tried to make.

BTW, here's what my ISP has to say about connecting multiple devices on their 8 Gb fibre service. So, they don't just give out a single IPv4 address. They even toss in a switch to help you share the connection.

johnpoz

@JKnott they give you an 8 port 10ge switch for "free" with the service - that is pretty freaking nice of them for sure.

And they hand out multiple public public IPv4 - without added charges, if the rates are reasonable those seem like great bonus to go with them over some other isp that doesn't do that..

JKnott

@johnpoz

I believe it's $300 (Cdn) per month for 8 Gb. And no, there's no extra charge for the IPv4 addresses. Just plug in whatever you want and it works. I don't know if there's a maximum, for example if you had a 48 port switch. But I have had 2 IPv4 addresses for years.

sef1414

@johnpoz

Hmm, yeah. Sorry. I missed that in your first post, thought you were just meaning 2nd pfsense should have the same settings. I did not try setting it to a static IP on secondary, I had it set as DHCP, as the IP is dynamic. I will try doing that though.

sef1414

@JKnott

I did try some packet captures, though I'm a bit inexperienced at that level of detail. All I could see was IP 0.0.0.0 sending requests to a subnet mask like 255.255.255.x and not getting any response. I will re-create and actually copy that info.

JKnott

@sef1414

At the start of the DHCP sequence, before it has received any info, the client will broadcast the request and use 0.0.0.0 as the source address.

sef1414

@johnpoz

Alright, I gave it a go. Changed secondary WAN config from DHCP --> Static, and input the IP fetched from DHCP on the primary box. No dice. Can't get any connectivity.

It showed up on assignments page, but no connectivity and gateway status page showed 100% packet loss.

I'm really stumped here, only way I can get an active connection is by removing the cloned MAC spoof, and then power cycling the modem.

sef1414

@JKnott Yep, this is all I saw on the packet capture

johnpoz

@sef1414 so you sure your clone mac is the same? If you say it works when you use the native mac and just power cycle the modem it points to your clone mac not being correct..

In your packet capture - you see the correct "cloned" mac?

sef1414

@johnpoz Pretty sure. It was copied from the MAC address field of the interfaces status page on the primary box. Just verified that it is correct. Perhaps I'm missing something.

Guess I will need to do a more verbose packet capture?

All I saw was these lines on repeat:

13:21:56.318459 IP 0.0.0.0.68 > 255.255.255.255.67: UDP, length 300

johnpoz

@sef1414 or just download it and look in say wireshark

sef1414

@johnpoz

Alright so I was able to do some more testing, and it is pretty bizarre compared to what I believe should be expected behavior.

I've checked a countless times at this point to ensure the MAC from my primary box is set on the WAN interface of the secondary box. I would expect that my modem can not tell the difference between the two boxes, and both pfsense boxes should be able to grab the same IP from my modem if I disable the WAN interface on one of them, or even perhaps both simultaneously, though I would expect issues in that scenario.

Here are the steps I took with the results:

On pfsense primary, put down WAN interface via ifconfig in SSH session
Physically plug ethernet cable from pfsense secondary into trunk port 5 on switch
Pfsense secondary shows WAN connection as up, but doesn't receive an IP address
Unplug pfsense secondary from trunk port 5, and plug it into trunk port 1 (where pfsense primary was previously plugged in)
Pfsense secondary obtains an IP quickly (without modem reboot) and has connectivity - With a different IP address!
Unplug pfsense secondary from trunk port 1, replace pfsense primary into trunk port 1. - Pfsense primary grabs the previous IP.

I could see it possibly being a switch configuration issue, though I've been over that a dozen times as well. That would not explain pfsense secondary getting a different IP though. My best guess is MAC spoofing is not working somewhere between the interface and modem.