Added second WAN but no traffic
-
I have the main WAN with Spectrum. They provided me with a static IP, which is assigned to the interface WAN and a Gateway IP which is assigned to the pfSense gateway WANGW. No problems there.
I added a new interface called WAN2_5G, which is connected to my T-mobile box. They provided me a static IP and I added that to the interface. The box is setup as bridge, so no DHCP. They did not provide me a gateway IP.
When I connect my t-mobile box to a regular computer, I get the gateway IP, but if I add that to pfSense for a gateway called WAN2_5G_GW, there is no traffic.
If I add the same static IP to the interface and gateway in pfsense, it shows the gateway as online, but no traffic.
For troubleshooting, I connected the box to a computer again and did a trace route to 8.8.8.8. The first step shown is 192.168.1.1. To my understanding, the first step should be the gateway, so I am confused, but the internet works fine in the computer.
I cannot use 192.168.1.1 as the gateway IP in pfSense because it is a different subnet. And I don't see an option in my Inseego web interface to change the gateway IP.
Any idea on how to solve this? -
@kashs The fact that you see 192.168.1.1 as the first hop on your PC, seems to indicate that the T-mobile box is not connected in bridged mode? What does ipconfig show on that PC, does it show a 192.168.1.NNN address?
And then internet should be working since you are going through the T-mobile box...Connecting pfsense to that same Eth port on the TMO box will not work as long as you have a conflict with the LAN IP range on pfsense (192.168.1.0/24).
Check that bridge mode is actually working on the box, and/or that you are using the correct Eth port. Sometimes bridge mode only works on one of the connections.
And since you will want to access the UI on the TMO box, I'd suggest changing the IP on it to something different from 192.168.1.1Finally, even though you have a static IP, they may still use DHCP to hand it out. So you may want to set WAN2_5G to DHCP...
-
@Gblenn, thanks for the reply.
The bridge mode was active. I even tested leaving pfSense as DHCP, and the static IP was assigned to the interface. I think I was not able to get internet exactly because the modem was going to 192.168.1.1 as the gateway and pfSense didn't like that.
When I connected to the PC and did a ipconfig, I got the correct info. Static IP (72.xxx.xxx.35) for the computer and gateway was correct (72.xxx.xxx.36). But all my attempts to ping the gateway failed, even though I had internet.
My pfSense internal network is on 10.10.xxx.xxx.
My spectrum modem/router is set as bridge, but for some reason they require me to keep the wireless router active in order to have a static IP. The spectrum router is using 192.168.xxx.xxx.
I did confirm that I am using the correct ethernet port.
Yesterday I was able to change the DHCP on the TMO box to 172.16.xxx.xxx and set a static IP for my pfSense in there. Then I change my interface in pfSense to that static IP and changed the gateway to the 172.16.1.1 IP and everything worked. I was trying to avoid that because of the double NAT, but I will monitor to see if I encounter any problems.
What is interesting is that now when I do a trace route to google in pfSense, the static IP does not show up at all, but if I go to what's my IP, the static IP is shown correctly. Isn't that strange? -
@kashs Ok so it works (as in getting the correct public IP) if you connect pfsense as DHCP, but no internet? Is pfsense showing 192.168.1.1 (or 172.16.1.1 now) as gateway if you connect it that way, or no gateway at all?
I was testing a ZTE 5G router some months back and had issues similar to yours. I ended up connecting a dumb switch between pfsense and the router to resolve the issue, which I think was VLAN related??
I also seem to remember I had to create a static route (in your case 172.16.1.1 to go out your WAN2 gateway) in order to be able to access it's UI. -
@Gblenn
With DHCP in pfSense it did not work even though the IP shown in the interface was correct. The pfSense gateway set as dynamic, but no trafic.
Here is how it looks like:
The IP's for the interface and GW are set as static. And in the TMO Box I reserved the IP 172.16.1.2 to my pfSense mac address.
-
@kashs Hmmm, I'm confused...
With DHCP in pfSense it did not work even though the IP shown in the interface was correct. The pfSense gateway set as dynamic, but no trafic.
But this is not what you are showing in the pictures, which is not the correct IP (72.xxx.xxx.35) ?
So in this case you do not have the TMO box in bridge mode then?The IP's for the interface and GW are set as static. And in the TMO Box I reserved the IP 172.16.1.2 to my pfSense mac address.
And pfsense is in fact not using DHCP for WAN2_5G?
And I assume this setup is working, although double NAT of courseTraceroute should show your ISP gateway, not your static IP, in the list. The fact that you get the correct IP from accessing "whatismyip" is simply because it can only "see" your exit point, which is your TMO IP.
But what I was asking was what information does pfsense show when you have the TMO box actually set up in bridge mode?? Not like it is set up right now...
-
@Gblenn said in Added second WAN but no traffic:
But this is not what you are showing in the pictures, which is not the correct IP (72.xxx.xxx.35) ?
So in this case you do not have the TMO box in bridge mode then?Correct. I had to remove the bridge mode and set it to DHCP in the TOM box.
@Gblenn said in Added second WAN but no traffic:
And pfsense is in fact not using DHCP for WAN2_5G?
And I assume this setup is working, although double NAT of courseWAN2_5G is setup as static IP 172.16.1.2. No way to avoid double NAT but so far no issues.
Here is what the traceroute shows:
None of these are my static IP or the ISP Gateway IP.
@Gblenn said in Added second WAN but no traffic:
But what I was asking was what information does pfsense show when you have the TMO box actually set up in bridge mode?? Not like it is set up right now...
The static IP is correctly assigned to the WAN2_5G interface, but the WAN2_5G_GW does not get an IP. If I set it manually to the ISP GW IP, no traffice and Offline status. When I tried the static IP in the GW, it shows it as Online, but no traffic.
-
@kashs When you say “no traffic” do you mean you can’t ping out or it’s not being used? You need to create rules or load balancing to use the second WAN.
https://docs.netgate.com/pfsense/en/latest/multiwan/strategies.html -
@SteveITS
No traffic when I ping google.
I also tried:- Setting WAN2_5G_GW as the default gateway
- Creating gateway groups WAN TO WAN2 and WAN2 to WAN, set them in the advanced settings of firewall rules.
No luck on getting data going through.
I was not able to see anything in the logs that could point me to a solution. The only thing I saw was "sendto error: 64" and "sendto error: 65", but I believe these were because I was testing the static IP in both the interface and gateway, which lead to nowhere. -
@kashs said in Added second WAN but no traffic:
Correct. I had to remove the bridge mode and set it to DHCP in the TOM box.
Ok got it..
WAN2_5G is setup as static IP 172.16.1.2. No way to avoid double NAT but so far no issues.
For simplicity, and for further testing, I'd keep pfsense as DHCP. It really doesn't matter what IP it gets from the TMO box, and you have already created a static entry in the box based on pfsense MAC.
Here is what the traceroute shows:
None of these are my static IP or the ISP Gateway IP.
When you log into the TMO box, you should be able to see the settings there, for "internet". So you would see what IP and Gateway it has received from TMO. Also, entry no 5 seems to start with 72.xx which is the same as the static IP you have been given by TMO?
The static IP is correctly assigned to the WAN2_5G interface, but the WAN2_5G_GW does not get an IP. If I set it manually to the ISP GW IP, no traffice and Offline status. When I tried the static IP in the GW, it shows it as Online, but no traffic.
What Inseego router is it that you have? I did some googling and found someone having similar problems on an FX2000and all that was required would be the following.
- Unplug everything on LAN side of Inseego and reboot it
- Set pfsense WAN2_5G back to dhcp
- Connect to the Inseego
https://www.reddit.com/r/tmobileisp/comments/11x7mgy/how_fx2000_in_bridge_mode_with_5g_business/