Pfsense with 2 modems as multiwan : cannot access webGUI of the secondary modem

trap16 · Nov 12, 2017, 7:03 PM

Hello,

I have actually only basic knowledge on routing and setting up network, I would have some hints on the following and some way to sort it out.

Actually my home network is under 10.x.x.x (LAN) on a pfsense router.
I own 2 router modems :
- a box for fiber connection, plugged on WAN1 its adress is 192.168.1.1 , the pfsense WAN1 interface is set as 192.168.1.100
- a modem for 4G connection, plugged on WAN2 its adress is 192.168.5.1 , the pfsense WAN2 interface is set as 192.168.5.100

I have a set my pfsense router as multiwan, I followed some tuttorial for multiwan and set a group gateway with tier1 for fiber, and tier2 for 4G, aim is having a failover on 4G (i have a limited traffic volume on it) and main gateway as fiber.
This seems to work.

But.

I noticed that i go on my fiber modem webpage without issue with http://192.168.1.1 or http://192.168.1.100
While it is impossible to get on my 4G modem webpage ( with http://192.168.5.1) when all my modems are up.

My understanding is when the fiber modem is up the traffic to 192.168.5.1 is routed to the fiber modem (which sound logical), while i want it to go on the 4G modem when i target 192.168.5.x network

Is there anyway to define a rule somewhere (firewall, route ?) to tell when i open a browser on http://192.168.5.1 I want to go on the 4G modem ?

Best regards
Olivier

jahonix · Nov 12, 2017, 11:33 PM

Create a firewall rule with destination 192.168.5.1 and use the 4G modem as gateway, maybe?
Rule has to be on Lan interface (or where-ever you're coming from) and order of rules should be verified.

trap16 · Nov 13, 2017, 3:25 AM

Thansk for your reply.

I added the rule in my firewall but it did not changed behaviour,

I found how to capture packet using diag and wireshark and found that it seems the packet are going on the correct gateway, but i do not see any reply , only tcp retransmission (I guess this is because i don't get a reply from the modem)

capture on my 4G wan + browser opened to 192.168.5.1 (to be clear the modem has static ip 192.168.5.1 -netgear LB1110 default ip- and my pfsense router interface has ip 192.168.5.100 on the gateway targetting that modem)

I guess this means the routing is correct but for some reason my 4G modem does not want to reply …

jahonix · Nov 14, 2017, 12:45 AM

Can you ping the modem's address?

trap16 · Nov 14, 2017, 4:57 AM

Hello,

I did a few more testing, which raised more questions … I added some screenshots to have a more friendly view of the problem.

To answer your question, I can't ping my 4G modem from the LAN but I can ping it from the pfsense box.

I also did some testing on my other modem (WAN1, the fiber modem-router) and found i can ping it from everywhere (LAN + pfsense box)
strangely I have a device plugged on the subnet of this modem (a tv decoder) that i can't ping from the LAN and can ping from the pfsense box.

See below :

From left to right windows, from :

pfsense
a LAN computer, running under ubuntu
another LAN computer running under windows

Test done : ping 192.168.5.1 (the 4G modem LB1110) = OK/KO/KO , ping 192.168.1.1 (the fiber modem-router) = OK/OK/OK , ping 192.168.1.10 (the TV receiver) = OK/KO/KO
Trying to access web GUI lead to same results as ping.

I would understand a 100% fail from my test on the LAN (or a 100% success) but not that one, does it means the difference can be explained by the modem settings (so no issue is coming from my pfsense settings) ?

Thanks a lot.

trap16 · Nov 14, 2017, 10:15 PM

Another test today , I tried to check why pfsense can ping the tv receiver and why a computer on the LAN cannot …

I used diagnostics on WAN1 interface to capture the packets for 192.168.1.10 and found that when i'm pinging from :

pfsense box (192.168.1.100) : the destination MAC is the MAC address of the tv receiver (the ping work in this case)
a computer on LAN (10.0.2.1) : the destination MAC is the MAC address of the fiber modem-router (and i get no reply)

trap16 · Nov 16, 2017, 7:06 AM

I sorted out half of the issue, it was due to bad settings.

I removed the gateway from the WAN1 (to fiber modem) and WAN2 (to 4G modem) interface
I added several LAN firewall rules to pass :
1) LAN net to WAN1 net with no gateway
2) LAN net to WAN2 net with no gateway
3) LAN net to any with my failover gateway group (192.168.1.1 tier 1 and 192.168.5.1 tier 2)

And now I can ping and access the tv receiver (192.168.1.10) from my LAN

So I can now ping correctly my fiber box modem, my tv receiver from my LAN, but the 4G modem is still failing (can access it from a shell on pfsense but not from my LAN)

riccio99 · Nov 22, 2017, 12:03 PM

hi, i had a similar problem with another firewall (Dlink).

i added a protocol binding rule:
Service: HTTP
Local Gateway: the wan of the 4g modem
Source Networks: Any
Destination network: ip adress of the wan port of the 4g modem.

I hope it help you

t__2 · Jan 15, 2018, 11:59 PM

Have almost the exact situation as this. Only difference my internal LAN is 192.168.1.1/24 and the address pfSense is usng for the Netgear LB1120 4G modem is 192.168.5.2 (gateway 192.168.5.1).
Failover works works great to the 4G modem if my primary FIOS gateway goes down for whatever reason.
I like the original poster cannot get to the 4G modem configuration web page from a computer on my LAN. I can ping the default web page address @ 192.168.5.1 and that works just fine. I have read all the posts I can find on this starting with the common one https://doc.pfsense.org/index.php/Accessing_modem_from_inside_firewall. That and others I found are either not exactly the same situation or are not specific enough instructions for a noob like me to translate the shorthand used into actual settings in pfSense. Using latest 2.4.2_1 version by the way.

Any help would be appreciated.

Derelict · Jan 16, 2018, 12:32 AM

Is the 4G modem webgui address the same address as the default gateway on that interface when it's up? Is it in the same subnet as the default gateway on that interface?

t__2 · Jan 16, 2018, 9:01 PM

Attached is what my Dashboard says . . .
Let me know if that is not the info you want.

t__2 · Jan 16, 2018, 9:24 PM

Yes the default 4G modem web page is at 192.168.5.1. Same as the Gateway.

Derelict · Jan 16, 2018, 11:49 PM

Hmm. Seems like that should work. Do you have good outbound NAT on the WAN_CELL interface? If connections to 192.168.5.1 are source-NAT to 192.168.5.2 and it isn't working the next thing to look at would probably be a packet capture on that interface filtered on 192.168.5.1 during a connection attempt.

t__2 · Jan 17, 2018, 4:49 PM

Sorry for my ignorance how do I tell if I "have good outbound NAT on the WAN_CELL interface"? I have not set up any special NAT, routing, or firewall rules to make this work yet. Mostly because I don't know what I am doing. The instructions I find searching Google are too generic like "create a new outbound NAT rule". Until I have done a few of them I will need detailed instructions on how to do things like that in the pfSense settings.

On a computer on the LAN when I put 192.168.5.2 in the browser URL field I get the pfSense box log in page. If I put 192.168.5.1 I eventually get "Problem loading page" and "The connection has timed out" warnings.

Derelict · Jan 17, 2018, 4:57 PM

Then just post a screenshot of your outbound NAT.

And having a LAN subnet of 10.0.0.0/8 is going to cause you nothing but grief. Why did you choose to do that?

t__2 · Jan 17, 2018, 5:29 PM

Here is my Firewall/NAT/Outbound page. (If that is what you were asking for.)

By the way I think you are getting postings from trap16 mixed up with mine. I don't have the 10.0.0.0/8 deal. Maybe I should have started a different thread but I have the same problem as trap16. Just a slightly different set up.

In my setup I think the 10.3.201.0/24 network rules in the Firewall were automatically set up when I tried to configure OpenVpn. I have deleted my attempts to set it up for now so if you think I should delete those rules let me know.

Derelict · Jan 17, 2018, 6:27 PM

Yeah, you're right. I was confused. Your NAT rules look fine.

So what you want to do is start a packet capture on WAN_CELL for all traffic for host 192.168.5.1 then try to connect to the modem then stop the capture.

It would probably be best if you attached the capture since MAC addresses might be important. Or at least set the detail to Full, View Capture again and post that.

t__2 · Jan 17, 2018, 7:21 PM

Is this the settings you want me to try for packet capture?

How long do I run the capture? Until the browser times out?

t__2 · Jan 17, 2018, 7:41 PM

Here is the capture with the settings I posted. Ran it until the browser timed out.

pfSense_capture.txt

Derelict · Jan 17, 2018, 7:51 PM

Your modem is not accepting the ARP Reply for whatever reason. You'll need to ask them. pfSense is doing everything it is supposed to there.

11:31:58.374001 0a:40:06:d2:5d:5d > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.5.2 tell 192.168.5.1, length 46
11:31:58.374014 00:08:a2:0b:8e:0e > 0a:40:06:d2:5d:5d, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Reply 192.168.5.2 is-at 00:08:a2:0b:8e:0e, length 28
11:32:14.614066 0a:40:06:d2:5d:5d > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.5.2 tell 192.168.5.1, length 46
11:32:14.614083 00:08:a2:0b:8e:0e > 0a:40:06:d2:5d:5d, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Reply 192.168.5.2 is-at 00:08:a2:0b:8e:0e, length 28

and on and on.