ISP Redirect to Payment Notification

Sensiva

Hello folks,
My service provider does this silly trick which is to notify me when a subscription payment is overdue. It redirects any website I visit to a page notifying about the payment and payment options..etc.
Once I pay (or clicking "I got it" link) , connectivity is back to normal.

After deploying pfSense, I wasn't able to get this notification, and once a bill is overdue, everything seems offline.
After searching around in this forum I found this solution.

https://forum.netgate.com/topic/89932/isp-uses-captive-portal/1

Which worked fine. Disabling DNSSEC and enabling Forwarding Mode in DNS Resolver made it possible to get this notification. However after upgrading to pfSense 2.4.4 I am not able to get this notification from my ISP. I double checked DNSSEC and forwarding mode options.

I tried doing a clean install of pfSense 2.4.4 , still the same problem persist.

The interesting part is installing pfSense 2.4.3 solves the problem. Which means that is likely something needs to be tweaked in 2.4.4 configurations.

Unfortunately I don't know where to start troubleshooting. Any directions or ideas are appreciated.

Thank you in advance

stephenw10

What happens if you have a client behind the firewall use the ISP supplied DNS servers directly?

Steve

Sensiva

@stephenw10 I will try that.
Please, note that the firewall itself is using the ISP automatically supplied DNS servers. So I am going to test if a client using ISP DNS servers directly in a manual interface configuration.

The problem should be reproduced in the next 48 hours. I will keep you posted.

Let me know if there is anything else to check.
Thanks

jclear

Wouldn't be simpler to set up an auto-pay of the ISP bill?

stephenw10

@jclear said in ISP Redirect to Payment Notification:

Wouldn't it be simpler to set up an auto-pay of the ISP bill?

Radical plan!

But they might also redirect you for other stuff and you want to see that. It's an ugly method of alerting you to things IMO but you work with what you got.

Steve

Sensiva

I received another notification. Here is what I tested.

Version 2.4.4 : network is offline, and the ISP redirect page is not accessible.
Version 2.4.4 with my computer DNS set to ISP supplied DNS, redirect isn't accessible
-Version 2.4.3: network is offline, redirect page is successfully accessible.
Version 2.4.3 with my computer DNS set to ISP supplied DNS, redirect page is successfully accessible.

Right now the the redirect will remain until I pay because it is 10 days over due, I will try not to pay for 8 hours so hopefully you guys see this update and let me know what else to do for further troubleshooting.

Hopefully we get all the required info to file an adequate bug report.

Thank you for your time

Gertjan

Hi,

Seems to me that your ISP is behaving like a captive portal when the cut you off.

Consider :
@sensiva said in ISP Redirect to Payment Notification:

Version 2.4.4 : network is offline, and the ISP redirect page is not accessible.
Version 2.4.4 with my computer DNS set to ISP supplied DNS, redirect isn't accessible

When you instruct your PC, or PfSense, to use the DNS supplied by your ISP, it should be accessible, even when they 'block" you.
This is how captive portals works and is a major condition.

A test : set the pfSense DNS to your ISP DNS and :
DNSLook ( Diagnostics => DNS Lookup) for example google.it
Did it resolve ?

Same test, even better :
Enter console mode - option 8.

dig @8.8.8.8 google.it

Change the "8.8.8.8" for the IPv4 of your ISP's DNS.
I should resolve, otherwise : you can't reach your ISP's DNS .... and that's no good - and you can't do nothing about that, the problem is upstream.

Also : 2.4.4 or 2.4.3 : nothing changed that is DNS related. If 2.4.3 works, then 2.4.4 should also work.
Except some special settings on your side that I can't imagine.

Btw : still, you somehow try to prove that not paying bills is never doing any good anywhere. That fact is already known. The solution also.

Derelict

It could be that they are being stupid and using DNS to return an RFC1918 address for any query.

pfSense's DNS rebinding protections prevent you from falling prey to that.

Hard to offer advice without knowing what, exactly, your browser was complaining about when you were shut off and trying to browse.

Far, far easier to just pay your bill on time.

stephenw10

They may cut off subscribers for reasons other than payment of course which would be useful to know about.

It's hard to see what could have changed between 2.4.3 and 2.4.4 that prevents this with the clients using the ISPs DNS servers directly.

Try to test what's actually happening. Try to resolve something. Try to traceroute to something.

And, yes, what actual error does the browser show when it fails to be redirected.

Steve

Sensiva

@Gertjan
I don't know if it is a captive portal or something else. In my first post I mentioned that I was able to be redirected to the ISP notification by using the configration illustrated in the link which was discussing an ISP using a captive portal.

When I mentioned DNS supplied by ISP I meant that I used DNS servers appearing in the PPPoE interface status in pfSense. Therefore in my test I just changed my computer network interface configuration to use ISP DNS IP instead of pfSense IP (DNS Resolver)

Btw my ADSL connection is a dynamic PPPoE made through a bridged ADSL router

@Gertjan & @Derelict
I am sorry, It seems that I didn't elaborate my point well enough. I understand that in order to go online again I must pay. That's totally out of question lol.
Some ISPs use this lousy mechanism to deliver various announcments, not only payment. Add for this online payment isn't possible when unable to get access to the redirected page.

Anyway my point is to find out why v2.4.3 is able to show these URL redirects/announcements while 2.4.4 can't do the same, given that both are configured exactly the same.

So I am deliberatily not paying to keep the issue active so we can go troubleshoot as much as I can (staying offline is an annoying thing, and using cell modems is painful with pfSense and costy too)
That's why I am mentioning I am not going to pay :D

Now the technical part:
v2.4.3 machine console:
- DNS resolves any domain correctly. However No ping replies
- Traceroute fails
- Gateway status is offline

Client behind v2.4.3 firewall:
- DNS resolves any domain correctly. However No ping replies (using cmd)
- Traceroute fails
- Opening any URL redirects to the ISP notification

v2.4.4 machine console:
- DNS resolves any domain correctly. However No ping replies
- Traceroute fails
- Gateway status is offline

Client behind v2.4.4 firewall:
- DNS resolves any domain correctly. However No ping replies (using cmd)
- Traceroute fails
- Opening any URL yields to error "Server timed out" or "Server took too long to respond"

Please, note:
- When changing DNS I flush DNS cache before doing the next test
- When checking redirected page in the browser, I use incognito mode and terminal browser lynx to avoid caching/cookies issues
- I am testing using pfSense 2.4.4 upgraded from 2.4.3 and a clean install of 2.4.4 which behaves the same way

I will not pay for the sake of doing further tests :D

Gertjan

@sensiva said in ISP Redirect to Payment Notification:

v2.4.4 machine console:

DNS resolves any domain correctly. However No ping replies

Traceroute fails => Normal ...

Gateway status is offline

Gateway down is your problem.
This is tested by sending a ping to an IP, somewhere up stream, that is considered your gateway - and could be any IP on the Internet.

When the ping doesn't work, your WAN will be considered down by pfSense.
Try changing the Gateway IP to an IP that is still reachable even when you didn't pay the bill, or disable Gateway Monitoring altogether.

Sensiva

@gertjan This gateway is automatically assigned by the PPPoE connection.

Note that although the gateway shows offline in v2.4.3 machine, the browser is redirected to the ISP notification page

johnpoz

@sensiva said in ISP Redirect to Payment Notification:

Note that although the gateway shows offline in v2.4.3 machine, the browser is redirected to the ISP notification page

Sorry not possible - unless in you 2.4.3 settings you have it set always consider gateway up on loss of monitor... How would it be possible for you to access anything if the gateway is DOWN...

Pfsense out of the box if can not ping its gateway, marks it down and won't send traffic out it. Unless you tell it to not count monitor as being down..

Also not possible for them to redirect if resolving correctly unless they using a transparent proxy on you.. A normal redirection would be done via dns and vis say resolving www.domain.tld to 1.2.3.4 they resolve it to 5.6.7.8 which they run and host up a default page that gives you some info.

If they are not intercepting your dns and changing it - then they are intercepting your traffic and redirecting.. Same sort of thing you can do with pfsense and "transparent" proxy.

What are your settings?
Gateway Monitoring
Disable Gateway Monitoring This will consider this gateway as always being up.
Gateway Action
Disable Gateway Monitoring Action No action will be taken on gateway events. The gateway is always considered up.

what about
State Killing on Gateway Failure
Flush all states when a gateway goes down The monitoring process will flush all states when a gateway goes down if this box is checked.

Sensiva

@johnpoz I understand that it seems illogical, but this is what is really happening. both v2.4.3 and v2.4.4 showing offline gateway, however v2.4.3 redirects to ISP and v2.4.4 does not redirect.
Both of them resolving domains correctly.
Regarding gateways settings, both are the same and default (not altered). Except that v2.4.3 showing an option to set the selected gateway as default, while this option is not available/listed in v2.4.4
I double checked and I am sure that both machines have "disable monitoring" and "disable monitoring action" unchecked.
I would like to remind you that there are three pfSense machines
1- v2.4.3
2- v2.4.4 which is a clone of machine 1 and upgraded
3- v2.4.4 which is a clean install
machines 2 & 3 are behaving the same way.

Also I guess that It doesn't have to be a v2.4.4 release bug, the ISP might be doing something wrong (I have seen horrible stuff they do)
Either way I guess pfSense should handle both cases because pfSense deals with whatever (no?)

regarding killing state settings , I can't find it in the web interface.

Thanks

Derelict

I suppose it could have something to do with the new default gateway scheme. What's that set to in System > Routing.

Sensiva

@derelict set to Automatic for both IPv4 and IPv6

Sensiva

@derelict said in ISP Redirect to Payment Notification:

I suppose it could have something to do with the new default gateway scheme. What's that set to in System > Routing.

I found it!
there is a second gateway which is the link between pfSense machine and the bridged ADSL router.
In v2.4.3 machine that gateway wasn't the default, but after upgrading to v2.4.4 default selection is automatically set.
After the PPPoE gateway (which is the logical default for automatic selection) gone offline, the new selected default is now the gateway to the bridged ADSL router which has no internet connectivity, hence blocking the URL redirect.

that's why when I checked what @derelict asked about, I found that the selected default gateway was the second gateway.
After setting the default gateway to the PPPoE connection instead of Automatic, I was able to see the redirected pages.

stephenw10

Ah, yup that will do it! There have been a number changes gone in for 2.4.5 to address that sort of incorrect gateway selection.

Usually if you only have one gateway is will continue to be used whether or not it's marked off-line. It's only if you have gateway groups or failover that it has any effect.

Steve