Managing Network Block Lists

voxmagna1 · Sep 28, 2021, 12:16 PM

Hi, I'm pretty new with PfSense and I'm struggling in a few areas for my small home network Starting at the end of my struggle I decided to use the simple firewall alias block lists to block some of the most frequent IPs pinging my public WAN address.

I created a list manually using the existing EasyRuleBlockHostsWAN name then saved the list as a text file. I looked at the exported file and it seemed to have no carriage returns or delimiters after /32 range? The list worked and I decided to re-import the saved text file. In my understanding of logical operation an 'export' is usually matched to a corresponding 'import' of what was exported. This doesn't seem the case here using the same Alias name to modify the network list.

I could create a new Alias name and copy my network list from file into the white space and the bulk import block list was accepted but not as a file. I tried the same thing using the list with all entries separated by CR and got double CRs after the import. It seemed that PF insisted on CRs separating entries, but worked with no CR separators for my windows raw text file.

I don't understand why a network bulk file exported by PFS in a format it wants can't be re-imported as the same text file?

My biggest struggle has been trying to use PFS to block web URLs, although blocking IP addresses is a lot easier. That seems to me to be the most fundamental thing I should be able to do Most sites are now HTTPS and I failed to get Squid filtering to work and not block the HTTPS sites I wanted. It did work for HTP sites and i could get the Squid warnings when I tested blocked sites. I didn't want to configure each client browser to use the PFS proxy server. My pc clients are configured manually with fixed IP addresses. I couldn't get DNS blocking to work reliably on unbound either. I have PFS configured for OpenVPN to my VPN provider and I think my problem might be in the DNS resolution area?

Additional packages like Pfblocker gave me grief when they went off to update their block lists and some connections failed leaving PfSense hanging. I'm now cautious using the webgui because the interface is shut down during reboots and some firewall rule changes, so you see no progress or lack of progress. For now I'm using IP address blocking which works without additional packages.

Any help or pointers would be appreciated - Thanks

johnpoz · Sep 28, 2021, 12:50 PM

@voxmagna1 said in Managing Network Block Lists:

lists to block some of the most frequent IPs pinging my public WAN address.

Why? Unless you set wan to answer ping - they are dropped anyway. Blocking by IP that you notice in your log, that you don't want pfsense to answer because you opened up ping is going to be a never ending game of wack-a-mole. So either do not allow ping at all, or allow who can ping you via whitelist or by country based IP list.. But trying to maintain a list of ips you see in your logs is going to be a never ending exercise in futility..

My biggest struggle has been trying to use PFS to block web URLs, although blocking IP addresses is a lot easier. That seems to me to be the most fundamental thing

URL is proxy thing.. while you can block domains and specific fqdn via pfblocker (dns based).. Trying to block say www.google.com becomes difficult via IP, because anything hosted off CDN which is pretty much most of the internet these days the IPs change..

While by default pfsense does a dns lookup for whatever.something.tld that you put in a alias every 5 minutes. Does your client use pfsense for dns? Its possible with todays very low TTLs that since pfsense looked it up, and 4 minutes later for example it has to be queried again and this time gets back a different IP.. So how would pfsense block that if its alias say 1.2.3.4 for something.whatever.tld, but now the IP is 4.5.6.7 and that is where the client looking to go.

I'm now cautious using the webgui because the interface is shut down during reboots and some firewall rule changes

Have no idea what your saying here - why would you be rebooting for a rule change.. Just no idea what your trying to say.. The only time you would ever need to reboot pfsense, is an update to pfsense version.

voxmagna1 · Sep 28, 2021, 2:17 PM

Thanks. I agree that ping blocking ends up being fruitless. So many probes coming from redyber.net and being blocked along with zillions from Russia and China!

OK I've now discovered what I think could be a fundamental problem with my DNS resolver settings. In order for DNS blocking to work I assume the resolver has to be set correctly. I'm using a virtual interface for my OpenVPN routing and their private DNS IPs. Most PfSense standard configurations are a walk through without VPN and I'm not using DHCP. This makes it easier for me to block clients and allow certain clients IPs like Smart TV yo be in a DMZ subnet and Bypass the VPN, a solution I found really hard to find. For a small home network with kids on portable wifi devices the Pfsense controls their traffic and setting a fixed IP is no hardship on a few phones and pads.

When I go to Diagnostics/DNS Lookup I didn't get the URL resolved. After changing the DNS resolver Interfaces to my LAN AND the VPN interface, PfSense now returns a reply to DNS and ping queries. Would that be important to get Pfblocker to work?

I still haven't found any definitive help on Windows Network settings and what to do with browser network proxy settings when not using DHCP. Tests so far show my IP is routed via VPN, all the bad boys probing my public IP are being blocked and my DNS isn't leaking.

johnpoz · Sep 28, 2021, 2:31 PM

@voxmagna1 said in Managing Network Block Lists:

PfSense now returns a reply to DNS and ping queries. Would that be important to get Pfblocker to work?

It would be fundamental for anything that needs to lookup something on pfsense.. Be that an alias set with fqdn, or pfblocker trying to download a list from https://blocklist.domain.tld, etc.

Pfsense needs working dns if you want it to lookup stuff, same with any packages running on pfsense, or even for pfsense to check for updates or packages that are available.

When pfsense dns doesn't work - the gui can also be very sluggish.. Because its trying to check for if there is an update, etc. And that can hold up display of the system page, etc..

Functional dns is a really a requirement for anything.. Internet doesn't work with out it.

stephenw10 · Sep 28, 2021, 4:17 PM

For pfBlocker DNS-BL to work clients must firts be using pfSense as their DNS. That happens by default if you're using DHCP but not if you aren't.

Then because you're routing traffic via a VPN you need to make sure Unbound is using the VPN interface for outbound queries otherwise it will resolve using the WAN dircetly. That will be reported as a 'leak'. As an alternative you can set it to forwarding mode and then set only the VPN providers servers in Sys > General Setup.

Steve

voxmagna1 · Sep 28, 2021, 6:29 PM

@stephenw10 Thanks. If I had one simple question it would be how can I block websites including HTTPS easily using their URL and still have a good browsing experience accessing sites including banks. If I use any kind of proxy or filtering they keep throwing Captchas at me. I never got transparent proxy and SSL filtering to work without blocking some important secure sites I needed to access.

I'm still at the bottom of the learning curve. My routes work with the VPN tunnel and I can choose which clients bypass or block them and lots of WAN to real IP probing is blocked as it should. But achieving even simple filtering has so far evaded me and as said it's because my web browsers aren't configured through the Pfsense proxy? I thought DNS filtering would be easiest to start with, but I need to do more work and rethink if I can use DHCP but still do selective routing in and out of a DMZ subnet and bypass the VPN on the LAN. At the moment I reserve fixed IP blocks, allocate the client IP address to one within a block and use firewall aliases containing the IP block address range to take the route I want. It has another advantage that 'visitors' jumping on my wifi get no IP address from DHCP.

I've had the GUI appear to freeze a few times and it's what you said, I couldn't see background activity unless I put a monitor on the VGA port. The backup/restore via the GUI can be a life saver but a couple of times the restored XML file via PC hasn't worked as well as the native restore list from the box. Thanks for the replies - Onwards and upwards I hope.

stephenw10 · Sep 29, 2021, 12:02 AM

DNS based blocking is by far the easiest way and that then applies to all connections. So using pfBlocker with DNS-BL. Obviously using that you cannot filter by full URL only by FQDN.

The only way to get full URL filtering for browsing is to use Squid in full intercept mode. In transparent mode that too only sees the FQDN in the header.

You can certainly still use DHCP. Just use static lease mappings so clients always get the same IP address. Then your existing policy routing still applies.

You don't need to use dhcp to use dns based filtering though. You just have to be sure clients are using pfSense for DNS. So set them to use it directly and/or redirect DNS queries that might be trying to use some other DNS server:
https://docs.netgate.com/pfsense/en/latest/recipes/dns-redirect.html

Steve

voxmagna1 · Sep 29, 2021, 11:49 AM

Thanks Steve, I'll do some more tests. I understand now I don't need DHCP but using it ensures portable clients will get the PfSense DNS resolver and not their default network settings. When they get on another network they will get their IP address and DNS as normal. I think this has been my problem - Knowing when DNS lookup is coming from PFsense or from the VPN private DNS servers direct. My Windows setup is using my preset fixed IPs and specified DNS servers which are Google or the ISP. Big mistake! I hadn't thought about DHCP with static lease mappings but I'll research. I don't know if that will allocate the IP address I want unless PfSense can use client MAC addresses. When I do a DNS leak test, all my external IPs show as the VPN provider along with their DNS addresses looking similar to their IP address. I don't see Google!

Without using the wrong network speak I'll explain what I and many others may want to achieve:

Small home networks using standard ISP supplied routers can be compromised by the addon 'Smart' clients that people are now just plugging in without considering DMZs or sub nets. Some U.K TV streaming (BBC & Netflix) and bank sites look for the public IP address and geolocation to allow access to their services. VPN providers using shared and rotated IP addresses are often blocked. In a family home network the ability to block websites or non-approved connected clients is important.

My pfsense setup at the moment is:

Local Lan for PCs using fixed IP addresses. A small block of IP addresses is assigned to 2 firewall Aliases - 'Pass to VPN' or 'Bypass VPN' to WAN public IP.

A DMZ interface with an IP address range for Smart TV, Media box and internet connected hard disc recorder. The DMZ accesses my Public IP. Any packets to or from the LAN are blocked. I regard the DMZ as low security.

A wifi interface connected to the LAN on a fixed IP using VPN.

This works for routing, blocking and allowing traffic, but I can't achieve DNS filtering. If I use DHCP fixed static mapping wouldn't there be uncertainty that another client could get the wrong IP? I may be dumb but it seemed to me that unless Pfsense could get a client MAC address it can't reliably use the rules set for it? I think that's why I concluded each client would need a fixed IP address and the pfsense DNS server address. On a small home network that's easy to configure for each client unless a wired laptop on a fixed IP moves elsewhere and won't connect wired to a DHCP router. WiFi connections aren't a problem because each connection on Windows defaults to DHCP. I'd like to use DHCP on PfSense, but I can't yet see how I can achieve selective routing?