Filtering incoming traffic based on IP address and URL

viragomann

@lifeboy said in Filtering incoming traffic based on IP address and URL:

For example:
URL:https://service.com/api/v2/payment
IP allowed to call this: 196.13.15.23
Allow this.

URL:https://service.com/api/v2/payment
IP not 196.13.15.23
Block this.

These are pretty the same URLs and source IPs. There is no possible rule, which can either allow or block it, just according your recent mood.

If you want to allow certain URLs though, you can do this with a reverse proxy. On pfSense you can install the HAproxy package for this aim.

Gertjan

@lifeboy said in Filtering incoming traffic based on IP address and URL:

some rule that checks the ip address and the URL

Ok, this might be surprise to you, but when a device using IP a.b.c.d connects to your public AP server, what will be known is the source IP, destination IP, source port and destination port, as these are part of the IP protocol.
There isn't any 'URL' in these packets. host names or URL are used by the sender, so it can, from the "https://service.com/api/v2/payment" take out the host name, == service.com, then resolve that, so it can do the final "https://a.b.c.d/api/v2/payment".
The file name path, /api/v2/payment will get transmitted in the https request.

Take note : On a higher level, because this is https, the URL gets send, for TLS needs but I'm not sure.
The server, your API server, will send back, among others, a certificate, and that certificate will contain "I am service.com" so the client knows he's connected to the right service - and here is TLS explained ^^

But I'm not sure why you want to include the URL ? Both the real client and fake client use exactly the same URL to reach your API host.
Only their IP differs.
Knowing that these known clients all have an upfront known IP, there is only one choice : You can make https://service.com/whatever available for everybody, but the URL https://service.com/api/v2/payment only by the list with known IPs.

Example : A Wordpress web site.
The main root .htaccess file contains :

# Block WordPress xmlrpc.php requests
<Files xmlrpc.php>
 Order deny,allow
 deny from all
 allow from 82.127.58.108
 allow from 2a01:dead:907:a6dc::/64
 allow from 2a01:daad:907:a6eb::/64
</Files>
......

so now I can only access the https://www.my-wordpress-site.tld/xmlrpc.php from the listed IPs (networks), and nobody else.

So, if you have a pfSense in front of your https://service.com/, then no, pfSense can't access/crack open the TLS payload to see if there is a URL.
If pfSense has a proxy, then, their might be a possibility .... but I've never set up a proxy.

lifeboy

@viragomann Note in the example that I want to allow traffic if the URL and ip match, but not allow if the URL matches but the ip not...

lifeboy

@Gertjan Yes, indeed, I realise that pfSense ito packets cannot know the URL, but I was hoping there's some filter / reverse proxy / something else that one could employ in this scenario or achieve the desired outcome.

In my example: https://service.com must be reachable by anybody. However, https://service.com/api/v2/payment must be accessible only for whitelisted addresses.

Since there are add-ons like Zenarmor that filter outgoing traffic, I was hoping someone knows of a tools that does this for incoming traffic...

We're using nginx for this service, so I think you're pointing me in the right direction. Let's see what options I have to whitelist and block there.

viragomann

@lifeboy said in Filtering incoming traffic based on IP address and URL:

Yes, indeed, I realise that pfSense ito packets cannot know the URL, but I was hoping there's some filter / reverse proxy / something else that one could employ in this scenario or achieve the desired outcome.

As mentioned, the HAproxy package is available for pfSense. Just install it and configure it accordingly to your needs.
There is nothing in pfSense out of the box, which can achieve this, however.

lifeboy

@viragomann said in Filtering incoming traffic based on IP address and URL:

As mentioned, the HAproxy package is available for pfSense. Just install it and configure it accordingly to your needs.
There is nothing in pfSense out of the box, which can achieve this, however.

Someone pointed me here and it seems HAProxy is indeed what I need!

viragomann

@lifeboy
I use it to achieve a similar URL restriction.

You can create a whitelist with allowed IPs just as an alias in pfSense.
Then in HAproxy create a "Source IP matches IP or Alias" ACL and assign the alias to it.
In a second ACL you state the URL to match.

And then add an action for what to do, block, pass, etc.

Jersyogg

To filter incoming traffic by IP and URL on pfSense, you can combine firewall rules for IP blocking with a proxy or web filtering tool (like Squid) for URL filtering. pfSense alone can block IPs easily, but URL filtering requires additional packages or external solutions.

pwood999

Could you simply put the API service on a different port to the public https, and then filter by source IP & destination port ?

Jersyogg

To filter incoming traffic by IP and URL on pfSense, use firewall rules for IP blocking and a proxy or web filter (like Squid) for URL control. pfSense handles IP filtering natively, but URL filtering requires extra tools. For consistent outbound IPs useful in more complex setups you can check here. LightningProxies offers IPv6 proxies with 2× /29 subnet pools, unlimited bandwidth and threads, HTTP/SOCKS5 support, sticky or rotating sessions, IP whitelisting, and global coverage.