Filtering incoming traffic based on IP address and URL

lifeboy

I'm looking for some direction on how to achieve the following:

We have an API endpoint service running on a public ip address. There is a public website running on this address as well. There is an API's however that should only be called by a limited number of users from fixed ip addresses. Of course, we can't block all ip's and just whitelist the allowed ones, since we don't know who accesses the public service. We do however know what the URL should look like when a legitimate source calls the API. So we'd like to construct some rule that checks the ip address and the URL that is being called and of both match our pattern, then the traffic must be allowed, otherwise we block (drop) the traffic.

For example:
URL:https://service.com/api/v2/payment
IP allowed to call this: 196.13.15.23
Allow this.

URL:https://service.com/api/v2/payment
IP not 196.13.15.23
Block this.

How can this be done?

viragomann

@lifeboy said in Filtering incoming traffic based on IP address and URL:

For example:
URL:https://service.com/api/v2/payment
IP allowed to call this: 196.13.15.23
Allow this.

URL:https://service.com/api/v2/payment
IP not 196.13.15.23
Block this.

These are pretty the same URLs and source IPs. There is no possible rule, which can either allow or block it, just according your recent mood.

If you want to allow certain URLs though, you can do this with a reverse proxy. On pfSense you can install the HAproxy package for this aim.

Gertjan

@lifeboy said in Filtering incoming traffic based on IP address and URL:

some rule that checks the ip address and the URL

Ok, this might be surprise to you, but when a device using IP a.b.c.d connects to your public AP server, what will be known is the source IP, destination IP, source port and destination port, as these are part of the IP protocol.
There isn't any 'URL' in these packets. host names or URL are used by the sender, so it can, from the "https://service.com/api/v2/payment" take out the host name, == service.com, then resolve that, so it can do the final "https://a.b.c.d/api/v2/payment".
The file name path, /api/v2/payment will get transmitted in the https request.

Take note : On a higher level, because this is https, the URL gets send, for TLS needs but I'm not sure.
The server, your API server, will send back, among others, a certificate, and that certificate will contain "I am service.com" so the client knows he's connected to the right service - and here is TLS explained ^^

But I'm not sure why you want to include the URL ? Both the real client and fake client use exactly the same URL to reach your API host.
Only their IP differs.
Knowing that these known clients all have an upfront known IP, there is only one choice : You can make https://service.com/whatever available for everybody, but the URL https://service.com/api/v2/payment only by the list with known IPs.

Example : A Wordpress web site.
The main root .htaccess file contains :

# Block WordPress xmlrpc.php requests
<Files xmlrpc.php>
 Order deny,allow
 deny from all
 allow from 82.127.58.108
 allow from 2a01:dead:907:a6dc::/64
 allow from 2a01:daad:907:a6eb::/64
</Files>
......

so now I can only access the https://www.my-wordpress-site.tld/xmlrpc.php from the listed IPs (networks), and nobody else.

So, if you have a pfSense in front of your https://service.com/, then no, pfSense can't access/crack open the TLS payload to see if there is a URL.
If pfSense has a proxy, then, their might be a possibility .... but I've never set up a proxy.

lifeboy

@viragomann Note in the example that I want to allow traffic if the URL and ip match, but not allow if the URL matches but the ip not...

lifeboy

@Gertjan Yes, indeed, I realise that pfSense ito packets cannot know the URL, but I was hoping there's some filter / reverse proxy / something else that one could employ in this scenario or achieve the desired outcome.

In my example: https://service.com must be reachable by anybody. However, https://service.com/api/v2/payment must be accessible only for whitelisted addresses.

Since there are add-ons like Zenarmor that filter outgoing traffic, I was hoping someone knows of a tools that does this for incoming traffic...

We're using nginx for this service, so I think you're pointing me in the right direction. Let's see what options I have to whitelist and block there.

viragomann

@lifeboy said in Filtering incoming traffic based on IP address and URL:

Yes, indeed, I realise that pfSense ito packets cannot know the URL, but I was hoping there's some filter / reverse proxy / something else that one could employ in this scenario or achieve the desired outcome.

As mentioned, the HAproxy package is available for pfSense. Just install it and configure it accordingly to your needs.
There is nothing in pfSense out of the box, which can achieve this, however.

lifeboy

@viragomann said in Filtering incoming traffic based on IP address and URL:

As mentioned, the HAproxy package is available for pfSense. Just install it and configure it accordingly to your needs.
There is nothing in pfSense out of the box, which can achieve this, however.

Someone pointed me here and it seems HAProxy is indeed what I need!

viragomann

@lifeboy
I use it to achieve a similar URL restriction.

You can create a whitelist with allowed IPs just as an alias in pfSense.
Then in HAproxy create a "Source IP matches IP or Alias" ACL and assign the alias to it.
In a second ACL you state the URL to match.

And then add an action for what to do, block, pass, etc.