Adding a Subnet to an Interface
-
@johnpoz I have updated my rules and have most things working now, - thanks, - the only issue being httpd lookups. I am not sure which ports httpd uses for its lookups, so I am looking into that now.
-
@nogosubnet httpd lookups? You mean port 80? Not sure what you mean by httpd lookups?
edit: btw I still don't show being able to talk to the ns listed in that IP space. So if those are authoritative for whatever fqdn your trying to use - its not going to work.. Atleast not publicly
Is that what you mean by "lookups"
edit2: Your dns for that domain.uk - looks jacked.. Your glue for that domain is different and you have ns1 and ns2 with 2 different addresses each, etc. Also is very BAD idea to run all authoritative in the same IP block..
-
@johnpoz Not quite, - Apache (or httpd on RHEL) does not use port 80 or 443 for address lookups, - it uses a hidden process which makes it very difficult to debug when there is a problem.
...and I have no idea what is happening with the webserver, - everything reports that everything is good (apart from httpd), but nothing is ...and nothing is getting past pfSense regardless of rules (except SSh, - which is about the only thing that is, or has).
BIND, for example, reports no problems ...but it is evident (using zonemaster.net, for example) that the report is fake and that, no, BIND definitely is not working as it should.
I cannot understand this at all: even with the webserver firewall disabled and completely open pfSense rules, the only thing that is definitely working is sshd.
-
@nogosubnet said in Adding a Subnet to an Interface:
Apache (or httpd on RHEL) does not use port 80 or 443 for address lookups
Your confused - or using the wrong terms.. Because that makes ZERO sense.. No shit 80 and 443 is not used for "address" lookups.. address lookup would be dns, which is UDP 53, sometimes tcp 53..
As I have already told you - your dns is jacked, for the domain that is set for IPs in your /29 - not going to post it on public forum for sake of your privacy. But you posted the /29 space - so anyone could look up the domain if they wanted too..
Currently nothing is going to work publically dns wise for yourdomain your running on IPs in your /29.. Because they can not be talked to from the public internet.. you have .26-.29 being listed for ns1 and ns2.. Which in itself is borked.. But can not talk to them from the public internet - did you allow for udp and tcp 53? To those IPs..
-
@nogosubnet said in Adding a Subnet to an Interface:
webpage not being displayed (holding page only).
So you're connecting successfully to the web server and just not seeing the expected content? That's not a routing issue, that's a successful connection.
For example most web servers are configured to share an IP and thus only show the site if one connects with the correct hostname. If connecting with an IP or the wrong hostname then the default page is displayed.
If you're using a hostname, is it resolving to the expected IP?
-
@johnpoz Thanks, - I know that there are issues with the DNS and cannot fix them. - I have a solution that normally works (outside of pfSense), and which reports back as good and working with Zonemaster. DNSSEC is out of the question because I can no longer get those records working with that domain and have not been able to for many months; so after much fruitless research and study I just leave the matter alone and do not bother with it.
As for pfSense, the only issue I am facing now is that of configuring the DNS bypass (I have BIND custom nameservers on the webserver and pfSense will not allow queries using those nameservers unless I make changes to System > General and Services > DNS Resolver, both of which, so far, are proving futile; although I have, at least, got internet access for the Windows PC and working firewall rules, now).
-
@steveits valid points and yes, the hostname is fine; but my DNS setup on pfSense is not: I have managed to make changes to System > General sufficient enough to allow internet access for my Windows PC, but the webserver uses custom BIND nameservers and, as I said in my reply to @johnpoz, my attempts, so far, to make pfSense allow their use have proved to be completely futile.
Basically, the /29 subnet I am using for the webserver (and only the webserver) needs to be served by the webserver's BIND DNS resolver which, in turn, would route its lookups through the same Cloudflare resolver IPs that I have set in System > General on the pfSense side of things; but getting pfSense to allow this is proving to be absolutely unworkable.
-
@nogosubnet said in Adding a Subnet to an Interface:
but the webserver uses custom BIND nameservers
And which servers are those? You list name servers only 2 names on 4 different IPs .26-.29
They resolve www.yourdomain.uk just fine before.. But now they are not again. But where are you webservers pointing too?
So you have clearly opened up wan firewall rules to allow access before. But what pfsense uses for dns has zero to do with your issue.. Where is your webserver pointing to for dns - itself?
None of these issues have anything to do with pfsense, never has.. And pfsense dns has zero to do with resolution of a domain via authoritative name servers..
-
@nogosubnet are you looking for the Host Overrides section of the DNS Resolver page?
-
@johnpoz currently I have a Hybrid NAT rule in place to remove NATting between WAN and the /29 network.
Some lookups, eg: dnf update, work, but anything involving port 443 will not, regardless of how many completely open (unrestricted) DNS and HTTPS rules I have in place.
-
@nogosubnet said in Adding a Subnet to an Interface:
but anything involving port 443 will not
What port your webserver listens on has ZERO to do with a dns lookup...
Your authoritative name servers ns1 and ns2 for your domain are currently not reachable.. So it is not possible to look up anything in that domain.
There were previous, but now they are not. So you had the rules working at one time. You are confused if you think 443 has anything to do with name resolution.. Unless you are talking about running DoH.. Which you can not to the public internet for authoritative name servers.
-
@steveits I think it could be, yes, - thanks, - but, so far, adding the /29 (or any addresses from) has not worked.
-
@nogosubnet Again - it is not going to be possible for anyone on the internet to lookup any records in your domain.. No matter what host overrides or dns setup you have on pfsense. If they can not talk to your name servers.
Allow udp/tcp 53 to the IPs running your dns .26-29 is what is listed for you domain. Which again is borked seutp.. You have 2 nameservers and their IPs listed for glue, but before when you could talk to them.. ns1 and ns2 had 2 different IPs each..
Post up your wan rules!
-
@johnpoz whatever, - it is obvious that this is never going to work, and that a decent Draytek router offers a vastly superior solution; so that is what I am going to stick with.
I had really hoped that pfSense lived up to the hype, but it does not: yes, it is capable of "novelty" value, gimmick, fanboy status; but it is clearly not suitable for deployment as anything more serious and most people, myself included, do not have the time to play pedantic pissing contests with things that they are never going to be able to make any progress with, no matter what.
I might also add that, yes, I do, both personally and as a business, support projects like this when I find one that actually works; but this is clearly not one of them and is never likely to be.
Paid support is available, yes; but $399 for something that might only take 5 minutes for the likes of yourself? Forget it.
-
What your trying to do is 2 minutes of configuration - tops! I am sorry you are confused on how any of this works..
What are you wan rules?? That is 10 seconds for you to do - but you can't seem to be bothered.
-
@johnpoz yes, I realise that, and no need to apologise; but I simply cannot make sense of what is required: I read the articles, find something that should work, try it, try something else, something else, and nothing gives ...whilst the articles just suggest progressively more hellish configurations, none of which work, either.
Then, just to really take the piss, I find that rules that should allow completely open, unrestricted, access for various services don't ...so I cannot be sure that I have eliminated various potential solutions even when I think I have (and that holds true with all the interfaces, not just that that I am working with!).
I have also identified the problem that needs addressing (that of being able to resolve DNS queries and access to the webserver BIND DNS resolver, ie: the /29 BIND DNS resolver needs to be able to access the internet or, at the very least, pfSense's resolver) but I simply cannot find any way of resolving it. I did investigate Host and Domain options under the DNS Resolver configuration (as suggested by @SteveITS), but that did not fix the problem, either.
...so you can see why this, to me, appears to be an excercise in completely futility.
-
You need a firewall rule that allows udp/tcp port 53 on your wan to the /29 IPs address bind is running on..It is that simple. And the box your running bind on needs to have gateway set pointing back to pfsense opt1 address x.x.x.25?? In your /29
So again I ask - post up your wan rules.. There should be no port forwards.. Lets see your outbound nat rules..
What pfsense has set for dns, has nothing to do with bind talking to the internet, or the internet talking to bind...
It was working for a bit - since I was able to query your bind servers. Maybe you had a any any rule on your wan? You didn't put back that nonsense port forward of ports 1-65k to the opt address??
Let me guess - you have 1 box.. Your webserver.. You don't actually have another box on x.x.x.26 and .27 and .28 and .29 running bind do you. What is registered for your domain name servers is .28 and .29 -- but then when you ask your name servers for ns1 or ns2 it says that ns1 is .26 and .28, and ns2 is .27 and .29
You currently have it working - as I am able to query to the .28 address
I can query all 4 of the IPs .26-29 and get a response..
When I go to other domain.info your also setup for I can get to that site
But your primary domain.uk I just get the apache page. Both present a acme cert with san for both domains. So your problem currently is with your apache setup and host headers displaying the correct site..
I have blocked out your domain names, for your privacy - but all of this info is available from the IPs you posted via a bit of simple detective work.. So you might want to remove your IP range you posted. Or I can remove/edit it you wish? Or we can stop hiding your domain names and make it easier to discuss.
edit: also seems you have .30 running bind as well - since I can query that as well - and apache loads on that IP as well.
edit2: looks like glue is ok and ns are only returning the .28 and .29 addresses, but you have bind listening on all the ips .26-.30 as well as apache.
edit3: While your bind is not allowing for open resolving which is good.. It is answering a query for ANY.. This has been deprecated many years ago.. And get back all the records.. I would really suggest you disable answering a ANY query..
You can get info about not answering any with every record here
https://datatracker.ietf.org/doc/html/rfc8482Here is a good blog post about it
https://blog.cloudflare.com/rfc8482-saying-goodbye-to-any/ -
@johnpoz My apologies for the late reply.
I revert the pfSense arrangement when not working on it, hence some of the problems and inconsistencies you may have been seeing.
Either way, though, thanks for the additional information, - your persistance in this matter is appreciated and I will try again to get things working, starting with the rules and further information you have asked for.
You can mention the domain if you like (it is long overdue for some major revisions, and is not the main project on my server). The BIND ANY query needs to stay: as stated by many web admins, BIND queries for public domains will not work otherwise (otherwise I would definitely have altered it). There is what looks like a possible workaround (involving placing the ANY in the zone definitions with, - presumably, - NONE as the global action) listed here:
https://serverfault.com/questions/330385/bind-would-not-work-unless-allow-query-is-any
...and I will follow-up on that once I have everything else working (along with the further linked information you provided, - thanks).
To clarify the situation with the IP addresses, x.x.x.26 and x.x.x.28 belong to NS1, whilst x.x.x.27 and x.x.x.29 belong to NS2. I cannot have less than two addresses, and trying to advertise both addresses in the zone files causes all kinds of inconsistency errors which appears to be related to there being two IPv4 addresses, as the issue does not happen when there is both an IPv4 and an IPv6 address assigned as glue to a nameserver.
For the rules, firstly the NAT arrangement (if this is wrong I may as well not worry about the rules because nothing is going to work):
WAN Rules:
The box that I am running BIND on does indeed specify x.x.x.25/29 as the gateway, - it was configured during install to only use the IP Routed Subnet /29 range of IPv4 addresses, so there should be no issues there provided that those addresses (the x.x.x.25 one, at least) map to WAN on the pfSense side of things.
What is really annoying about this is that everything on the HTTP side of things will work on startup, but will fail as soon as I restart Apache ...and I know that the problem is related to the pfSense configuration because as soon as I revert the setup back to how it was (webserver directly connected to the router, with no pfSense) the problem immediately disappears).
-
@johnpoz I will be leaving the pfSense arrangement in place until Monday in order for you to be able to check things out properly; otherwise you will seeing the alternative, non-pfSense directly-connected arrangement (which does work and which will allow you to navigate to the website).
This is an example of the command-line output when querying the status of httpd (yes, the domain names have been edited in order to obfuscate them):
Sep 26 11:30:38 localhost.localdomain systemd[1]: Starting The Apache HTTP Server... Sep 26 11:30:38 localhost.localdomain httpd[1100]: [Sun Sep 26 11:30:38.644763 2021] [core:error] [pid 1100:tid 1100] (EAI 2)Name or service not known: AH00547: Could not resolve host name some.uk -- ignoring! Sep 26 11:30:38 localhost.localdomain httpd[1100]: [Sun Sep 26 11:30:38.646452 2021] [core:error] [pid 1100:tid 1100] (EAI 2)Name or service not known: AH00547: Could not resolve host name some.info -- ignoring! Sep 26 11:30:38 localhost.localdomain httpd[1100]: [Sun Sep 26 11:30:38.990771 2021] [core:error] [pid 1100:tid 1100] (EAI 2)Name or service not known: AH00547: Could not resolve host name some.uk -- ignoring! Sep 26 11:30:38 localhost.localdomain httpd[1100]: [Sun Sep 26 11:30:38.992785 2021] [core:error] [pid 1100:tid 1100] (EAI 2)Name or service not known: AH00547: Could not resolve host name some.info -- ignoring! Sep 26 11:30:39 localhost.localdomain httpd[1100]: Server configured, listening on: port 443, port 80 Sep 26 11:30:39 localhost.localdomain systemd[1]: Started The Apache HTTP Server....and this is the content of my resolv.conf file (127.0.0.53 is auto-configured and has not been added, or edited, by myself):
nameserver 127.0.0.53 options edns0 trust-ad -
@johnpoz Further to the last post: the above details (for resolv.conf and httpd) have been added purely to give you a clearer idea of what the problem is and to show that the problem is not simply a stupid manual edit that I have made to, in this case, resolv.conf.
127.0.0.53 in resolv.conf could explain what the problem is, because surely that will simply 'resolve' to the webserver localhost? In other words any DNS lookups would go nowhere and do nothing, ie: they would be nullified. What are your thoughts on this?
