Really slow after restore
He everyone. This is my first post here in the PFsense forums. I'm still pretty new to PFSense. I have been using IPCop and smoothwall for years, as well as command line IPchains and IPtables. So I'm not new to concept of nat or firewall.
Recently, I installed PFSense on an old PC. This PC had a nehemiah chip and only 256 mb of ram. It was a temporary measure. Pfsense performed like a champ on that system for about 6 weeks.
I was able to open up a more powerful system that wasn't so old. So, I did a backup using the PFsense backup utility under Diagnostics > Backup & Restore. I installed PF sense on the new hardware, restored the settings from the backup, and the whole thing went pretty smoothly. I was very pleased.
But, since then something very strange has happened. I run my own email server with the Horde suite. Ever since I migrated to the new server, my horde performance has really gotten bad. Every click seems to take 30-60 seconds now for the page to load (emails, address book, whatever). The really strange thing is that this only happens on the LAN. Accessing Horde from the Internet (WAN interface), performs just as well as it always did. I am truly baffled.
My plan was to get familiar with HA proxy to route external requests to my horde box… but I don't understand it well at all yet (no time). So, I had just been using NAT to take care of https, http, pop3-s, imap-s, submission, smtp over ssl, etc.
I have nat reflection turned off. I'm using split DNS so that the inside systems can access email via hostname. I have pfsense configured for dhcp, setting itself as the primary dns server, with my ISP's and secondary and tertiary. I have setup the system under DNS Resolvers > General settings > Host overrides. I even created a host entry on the email server itself (which also hosts horde & imap services) to rule out any DNS issues. Nothing helps. I cannot see any errors in any log files on the mail server or on the pfsense box. I have pinged everything from the email server, and it recognizes itself appropriately... and resolves instantly. So I don't think this is a DNS issue.
Since horde (apache) lives on the server as sendmail, courier etc., and there appear to be absolutely no issues with name resolution (IP address is obtained dynamically from pfsense), I don't have clue what could be happening here.
The only thing that changed on the network was the migration from the old hardware to the new for the pfsense box.
Any ideas? I appreciate any help here.
Just a guess, but any chance your WAN IP address changed with the new pfSense box and something in your Horde setup is defaulting to the old one?
If you know the MAC address of the old WAN NIC you could try and spoof it in the new one to try and get your old WAN IP back - perhaps eliminate a variable in the problem….
No IP address change. It is a static IP for the WAN (and for the LAN, for that matter).
Yes, I could try to spoof the MAC.. but I don't see why that would change anything… The slowness is only when I'm on the LAN. It works at full speed when hitting it from the WAN... If things are working properly, the traffic should never even hit WAN interface when connecting to the email server from the LAN, right?
I cannot see anything weird in my configuration of PFSense... I think there is something strange going on under the hood here... I don't see anything in any logs or when I look at a tcpdump...
So far I agree with your analysis, the WAN address was just a WAG given so little to go on.
I wouldn't expect WAN to be directly involved with a local LAN request for mail either, but Horde might be doing some checks??
Without any log info on pfSense (or Horde? can you bump up the verbosity on its logs?) you're back to troubleshooting 101.
My first step would be to try and reinstall the previous pfSense box temporarily to see if the problem actually goes away or if you're fighting something else.
It would be helpful to "prove" that the pfSense change is the actual cause of this issue.
It may also be worthwhile to verify it's only Horde and not something to do with Apache, any other Web services on that box?
Honestly my first instinct is this isn't a pfSense problem, but without more info (for or against) it's hard to be definitive.
Hello, and thank you for your replies! Sorry it took a while to write back.. I have a hard time coming up with time to deal with this during the week.
Yes, I did swap-in the old firewall. The problem definitely abates when I use the temporary firewall. I swap the current firewall back in, and the problem persists. And yes, I rebooted everything… right down tot he network switch.
I think that this had something to do with the restore... maybe it took some setting or debris from the old system.... I don't know... a mac address or something that doesn't match the current network cards... I cannot really say... I don't know about anything that happens under the hood on pfsense. But my settings all look good, and the log files tell me nothing.... no errors anywhere...
Of course, I agree with you... instinct says that it can't be pfsense... butI have looked my horde box up and down... my apache configs... my resolv.conf, my hosts file, all the horde configs... everything looks normal. And with switching out the two pfsense boxes, I'd say it is pretty conclusive at this point: it has to be pfsense related.
Anyone know any tricks I can do... some kind of litmus test, or config files I can examine somewhere? Something that might not have translated properly from the restore?
Hello. I have this one figured out now.
It was definitely the pfsense box… or rather my misconfiguration of it. I went off a different reference sheet from my ISP. They had the secondary dns server written down wrong, though they look very similar. The temporary pf sense box had the correct secondary server. The new pfsense box had the wrong secondary server. So in my DNS Server settings, I was passing out three DNS servers: the pfsense box, the first from my isp, and the second from my isp (with typo).
I had long though that nameservers listed in resolv.conf were parsed in sequence; meaning that the second was only used if the first timed-out, and the third was only used when the first and second timed-out and so on. Clearly, that is not the case (at least, not where horde is concerned). And, it is likewise very interesting that the Horde server was still using DNS even though I had the same host entries in my /etc/hosts file... It is probably hard-coded in the php... probably using that net_DNS module in pear... either way, the code in horde seems to prefer DNS to /etc/hosts. Just weird that nothing else on the network (not even other LAMP applications) were affected by this... just Horde.
Anyway, that mystery is solved! My email is zinging along nicely now.
Thank you both for your input!
Don't you just love those "interesting" coding issues that collaborate to muck upo your network at will…..
Glad you got it sorted out.