[Solved] pfsense is not making sense
-
I have the weirdest issue which is driving me crazy. I have been working on this for days trying to isolate what the exactly problem is. Here's where I'm at,
Running pfsense 2.4.1 - Release (amd64)
ISP provides 120Mbps down/ 40Mbps upI've had pfsense running in the office for months. I originally started with a simple setup, no packages installed. Everything was fine, speeds tests were reporting full bandwidth in both down/up.
Since then, the network topology has not changed. I have installed pfsense OS updates along the way, Snort, squid (with cache and AV), and pfblocker. I have been running speed tests recently and my upload is consistently fine. The issue is with my download speeds. I can't get above ~97 Mbps. I know this sounds like an auto-negotiation or cabling issue, but I can assure you that's not the problem. Windows shows a 1 Gig connection and I have tried several PC's.
I have disabled all packages mentioned above which I suspected could have caused issues. I even disabled some packages like nut just because I'm running out of ideas. I have tried at least 5 or more different client PC's on the network and they all have the issue. Here are a few scenarios:
Client 1 (Windows 10) - This is a client native to the pfsense network and it has the issue on the pfsense network. When connected to a completely different LAN with a different non-pfsense router (Same upstream ISP gateway), I get full 120 Mbps download speeds.
Client 2 (Windows 7 64 bit) - This is a client native to the pfsense network and it has the issue on the pfsense network. I tried using a second NIC port on this same PC, it then got full 120 Mbps download speeds on the same pfsense network with the exact same cable that was plugged into the original NIC port!
Client 3 (Windows 7 64 bit) - This client is NOT native to the pfsense network. When I connected it to the pfsense network, it got the full 120 Mbps!
It's almost as if pfsense is remembering the MAC or the specific NIC and not allowing it to go above a certain speed. I tried rebooting pfsense, but that didn't help. For client 1, I tried removing the static DHCP lease and giving it a new IP. That didn't help. I don't think it has to do with static DHCP leases anyway because client 2 did not have a static lease and still had the issue.
I'm sure there must be a way to help narrow this down further. Would Wireshark be any help in a situation like this? If so, what do I look for?
I am trying to avoid reinstalling pfsense from scratch since the office is running off this system. Plus, I would like to understand what's going on and hopefully contribute to the cause with some useful diagnostics.Sorry for the long post. If you got this far, thanks for reading!
-
What kind of system do you have?
-
Do you have any sort of limiters or qos setup on pfsense?
-
johnpoz, good question. No, I don't have any QOS or limiters setup.
kejianshi, my system info is attached.
Thanks
Raffi
 -
What kind of system do you have?
In case you were wondering if this is running on a VM. It's a full install on the actual hardware.
Raffi
-
No. I want to know if your processor is a wimp or a brute.
The services you have running are difficult for weak processors.
-
It's definitely not a high end system, but the processor never breaks a sweat. The CPU usage is almost always close to no usage at all even when I had all those services running during the day when most users were on the network. I have disabled all the services mentioned since then, but no joy.
-
You need to provide system specs including hard drive type and amount of memory as well as squid config and processor type. As well as interface speeds.
(I see your cpu is enough and so is ram)
I'm leaning towards you problem being squid.
-
I have a 120GB Samsung evo SSD installed. The processor and RAM info are in the screen shot I provided.
Intel(R) Celeron(R) CPU 1017U @ 1.60GHz
2 CPUs: 1 package(s) x 2 core(s)
AES-NI CPU Crypto: No4GB RAM
I disabled squid to help isolate the issue and make troubleshooting simpler. Would the squid settings still be a factor even if the service is disabled?
I only have two NICs on the system. One for WAN and another for LAN. Both interfaces are Gigabit.
Raffi
-
Is squid still running? Check processes.
-
I double checked it, squid is not running. I also attached a screen shot of my services
-
I don't know.
Burn the install. Reinstall. Test again.
-
lol I wanna do what your avatar is doing right now. I think you're right though, I may have no choice.
-
BTW - I meant check it at a real console. ps -aux
-
[2.4.1-RELEASE][admin@pfsense.telebyte]/root: ps -aux
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
root 11 200.0 0.0 0 32 - RL 16:59 2659:43.17 [idle]
root 0 0.0 0.0 0 208 - DLs 16:59 0:00.19 [kernel]
root 1 0.0 0.0 5024 908 - ILs 16:59 0:00.01 /sbin/init –
root 2 0.0 0.0 0 16 - DL 16:59 0:00.00 [crypto]
root 3 0.0 0.0 0 16 - DL 16:59 0:00.00 [crypto retur
root 4 0.0 0.0 0 32 - DL 16:59 0:00.01 [cam]
root 5 0.0 0.0 0 16 - DL 16:59 0:00.01 [soaiod1]
root 6 0.0 0.0 0 16 - DL 16:59 0:00.01 [soaiod2]
root 7 0.0 0.0 0 16 - DL 16:59 0:00.01 [soaiod3]
root 8 0.0 0.0 0 16 - DL 16:59 0:00.01 [soaiod4]
root 9 0.0 0.0 0 16 - DL 16:59 0:00.00 [sctp_iterato
root 10 0.0 0.0 0 16 - DL 16:59 0:00.00 [audit]
root 12 0.0 0.0 0 272 - WL 16:59 4:41.33 [intr]
root 13 0.0 0.0 0 32 - DL 16:59 0:00.00 [ng_queue]
root 14 0.0 0.0 0 48 - DL 16:59 0:00.01 [geom]
root 15 0.0 0.0 0 256 - DL 16:59 2:36.05 [usb]
root 16 0.0 0.0 0 16 - DL 16:59 0:24.10 [pf purge]
root 17 0.0 0.0 0 16 - DL 16:59 0:13.27 [rand_harvest
root 18 0.0 0.0 0 16 - DL 16:59 0:02.78 [acpi_thermal
root 19 0.0 0.0 0 16 - DL 16:59 0:00.32 [acpi_cooling
root 20 0.0 0.0 0 16 - DL 16:59 0:00.07 [enc_daemon0]
root 21 0.0 0.0 0 48 - DL 16:59 0:04.35 [pagedaemon]
root 22 0.0 0.0 0 16 - DL 16:59 0:00.00 [vmdaemon]
root 23 0.0 0.0 0 16 - DL 16:59 0:00.00 [pagezero]
root 24 0.0 0.0 0 16 - DL 16:59 0:00.40 [bufspacedaem
root 25 0.0 0.0 0 32 - DL 16:59 0:02.04 [bufdaemon]
root 26 0.0 0.0 0 16 - DL 16:59 0:00.38 [vnlru]
root 27 0.0 0.0 0 16 - DL 16:59 0:07.44 [syncer]
root 60 0.0 0.0 0 16 - DL 16:59 0:00.08 [md0]
root 300 0.0 0.7 282676 29264 - Ss 16:59 0:02.47 php-fpm: mast
root 338 0.0 0.1 19436 4400 - INs 16:59 0:00.02 /usr/local/sb
root 340 0.0 0.1 19436 4216 - IN 16:59 0:00.00 check_reload_
root 353 0.0 0.1 9556 5516 - Ss 16:59 0:00.04 /sbin/devd -q
root 4772 0.0 0.1 19324 3196 - Ss 17:00 0:00.37 /usr/local/sb
root 5504 0.0 0.1 13084 2776 - IN 00:01 0:00.00 /bin/sh /etc/
root 5543 0.0 0.0 6172 1928 - IN 00:01 0:00.00 sleep 81230
root 7987 0.0 0.2 20348 6116 - Ss 16:59 0:10.19 /usr/local/sb
root 8940 0.0 0.1 12696 2392 - Ss 16:59 0:06.17 /usr/local/sb
root 12193 0.0 0.2 53488 6968 - Ss 16:59 0:00.00 /usr/sbin/ssh
root 12368 0.0 0.1 10580 2180 - Is 16:59 0:00.00 /usr/local/sb
root 14985 0.0 0.1 15076 2384 - Is 16:59 0:11.32 /usr/local/bi
root 19768 0.0 0.1 13084 2844 - IN 13:29 0:01.18 /bin/sh /var/
root 33534 0.0 0.0 8224 2004 - Is 17:00 0:00.00 /usr/local/bi
root 33889 0.0 0.0 8224 2020 - I 17:00 0:00.03 minicron: hel
root 34129 0.0 0.0 8224 2004 - Is 17:00 0:00.00 /usr/local/bi
root 34552 0.0 0.0 8224 2016 - I 17:00 0:00.00 minicron: hel
root 34737 0.0 0.0 8224 2004 - Is 17:00 0:00.00 /usr/local/bi
root 35020 0.0 0.0 8224 2016 - I 17:00 0:00.00 minicron: hel
root 37355 0.0 0.0 6172 1928 - IN 15:39 0:00.00 sleep 60
root 37366 0.0 0.2 78836 8140 - Ss 15:39 0:00.03 sshd: admin@p
root 48169 0.0 0.2 25416 6724 - Is 17:00 0:00.00 nginx: master
root 48399 0.0 0.2 27464 7768 - I 17:00 0:00.59 nginx: worker
root 48521 0.0 0.2 27464 8188 - I 17:00 0:01.90 nginx: worker
root 48884 0.0 0.1 12496 2368 - Is 17:00 0:00.50 /usr/sbin/cro
root 49416 0.0 0.3 24604 12424 - Ss 17:00 0:04.41 /usr/local/sb
root 60609 0.0 0.7 282676 29268 - I 15:37 0:00.00 php-fpm: pool
root 65254 0.0 0.1 10368 2088 - Ss 17:00 0:11.20 /usr/sbin/pow
root 70050 0.0 0.1 10580 2308 - Ss 17:00 0:00.00 /usr/local/sb
root 71912 0.0 0.0 10288 2012 - Is 13:37 0:00.00 /usr/local/sb
dhcpd 74470 0.0 0.2 16648 7836 - Ss 15:22 0:00.06 /usr/local/sb
root 78540 0.0 0.2 41504 7588 - I 13:34 0:00.00 /usr/local/sb
root 78860 0.0 0.2 52880 9108 - Ss 13:34 0:01.14 /usr/local/sb
unbound 79886 0.0 0.8 64468 33648 - Ss 09:58 0:17.38 /usr/local/sb
root 80737 0.0 0.1 10472 2532 - Ss 17:00 0:09.21 /usr/sbin/sys
root 68908 0.0 0.1 39432 2836 v0 Is 17:00 0:00.01 login [pam] (
root 70053 0.0 0.1 13084 2924 v0 I 17:00 0:00.00 -sh (sh)
root 70341 0.0 0.1 13084 2800 v0 I+ 17:00 0:00.00 /bin/sh /etc/
root 69122 0.0 0.1 10388 2128 v1 Is+ 17:00 0:00.00 /usr/libexec/
root 69382 0.0 0.1 10388 2128 v2 Is+ 17:00 0:00.00 /usr/libexec/
root 69546 0.0 0.1 10388 2128 v3 Is+ 17:00 0:00.00 /usr/libexec/
root 69647 0.0 0.1 10388 2128 v4 Is+ 17:00 0:00.00 /usr/libexec/
root 69652 0.0 0.1 10388 2128 v5 Is+ 17:00 0:00.00 /usr/libexec/
root 69953 0.0 0.1 10388 2128 v6 Is+ 17:00 0:00.00 /usr/libexec/
root 70040 0.0 0.1 10388 2128 v7 Is+ 17:00 0:00.00 /usr/libexec/
root 37841 0.0 0.1 13084 2800 0 Ss 15:39 0:00.00 /bin/sh /etc/
root 40476 0.0 0.1 13392 3632 0 S 15:39 0:00.01 /bin/tcsh
root 42749 0.0 0.1 21104 2716 0 R+ 15:39 0:00.00 ps -aux -
The "idle" process is using way too much processor… (kidding)
Don't see anything odd. I'd reinstall and test again.
-
haha tech humor. I'm going to hold off a reinstall for now since it's not a show stopper, but I have a feeling that may be the only option. I'll have to find a good time to get it done.
Thanks for the help.
Raffi
-
Yeah - I'd wait for a good time. It could take seconds or perhaps minutes to hit the "default settings" button in the console.
Might work as well as a fresh install.
-
lol good idea, I'll try that first.
Have you had any experience with a reinstall when an issue came up? I wonder if restoring my config on a fresh install would also "restore" the issue? I guess, I'll only know by trying.
-
Likely so. I've noticed that when I screw up my settings, save them and then restore them, they are still screwed up. Maybe its just me.
