Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Massive performance issues on some websites

    Scheduled Pinned Locked Moved Problems Installing or Upgrading pfSense Software
    27 Posts 6 Posters 6.0k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • J
      justin.j
      last edited by

      I'm in the process of migrating away from our current UTM (Endian) to pfSense but am having a number of issues in the process, the biggest one being extremely slow page loads, but only on some websites.  When I switch back to Endian from pfSense the websites load quickly as expected.  I've tried disabling snort on the WAN interface to no avail.
      Both UTMs are running on ESXi 5.5, Endian with 1gb RAM and 2vCPU and pfSense with 2gb/2vCPU.  Both UTMs use the same DNS, physical NICs, and 10mb WAN connection.  Endian is running VMXNET3 NICs and pfSense running E1000 as per the virtualisation guidelines.
      During the page loads for these sites I have esxtop open monitoring the network and vSphere performance monitor open showing realtime overview.
      During the page loads the CPU usage remains low (About 150mHz) and memory remains mostly consumed (I'm guessing like SQL boxes it just uses the maximum amount of memory available to it?).

      As a comparison Youtube over HTTPs takes about 2 seconds to fully load the page on Endian compared to about 15 seconds on pfSense.
      Some sites fail to load at all (Firewall shows no blocks relating to these sites) or sometimes after multiple refreshes may load but can take 30 seconds or more to load a page that would normally load in <5.

      Also not sure if it is related but the auto update in pfSense fails, but if I copy the URL to my browser it loads quite fine.

      Downloading new version information…done
      Unable to check for updates.
      Could not contact pfSense update server https://updates.pfsense.org/_updaters/amd64

      If anyone could give some pointers on why these issues might be occurring it would be greatly appreciated as being unable to migrate to pfSense means we're unable to complete some other planned projects as well (Due to Endian's 4 interface limits).

      1 Reply Last reply Reply Quote 0
      • J
        jfranco
        last edited by

        Seems you have DNS issues.
        I will recommend you edit your /etc/resolv.conf and add your DNS server in the first line:

        nameserver 10.0.0.1
        

        Do not use unbound for DNS.
        If you don't have a local DNS server, use one that's near you, or google's 8.8.4.4

        1 Reply Last reply Reply Quote 0
        • J
          justin.j
          last edited by

          @jfranco:

          Seems you have DNS issues.
          I will recommend you edit your /etc/resolv.conf and add your DNS server in the first line:

          nameserver 10.0.0.1
          

          Do not use unbound for DNS.
          If you don't have a local DNS server, use one that's near you, or google's 8.8.4.4

          Definitely not a DNS issue.  Our internal network references an internal DNS which is setup with forwarders to our ISP and OpenDNS so that wouldn’t stop machines internally from connecting.  Because of the auto update issue I double checked the DNS settings in pfSense and they are configured correctly (To OpenDNS via WAN gateway), just in case pfSense didn’t update its host OS I checked the resolv.conf and it’s applied correctly there too (127.0.0.1 followed by the 2 OpenDNS servers).

          1 Reply Last reply Reply Quote 0
          • DerelictD
            Derelict LAYER 8 Netgate
            last edited by

            I'm out of my lane but those who know will want to know what packages you're running (snort, squid, squidguard, havp, etc.)

            Chattanooga, Tennessee, USA
            A comprehensive network diagram is worth 10,000 words and 15 conference calls.
            DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
            Do Not Chat For Help! NO_WAN_EGRESS(TM)

            1 Reply Last reply Reply Quote 0
            • J
              justin.j
              last edited by

              @Derelict:

              I'm out of my lane but those who know will want to know what packages you're running (snort, squid, squidguard, havp, etc.)

              Only Snort and Open VM Tools

              1 Reply Last reply Reply Quote 0
              • K
                kejianshi
                last edited by

                Only snort huh?

                1 Reply Last reply Reply Quote 0
                • S
                  Supermule Banned
                  last edited by

                  Disable Snort and report back

                  1 Reply Last reply Reply Quote 0
                  • J
                    justin.j
                    last edited by

                    @kejianshi:

                    Only snort huh?

                    And Open VM tools.

                    packages.png
                    packages.png_thumb

                    1 Reply Last reply Reply Quote 0
                    • S
                      Supermule Banned
                      last edited by

                      Leave it on one at a time….

                      1 Reply Last reply Reply Quote 0
                      • J
                        justin.j
                        last edited by

                        @Supermule:

                        Disable Snort and report back

                        That was one of the first things I tried.  Mentioned that in my OP

                        @Supermule:

                        Leave it on one at a time….

                        I'm not sure what you mean by that?

                        1 Reply Last reply Reply Quote 0
                        • K
                          kejianshi
                          last edited by

                          I agree with super mule - Turning of snort has occasionally been known to help performance - From time to time.
                          The same way not being DDOSed helps with performance….

                          You said you are running two packages.
                          I think supermule meant try disabling one, and then the other separately to see which is causing the problem.

                          1 Reply Last reply Reply Quote 0
                          • J
                            justin.j
                            last edited by

                            @kejianshi:

                            I agree with super mule - Turning of snort has occasionally been known to help performance - From time to time.
                            The same way not being DDOSed helps with performance….

                            I've tried with Snort enabled (When I first tried to switch over to pfSense) and then with it disabled to try and narrow down the cause of the issues I'm having.
                            Considering it's a fresh install with minimal packages added I'm a bit stumped as to why these issues are occurring, and even more stumped that I can't see anything being blocked or high use in any resources >:(

                            1 Reply Last reply Reply Quote 0
                            • K
                              kejianshi
                              last edited by

                              Pfsense works well and works fast.  Its either a package, a misconfiguration, your DNS or something like that that is causing your problems.

                              What are you using for DNS?  Can we see the list that appears on the main page to the left?

                              1 Reply Last reply Reply Quote 0
                              • J
                                justin.j
                                last edited by

                                @kejianshi:

                                Pfsense works well and works fast.  Its either a package, a misconfiguration, your DNS or something like that that is causing your problems.

                                What are you using for DNS?  Can we see the list that appears on the main page to the left?

                                As mentioned previously, it is 127.0.0.1 followed by the two OpenDNS servers.
                                Here is a screenshot that shows details I have already given.

                                General.png
                                General.png_thumb

                                1 Reply Last reply Reply Quote 0
                                • K
                                  kejianshi
                                  last edited by

                                  Try changing DNS to 8.8.8.8 and 8.8.4.4 just for a little while and reboot.

                                  See if it helps.

                                  If thats not the issue, can we see your lan and wan firewall rules?

                                  "unable to check for updates" seems like something very basic is broken.

                                  1 Reply Last reply Reply Quote 0
                                  • S
                                    Supermule Banned
                                    last edited by

                                    Why are you unable to check for updates??

                                    1 Reply Last reply Reply Quote 0
                                    • J
                                      justin.j
                                      last edited by

                                      @kejianshi:

                                      Try changing DNS to 8.8.8.8 and 8.8.4.4 just for a little while and reboot.

                                      See if it helps.

                                      If thats not the issue, can we see your lan and wan firewall rules?

                                      "unable to check for updates" seems like something very basic is broken.

                                      100% sure that won't help.  We've used OpenDNS for years without an issue and it works fine in Endian.
                                      Not to mention that internally the DNS saved in the firewall isn't even used as we use our own internal DNS server.

                                      1 Reply Last reply Reply Quote 0
                                      • J
                                        justin.j
                                        last edited by

                                        @Supermule:

                                        Why are you unable to check for updates??

                                        I'm not sure.  If I copy and paste the update link from the failure message it loads on my PC fine and the package list updates on pfSense fine so I'm still really dumbfounded.

                                        Offtopic:  Slightly jealous of your speed there  :-\  Management won't spend more on a faster link (Currently 10mb) and home internet is worse :(

                                        1 Reply Last reply Reply Quote 0
                                        • K
                                          kejianshi
                                          last edited by

                                          Please just try google DNS just to placate my silly whims.

                                          1 Reply Last reply Reply Quote 0
                                          • S
                                            Supermule Banned
                                            last edited by

                                            Pls. try Google DNS to see if the issue persits…

                                            @justin.j:

                                            @Supermule:

                                            Why are you unable to check for updates??

                                            I'm not sure.  If I copy and paste the update link from the failure message it loads on my PC fine and the package list updates on pfSense fine so I'm still really dumbfounded.

                                            Offtopic:  Slightly jealous of your speed there  :-\  Management won't spend more on a faster link (Currently 10mb) and home internet is worse :(

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.