Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    CARP/VIPS issue in downloading large file

    HA/CARP/VIPs
    5
    17
    8.2k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      mypal
      last edited by

      I have two pfsense firewalls set up with 4 interfaces each (Public, Private, Sync and DMZ). The interfaces are all running with CARP/VIPs. The issue that I have downloading or uploading a large size file (1MB +) to a webserver/qmail server is located at the DMZ from the public internet always hit a timeout when both firewalls are active. If I down one of the firewall, this issue does not occur.

      Both firewall are Dell Dual Xeon/2GB RAM. The pfsense version is 1.0-Snapshot-09-12-06. Any ideas on what should I check?

      Thanks
      Mypal

      1 Reply Last reply Reply Quote 0
      • S
        sullrich
        last edited by

        Take a look at this document: http://doc.pfsense.org/index.php/Setting_up_CARP_with_pfSense and see if it brings up anything that may be setup wrong.

        1 Reply Last reply Reply Quote 0
        • M
          mypal
          last edited by

          One additional thing that I forgot to mention, when I transfer file from a PC at the Private network (using VIPs gateway at Private) to DMZ network, I have no such issue.

          I suspect there must be some timeout when doing it from the Internet. How can I check this?

          Thanks
          Mypal

          1 Reply Last reply Reply Quote 0
          • B
            billm
            last edited by

            @mypal:

            One additional thing that I forgot to mention, when I transfer file from a PC at the Private network (using VIPs gateway at Private) to DMZ network, I have no such issue.

            I suspect there must be some timeout when doing it from the Internet. How can I check this?

            Thanks
            Mypal

            Sounds like your CARP address(es) on the internet side are fighting for master status to me.  Make sure that all the CARP IPs on the secondary are in BACKUP, not MASTER mode.

            –Bill

            pfSense core developer
            blog - http://www.ucsecurity.com/
            twitter - billmarquette

            1 Reply Last reply Reply Quote 0
            • S
              sullrich
              last edited by

              No, there is no timeout.  I really have no idea what you mean.

              Take a look at the wiki, I just updated the graphic so that it may be a bit better now.

              1 Reply Last reply Reply Quote 0
              • H
                hsiang
                last edited by

                I also facing the same issue from the WAN link. I had set the CARP as describe. When the issue occur the secondary firewall status is showing backup.

                However the external user is having time out problem when sending attachment email (150kb only), connecting to the mail server in dmz. Thing works fine when i shut down the secondary firewall totally.

                1 Reply Last reply Reply Quote 0
                • S
                  sullrich
                  last edited by

                  Sorry but this is not enough information to go on.

                  Please show us this command on both machines:

                  ifconfig

                  1 Reply Last reply Reply Quote 0
                  • M
                    mypal
                    last edited by

                    #ifconfig on machine1

                    em0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>mtu 1500
                            options=b <rxcsum,txcsum,vlan_mtu>inet 192.168.0.229 netmask 0xffffff00 broadcast 192.168.0.255
                            inet6 fe80::214:22ff:fe18:8d47%em0 prefixlen 64 scopeid 0x1
                            ether 00:14:22:18:8d:47
                            media: Ethernet autoselect (1000baseTX <full-duplex>)
                            status: active
                    em1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>mtu 1500
                            options=b <rxcsum,txcsum,vlan_mtu>inet 202.184.208.229 netmask 0xffffff00 broadcast 202.184.208.255
                            inet6 fe80::214:22ff:fe18:8d48%em1 prefixlen 64 scopeid 0x2
                            ether 00:14:22:18:8d:48
                            media: Ethernet autoselect (100baseTX <full-duplex>)
                            status: active
                    em2: flags=8843 <up,broadcast,running,simplex,multicast>mtu 1500
                            options=b <rxcsum,txcsum,vlan_mtu>inet 10.0.0.229 netmask 0xffffff00 broadcast 10.0.0.255
                            inet6 fe80::204:23ff:fed4:c338%em2 prefixlen 64 scopeid 0x3
                            ether 00:04:23:d4:c3:38
                            media: Ethernet autoselect
                            status: no carrier
                    em3: flags=8943 <up,broadcast,running,promisc,simplex,multicast>mtu 1500
                            options=b <rxcsum,txcsum,vlan_mtu>inet 10.10.10.229 netmask 0xffffff00 broadcast 10.10.10.255
                            inet6 fe80::204:23ff:fed4:c339%em3 prefixlen 64 scopeid 0x4
                            ether 00:04:23:d4:c3:39
                            media: Ethernet autoselect (1000baseTX <full-duplex>)
                            status: active
                    em4: flags=8802 <broadcast,simplex,multicast>mtu 1500
                            options=b <rxcsum,txcsum,vlan_mtu>ether 00:04:23:d4:c3:bc
                            media: Ethernet autoselect (1000baseTX <full-duplex>)
                            status: active
                    em5: flags=8802 <broadcast,simplex,multicast>mtu 1500
                            options=b <rxcsum,txcsum,vlan_mtu>ether 00:04:23:d4:c3:bd
                            media: Ethernet autoselect
                            status: no carrier
                    pflog0: flags=100 <promisc>mtu 33208
                    enc0: flags=0<> mtu 1536
                    lo0: flags=8049 <up,loopback,running,multicast>mtu 16384
                            inet 127.0.0.1 netmask 0xff000000
                            inet6 ::1 prefixlen 128
                            inet6 fe80::1%lo0 prefixlen 64 scopeid 0x9
                    pfsync0: flags=41 <up,running>mtu 1348
                            pfsync: syncdev: em2 maxupd: 128
                    carp0: flags=49 <up,loopback,running>mtu 1500
                            inet 202.184.208.231 netmask 0xffffff00
                            carp: MASTER vhid 1 advbase 1 advskew 200
                    carp1: flags=49 <up,loopback,running>mtu 1500
                            inet 192.168.0.232 netmask 0xffffff00
                            carp: MASTER vhid 2 advbase 1 advskew 200
                    carp2: flags=49 <up,loopback,running>mtu 1500
                            inet 10.10.10.231 netmask 0xffffff00
                            carp: MASTER vhid 3 advbase 1 advskew 200

                    ifconfig on machine2

                    em0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>mtu 1500
                            options=b <rxcsum,txcsum,vlan_mtu>inet 10.10.10.230 netmask 0xffffff00 broadcast 10.10.10.255
                            inet6 fe80::204:23ff:fed4:bcfa%em0 prefixlen 64 scopeid 0x1
                            ether 00:04:23:d4:bc:fa
                            media: Ethernet autoselect (1000baseTX <full-duplex>)
                            status: active
                    em1: flags=8843 <up,broadcast,running,simplex,multicast>mtu 1500
                            options=b <rxcsum,txcsum,vlan_mtu>inet 10.0.0.230 netmask 0xffffff00 broadcast 10.0.0.255
                            inet6 fe80::204:23ff:fed4:bcfb%em1 prefixlen 64 scopeid 0x2
                            ether 00:04:23:d4:bc:fb
                            media: Ethernet autoselect (1000baseTX <full-duplex>)
                            status: active
                    bge0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>mtu 1500
                            options=1b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging>inet 192.168.0.230 netmask 0xffffff00 broadcast 192.168.0.255
                            inet6 fe80::206:5bff:feef:61f0%bge0 prefixlen 64 scopeid 0x3
                            ether 00:06:5b:ef:61:f0
                            media: Ethernet autoselect (1000baseTX <full-duplex>)
                            status: active
                    bge1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>mtu 1500
                            options=1b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging>inet 202.184.208.230 netmask 0xffffff00 broadcast 202.184.208.255
                            inet6 fe80::206:5bff:feef:61f1%bge1 prefixlen 64 scopeid 0x4
                            ether 00:06:5b:ef:61:f1
                            media: Ethernet autoselect (100baseTX <full-duplex>)
                            status: active
                    em2: flags=8802 <broadcast,simplex,multicast>mtu 1500
                            options=b <rxcsum,txcsum,vlan_mtu>ether 00:04:23:d4:c3:36
                            media: Ethernet autoselect
                            status: no carrier
                    em3: flags=8802 <broadcast,simplex,multicast>mtu 1500
                            options=b <rxcsum,txcsum,vlan_mtu>ether 00:04:23:d4:c3:37
                            media: Ethernet autoselect
                            status: no carrier
                    pflog0: flags=100 <promisc>mtu 33208
                    enc0: flags=0<> mtu 1536
                    lo0: flags=8049 <up,loopback,running,multicast>mtu 16384
                            inet 127.0.0.1 netmask 0xff000000
                            inet6 ::1 prefixlen 128
                            inet6 fe80::1%lo0 prefixlen 64 scopeid 0x9
                    pfsync0: flags=41 <up,running>mtu 1348
                            pfsync: syncdev: em1 maxupd: 128
                    carp0: flags=49 <up,loopback,running>mtu 1500
                            inet 202.184.208.231 netmask 0xffffff00
                            carp: BACKUP vhid 1 advbase 1 advskew 200
                    carp1: flags=49 <up,loopback,running>mtu 1500
                            inet 192.168.0.232 netmask 0xffffff00
                            carp: BACKUP vhid 2 advbase 1 advskew 200
                    carp2: flags=49 <up,loopback,running>mtu 1500
                            inet 10.10.10.231 netmask 0xffffff00
                            carp: BACKUP vhid 3 advbase 1 advskew 200
                    carp3: flags=49 <up,loopback,running>mtu 1500
                            inet 202.184.208.201 netmask 0xffffff00
                            carp: BACKUP vhid 6 advbase 1 advskew 200</up,loopback,running></up,loopback,running></up,loopback,running></up,loopback,running></up,running></up,loopback,running,multicast></promisc></rxcsum,txcsum,vlan_mtu></broadcast,simplex,multicast></rxcsum,txcsum,vlan_mtu></broadcast,simplex,multicast></full-duplex></rxcsum,txcsum,vlan_mtu,vlan_hwtagging></up,broadcast,running,promisc,simplex,multicast></full-duplex></rxcsum,txcsum,vlan_mtu,vlan_hwtagging></up,broadcast,running,promisc,simplex,multicast></full-duplex></rxcsum,txcsum,vlan_mtu></up,broadcast,running,simplex,multicast></full-duplex></rxcsum,txcsum,vlan_mtu></up,broadcast,running,promisc,simplex,multicast></up,loopback,running></up,loopback,running></up,loopback,running></up,running></up,loopback,running,multicast></promisc></rxcsum,txcsum,vlan_mtu></broadcast,simplex,multicast></full-duplex></rxcsum,txcsum,vlan_mtu></broadcast,simplex,multicast></full-duplex></rxcsum,txcsum,vlan_mtu></up,broadcast,running,promisc,simplex,multicast></rxcsum,txcsum,vlan_mtu></up,broadcast,running,simplex,multicast></full-duplex></rxcsum,txcsum,vlan_mtu></up,broadcast,running,promisc,simplex,multicast></full-duplex></rxcsum,txcsum,vlan_mtu></up,broadcast,running,promisc,simplex,multicast>

                    1 Reply Last reply Reply Quote 0
                    • S
                      sullrich
                      last edited by

                      See http://doc.pfsense.org/index.php/Setting_up_CARP_with_pfSense

                      Machine #1's advskew needs to be lower.

                      1 Reply Last reply Reply Quote 0
                      • H
                        hsiang
                        last edited by

                        From the GUI i had set the master CARP advertising frequency as 0 and is shown as 0 in config.xml. However from ifconfig the advskew still show as 200.

                        To change the value, i went to /etc/inc/interface.inc to manually change the advskew to 0. Now from ifconfig the master advskew is showing 0

                        carp0: flags=49 <up,loopback,running>mtu 1500
                                inet 202.184.208.231 netmask 0xffffff00
                                carp: MASTER vhid 1 advbase 1 advskew 0
                        carp1: flags=49 <up,loopback,running>mtu 1500
                                inet 192.168.0.232 netmask 0xffffff00
                                carp: MASTER vhid 2 advbase 1 advskew 0
                        carp2: flags=49 <up,loopback,running>mtu 1500
                                inet 10.10.10.231 netmask 0xffffff00
                                carp: MASTER vhid 3 advbase 1 advskew 0
                        carp3: flags=49 <up,loopback,running>mtu 1500
                                inet 202.184.208.201 netmask 0xffffff00
                                carp: MASTER vhid 6 advbase 1 advskew 0

                        Result:
                        after i reboot the master - the CARP Status will remain as backup, until i shut down the slave.

                        Question:
                        1. shouldn't the master take charge back as master when it alive?
                        2. is it a bug in the program where the advskew is binded to 200 as default??

                        Thanks</up,loopback,running></up,loopback,running></up,loopback,running></up,loopback,running>

                        1 Reply Last reply Reply Quote 0
                        • S
                          sullrich
                          last edited by

                          After a reboot operation the CARP holdup timer will keep the item at 200 for atleast 2-3 minutes.

                          Afterwards the advskew is set to whatever is in config.xml.

                          1 Reply Last reply Reply Quote 0
                          • H
                            hsiang
                            last edited by

                            It seems like is not that case where the advskew will follow whatever in config.xml after 3 minutes

                            Here is the setting of in my config.xml for my Slave/Secondary Pfsense

                            <vip><mode>carp</mode>
                                                    <interface>wan</interface>
                                                    <vhid>1</vhid>
                                                    <advskew>100</advskew>
                                                    <password>carp</password>
                                                    <descr>WAN CARP</descr>
                                                    <type>single</type>
                                                    <subnet_bits>24</subnet_bits>
                                                    <subnet>x.x.x.x</subnet></vip>
                                            <vip><mode>carp</mode>
                                                    <interface>lan</interface>
                                                    <vhid>2</vhid>
                                                    <advskew>100</advskew>
                                                    <password>carp</password>
                                                    <descr>LAN CARP</descr>
                                                    <type>single</type>
                                                    <subnet_bits>24</subnet_bits>
                                                    <subnet>192.168.0.232</subnet></vip>
                                            <vip><mode>carp</mode>
                                                    <interface>opt2</interface>
                                                    <vhid>3</vhid>
                                                    <advskew>100</advskew>
                                                    <password>carp</password>
                                                    <descr>DMZ CARP</descr>
                                                    <type>single</type>
                                                    <subnet_bits>24</subnet_bits>
                                                    <subnet>10.10.10.231</subnet></vip>

                            which show all is having advskew 100.

                            however when i issue ipconfig after the said time (actually more than 1/2 hour) it still show as below:

                            carp0: flags=49 <up,loopback,running>mtu 1500
                                    inet 202.184.208.231 netmask 0xffffff00
                                    carp: BACKUP vhid 1 advbase 1 advskew 200
                            carp1: flags=49 <up,loopback,running>mtu 1500
                                    inet 192.168.0.232 netmask 0xffffff00
                                    carp: BACKUP vhid 2 advbase 1 advskew 200
                            carp2: flags=49 <up,loopback,running>mtu 1500
                                    inet 10.10.10.231 netmask 0xffffff00
                                    carp: BACKUP vhid 3 advbase 1 advskew 200</up,loopback,running></up,loopback,running></up,loopback,running>

                            1 Reply Last reply Reply Quote 0
                            • M
                              moffl
                              last edited by

                              Did you find out what was causing your email/file timeout errors as i to am having this problem. The master has a advskew "0" and backup has "100" everything seems to sync up but emails over 150 - 200k hang the email client and cuases a timeout error and files over 500k just time out

                              1 Reply Last reply Reply Quote 0
                              • H
                                hsiang
                                last edited by

                                i had created CARP for each IP i am using for my servers. then it works fine after that. have a try.

                                1 Reply Last reply Reply Quote 0
                                • M
                                  moffl
                                  last edited by

                                  @hsiang:

                                  i had created CARP for each IP i am using for my servers. then it works fine after that. have a try.

                                  Does this mean that you created virtual ips for each server and if so i have already done that

                                  1 Reply Last reply Reply Quote 0
                                  • M
                                    moffl
                                    last edited by

                                    I also keep getting this in the system logs
                                    kernel: arp_rtrequest: bad gateway 10.190.10.7 (!AF_LINK)
                                    Dec 12 00:16:22 kernel: arp_rtrequest: bad gateway 10.190.10.6 (!AF_LINK)
                                    Dec 12 00:16:21 kernel: arp_rtrequest: bad gateway 10.190.10.4 (!AF_LINK)
                                    Dec 12 00:16:18 kernel: arp_rtrequest: bad gateway 10.190.10.2 (!AF_LINK)
                                    Dec 12 00:16:17 kernel: arp_rtrequest: bad gateway 198.169.176.2 (!AF_LINK)
                                    Dec 12 00:16:16 kernel: arp_rtrequest: bad gateway 10.190.10.25 (!AF_LINK)

                                    1 Reply Last reply Reply Quote 0
                                    • S
                                      sullrich
                                      last edited by

                                      http://faq.pfsense.com/index.php?action=artikel&cat=1&id=167&artlang=en&highlight=bad%20gateway

                                      1 Reply Last reply Reply Quote 0
                                      • First post
                                        Last post
                                      Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.