Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    New firewall no ipsec traffic

    Scheduled Pinned Locked Moved IPsec
    6 Posts 2 Posters 4.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      Sylhouette
      last edited by

      Hello all
      I have 4 Soekriss boxes  with 2 SDSL connections, and they are configured with fail over

      Both have 2 IPSEC tunnels to 2 remote locations.

      All is working well.
      Now we have a new Fiber connection, and i use a Core2 machine with 2 fxp0 cards.I use the same firewall rules as the soekriss.
      Now when i switch the IPSEC tunnel from one of the 2 Soekriss boxes it all looks ok.
      i can ping the other side, i can take over the GUI on the remote Firewall (pfsense on soekriss also), but sending mail to the mail server (location on the fiber side) is not working.
      ALso a csup from the remote side to the side behind the fiber stalls.
      It looks like the connection is established but it stalls.

      On the Core2 server i use a rule on the IPSEC interface that allows any protocol from any source to any destination.
      Still i get drop in the firewall log.

      
Jun 11 21:31:57 	pf: 64\. 004877 rule 154/0(match): block out on fxp0: (tos 0x10, ttl 62, id 15453, offset 0, flags [DF], proto TCP (6), length 564) 192.168.9.6.22 > 192.168.1.231.1349: P 0:524(524) ack 1 win 65535
Jun 11 21:30:53 	pf: 64\. 003898 rule 154/0(match): block out on fxp0: (tos 0x10, ttl 62, id 32602, offset 0, flags [DF], proto TCP (6), length 564) 192.168.9.6.22 > 192.168.1.231.1349: P 0:524(524) ack 1 win 65535
Jun 11 21:29:49 	pf: 11\. 175028 rule 154/0(match): block out on fxp0: (tos 0x10, ttl 62, id 14489, offset 0, flags [DF], proto TCP (6), length 564) 192.168.9.6.22 > 192.168.1.231.1349: P 0:524(524) ack 1 win 65535
Jun 11 21:29:38 	pf: 11\. 036976 rule 153/0(match): block in on enc0: (tos 0x0, ttl 64, id 61382, offset 0, flags [none], proto ICMP (1), length 84) 192.168.9.241 > 192.168.1.2: ICMP echo request, id 13419, seq 0, length 64
Jun 11 21:29:26 	pf: 1\. 010003 rule 153/0(match): block in on enc0: (tos 0x0, ttl 64, id 19384, offset 0, flags [none], proto ICMP (1), length 84) 192.168.9.241 > 192.168.1.2: ICMP echo request, id 12139, seq 2, length 64
Jun 11 21:29:25 	pf: 1\. 002979 rule 153/0(match): block in on enc0: (tos 0x0, ttl 64, id 50084, offset 0, flags [none], proto ICMP (1), length 84) 192.168.9.241 > 192.168.1.2: ICMP echo request, id 12139, seq 1, length 64

      i really do not understand this.
      the version is 1.2.2 but i just updated to 1.2.3RC1 and it still has the same issue.
      it really drives me mad.

      i can use ssh to a freebsd server on the other side
      this is what i get from the csup

      
jailhost_pl ~ # csup -g -L2 /usr/local/etc/cvsup/ports-supfile
Parsing supfile "/usr/local/etc/cvsup/ports-supfile"
Connecting to 192.168.1.22
Connected to 192.168.1.22
Server software version: SNAP_16_1h
Negotiating file attribute support
Exchanging collection information
Establishing multiplexed-mode data connection
Running

      If i switch the tunnel back to one of the SDSL pfsense firewalls it immediatly start working again.

      pings work

      
 ping 192.168.1.22
PING 192.168.1.22 (192.168.1.22): 56 data bytes
64 bytes from 192.168.1.22: icmp_seq=0 ttl=61 time=58.399 ms
64 bytes from 192.168.1.22: icmp_seq=1 ttl=61 time=59.318 ms
64 bytes from 192.168.1.22: icmp_seq=2 ttl=61 time=58.209 ms
^C
--- 192.168.1.22 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 58.209/58.642/59.318/0.484 ms

      a telnet session to my mysql server looks ok but fails later on

      
telnet 192.168.1.16 3306
Trying 192.168.1.16...
Connected to mysql.schavemaker2.local.
Escape character is '^]'.
4
5.0.75XmJfD_5',b=F+@Cm{C{dcquit
Bad handshakeConnection closed by foreign host.

      Am i overlooking something.
      i did do a reset of all states, but no succes also.
      if i switch the remote location between either SDSL all is working like it should, but it is not working with the new box.

      Thanks for your time
      regards,
      Johan

      1 Reply Last reply Reply Quote 0
      • F
        fastcon68
        last edited by

        Just a few questions?
        How far apart are the two sites?
        What verisons of PF-Sense are you using on the two boxes?
        Are you converting the fiber to ethernet or are going to straight into the Soekriss box?

        Possible things to check/test?
        Test your fiber patch cables (I just had one that was bad from the factory It tested fine but was actually bad)
        Check your gbics if your using any?
        Make sure the speed and duplex settings are the same over the class?
        Have the fiber retested to make sure the pairs that you are using is good and solid.
        RC

        1 Reply Last reply Reply Quote 0
        • S
          Sylhouette
          last edited by

          one location is in Poland, the other 3 are in The Netherlands
          2 locations in Holland are about 200 Km away, and one is 4 Km away.

          The home location is in the Netherlands also.
          The fiber is converted by a cisco router, which i do not have control over.

          On the remote sites i use PFSense 1.2
          On the home location i use 1.2.2 , the same version as the soekriss boxes behind the SDSL lines.

          I think the line is ok, i can down and upload a FreeBSD DVD iso without getting it corrupt.I can do this multiple times from the internet.

          regards,
          Johan

          1 Reply Last reply Reply Quote 0
          • S
            Sylhouette
            last edited by

            Well i have configured one of the SDSL pfsense boxes to the fibers configuration, only had to change the WAN IP and carp IP's
            Then this box shows the same issue, so i think they do not fully open there cisco to my public ip's , i will call my ISP monday and ask about this.

            thanks for your time.

            regards,
            Johan

            1 Reply Last reply Reply Quote 0
            • F
              fastcon68
              last edited by

              I agree a quick call to the ISP may get you headed in the right direction.
              RC

              1 Reply Last reply Reply Quote 0
              • S
                Sylhouette
                last edited by

                Well we contacted the ISP and they say nothing is wrong with the connection.
                I had one location not far away from the Fiber Site witch has monowall running, but did not needed to connect to the main site.
                I tried to setup an IPSEC tunnel from there (with monowall) and it worked like it should.   ???

                So now i  have a fiber site with pfsense 1.2.3RC1 that cannot transfer data with other pfsense locations(1.2, 1.2.1 and 1.2.2), but can with a monowall site.

                i am totaly lost here.

                one of my SDSL routers is also 1.2.3 RC1 so it could not be the version i guess.

                if i let the csup go i get this a s a result.

                
csup -g -L2 /usr/local/etc/cvsup/ports-supfile
Parsing supfile "/usr/local/etc/cvsup/ports-supfile"
Connecting to 192.168.1.22
Connected to 192.168.1.22
Server software version: SNAP_16_1h
Negotiating file attribute support
Exchanging collection information
Establishing multiplexed-mode data connection
Running
Receiver: Operation timed out

                hense is can remotly ssh to that cvsupd deamon machine and stay conected for more than one hour giving commands and so on.

                what i also see on the Fiber site is the following in a tcpdump on the wan side of the firewall.
                do not know if it could be something with that.
                I did the following.
                start the capture, then hit the command csup -g -L2 /usrlocal/etc/cvsup/port-supfile (wich connect to 192.168.1.22 on the main fiber side.)
                When it says Running i stopped the capture.

                
21:25:50.556717 IP (tos 0x10, ttl 255, id 55615, offset 0, flags [DF], proto VRRP (112), length 56, bad cksum 0 (->c893)!) 193.173.YYY.YYY > 224.0.0.18: VRRPv2, Advertisement, vrid 53, prio 0, authtype none, intvl 1s, length 36, addrs(7): 244.201.250.45,239.151.131.255,91.102.75.234,67.104.75.221,239.214.110.143,61.144.15.165,67.129.197.78
21:25:50.626712 IP (tos 0x10, ttl 255, id 55361, offset 0, flags [DF], proto VRRP (112), length 56, bad cksum 0 (->c991)!) 193.173.YYY.YYY > 224.0.0.18: VRRPv2, Advertisement, vrid 54, prio 0, authtype none, intvl 1s, length 36, addrs(7): 56.54.235.42,173.40.246.67,169.202.181.189,144.245.123.176,201.113.242.40,255.220.146.215,47.168.165.213
21:25:51.276823 IP (tos 0x10, ttl 255, id 52523, offset 0, flags [DF], proto VRRP (112), length 56, bad cksum 0 (->d4a7)!) 193.173.YYY.YYY > 224.0.0.18: VRRPv2, Advertisement, vrid 50, prio 0, authtype none, intvl 1s, length 36, addrs(7): 233.106.174.249,155.20.150.202,211.34.143.254,15.186.44.85,158.20.184.87,103.232.91.113,201.21.228.48
21:25:51.456717 IP (tos 0x10, ttl 255, id 16938, offset 0, flags [DF], proto VRRP (112), length 56, bad cksum 0 (->5fa9)!) 193.173.YYY.YYY > 224.0.0.18: VRRPv2, Advertisement, vrid 52, prio 0, authtype none, intvl 1s, length 36, addrs(7): 233.153.49.198,230.152.10.223,42.57.100.58,141.190.174.130,26.119.72.102,234.42.140.127,40.41.89.109
21:25:51.566720 IP (tos 0x10, ttl 255, id 8805, offset 0, flags [DF], proto VRRP (112), length 56, bad cksum 0 (->7f6e)!) 193.173.YYY.YYY > 224.0.0.18: VRRPv2, Advertisement, vrid 53, prio 0, authtype none, intvl 1s, length 36, addrs(7): 203.193.59.95,236.64.210.109,202.60.129.232,98.65.160.2,188.182.97.39,133.249.204.141,146.127.198.175
21:25:51.636722 IP (tos 0x10, ttl 255, id 37934, offset 0, flags [DF], proto VRRP (112), length 56, bad cksum 0 (->da5)!) 193.173.YYY.YYY > 224.0.0.18: VRRPv2, Advertisement, vrid 54, prio 0, authtype none, intvl 1s, length 36, addrs(7): 142.169.150.134,96.208.150.73,59.63.216.151,199.179.137.75,13.57.85.140,44.126.209.202,250.61.160.113
21:25:52.086603 IP (tos 0x0, ttl 57, id 43754, offset 0, flags [none], proto ESP (50), length 112) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x40e), length 92
21:25:52.096848 IP (tos 0x0, ttl 64, id 65102, offset 0, flags [none], proto ESP (50), length 112, bad cksum 0 (->fcfa)!) 193.173.XXX.XXX > 217.166.XXX.XXX: ESP(spi=0x0a842405,seq=0x34d), length 92
21:25:52.106606 IP (tos 0x0, ttl 57, id 33443, offset 0, flags [none], proto ESP (50), length 104) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x40f), length 84
21:25:52.126803 IP (tos 0x0, ttl 64, id 60469, offset 0, flags [none], proto ESP (50), length 144, bad cksum 0 (->ef4)!) 193.173.XXX.XXX > 217.166.XXX.XXX: ESP(spi=0x0a842405,seq=0x34e), length 124
21:25:52.136605 IP (tos 0x0, ttl 57, id 42443, offset 0, flags [none], proto ESP (50), length 104) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x410), length 84
21:25:52.136721 IP (tos 0x0, ttl 57, id 33521, offset 0, flags [none], proto ESP (50), length 128) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x411), length 108
21:25:52.146781 IP (tos 0x0, ttl 64, id 20094, offset 0, flags [none], proto ESP (50), length 120, bad cksum 0 (->acc3)!) 193.173.XXX.XXX > 217.166.XXX.XXX: ESP(spi=0x0a842405,seq=0x34f), length 100
21:25:52.156606 IP (tos 0x0, ttl 57, id 10201, offset 0, flags [none], proto ESP (50), length 104) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x412), length 84
21:25:52.156720 IP (tos 0x0, ttl 57, id 7111, offset 0, flags [none], proto ESP (50), length 152) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x413), length 132
21:25:52.176779 IP (tos 0x0, ttl 64, id 1369, offset 0, flags [none], proto ESP (50), length 120, bad cksum 0 (->f5e8)!) 193.173.XXX.XXX > 217.166.XXX.XXX: ESP(spi=0x0a842405,seq=0x350), length 100
21:25:52.176794 IP (tos 0x0, ttl 57, id 25585, offset 0, flags [none], proto ESP (50), length 184) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x414), length 164
21:25:52.176965 IP (tos 0x0, ttl 57, id 56203, offset 0, flags [none], proto ESP (50), length 104) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x415), length 84
21:25:52.186809 IP (tos 0x0, ttl 64, id 37728, offset 0, flags [none], proto ESP (50), length 104, bad cksum 0 (->67f1)!) 193.173.XXX.XXX > 217.166.XXX.XXX: ESP(spi=0x0a842405,seq=0x351), length 84
21:25:52.186986 IP (tos 0x0, ttl 64, id 65076, offset 0, flags [none], proto ESP (50), length 104, bad cksum 0 (->fd1c)!) 193.173.XXX.XXX > 217.166.XXX.XXX: ESP(spi=0x0a842405,seq=0x352), length 84
21:25:52.187000 IP (tos 0x0, ttl 57, id 39913, offset 0, flags [none], proto ESP (50), length 104) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x416), length 84
21:25:52.187111 IP (tos 0x0, ttl 57, id 8360, offset 0, flags [none], proto ESP (50), length 120) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x417), length 100
21:25:52.196772 IP (tos 0x0, ttl 64, id 51020, offset 0, flags [none], proto ESP (50), length 112, bad cksum 0 (->33fd)!) 193.173.XXX.XXX > 217.166.XXX.XXX: ESP(spi=0x0a842405,seq=0x353), length 92
21:25:52.196786 IP (tos 0x0, ttl 57, id 4520, offset 0, flags [none], proto ESP (50), length 104) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x418), length 84
21:25:52.206605 IP (tos 0x0, ttl 57, id 40382, offset 0, flags [none], proto ESP (50), length 104) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x419), length 84
21:25:52.206720 IP (tos 0x0, ttl 57, id 64645, offset 0, flags [none], proto ESP (50), length 136) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x41a), length 116
21:25:52.216801 IP (tos 0x0, ttl 64, id 37695, offset 0, flags [none], proto ESP (50), length 136, bad cksum 0 (->67f2)!) 193.173.XXX.XXX > 217.166.XXX.XXX: ESP(spi=0x0a842405,seq=0x354), length 116
21:25:52.226609 IP (tos 0x0, ttl 57, id 27029, offset 0, flags [none], proto ESP (50), length 104) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x41b), length 84
21:25:52.226743 IP (tos 0x0, ttl 57, id 15567, offset 0, flags [none], proto ESP (50), length 136) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x41c), length 116
21:25:52.246909 IP (tos 0x0, ttl 64, id 12856, offset 0, flags [none], proto ESP (50), length 200, bad cksum 0 (->c8b9)!) 193.173.XXX.XXX > 217.166.XXX.XXX: ESP(spi=0x0a842405,seq=0x355), length 180
21:25:52.256608 IP (tos 0x0, ttl 57, id 45255, offset 0, flags [none], proto ESP (50), length 104) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x41d), length 84
21:25:52.256740 IP (tos 0x0, ttl 57, id 61108, offset 0, flags [none], proto ESP (50), length 112) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x41e), length 92
21:25:52.286735 IP (tos 0x10, ttl 255, id 21571, offset 0, flags [DF], proto VRRP (112), length 56, bad cksum 0 (->4d90)!) 193.173.YYY.YYY > 224.0.0.18: VRRPv2, Advertisement, vrid 50, prio 0, authtype none, intvl 1s, length 36, addrs(7): 233.106.174.249,155.20.150.203,76.160.207.216,86.179.189.21,77.227.123.119,46.169.255.8,192.68.9.89
21:25:52.376795 IP (tos 0x0, ttl 64, id 10790, offset 0, flags [none], proto ESP (50), length 104, bad cksum 0 (->d12b)!) 193.173.XXX.XXX > 217.166.XXX.XXX: ESP(spi=0x0a842405,seq=0x356), length 84
21:25:52.386611 IP (tos 0x0, ttl 57, id 63728, offset 0, flags [none], proto ESP (50), length 112) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x41f), length 92
21:25:52.396781 IP (tos 0x0, ttl 64, id 44904, offset 0, flags [none], proto ESP (50), length 112, bad cksum 0 (->4be1)!) 193.173.XXX.XXX > 217.166.XXX.XXX: ESP(spi=0x0a842405,seq=0x357), length 92
21:25:52.406607 IP (tos 0x0, ttl 57, id 40878, offset 0, flags [none], proto ESP (50), length 104) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x420), length 84
21:25:52.406722 IP (tos 0x0, ttl 57, id 18343, offset 0, flags [none], proto ESP (50), length 112) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x421), length 92
21:25:52.416778 IP (tos 0x0, ttl 64, id 5916, offset 0, flags [none], proto ESP (50), length 112, bad cksum 0 (->e42d)!) 193.173.XXX.XXX > 217.166.XXX.XXX: ESP(spi=0x0a842405,seq=0x358), length 92
21:25:52.426607 IP (tos 0x0, ttl 57, id 34180, offset 0, flags [none], proto ESP (50), length 104) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x422), length 84
21:25:52.426720 IP (tos 0x0, ttl 57, id 5273, offset 0, flags [none], proto ESP (50), length 120) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x423), length 100
21:25:52.436779 IP (tos 0x0, ttl 64, id 47120, offset 0, flags [none], proto ESP (50), length 112, bad cksum 0 (->4339)!) 193.173.XXX.XXX > 217.166.XXX.XXX: ESP(spi=0x0a842405,seq=0x359), length 92
21:25:52.446606 IP (tos 0x0, ttl 57, id 19897, offset 0, flags [none], proto ESP (50), length 104) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x424), length 84
21:25:52.446720 IP (tos 0x0, ttl 57, id 64944, offset 0, flags [none], proto ESP (50), length 112) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x425), length 92
21:25:52.456782 IP (tos 0x0, ttl 64, id 39268, offset 0, flags [none], proto ESP (50), length 112, bad cksum 0 (->61e5)!) 193.173.XXX.XXX > 217.166.XXX.XXX: ESP(spi=0x0a842405,seq=0x35a), length 92
21:25:52.456906 IP (tos 0x0, ttl 57, id 39372, offset 0, flags [+], proto ESP (50), length 1492) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x426), length 1472
21:25:52.456934 IP (tos 0x0, ttl 57, id 39372, offset 1480, flags [none], proto ESP (50), length 72) 217.166.XXX.XXX > 193.173.XXX.XXX: esp
21:25:52.456947 IP (tos 0x0, ttl 57, id 53480, offset 0, flags [+], proto ESP (50), length 1492) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x427), length 1472
21:25:52.456968 IP (tos 0x0, ttl 57, id 53480, offset 1480, flags [none], proto ESP (50), length 72) 217.166.XXX.XXX > 193.173.XXX.XXX: esp
21:25:52.466616 IP (tos 0x0, ttl 57, id 29348, offset 0, flags [+], proto ESP (50), length 1492) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x428), length 1472
21:25:52.466638 IP (tos 0x0, ttl 57, id 29348, offset 1480, flags [none], proto ESP (50), length 72) 217.166.XXX.XXX > 193.173.XXX.XXX: esp
21:25:52.466650 IP (tos 0x0, ttl 57, id 59795, offset 0, flags [+], proto ESP (50), length 1492) 217.166.XXX.XXX > 193.173.XXX.XXX: ESP(spi=0x0a7d5c27,seq=0x429), length 1472
21:25:52.466669 IP (tos 0x0, ttl 57, id 59795, offset 1480, flags [none], proto ESP (50), length 72) 217.166.XXX.XXX > 193.173.XXX.XXX: esp

                193.173.XXX.XXX is my faulty fiber site(CARP0) 217.166.XXX.XXX is my other site (in this case fiber also, from the same ISP)
                193.173.YYY.YYY is the WAN address itself

                On the other side 217.166.XXX.XXX i do not see those bad cksum's

                I hope someone can shed a light on this.
                And sorry for my poor explanation capability's in english

                regards, and thanks for your time reading this
                Johan

                1 Reply Last reply Reply Quote 0
                • First post
                  Last post
                Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.