static route to a network, but only part of it is connectable



  • Hi guys,
    I have a dual WAN setup:
    WAN1 is a fiber connection that goes to a 100.84.x network with a 255.255.240.0 subnet mask. WAN1 Gateway is 100.84.a.b
    WAN2 is a wireless that goes to a 192.168.100.0/24 subnet.
    WAN2 Gateway is 192.168.100.1

    WAN2 Gateway provides access to a network called 10.0.0.0/8.

    I've set up static routing in pfsense and created a firewall rule that allows LAN-> 10.0.0.0/8

    Now it gets weird:
    tba@emu:~$ ping 10.64.1.162
    PING 10.64.1.162 (10.64.1.162) 56(84) bytes of data.
    64 bytes from 10.64.1.162: icmp_seq=1 ttl=56 time=4.77 ms
    64 bytes from 10.64.1.162: icmp_seq=2 ttl=56 time=4.88 ms
    ^C
    --- 10.64.1.162 ping statistics ---
    2 packets transmitted, 2 received, 0% packet loss, time 1001ms
    rtt min/avg/max/mdev = 4.771/4.826/4.881/0.055 ms
    [control-c]
    tba@emu:~$
    tba@emu:~$ ping 10.113.64.10
    PING 10.113.64.10 (10.113.64.10) 56(84) bytes of data.
    64 bytes from 10.113.64.10: icmp_seq=1 ttl=60 time=8.48 ms
    [control-c]
    --- 10.113.64.10 ping statistics ---
    1 packets transmitted, 1 received, 0% packet loss, time 0ms
    rtt min/avg/max/mdev = 8.482/8.482/8.482/0.000 ms
    [control-c]
    tba@emu:~$

    As you can see, both hosts reply.

    Now, if I try to ssh to them, only one works:
    tba@emu:~$ ssh root@10.64.1.162

    root@10.64.1.162's password:
    [control-c]
    tba@emu:~$
    tba@emu:~$ ssh root@10.113.64.10
    ssh: connect to host 10.113.64.10 port 22: Connection refused
    tba@emu:~$

    What's going on?

    If I connect my laptop to another port on the 192.168.100.1 gateway, I can ssh into both hosts.

    So something is worng in my box, but I can't figure out what it is.

    Any suggestions?

    Routing tables

    Internet:
    Destination Gateway Flags Netif Expire
    default 100.84.96.1 UGS nfe0
    10.0.0.0/8 192.168.100.1 UGS igb1
    100.84.a.b link#1 U nfe0
    100.84.x.y link#1 UHS lo0
    127.0.0.1 link#6 UH lo0
    192.168.1.0/24 link#2 U igb0
    192.168.1.1 link#2 UHS lo0
    192.168.2.0/24 link#5 U igb3
    192.168.2.1 link#5 UHS lo0
    192.168.3.0/24 link#4 U igb2
    192.168.3.1 link#4 UHS lo0
    192.168.100.0/24 link#3 U igb1
    192.168.100.10 link#3 UHS lo0
    193.162.159.194 192.168.100.1 UGHS igb1
    194.239.134.83 100.84.96.1 UGHS nfe0
    208.67.222.222 100.84.96.1 UGHS nfe0

    Thanks,

    /tony



  • You didn't tell us from which network you make your ssh connections. So the IP range from the lan.

    I would try to ssh into pfsense and open a shell. The use tcpdump to see if the ssh response reach the pfsense box. If the ssh answer packages don't make it to port wan2 oder don't go out there the problem is in pfsense. Else you need to search on a nether place.

    Also you might check firewall log. Maybe you got an asynchronous routing and there fore the firewall block packages because of unknown or wrong state. Icmp does not have session states.

    Hope that helps you.



  • Thanks surf

    Yes, sorry about that. LAN is 192.168.1.0/24

    This is the tcpdump of the one that's working:
    07:28:07.291298 IP 192.168.100.10.55145 > 10.64.1.162.ssh: Flags [S], seq 4185587354, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 1543212549 ecr 0], length 0
    07:28:07.295556 IP 10.64.1.162.ssh > 192.168.100.10.55145: Flags [S.], seq 3341411139, ack 4185587355, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
    07:28:07.295633 IP 192.168.100.10.55145 > 10.64.1.162.ssh: Flags [.], ack 1, win 513, length 0
    07:28:07.297550 IP 192.168.100.10.55145 > 10.64.1.162.ssh: Flags [P.], seq 1:39, ack 1, win 513, length 38
    07:28:07.301223 IP 10.64.1.162.ssh > 192.168.100.10.55145: Flags [.], ack 39, win 229, length 0
    07:28:07.305306 IP 10.64.1.162.ssh > 192.168.100.10.55145: Flags [P.], seq 1:40, ack 39, win 229, length 39
    07:28:07.305361 IP 192.168.100.10.55145 > 10.64.1.162.ssh: Flags [.], ack 40, win 512, length 0
    07:28:07.307958 IP 192.168.100.10.55145 > 10.64.1.162.ssh: Flags [P.], seq 39:1375, ack 40, win 513, length 1336
    07:28:07.309512 IP 10.64.1.162.ssh > 192.168.100.10.55145: Flags [P.], seq 40:992, ack 39, win 229, length 952
    07:28:07.309576 IP 192.168.100.10.55145 > 10.64.1.162.ssh: Flags [.], ack 992, win 505, length 0
    and so on.

    This is the one that's not:
    07:36:34.211123 IP 192.168.100.10.27377 > 10.113.64.10.ssh: Flags [S], seq 3380059398, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 1543719469 ecr 0], length 0
    07:36:34.222752 IP 10.113.64.10.ssh > 192.168.100.10.27377: Flags [R.], seq 0, ack 3380059399, win 0, length 0
    And that's it. So it sends a reset-package, but I don't know why.



  • I also tried http, same result:

    07:58:12.295711 IP 192.168.100.10.56799 > 10.113.64.10.http: Flags [S], seq 234521428, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 1545017553 ecr 0], length 0
    07:58:12.303827 IP 10.113.64.10.http > 192.168.100.10.56799: Flags [R.], seq 0, ack 234521429, win 0, length 0



  • Http is TCP on top of IP. So http ist just one step up in osi layer.


  • Rebel Alliance Global Moderator

    If your getting back RST, points to the device your connecting to sending that back.. Pfsense would not send back RST out of the box. Where exactly are you sniffing that at?



  • For me this looks like your box ssh box to which you connect did send the reset.



  • Hi guys,

    Thanks for your help, I guess it's because of a NAT thing, there is an extra layer between me and the host I cannot connect to.
    I'm off to go move some cables around :)

    Thx again

    /tony