Can't ping outside dns server from VLAN interface

[cesjr]

Hi guys
Below is my scenario.
I have a little problem about the ping issue.

On L3 switch (Cisco 3750)

• ping outside global server fail

• ping inside server success

• client reside on VLAN can access the internet. EX: browsing webpage

On FreeBSD router

• ping outside global server success

• ping inside server success

• client can access the internet. EX: browsing webpage

The Pfsense rules on LAN and outbound NAT are :

• Allow 10.1.1.0/24 to any

• Allow 10.1.2.0/24 to any

• Allow 10.5.1.0/24 to any

• Allow 192.168.1.0/24 to any

The attachment are tesing results.


[johnpoz]

so your vlan 1 is your transit?  To your L3, and you have network behind it.

So what are your rules on your pfsense interface that is connected to your L3 switch via this transit network?

So 192.168.1.247 is your gateway for this 10.1.2 network as well?

I am confused why you call it vlan 1 if its actually on your LAN interface?

[cesjr]

Sorry, The problem description is not clear enough.
The article has been updated .
Thanks your reply.

[johnpoz]

your ping sniff shows icmp unreachable?  Why is that from your dns box at .254??

Did you sniff on pfsense that your ping actually gets to pfsense?

[heper]

Rule limited to TCP/UDP ? That might explain why pings (ICMP) fail

[cesjr]

@heper:

Rule limited to TCP/UDP ? That might explain why pings (ICMP) fail

Thanks , there is no rules limited to TCP/UDP on LAN interface , should I allow ICMP permission on WAN to LAN ?

@johnpoz:

your ping sniff shows icmp unreachable?  Why is that from your dns box at .254??

Did you sniff on pfsense that your ping actually gets to pfsense?

Tracert from 192.168.1.254 to www.google.com was ok!

traceroute -d www.google.com.tw
traceroute to www.google.com.tw (172.217.27.131), 64 hops max, 40 byte packets
1  192.168.1.1 (192.168.1.1)  0.281 ms  0.376 ms  0.307 ms
2  h254.s98.ts.hinet.net (168.95.98.254)  0.793 ms  0.807 ms  0.787 ms
3  fyfy-3302.hinet.net (168.95.151.22)  0.796 ms  0.769 ms  0.758 ms
4  tchn-3012.hinet.net (220.128.17.38)  5.058 ms
    chch-3012.hinet.net (220.128.19.18)  4.901 ms
    tchn-3012.hinet.net (220.128.17.38)  2.019 ms
5  chch-3301.hinet.net (220.128.19.205)  2.954 ms
    chch-3012.hinet.net (220.128.17.29)  4.658 ms  6.603 ms
6  72.14.243.254 (72.14.243.254)  6.951 ms
    chch-3301.hinet.net (220.128.19.205)  2.437 ms
    72.14.243.254 (72.14.243.254)  7.400 ms
7  72.14.243.254 (72.14.243.254)  6.555 ms
    108.170.244.97 (108.170.244.97)  5.975 ms
    72.14.243.254 (72.14.243.254)  6.551 ms
8  209.85.142.13 (209.85.142.13)  6.872 ms
    209.85.240.15 (209.85.240.15)  7.366 ms
    209.85.142.13 (209.85.142.13)  6.673 ms
9  tsa03s02-in-f3.1e100.net (172.217.27.131)  8.904 ms
    209.85.142.13 (209.85.142.13)  6.306 ms
    tsa03s02-in-f3.1e100.net (172.217.27.131)  13.638 ms

I using sniffer to watch the icmp packet was actually gets to pfsense  but no response .
Either 192.168.1.247 to 192.168.1.1 or 192.168.1.1 to 192.168.1.247
See  corswitch_1.jpg / corswitch_2.jpg
It is strange  that code of icmp was 0 and it should be normal but still no response , the value of TTL gots no change.

I can ping  virtual IP on the other pfsense but the result of tracert still stop on 192.168.1.1

CorSwitch1#ping 59...25
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 59...25, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/10/25 ms

CorSwitch1#tracer 59...25
Type escape sequence to abort.
Tracing the route to domain.com.tw (59...25)

1 192.168.1.1 0 msec 8 msec 0 msec
  2  *  *  *
  3  *  *  *
  4  *  *  *
  5  *  *  *
  6  *  *  *
  7  *  *  *
  8  *  *  *
  9  *

Is that possible pfsense don't recognizant VLAN interface ?

[johnpoz]

"Is that possible pfsense don't recognizant VLAN interface ?"

Again what vlan interface would pfsense have setup???  Pfsense does not give a shit about any vlans or tagging?  Are you trying to tag your vlan 1 which is your transit network here??

Post your LAN rules - post your outbound Nat rules!

So devices on same transit network this 192.168.1/24 network can not ping pfsense???  And pfsense can not ping them??  .254, .247 ??  Then there is something wrong in your switching setup or firewall rules.

According to your sniffs pings were working!

Edgeswitch 1 is what?  Just a dumb switch since this is vlan 1 only (untagged traffic) L2???

Not sure why you have your freebsd router hanging off your L3??  Why not just connected to your L2 switch?  since this L2 is just doing your 192.168.1/24 transit network???  Your going to have issues with asymmetrical routing with devices in your downstream networks talking to stuff in the transit network!  And routing stuff to network connected to freebsd router bouncing off pfsense is wrong…  So freebsd has no route for 10.5.1/24?

What are you clients asking for dns??  Pfsense or .254??  So ask .254 from downstream - L3 sends it directly to .254 since attached.  But where does .254 send to get to downstream networks - has to send to pfsense.. Asymmetrical!!

Your dns is HOST, hosts do not belong in a transit network.. your 192.168.1/24 is transit - freaking HUGE transit, but putting stuff like dns in the transit is going to have issues with asymmetrical routing!!!  Why is your dns not on one of your other vlans?  And any router connected to via your transit need to have routes for all the other networks connected to other routers.  You have 3 routers connected.  But your freebsd is missing routes to network attached to your L3..  What is 10.1.1 wants to talk to 10.1.2??  Follow your path..  Its asymmetrical

[cesjr]

what vlan interface would pfsense have setup ?

There is no vlan interface on pfsense.

Pfsense does not give a shit about any vlans or tagging?

I don't give any vlans or tagging on pfsense.

Are you trying to tag your vlan 1 which is your transit network here

I don't tag vlan 1 on my network,vlan 1 is untagged native vlan.

So devices on same transit network this 192.168.1/24 network can not ping pfsense???  And pfsense can not ping them??  .254, .247 ??  Then there is something wrong in your switching setup or firewall rules.

Devices on same transit network can ping pfsense ,pfsense also can ping them.
.254 and .247 can ping each other including pfsense.

Edgeswitch 1 is what?  Just a dumb switch since this is vlan 1 only (untagged traffic) L2???

Edgeswitch1 is a cisco 2960 series switch , I have some vlan on them and vlan 1 is untagged vlan.

EdgSwitchB1#show vlan

VLAN Name                            Status    Ports
–-- -------------------------------- --------- -------------------------------
1    default                                active    Fa0/1, Fa0/2, Fa0/3, Fa0/4, Fa0/5, Fa0/6, Fa0/7, Fa0/8, Fa0/9, Fa0/10, Fa0/11, Fa0/12, Fa0/13, Fa0/14,
                                                Fa0/15 Fa0/16, Fa0/17, Fa0/18, Fa0/19, Fa0/21, Fa0/22, Fa0/23, Fa0/24, Fa0/25, Fa0/26, Fa0/27, Fa0/28,
                                                Fa0/29, Fa0/30 Fa0/31, Fa0/32, Fa0/33, Fa0/34, Fa0/35, Fa0/36, Fa0/37, Fa0/38, Fa0/39, Fa0/40, Fa0/41,
                                                Fa0/42, Fa0/43,  Fa0/47, Fa0/48
11  ACCOUNTING                      active
12  RD                                      active
13  MANAGE                              active
14  TESTING                              active
15  WIFI                                    active
51  PANTEC                              active    Fa0/20, Fa0/45, Fa0/46
1002 fddi-default                        act/unsup
1003 trcrf-default                      act/unsup
1004 fddinet-default                  act/unsup
1005 trbrf-default                      act/unsup

I'm going to move FreeBSD router to L2 , finally change 10.1.2.0/24 to L3 and let FreeBSD just doing DHCP job.
FreeBSD has a static route to 10.5.1.0/24 .
What does it mean of routing stuff to network connected to freebsd router bouncing off pfsense is wrong ?
Does this mean that if a packet is sent to the router it should be processed , rather than resent to pfsense ? (hairpin network?)

Client asking for dns is .254 , and .254 has stacic route to downstream networks.

#netstat -nr
Routing tables

Internet:
Destination        Gateway                Flags        Netif Expire
default                192.168.1.1          UGS        em0
10.1.1.0/24        192.168.1.247      UGS        em0
10.1.2.0/24        192.168.1.244      UGS        em0
10.5.1.0/24        192.168.1.247      UGS        em0
127.0.0.1              link#2                UH          lo0
192.168.1.0/24    link#1                  U          em0
192.168.1.254      link#1                UHS        lo0

How should I fix freaking HUGE transit network?
What does it mean of dns not on one of your other vlans ?
Does this  represents 10.1.2.0/24 to has its own dns server, for example 10.1.2.1.
If this is true , then I have to create three dns server for three different networks.

Sorry I forgot to say that there is a static route to 10.1.2.0/24 via 192.168.1.244 on L3.
So far, I manually add static route on all the routers.
In the end  I will only routing on L3.

Thanks your reply and suggestion. and sorry for my bad English and grammar.