Static route configuration: no Internet access on Pfsense + smart switch

[lfred]

Hello to the Members.

Below is a technical question from a learning person (a bit more than a beginner...), but first my true compliments to the creators of Pfsense and the possibility to use the software, which in my case for a basic home use.

I decided to make a donation to Netgate, but because of the post from Netgate’s website (17-02-2015) which implies that its better to give a bit support to FreeBSD, so I did it.

Lets back to the topic. After installing of Pfsense, which seems to run very stable on my old PC I have difficulties to configure a static route in Pfsense, which is needed to let function a VLAN made on another switch.

The idea is put some unsafe ioT's ‘away’.
Here is a kind of topography map I wish to realize:

Internet
:
:
WAN = 103.18.10x.x
[pfsense 2.5]
LAN = 192.168.1.1
:
:
[“smart” switch Netgear MX510] connected to the LAN interface in Pfsense
IP = 192.168.1.100
GW = 192.168.1.1
SubNetMask = 255.255.255.0
:
:
Default Vlan1 = 192.168.1.100, SubNetMask = 255.255.255.0,
Vlan60 (IP interface) = 192.168.60.1, SubNetMask = 255.255.255.0
:
:
[test PC, with manual settings]
IP = 192.168.60.2
GW (or next hop) = 192.168.60.1
SubNetMask = 255.255.255.0

I followed the steps described here to prepare the static routes on Pfsense:
https://docs.netgate.com/pfsense/en/latest/routing/static.html#figure-static-route-configuration

The settings in the System > Routing > Gateways are:
Interface: LAN
Addr Fam: IPv4
GW: 192.168.1.100

Settings in System > Routing > Static Routes are:
Destination Network: 192.168.60.0/24
GW: 192.168.1.100

The settings above seem to have some connection with the VLAN60 on the managed switch, I made these tests:

1. I can ping Pfsense IP (192.168.1.1)
2. I can ping 8.8.8.8 or sim.
3. I can see some relevant ICMP traffic (from IP of VLAN60) while looking at the Status>System Logs>Firewall>Dynamic View in Pfsense, which is ok - green (allow), but only if I use Ip numbers to ping.

And, I can not ping google.com or netgate.com, I can not see those ping-attempts in Pfsense’s System Logs and there is also no Internet on Vlan60.
I can not find it out why! I did temporary turn off the firewall to test if the firewall would block my VLAN60, but without any progress. It seems that only ping-traffic to P numbers goes there through… And after many days of trying and testing (also reading posts and manuals) it brings much disappointment.

The main questions are: how can I find out what blocks my Vlan60 from Internet? Is that still some rule in the Firewall of Pfsence? Or did I made mistakes while configuring Vlan60 on the switch itself? Or do you have other tips?

I really hope that you can help me with this.

Your assistance is highly valued!

LFred

[lfred]

Hello Pfsense-friends!

I’m sorry to ask again, but my level of knowledge does not permit me to solve the problem by myself. I put a lots of time in this issue already, because I’m not lazy at all and I try to read here as much as possible. But somehow other similar posts on the forum did not work for me….

Can someone just give me a hint where I should continue to search to find what blocks Internet traffic from my static route: wrong settings in Pfsense (like wrong Gateway...) or something in the switch? At least Pfsense would give me a kind of monitoring about where those static route-attempts go (if those attempts would leave the ports of the switch).

I made a print-screen of the routes (Diagnostics > Routes), maybe it could give you an idea…

As you can see there, the ping-traffic (under column Use) is active: if I try to ping IP’s - it counts the numbers of pings... but not if I try to ping a domain.

Please assist a bit.

Thank you in advance.

LF

[viragomann]

@lfred
Seems the VLAN60 cannot resolve hostnames.
You may have to provide a DNS.

Also make sure that the firewall rules on LAN interface are allowing access. The default LAN rule may only allow traffic from LAN network.

BTW: Why do you route the VLAN on the switch? You may get a higher usability when you terminate it on pfSense.

[lfred]

@viragomann
Hello VG,

Thank you greatly for the input!

I’ve checked the rules on LAN: there is a rule (made by clicking on + in Easy Rule: Passed from Firewall Log View) allowing all traffic to LAN from VLAN20.
I’ve checked the DNS setting in System > General Setup. It seams ok.

I’m adding some screens, those give some additional info. There is one thing: in column Source “cannot resolve”... Do you think that there are still wrong settings? (Pff, getting tired of this…. Many hours of trying for today are away...)

Next, I wish to answer your question about the reason of using a switch.

I’m music-man and wish to keep audio-samples on my NAS while rehearsing, but I experience regularly jumps in latency (and therefore difficulties to playback samples in real-time, I need to buffer it every time locally, which mostly not necessary). Somehow it starts (I believe) when other family members use the network/router as well, even if they do not sit on the same interface (subnet)...

So, my motivations to use the switch are:
1, Hope to unload the Pfsense-desktop and use a smart switch for simple static routing, which would only forward the packets between my NAS and DAW (of course the DAW needs sometimes access to Internet);
2, other family-members use a few ioT’s and I wish to separate those from my lab;
3: my Pfsense system is an old-desktop, I can not upgrade it for a better one yet, but seems perfect for the firewall/routing and PfblokerNG.

Do you believe that I do not need a smart switch to achieve this? I wish to keep a fast LAN response.
That switch is here for testing, if it doesn't needed, I’ll preferably remove it and I’ll use an unmanaged switch… Keep it simple.

I you have some ideas, please share it, at least in a few words.

Thankful regards,

LF

[viragomann]

@lfred said in Static route configuration: no Internet access on Pfsense + smart switch:

I’m adding some screens, those give some additional info. There is one thing: in column Source “cannot resolve”... Do you think that there are still wrong settings?

If the client complains that he cannot resolve names, so there is either no DNS configured, the configured DNS server does not respond or is not reachable.
Depends on the network settings on the VLAN NIC on the device.
Since your switch provides that VLAN, if it also provides DHCP and the client is set for automatic NIC config, you have also to provide the DNS on the switch.

Of course with the switch you can relieve the router. But there must be really much traffic to overload it. I think it may be more likely that there is something set up wrong on the router. But to find out, you have to watch some parameters like CPU usage, memory, MBUF, NIC traffic.

@lfred said in Static route configuration: no Internet access on Pfsense + smart switch:

I’ve checked the rules on LAN: there is a rule (made by clicking on + in Easy Rule: Passed from Firewall Log View) allowing all traffic to LAN from VLAN20.

That rule only allows traffic from one single source IP.

[lfred]

@viragomann
Thank you VM a lot for the extended replay!

I'm sitting for hours and checking the manuals and the system configurations both from the switch and Pfsense, but can not find other (than there already were) DNS settings which would prevent the traffic from the Internet... Also, I've adapted the rules in the Firewall to a Network alias instead of one IP. All without much success yet...
Tomorrow I will add more print-screens because its late here...
In any case, Thank you again for your input!

LF

[lfred]

Hello VM,

I listened to your advice (about creating static routing all in the Pfsense without the switch) and spend some time with testing/monitoring of the system (especially the performance state of the CPU / RAM) and I discovered, indeed, that the memory state is sometimes getting bottlenecked while using PfblorerNG and especially TLD! So I disabled the TLD and the system runs now acceptable without of annoying latency.

It still did not solve my issue yet, but I think I will remove the switch form the setup and go to learn how to configure it all in one single system.

If it will not work, I will let ask again, but I hope it won't needed.

Many thanks for your time and all shared information, its already very useful.

LF

[johnpoz]

When you create downstream networks.

Unbound, acls will not auto allow these downstream networks via its auto acls.

The auto acls only allow networks directly attached to pfsense to query it.

If you want downstream networks to query for dns on pfsense, then you would have to create acls in unbound to allow that.

[lfred]

Oh, thank you JP!

I expected that somehow... but not in Pfsense...

Few days ago, while trying config that switch, I found some info Internet about ACL's and I did configure it on the switch (just a basic ACL that lets traffic from the IP of VLAN60 through, print screen attached).

But I didn't know what further is blocking the Internet-access and I could not find any hints in the logs of Pfsense! I started to believe that the switch is broken... Its really pain...

Ok, I will try to make an ACL in Pfsense, because I spend many days for it, all without success. Really.

Thankfully,

LF

[lfred]

still no success...
ping: cannot resolve yahoo.com: Unknown host

Only IP pings come through.

Here is my basic ACL on Pfsense, tried both Allow and Allow Snoop:

[JonathanLee]

@lfred

Did you enable port 53 for DNS?
Domain Name Service it will resolve URLs to IP addresses without it no Internet will work.

Here is a copy of my very basic firewall rules. I have DNS set up for anything from Lan can access port 53.

[lfred]

Hello

Jonathanlee

Thank you a lot for this contribution, please excuse me for such delay... (I was too busy with teaching music..).

Unfortunately I could not configure that portion of my network (also because I decided after many days of trying not to use VLANs, but make everything as easy as possible in Pfsense only), so I sent that smart switch back to the store.
Still I hope that your contribution will aid someone with the same question, because I've burned too many hours and could not manage it alone, so it still hurts...

Again, thank you!
All the best,

@lfred

[johnpoz]

@lfred said in Static route configuration: no Internet access on Pfsense + smart switch:

Netgear MX510

I think your problem was most likely due to your switch not actually being a L3 switch..

From this thread
https://community.netgear.com/t5/Smart-Plus-and-Smart-Pro-Managed/MS510TX-trick-to-getting-routing-to-work-correctly/m-p/1885049#M15144

This stated..
This type of switch can only work with VLAN routing per interface. It does not support all the members of the VLAN as it is only a L2 switch. If you wanted to have VLAN routing working on all of the devices, then you may need a fully manageable switch.

So if you would of just used it as L2 and let pfsense route the vlans you wouldn't had any of your issues you were running into.

[lfred]

@johnpoz

Thank you Johnpoz!

I suspected that there was something wrong with the switch, really. But they told me that its a really L3 layer device and therefore I was trying it for too long... So its actually a L2+ one. Hmmm...
In any case, that simpler setup within Pfsence only compensates all my earlier frustration, because it works better any smart switch (in my little music-lab).

With many thanks and greetings to this forum,

@lfred

[johnpoz]

@lfred yeah in the data sheet they use the term "Layer 2+/Lite L3 features"

If you would of just used it as L2 and done routing on pfsense between your vlans/networks you would of had far less trouble..

Routing at the switch level is almost never needed in any sort of home setup.. Unless what you have doing your routing is not really capable of routing at wirespeed.. And you really want some devices on their own vlans. But your really going to have way less ability to actually firewall between these segments. Even with a fully managed L3 switch, I have one the ability to limit traffic between these vlans is difficult and convoluted.

If you want to try vlans again - there are many entry level smart switches that can do vlans in the $40 price range.. which prob way less than that netgear you had.