@johnpoz said in datatransfer rate not as high as it should be:

So this was never about file copy speed, but only iperf tests?

I think it was actually about file transfers as well...
Here's from one of the earler posts..

@pleaseHELP said in datatransfer rate not as high as it should be:

I've noticed that sending data to that TrueNAS is always capped at 80 Mb/s, but getting data from it runs on about 1Gbit/s, as it also should for upload. This isn't just a TrueNAS thing.

@Gblenn said in Strange behaviour with pfSense and 4G:

There is no way any outside IP can ping your local devices, either through your ISP router or pfsense. Your mobile ISP would have no clue where to route such a request...

I agree with you, because these are non routable IP adresses.

I think there is only one explanation... that the phone is actually connected to the 172.16.10.1/24 network.
Either you have some VPN/Tailscale running which can connect it to your LAN. Or it is in fact actually on wifi although you think it's off.

Even though I didn't detect anything, this seems the most logical.

Or perhaps another explanation is that the mobile ISP is using 172.16.NN subnet for their mobile clients and you are pinging someone elses phone or tablet... Check your IP on the phone...

This hypothesis is not possible because I could ping only my devices. And I just checked again, my phone IP is on a different range : 92.184.x.y

I didn't indicate it, because I uninstalled the server in the meantime, but I did some tests with OpenVPN. Even if it seems impossible to me, it gives the impression of a persistence of the tunnel

Anyway, I just checked and everything is back to normal.

Thank you for looking into this situation !

@michmoor I have a pile of /32 routes I may try the xml file. Thnx.

I should add, if I add an entry into my computer's hosts file to directly resolve to the internal IP of the NPM server, the loading is very fast, so it's not an issue with the Nginx server or the NPM in front of it. This makes me think that maybe the solution is to put manual entries into PF's resolver, but it still bothers me that it would only happen on one set of sites. Also, I am trying to keep VLAN 10 completely isolated from the rest of the VLANs, so having to resolve to it goes against what I'm trying to accomplish.

@khb FWIW, I ran speedtest-go with the default options every 15min for a couple of days.

avg stdev. min max download 128.3 49.5 40.3 309 upload. 11.6 5.4 5.7 33

The test was executed from the netgate, using only the starlink linked interface. A few sanity checks running the starlink native (naive, not advanced) test within a minute or so of the speedtest runs varied (sometimes matching a result from 15m before or after). The variability is large, which I expected, but varies faster than I'd have guessed.

Couldn't get dante to work until I found this. For those of you sportsfans keeping score at home, this is still valid/needed for pfSense version 2.7.2-CE and dante-1.4.3_2 circa 2/2025.

@phantom99b And what is the 2nd hop from your 192.168.6.1 if that is your pfsense box at 6.1 that is your isp - get with them.. who is 192.168.8.11 ? is that you you're saying? as you see you got an answer from it - so its not "blocked" That bogon block is inbound only when that is the source traffic and unsolicited into your wan.

Those could be loopback in your isp network, or where your pfsense box sits. But pfsense does not use that IP out of the box that is for sure. And just because its meant for documentation doesn't mean it stops someone from using it in their network.

pfsense will try and get there - it will send that traffic out your default gateway.. Here

traceisp.jpg

That 2nd hop is in my isp network. I use to get rfc1918 back address from my isp. on the 2nd hop.. Your isp can use those IP ranges in their network.. doesn't mean they will route past your isp network, etc. Then my isp was bought by another and they changed up their network, and no longer get those. Might have only been during the transition after the purchase.

rfc1918, bogon and special networks can be used in your own local network, so yeah your isp could use them as transit networks or loopback addresses on their equipment. But you creating traffic to them would be allowed to be answered by the state pfsense creates to try and get there.. That bogon rule only blocks unsolicited inbound traffic to where you enable it.. Out of the box that is only on the wan interface of pfsense.

Use to do a lot of work with a SDwan company - their devices would use 192.0.2 as their tunnel address space.. So they were sure it shouldn't step on any address space customer was using.

look in your arp table on pfsense right after you see that - what is the mac address of that 192.0.2.1 address - that would be the mac of the device connected to your pfsense out your interface you sent the traffic.. You mention vpn, that could be IP address used in the tunnel, etc.

Okay, so after a very long night and few coffees this morning, its now fully working without an issue, I can unplug the WAN cable and reconnect it, it jumps back up straight away. Rebooting the modem, the same, it waits for the link - once its there it reconnects and remains connected.

I first did a straight forward Factory Defaults reset, that didn't work. Next, did a fresh pfSense CE install, and that also has not helped, the same behaviour was repeating itself. I also tried the "forbidden fruit" fork, that didn't work either - same behaviour.

I still had the option to install pfSense plus from the days when I had the home lab license, so I installed that and the issue disappeared.... Rock solid. I set everything back up, all the packages I use and it still is working as expected. I have scanned through the changelogs / tickets on Redmine but couldn't see anything specific relating to this issue so my only conclusion is that something must have changed within FreeBSD itself.

How the logs look now, I have highlighted the event that never showed up before.

Feb 11 01:36:20 php-fpm 540 /rc.linkup: Hotplug event detected for WAN(wan) dynamic IP address (4: dhcp)
Feb 11 01:36:20 php-fpm 540 /rc.linkup: DEVD Ethernet attached event for wan
Feb 11 01:36:20 php-fpm 540 /rc.linkup: HOTPLUG: Configuring interface wan
Feb 11 01:36:20 check_reload_status 653 rc.newwanip starting ix0
Feb 11 01:36:20 php-fpm 540 /rc.linkup: Gateway, none 'available' for inet, use the first one configured. 'WAN_DHCP'
Feb 11 01:36:20 php-fpm 540 /rc.linkup: Gateway, none 'available' for inet6, use the first one configured. 'WAN_DHCP6'
Feb 11 01:36:20 check_reload_status 653 Restarting IPsec tunnels
Feb 11 01:36:21 php-fpm 20277 /rc.start_packages: Restarting/Starting all packages.
Feb 11 01:36:21 php-fpm 20277 /rc.newwanip: rc.newwanip: Info: starting on ix0.
Feb 11 01:36:21 php-fpm 20277 /rc.newwanip: rc.newwanip: on (IP address: xxx.xxx.xxx.xxx) (interface: WAN[wan]) (real interface: ix0).
Feb 11 01:36:22 php-fpm 63053 /rc.newwanip: Resyncing OpenVPN instances for interface LAN.
Feb 11 01:36:22 php-fpm 63053 /rc.newwanip: Creating rrd update script
Feb 11 01:36:23 php-fpm 20277 /rc.newwanip: Gateway, NONE AVAILABLE
Feb 11 01:36:23 php-fpm 20277 /rc.newwanip: Gateway, none 'available' for inet6, use the first one configured. 'WAN_DHCP6'
Feb 11 01:36:23 php-fpm 20277 /rc.newwanip: IP Address has changed, killing states on former IP Address 0.0.0.0.
Feb 11 01:36:24 php-fpm 63053 /rc.newwanip: Netgate pfSense Plus package system has detected an IP change or dynamic WAN reconnection - 192.168.2.1 -> 192.168.2.1 - Restarting packages.
Feb 11 01:36:24 check_reload_status 653 Starting packages
Feb 11 01:36:24 check_reload_status 653 Reloading filter
Feb 11 01:36:24 check_reload_status 653 Reloading filter
Feb 11 01:36:25 php-fpm 3134 /rc.start_packages: Restarting/Starting all packages.
Feb 11 01:36:25 check_reload_status 653 updating dyndns wan
Feb 11 01:36:25 check_reload_status 653 Reloading filter

I have also followed your suggestions and removed remote DNS servers, only using Resolver now.

Thank you for your help!

@viragomann

wan1 fiber is the most stable and fastest connection. 1gbps up/down
wan2 cable is a more unstable 1gbps/50mbps connection.

vlan01 faculty has to have the fastest connection both up/down
vlan02 guest can live with cable modem.

now cable the cable modem goes down about 20-40 times a year. And cannot support entire campus (amount of sessions, isp problem)
fiber we only have this school year has been down once because of road works.

faculty schould be guaranteed online so fiber -failover to cablemodem.
Cablemodem cannot support total campus so guest network should be disabled in this event.

@Bob-Dig You can set the gateway to any IP you want, but its not a normal, typical setup - and like the warning I showed you - the OS thinks you must of made a typo ;) It might still arp for it since its gateway IP and has to be reachable on the same L2

arp.jpg

But you can not be sure device or OS will do that. Nor can you be sure what your arping for will answer if the IP is not on the gateways network IP range.

@lowbug

Before the rule you can see 'counters'.
Like these :

ec743d53-134a-416d-9504-ad11fa52b0f6-image.png

If it stays at 0/0, the rule wasn't used ... = no matching traffic.

The current routes too
f7e0340e-76a2-4ded-aa95-4eafa084847e-image.png

@Zerejekim said in Routing across PFSense Interfaces or VLANs not working:

Outstanding!!!!!

Can't thank you enough.

Great, I also have a Synology here, it is old, but still works, DS218+ with two Ironwolf 2TB drives in a RAID 1 config.
More than 5 years with this guy and never lost a file, scrub once a month and that is it..

@jagradang
Let us know what you end up with.

What's the work-around that you have with aliases?

I'm currently experimenting trying to see whether I can get the Unbound for pfsense, to send queries down different VPNs - depending on which VLAN sent in the request. ChatGPT says it's possible, but, I'm yet to see it work. My workaround, was spinning up different adguard instances for each VLAN, and they work fine per vlan - per vpn. Prefer to keep things on pfsense if that's even possible. But don't know.

@padrino121
Drop the WAN circuits on a switch will work as far as not having to physically go on site but unfortunately the FRR configuration still requires manual intervention.
8300s is expensive gear. I would follow up with sales and see if they can provide a better solution. The money you paid it’s really unacceptable that there is this level of shortcomings especially with something so basic

This was too disruptive to the client to troubleshoot this further. We sent someone out to do a clean install of 2.7, then load the config back in. No issues since.

@duvel said in Can't ping 8.8.8.8 or google.com:

I would rather select TCP/UDP & ICMP than allow Al

This works just fine, and is the default :

828b4fd3-e69a-4749-9271-5aa975251d92-image.png

because you don't trust the other 252 ?:

@chrisjx Hi,
I also have a location with two ISPs, one is the primary and the second is a Starlink.
So I know how to setup the LAN4 as a OPT and assigned VLAN 40 to it. But how do I make sure the Starlink is on VLAN 40 then?

Did you managed to get this working?

BR
Nick

@manjotsc great job finding the issue, I had this machine that had a line on the monitor once, guys before me replaced the monitor the cable, I got there and took the cpu off it had a bent pin no lies reseated it or got a new one I can’t remember but that fixed the issue, it is amazing I didn’t understand why everything else worked, one pin caused the issue, also over doing heat compound can cause issues when it gets on pins.