@chrisjx Hi,
I also have a location with two ISPs, one is the primary and the second is a Starlink.
So I know how to setup the LAN4 as a OPT and assigned VLAN 40 to it. But how do I make sure the Starlink is on VLAN 40 then?

Did you managed to get this working?

BR
Nick

@manjotsc great job finding the issue, I had this machine that had a line on the monitor once, guys before me replaced the monitor the cable, I got there and took the cpu off it had a bent pin no lies reseated it or got a new one I can’t remember but that fixed the issue, it is amazing I didn’t understand why everything else worked, one pin caused the issue, also over doing heat compound can cause issues when it gets on pins.

@Bob-Dig said in Bogon Alias address created for pppoe WAN:

You can see, there is no relationship between the IP-address on my WAN and its gateway.

I wouldn't say that - that is not how a typical network is setup.. Normally the gateway off a network, needs to be on the network your trying to get off of, how else would you get there. But yes you can go against normal practice. Multiple layer3 on the same layer 2.. Where the IP is different layer 3 but your on the same layer 2 network so the client can get the mac of its gateway.

PPPoe is a tunnel so its a bit different. But normally the gateway would and should be on the same network space.. If its not you normally have to let the client know this.

If you go into routing and, and look at your dhcp gateway - if that IP is outside your network you get - you would normally have to set this.

gateway.jpg

https://docs.netgate.com/pfsense/en/latest/routing/gateway-configure.html#advanced-gateway-settings

The Use non-local gateway through interface specific route option allows a non-standard configuration where a gateway IP address exists outside of an interface subnet. Some providers attempting to scrape the bottom of the IPv4 barrel have resorted to this in order to not put a gateway into each customer subnet. Do not activate this option unless required to do so by the upstream provider.

Normally if you manually set that - the OS should scream at you.

warning.jpg

So while it "can" be done - it is not your typical setup..

When a client decides that hey this IP is not on my network, I need to send it to my gateway - it would arp for the mac address of the gateway (that is on its network) and send the traffic there.. In a typical setup if the gateway is not on the same network, how would the client get the mac to actually send the traffic to.. So there are some caveats to be sure if doing such a setup.. So please don't get the idea that you can make your network 192.168.1.0/24 and just set a gateway as 192.168.2.1 and it will work.

@Bob-Dig Because I still saw all the traffic still going out my WAN1 interface and my WAN2 interface is idle.

@0x010C LAN should typically NOT show up as a gateway in that list... You can have a gateway in the LAN segment, like a standalone VPN server or similar. In that case you set up a static route to it though...

Are you saying that C automagically became a default gateway when you created it? Have you tried changing the default, saving and changing back again?
Also, under gateway group you can create like a failover group, using A, B and C, and setting A to Tier 1 and the others at some higher Tier 2 and 3. Then use this group as the default gateway. All normal traffic wil then go through A, unless A is down. All policy routed traffic will go as per the policy... through B or C.

Hi, I made the 11. address up, the customer does run a range.
Sorted it, I required a rule in the netgate firewall to allow the 192.168.2.x,192.168.253.x, 192.168.6.x in from the lan side. The netgate was only expecting its own lan subnet not the subnets in the diagram to the left of that.

yay! Thanks all :)

@totalimpact said in Pfsense cannot port forward to Layer 3 switch:

having static routes requires a gateway on the Transit network.

Not on the interface - you create a gateway to the IP on the transit network, but you don't actually put that gateway on the interface of pfsense on the transit.. Or pfsense thinks a wan interface and creates an outbound nat on it.

You create the gateway in the routing gateway section not on the specific interface.

pfsense-layer-3-switch.png

@seanr22a said in IPsec routing problem:

I have three web sites including Nextcloud on the server at siteB and they are behind Cloudflare CDN (free version). I use an Apache reverse proxy at siteA now to get around the port blocking issue (Sending the traffic over the IPsec to the server at siteB). The ping time is around 230ms and I get around 10Mb up and 45Mb down from siteB to siteA. I spend most of my time in Thailand so the speed I get here is most important.

I get the Proxy setup, that's what I use to access my NextCloud server, as well as my Homeassistant and some other stuff. I just happen to use Nginx.

But I'm not sure I understand how Cloudflare CDN fits into this setup that you have?
If you host your server at your home in Thailand, and you access it via Sweden using some DynDNS service to find your Swedish IP, then you go directly via the VPN to site B. Where does Cloudflare come into play?

And I'm curious, which ISP is it, and which ports do they block? And what ports don't they block?

I've seen that many users say nginx is faster and use less resources but in my very small setup I really don't think it matters.

I agree, probably wouldn't make a noticeable difference if you changed. If you are curious however, and use docker, it's actually super simple to set up and has a very intuitive UI...

BUT, what could potentially improve performance quite a bit is if you change VPN to Wireguard. Depends on what HW you run pfsense on of course, but on smaller machines I can see a real difference even at moderate speeds.
I have a site with pfsense running on a tiny PC Engines APU2 and I can saturate the 250 Mbit connection to that site over Wireguard. But on an IPSec connection I can perhaps get 80-90 Mbit when testing with e.g. iperf or openspeedtest.

Looks like this is starting to happen again.

However it is limited, only some traffic is being routed over the backup connections.

@patient0 said in What is wrong with my routing?:

fgrep 62.155.245.31 /cf/conf/config.xml

shows no (=empty) output

but a 'cat /cf/conf/config.xml' reveals that the version of the config file (line 3) is "23.6".

@johnpoz well, yeah unfortunately. I was looking to strengthen my current setup but it seems there is nothing I can do for now.

Make sure when you are switching devices behind the modem that you hard reboot the modem as it will stick to one MAC address at a time.. when it is not in bridge mode it becomes that one MAC address by itself so you don't have to worry about the reboot process. But in this case pfSense is the router and the interface of your win needs to be that MAC address..

@Bob-Dig I am not said wireguard, i am said the WAN.

@stephenw10 said in Routing a service to non-default WAN:

When the other side initiates reply-to should apply.

However the auto rules do not appear to add reply-to. Which is interesting.

Pfsense is the responder in my case. Yes, it confused me a lot :) I was about to introduce some wild workarounds. So glad we got it working! Thanks a lot to both of you guys.

@tgl I looked into that. They combine TV and Internet service and the non-residential TV service sucks. That's why I went this way. At present, it is only personal playing with software development and the extra expense was not warranted for having 2 internet services.