Strange Routing Issue

[stephenw10]

Unclear what you're asking. You should be able to establish the tunnel from the cell modem side as long as the other end has a known public IP it can listen on. Once the tunnel is up you can access things across it either way.

[ahole4sure]

@stephenw10
OK. thank you - I know stupid question - but why do a site to site in the first place then if all can be accomplished the other way?

[stephenw10]

What other way do you mean? Using Tailscale? No reason if that works for you. As you said it may be faster with a direct WG tunnel. But you may not need that additional bandwidth.

[ahole4sure]

@stephenw10
No I meant that I didn't know I could establish a peer to site with the peer being behind CGNAT (in pfsense). -- is that what you are saying that I should be able to do that ?
But does that allow access from the non CGNAT site into the network of the CGNAT site ??

[stephenw10]

It's still site to site it's just that the tunnel can only be established from the side behind CGNAT. But, yes, once the tunnel is established connections can be opened across it both ways.

[ahole4sure]

@stephenw10

Thanks as always for your help!!!!

I am back to the setup again - and still trying to get past the same issue that I had initially (whhere I can't seem to get a hanshake)

I have a screenshot of the status page of the device that is behind CGNAT that should be initiating the tunnel

I relaize that there are many things like the WAN rule allowing 51825 in , etc
But is there a good way to troubleshoot WG connections that won't do a "handshake". (now that I know it should happpen going out from the CGNAT device)

1st screen is the remote CGNAT device status
2nd is the one with public IP

I can send screenshots of Rules or anything just to get me past the 15 to 20 hrs I've spent on this.
As you might can see -- I have been succesful in setting up WG connections before - both site to site and client to server

[stephenw10]

Are you now able to ping from the client to the server outside the tunnel?

I would check the states at both ends and filter by :51825. Make sure you're seeing WG opening states at both sides. And that they have two way traffic on them.

[ahole4sure]

@stephenw10

Going crazy here
I , TBH, am not sure whether I have been able to ping my public IP in the past or not. But I made sure I had created a WAN rule for allowing ping.
I am not able to ping from my laptop using another internet source.
I am not able to ping from the "client" pfsense box that is behind CGNAT

As far as the States - I sent screen shots form the running site to site on port 51821
AND I ssent screenshots from the non working connection with the client behind CGNAT 51825

Looks there is some partial traffic but not 2 -way traffic!

NOT SURE about the ping thing -- I have seemingly enabled ping to pass but I can't seem to ping my firewall from any device or any internet source BUT I have running services that are easily accesible from outside - that are coming in via https and haproxy

[SteveITS]

@ahole4sure ICMP to the public WAN IP from "any"/Internet would be allowed on the WAN interface which doesn't have that rule in your screenshot (may not be the right router).

ICMP to the public WAN IP from LAN would be a rule on the LAN interface, though that has a default allow all rule.

ICMP from within the VPN would be a rule on the Wireguard interface.

ICMP/ping is not required for other services to work.

[ahole4sure]

@stephenw10

Ok so some weird info --

I continued to not be able to ping - even with the rule (I think the screenshot you were referring to was my other router that I had not made that rule yet)
That made no sense -- so

I just temporarily disabled packet filtering all together -- result

1. I was then able to ping my main router
2. the previously "non-connecting" tunnel from the router behind CGNAT made the connection

I then renabled packet filtering

• became unable to ping my main router - depspite rule being in place
• the WG tunnel remained connected!

Seems like whatever is responsible for my "firewall" is not working correctly. I had rebooted last night and that didn't cause the incoming from CGNAT 51825 traffic to start -- but disabling the filtering did
?? - any suggestions
PS -- this pfsense install was from a complicated rough reinstall because the interface names were different -- the backup was hard to use ( that restore was a few months ago)

[SteveITS]

@ahole4sure What are the 42 alerts at the top? ;)

The 0/0 in the Status column means no matching connections at least since counting started.

Try a Diagnostics>Filter Reload.

https://docs.netgate.com/pfsense/en/latest/troubleshooting/firewall.html#new-rules-are-not-applied

[ahole4sure]

@SteveITS

I'll try the filer reload thing
Thanks

The alerts are an error that kept coming up after I restored a backup - with some mismatched interface assignments - etc

Do you have any suggestions on how to get rid of the error?
Can I get rid of the issue at an ssh level ??

If I recall right it is a rule on an interface or network that doesn't even exist and is not shown in the GUI

[stephenw10]

The tailscale interface should not be assigned.

[ahole4sure]

@stephenw10

While you guys are following this thread
Off topic
But an alternative for me to keep working on my new setup — can you guide me on how I could connect the WAN port on my second pfsense device to my existing LAN so that I at least have better internet but get to stay in my “new” and separate network
When I connect the wan (set on dhcp) port to a switch connected to my first pfsense lan I am not able to get internet
Is there something special that needs to be done to access internet from the second network and pfsense device?

[stephenw10]

Could be a subnet conflict if they both have the same LAN subnet. That will be an issue when you try to tunnel traffic between them also if so. Change the LAN subnet on the new device to something unused.

Otherwise nothing special should be required there. The default WAN setup as DHCP should work fine behind another pfSense install.