Epyc 3251 and Wireguard
-
What's the WG tunnel connected to there? Another pfSense install?
-
@stephenw10 said in Epyc 3251 and Wireguard:
Hmm, still showing issues in the Chelsio driver.
I assume that's only because my WAN is on the chelsio at the time. I didn't check when I disconnected the chelsio card but I would also assume it would've shown as the igb0 at that time.
But wireguard was no longer running on it when happened?
Probably the cause right there. WG shutting down when I try to use it?
-
@stephenw10 said in Epyc 3251 and Wireguard:
What's the WG tunnel connected to there? Another pfSense install?
Unfortunately not.
That tunnel goes to an opnsense box. At least until the vlan0 is fixed.
-
Hmm, so the encrypted WG traffic still runs over the Chelsio NIC, the WAN?
-
@stephenw10 Not really sure what you're asking there.
My WAN is on the chelsio card (cxl3), the WG tunnel comes up with handshakes, but as soon as I try to access the other side it crashes. -
Mmm, I'm unsure what you moved to igb0. I would have expected that to have to be the WAN for the WG interface to be running on it.
-
@stephenw10 I moved the WAN to igb0 and disconnected the chelsio card from the motherboard as a test.
The trouble still happened.
So I don't think focusing on the chelsio is the way to go.
It happens with the onboard nics also.
Because it still happened with the onboard nics, I reinserted the chelsio and moved WAN back to it. -
Right, I would agree except that it appeared the error was still on the Chelsio NIC even when it was not carrying WG traffic as I understand it.
It would be good to get a crash report from the igb0 as WAN setup if that's possible. It would be very surprising to see the same error on igb sicne many people are running WG with an igb parent.
-
@stephenw10 said in Epyc 3251 and Wireguard:
Right, I would agree except that it appeared the error was still on the Chelsio NIC even when it was not carrying WG traffic as I understand it.
How are you coming up with that?
-
@jarhead said in Epyc 3251 and Wireguard:
But wireguard was no longer running on it when happened?
Probably the cause right there. WG shutting down when I try to use it?
I may have read that wrong. But what I meant to ask there was; was WG running on the Chelsio NIC when that crash report was generated?
-
@stephenw10 I'll go through the whole thing again, trying to be more clear.
New router. Backed up old, restored on new changing interfaces as needed.
Wireguard would crash.
Moved WAN and LAN to onboard igb nic's.
Wireguard would crash.
Since this proves it's not related to the chelsio card, as it wasn't even plugged in to the motherboard, I reinstalled the chelsio and moved WAN and LAN back to it.
Wireguard would crash.
I found some weird errors in my gateways, as in network 1 was using gateway 2, and network 2 using gateway 1 when they should be 1 to 1 and 2 to 2, so I uninstalled wireguard then reinstalled it and recreated one tunnel.
Wireguard crashed and that's the dump I posted here.So focusing on the chelsio card seems to be not the way to go.
Have you guys used an Epyc 3251 in the office for testing at all?
-
Ah, OK. Sorry I misinterpreted the responses there then.
Is it possible to switch back to igb and try to generate a crash report?
The crash you saw there in cxgbe looks very similar to some we have seen in other drivers but that I expect to be fixed in igb.I'm not aware of any testing that has been done on an Epyc platform device.
Steve
-
@stephenw10 Definitely possible, just don't know when I can get to it. Got a lot of painting to do tonight so maybe I can play around as I watch the paint dry.
-
Ha, sounds like a good option!
I'll keep digging here, see if anyone has any suggestions.
-
@stephenw10 Boy that paint took a long time to dry!
Gave me a lot of time to try this out.
Kept going back to that config being messed up so I started from scratch.
New install, chelsio card not connected, just changed my network address with WAN on igb0 and LAN on igb1.
Installed Wireguard. It ran fine.
Installed Chelsio. still ran fine on igb interfaces.
Moved WAN and LAN to chelsio. Still ran fine!
So the Chelsio is not causing the issue.
Spent some time (a lot! long night) setting the config back to my usual. Not importing anything, had my old router up and used it as a reference.
Wireguard crashed.
In the process of setting up my old router as the new endpoint so I can rule out opnsense being the cause. (secretly hoping it is so I can get rid of it!!)Did get a new crash dump that might show something. This is still on the chelsio though.
-
Mmm, pretty much identical crash and it's in the cxgbe driver.
I have no idea how the WG encrypted traffic could be triggering it though.I'd be willing to bet it would not crash with igb as WAN. Though you said you were still seeing the errors logged with igb?
After reconfiguring it with your previous settings did it start crashing immediately?
Was it actually panicking and rebooting or just Wireguard erroring out?Steve
-
@stephenw10
But it did crash with the igb driver.
When I have a clean config (meaning no other interfaces assigned) it runs fine on both igb and cxgbe.
When I put my config back on it, it crashes on igb and cxgbe.
Gonna try disabling all other interfaces when I get a chance later. -
The firewall itself crashed and rebooted with the igb NIC as WAN?
-
@stephenw10
Yes, pointed that out many posts ago. That's why I keep saying stop focusing on the chelsio.
But I did make some progress.
I have WG working from pfSense to pfSense.
Don't think that matters because I did have it running to this same opnsense box.
What I'm thinking, and about to try, is that I've been recreating the tunnel I used with the old router to this opnsense junk.
I now created a whole new tunnel as a test between pfSenseseses.Gonna now try to create a new tunnel to the opnsense.
-
Hmm, no crash report from when it was running over igb though?
Just very odd that it shows as a crash in the driver not in Wireguard.
It must be some very unusual traffic the WG is creating and driver is trying to do something with. Hard to imagine what that could be though.
