Hmm, thats shown as mbps but can I assume it's actually Mbps?

Does the traffic graph in pfSense itself also show traffic during that iperf test?

If so It sounds like one of those devices on VLAN2 has the wrong subnet mask set and is sending traffic to it's gateway rather than directly.

@stephenw10 said in Comcast email doesn't load on iPhones when connected to network - works on PCs with same settings:

Ultimately try running a pcap on pfSense for the IP of the phone then try to check the email and see what it's sending.

I'll try - I haven't actually used pcap previously so will have to figure it out.

@BobL4002 so you can't go here?

https://currently.att.yahoo.com

does it resolve from your client?

$ dig currently.att.yahoo.com ; <<>> DiG 9.16.50 <<>> currently.att.yahoo.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 41641 ;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 1232 ;; QUESTION SECTION: ;currently.att.yahoo.com. IN A ;; ANSWER SECTION: currently.att.yahoo.com. 3532 IN CNAME atsv2-fp-shed.wg1.b.yahoo.com. atsv2-fp-shed.wg1.b.yahoo.com. 3532 IN A 74.6.143.26 atsv2-fp-shed.wg1.b.yahoo.com. 3532 IN A 74.6.231.20 atsv2-fp-shed.wg1.b.yahoo.com. 3532 IN A 74.6.231.21 atsv2-fp-shed.wg1.b.yahoo.com. 3532 IN A 74.6.143.25 ;; Query time: 12 msec ;; SERVER: 192.168.3.10#53(192.168.3.10) ;; WHEN: Tue Sep 03 13:21:59 Central Daylight Time 2024 ;; MSG SIZE rcvd: 159

what about in pfsense dns host lookup?

dns.jpg

@stephenw10 I just resorted to using opnSense, WiFi interface is detected straight out of the box through iwlwifi (It's FreeBSD 14) and no weird boot problems thus far. Thank you for the quick response, but I believe I might need some missing software/drivers or tweak the configuration accordingly, and I'm not getting much information on what's missing either, so I'm not going through that unnecessary rabbit hole.

@stephenw10 nope, there is no static routes defined.

@stephenw10 copy that. Along with over 300 APIs 🙌

@Matt2 That looks like https://redmine.pfsense.org/issues/15684
The fix is already in 24.08.

Well there are quite a few Unifi users here so you'll likely see more assistance setting that up. If you need it.

Both should work for this though.

@ILO_EWS traffic comes into the wan - that is what is connected to the internet.. Then pfsense would port forwarded it to the IP you set to forward to on your dmz..

If the traffic never hits your wan, kind of hard to forward it to your IP in your dmz.

This an example of a port forward.. You should have a port forward rule

portforward.jpg

Which would by default create the firewall rule you need

wanrule.jpg

So the port forward says any tcp traffic on port 23040, send it to my plex server that is on 192.168.9.10 on port 32400

The firewall rule allows this.. The only difference really in mine is that I limit what IPs can be forwarded to IPs that are in my pfb_allowPFb alias, which is the IPs that check if plex is available, both from plex check, and my own checks to notify me if plex goes down. And your IP also has to be a listed US ip.. All of my users are in the US.

But traffic never hits my wan ip, pfsense could never forward this. You will notice have forwarded 6.22GB of traffic through this rule since the last time those counters were reset.

If when you do a packet capture on pfsense - and do say a test from that can you see me site and you never see anything hit pfsense wan interface how could it ever forward it?

here via a packet capture I can see on my wan traffic come in on port 23040, and in my case an answer back - but if that traffic never comes in on 23040 pfsense could never forwarded it to be answered.

trafffic.jpg

@johnpoz nice 👍
Appreciate the quick reply

@stephenw10 Thank you!

I would probably setup VPN between the two locations so you can access it using the internal private IP. The client side can be at the dynamic site where it connect out to he fixed site.

I would also connect from home over VPN too.

Steve

@stephenw10

Hello Stephen, thank you for the feedback.

I read about the part here :

https://www.marvell.com/content/dam/marvell/en/public-collateral/transceivers/marvell-phys-transceivers-alaska-88e1543-product-brief-2012-07.pdf

I am not an electronics expert but what I read mentions only the eithernet ports, so yes, replacing it might not fix the problem completely.

My friend indicated that from the start but at this point, it might be worth the TRY if the part and labour are not too expensive.

I will keep you posted, thank you.

Jean-Pierre

Right. I see two workarounds.

Use netgraph ; this would achieve the result of stripping vlan0 from any traffic but also adds overhead. The whole goal over the last few years was to get away from netgraph (Thanks @cmcdonald ).

Use a switch between the ont and wan ports that strips vlan 0 tags. There's several out there that are known to work (https://github.com/owenthewizard/opnatt). This is the route I went.

This solution works well because the ONT is external. This won't work for those on gpon with sfp direct to firewall. Would need a similar arrangement for something in between the sfp and firewall to handle the vlan 0 stripping.

Hmm, might need a diagram here! If Proxmox is also tagging/untagging the issue could be in the config there.

Ok those settings are good. But you can see it has added automatic rules on the OPT1_TV intreface which implies there is a gateway defined on it still. There should not be a gateway on OPT1_TV.

@stephenw10
You were right, it was the ram !

I did a memetest and ended up with 4 passes but 0 errors. That was strange.

I ended buying a random stick of ram with same specs and replace it and also putting it in other ram slot. It's solid for 4 days now.

So what was shut down here, the Proxmox server? Not just the VM?

Does pfSense respond at the virtual console?

Does it show the expected IPs there?

Are you sure your traffic from LAN side clients is actually going through pfSense? Does it stop if you shut down the pfSense VM again?

Steve

Done.

em NICs are single queue. Only one CPU core can service the incoming and outgoing traffic queues. That means that on a 4 core CPU like you have it can never load all the cores.

igb NICs are multiqueue and here are attaching with 4 queues. Enough to load all the CPU cores sufficiently to prevent other services running.

You could override that by setting:

dev.igb.0.iflib.override_nrxqs=1 dev.igb.0.iflib.override_ntxqs=1

Or you could try to set a lower max interrupt rate like maybe:
hw.em.max_interrupt_rate=2000

But just swapping the WAN and LAN NIC assignments so LAN in em0 is probably easier. Unless you're not local to the box.