Trouble with WAN not being able to get correct speed

kilasin

@stephenw10 i tried testin iperf from another VM which i get the real speeds then i tried connectin my makeshift router to my machine directly good speeds just the WAN. However, when i put the isps router i get the exact 2G speeds but unfortuantely the ports are only 1 G only. Hence, I made this router from my old machine 4 10gs cards and 1 2.5G but im gettin capped at 1G speeds which suck. I will try that command thanks.

kilasin

@stephenw10 There it is.. You are correct i was only connecting at 1G speeds check this out :

Now how to change that is the next step lol any help would be greatly appreciate it

Jarhead

@kilasin As I said, verify the device you're plugging into first.
There may be a 1g port and a 10g port on it.

stephenw10

So you're passing that NIC through to the VM?

What is the actual connection here, the physical link?

Try running: ifconfig -vvvm ix0 to see what speeds it can link at.

kilasin

@stephenw10 Yes, I am passing the 10G nic through the VM so Unraid does not see it.

The actual connection is being connected to 10G nic card. Here is what I ran from your command

ix0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=48138b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,HWSTATS,MEXTPG>
capabilities=4f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
ether 6c:92:bf:48:c0:a0
inet6 fe80::6e92:bfff:fe48:c0a0%ix0 prefixlen 64 scopeid 0x1
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
supported media:
media autoselect
media 100baseTX
media 1000baseT
media 10Gbase-T
nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>

It appears to auto select the 1G speed, how do I change that into the 10G speed?

stephenw10

It's set to auto negotiate and is negotiating 1G. So the port at the other end might be 1G or the cable might not support 10G.

If you set the port to 10G fixed you may need to set the other end to 10G fixed also and that may not be possible.

Try running: sysctl dev.ix.0.advertise_speed to see what link rates it's trying to negotiate to.
You might be able to restrict that to 10G only using: sysctl dev.ix.0.advertise_speed=4

kilasin

@stephenw10 said in Trouble with WAN not being able to get correct speed:

sysctl dev.ix.0.advertise_speed=4

Ok this what I am getting when using the command sysctl dev.ix.0.advertise_speed: dev.ix.0.advertise_speed: 7

and this is what i got when using this command sysctl dev.ix.0.advertise_speed=4 : dev.ix.0.advertise_speed: 7 -> 4 (It disconnected my connection) So i just changed it back to 7 and i got connection back.

stephenw10

Ok, so it fails to link when it can only negotiate 10G. Check/replace the cable. Try linking to something else if you can.

[23.09.1-RELEASE][admin@6100.stevew.lan]/root: sysctl -d dev.ix.0.advertise_speed
dev.ix.0.advertise_speed: 
Control advertised link speed using these flags:
	0x1 - advertise 100M
	0x2 - advertise 1G
	0x4 - advertise 10G
	0x8 - advertise 10M

	0x10  - advertise 2.5G
	0x20  - advertise 5G

	100M and 10M are only supported on certain adapters.

advertise_speed=7 implies 10G, 1G and 100M.

kilasin

@stephenw10 said in Trouble with WAN not being able to get correct speed:

sysctl -d dev.ix.0.advertise_speed

Ok so I tried 10 and 20 and none worked unfortunately. I did see something curious though when the cable gets connected to my switch i see this :

The LAN from the WAN nic card is connected at 10G to my switch and my computer is only at 2.5G which should be 10G but that might just be a cable issue. Sorry for digressing.

Back to the WAN its weird i tried
0x1 - advertise 100M (works)
0x2 - advertise 1G (works)
0x4 - advertise 10G(not working)
0x8 - advertise 10M(invalid arguement)

x10(invalid arguement)
x20(invalid arguement)

stephenw10

That NIC likely doesn't support 2.5G or 5G so you can't set those advertise speeds.

I assume that switch is on the LAN side?

The WAN is connected to the ISP/ONT directly?

kilasin

@stephenw10 Correct on both questions .

kilasin

@stephenw10 I do have another 10G card that i can physically put in the machine but then that only leaves me with a 2.5G nic however I want my speeds lol i have another 2.5G card installed in the server but that is an rtl and pfsense does not like rtl cards as i tried implementing it into pfsense to no avail

stephenw10

Does the ISP/ONT device support 2.5G?

A Realtek 2.5G NIC in pfSense requires the realtek-kmod driver package.

kilasin

@stephenw10
the ISP/ONT supports it as their router gets those speeds. I tested their speed test and im getting exactly what I pay for but only in Wifi not Ethernet cuz of the port which makes no sense to me but that is how it is.

I see il try to install that package and try to use that nic as my WAN ..But I think i will bite the bullet and buy another 2.5G card that is compatible with pfsense

stephenw10

Ah, their router links to the ONT at 2.5G?

Are you sure the ONT actually supports 10G links then? That would explain why the 10G NIC only links at 1G.

kilasin

@stephenw10

I am pretty sure but I cannot be 100% as all I have is their router. So their router is messed up because their ports only do 1G but in WiFi i get 2G so I am guessing the 2.5G. I just installed the rltk package and try that nic as the WAN but seperating that nic from UnRaid is a pain and installing trying to make it work with pfsense was imposssible. I will try that. I have another option which is taking my 10G card that i have in my PC and installing it into the pfSense since i know this nic can pull 2.5G, which I have done before.

kilasin

@stephenw10

By the way is there a way i can check if the ONT can handle 2.5G connections?

stephenw10

What is their router, what model? I imagine the hardware will be documented somewhere.

You will need to add the loader values to load the Realtek module at boot if you have not already:

echo 'if_re_load="YES"' >> /boot/loader.conf.local
echo 'if_re_name="/boot/modules/if_re.ko"' >> /boot/loader.conf.local

kilasin

@stephenw10

Ya i dont that actually .. do i have to put echo ? I just added the lines to the config file like so:

kern.cam.boot_delay=10000
kern.geom.label.disk_ident.enable="0"
kern.geom.label.gptid.enable="0"
kern.ipc.nmbclusters="1000000"
kern.ipc.nmbjumbo9="524288"
kern.ipc.nmbjumbop="524288"
opensolaris_load="YES"
zfs_load="YES"
opensolaris_load="YES"
zfs_load="YES"
kern.cam.boot_delay=10000
kern.geom.label.disk_ident.enable="0"
kern.geom.label.gptid.enable="0"
kern.ipc.nmbclusters="1000000"
kern.ipc.nmbjumbo9="524288"
kern.ipc.nmbjumbop="524288"
if_re_load="YES"
if_re_name="/boot/modules/if_re.ko"
boot_serial="NO"
autoboot_delay="3"
hw.hn.vf_transparent="0"
hw.hn.use_if_start="1"
net.link.ifqmaxlen="128"
machdep.hwpstate_pkg_ctrl="1"
net.pf.states_hashsize="524288"

kilasin

@stephenw10

and yes their WAN is 2.5G i just confirmed as shown here