We did some more testing - I installed one FreeBSD 12.3 and one pfsense dev on few of the spare HP boxes we had lying around. I wanted to be sure what is the problem exactly, the netgate hardware or the OS.
As far as I can tell this is definitely something related to how FreeBSD handles these smart USB devices that have both storage and modem modules.
I found this 100g router, which I received last week
will test to see how it goes.
in regards to saturating a 100g local link and a 40g internet link.
For this kind of speeds better to use Emerson, Nokia, Huawei routers from ISP-grade lines, or a little bit cheaper F5, Extreme, Juniper.
But not SOHO like Ubiquity, Microtik, D-Link, etc...
As a You demonstrate, money, electricity uplinks and rack space - not a big problem for You. Because this equipment are ISP-grade, anyone local distributor would be happy ship, install and give You 1-2 weeks for test, for free.
@stephenw10 Thanks, good to know it's somewhat normal. I tried that but unfortunately it didn't seem to have an effect. I guess I can either live with it or turn of BIOS redirection, unless anyone has any particularly clever ideas.
Wanted to update this post. it was the card.
I had a sas LSI pcie card handy stuck in there and did the pciconf -lv and it found it. As a last ditch effort got windows on this machine and even windows could not see it.
All these test were done with the riser installed so my stuff was working.
I'll assume you have a 5000 model.
It looks like a standard intel machine , with industrial specs
It doesn't mention much about the network cards (just dual + gigabit)
But there are drivers for realtek network cards on Dell, for that model.
Realtek isn't the "best" networkadapter for pfSense, but with a little driver trickery, it might work perform reasonably well.
On the installed Ubuntu , try to run the commands :
And paste the outputs here.
Good info here
I would just make a pfSense USB install stick , and give it a try.
Or pfSense Plus only supports this one from NetGate:
pfSense plus comes with one driver that is supporting many
but not all cards and/or chips. If you get hands on a support
card (chip) it will running out of the box! Coding a driver means you should be also hands on a device or hardware
and such of the Intel QAT cards are often high in price!!!
So if someone is coding that driver, that should be taking
care on all available QAT things on the market he should be sorted with money or hardware for doing this.
Spend some money to the FreeBSD Project and/or support
coders with hardware for getting the maximum out.
pfSense comes with enough LAN port and you might be
able to insert a dump switch on each, and you don´t need
VLANs, pure routing is here the entire job of the pfSense
pfSense is sorted with one or more VLAN capable switch
and is doing the entire routing between the VLANs on top
of its other work!
pfSense is sorted with one or more Layer3 Switches and
the switch(es) are routing the entire workload self, this free´s up your pfSense for doing other work, or you may be able to install some more packets without problems.
So this might be the first problem, for VLAN or not VLAN
usage. And the other thing is how many Volt/Watt are all the cameras are needing, so you should be looking for
two different numbers their;
Volt/Watt per port that all cameras will be getting enough per port
the entire electric budget must be also covering all port
with "xyz" Watt in total!
So if your power budget total and per port is right and
you have a really let us say powerful pfSense you can
also go with dump or layer2 switches.
NETGEAR GS728TP 28 Port Gigabit Ethernet LAN PoE Switch Smart (Netzwerk Switch Managed mit 24x PoE+ 190W, 4x 1G-SFP for ~450 €
NETGEAR GS324TP PoE Switch 24 Port Gigabit Ethernet LAN Switch Smart (24x PoE+ 190W & 2x 1G-SFP, Managed Switch mit WebGUI, VLAN, IGMP, QoS, PoE Switch 19 Zoll Rack-Montage) for ~350 €
Netgear GS524PP Switch 24 Port Gigabit Ethernet LAN PoE Switch (mit 24x PoE+ 300W for ~400 €
NETGEAR JGS524PE PoE Switch 24 Port / 16 PoE Ports (100W) ports for ~200 €
12 PoE ports
Netgear (GS524UP) unmanaged but PoE++ for ~450 €
16 PoE ports
While I see how those could lead you to your conclusion.. But I take it when your device is directly connected to your modem you have a different IP with your isp. Or when you have the other edge router as well common for this IP to be different.
They get an IP in the same subnet, and the hops are the same.
I understand that it's not ideal to test this to the internet. But as I only have issues when going out to my WAN and the only variable that I changed was my firewall/router it leads me to believe there's something wrong with my hardware choices.
I'll run the tests during the weekend as to not affect my network so much during the weekdays
Thanks. I've already got one though. Someone else might.
I'm not sure there's much on it we can use. What there is there is detailed in this thread, mostly here.
It would be nice to know where the LEDs are connected. I could never find them in the usual places.
For big concerns (large files) and routing much traffic
and on top what installed packets will be there in game too! Do you plan using IDS or IPS (inline mode) and if so
where you are want to use it. Is there one or more radius servers in game too? Is there another ids instance inside
this setup, like OSSec or so?