Hardware recommendations for 40gb internet, 100gb lan

Cool_Corona

Really.... this is nowhere a home setup.

Way to much power alone to support that kind of hardware.

I just dont get why......

If you run LB on ESXi, then enough hosts can do what you need on pfsense.

Patch

@mercer2 said in Hardware recommendations for 40gb internet, 100gb lan:

so would like something easy to setup and monitor.

A Lan with 40-100Gb capacity sounds generous but possibly useful.

To size your Wan, do not use what your ISP can provide and what Speedtest can measure (although at that speed you would probably have to run multiple concurrently).

Instead look at how many concurrent video channels you will want to access remotely. Or what is the actual high bandwidth concurrent tasks you will actually do.

The result of this calculation is likely to be within high end pfsense hardware capacity. If not someone is going to have to manage an enterprise system.

If you want it just for fun, then try playing with TNSR

mercer2

@patch
Thank you very much for your answer.

This is mostly for fun, and nothing I do needs 1gb, but I love tech so is always fun to push the edge for me.

our biggest data use is watching netflix.

Still it would be fun to test and play with 40gb bandwidth.

all of our neighborhood is getting 10g. so looking for ideas on how to better use such a big amount of capacity.

Patch

@mercer2
Sounds reasonable to me.

If it was me I would get hardware of at least Netgate 6100 level performance (but you may want a Netgate 1537 or equivalent) and load pfsense on it and get that running.

I assume you also have a hardware failure continuity plan, which involves having access to backup router hardware. After checking you can load pfsense and your configuration on it, load TNSR and experiment with it.

If you get board with TNSR then set up the two routers in a HA configuration.

stephenw10

You basically need something about as fast as you can get! And obviously it will need to have 40G NICs. But it still won't pass 40Gbps with pfSense. You would need to be running TNSR to actually get there.

Steve

luckman212

@mercer2 your IT room is about the size of my entire NYC apartment. How did you accumulate so much hardware?

stephenw10

@luckman212 said in Hardware recommendations for 40gb internet, 100gb lan:

your IT room is about the size of my entire NYC apartment.

Ha. Yup could fit some bunks in there and put it on AirBnB!

The power consumption would drain my experimenting fund in a few days I suspect.

Steve

mercer2

it is kind of crazy, our house is 15k sqft, with 30 tons of ac. our electrical service panel is 1200amps and this 3 racks have 7 x 30 amps dedicated circuits plus a dedicated 2 ton ducted ac.

any recommendations on hardware components for a pfsense capable that also could be migrated to tnsr and snort?

thanks

flat4

@mercer2
Sir what do you do for a living?
15k House, 30 tons of ac,
going to assume that's in a basement?

luckman212

@mercer2 said in Hardware recommendations for 40gb internet, 100gb lan:

any recommendations on hardware components

uh yeah this should fit nicely in your 15k sqft home: PA-7080 (and will only set you back $170K)

mercer2

I wish!!!

trying to be cost effective and wife not mad about it.

luckman212

@mercer2 Ehh.. if she's not mad about 3 full racks, the 600lb workgroup printer or the electric bill from those 2100 Amps...for your homelab, I can't see her complaining about anything.

My wife got mad at the little LED on our Apple TV, I had to cover it with electrical tape

stephenw10

In all seriousness firewall/routing at 40Gbps is beyond pfSense scale. TNSR can do it though and is very cost effective at those bandwidths compared with other enterprise routers, like that PA.

Steve

Derelict

@derelict said in Hardware recommendations for 40gb internet, 100gb lan:

@mercer2 What does the ISP providing the 40Gb service recommend?

Still no answer to this. Genuinely curious.

Patch

@mercer2 said in Hardware recommendations for 40gb internet, 100gb lan:

any recommendations on hardware components for a pfsense capable that also could be migrated to tnsr and snort

As reference point I would start with Netagate characterised hardware
https://www.netgate.com/tnsr-software/how-to-buy#appliances
https://docs.netgate.com/tnsr/en/latest/platforms/ and
https://www.netgate.com/tnsr-software/performance#get-to-know

For software in a home lab; Netgate have a free option which would be a reasonable place to start if you are just trying it for fun https://www.netgate.com/pricing-tnsr-software

Similarly pfsense is available in a free community edition https://www.pfsense.org/download/ or a free home lab versions https://www.netgate.com/pricing-pfsense-plus

If you want to start with more capacity you will need to scale it up from there, which probably involves adding a 40Gb network interface and more Xeon processes.

Personally I would not push the hardware specs till I had experimented with pfsense & tnsr to see if I was comfortable working with them. Both pfsense and tnsr run on most recent 64-bit x86 processor with SSE4.2 extensions. You may already have under used hardware meeting the minimum requirements.

Sergei_Shablovsky

@mercer2 said in Hardware recommendations for 40gb internet, 100gb lan:

I’m extremely lucky to have a 40gb internet fiber installed in my house

I know this is way over the top, and greatly underused. I’m a typical user, but my hobby is home automation and homelab

I would like to hit a Speedtest alternative service and get the full 40gb, with a chance to maybe upgrade to 100gb internet in the future.

According equipments in a racks more looks like this is a online + terrestrial radio station site:
0. EXTRA-HUGE 40Gb / 100Gb UPLINK (extremely too much for ANY work, even online trading need only 200Mb but with small jitter/delay, only ISP need speeds like that...)

A lot of UPS (ok, not Symmetra or 50kva GE, but anyway this is a Houston, not a Siberia or India with a “electricity cut-offs”...);
Rackmount PROFESSIONAL sound processors (I recognize 2 big (and costly!!!!) (Only this two may cost much more than all this setup, even full of Dell Servers...)
Not so much powerful Servers and Svitches in a racks, no fiber connectors.. (Why You need even 40G if a You have no fiber-connected rack equipment?)
Audio streamer amplifiers (or may be CD changers, but no one wiring even middle-class amplifier from underground to living room 2 floor ahead, because even You using $100/ft cable to Dali's sound speaker, as a result You receive a CRAPPY sound). I could recognize some modern DENONs and SONOS..
Rackmount storage (I not see the lights on HDD caddies, may be not filled or switched on...)
Some sort of car shortwave radio stations (or may be receiver/processors for wireless studio mic)
A huge copy/print station on a background
A lot of LCD remote for climate control systems on a wall

But also I see

unlabeled patches at the top of each rack;
no any fire-stop system (strongly needed if You have such amount of UPS in same room) near racks;
NO PIPES FOR COOLING SYSTEM for this amount of equipment (especially for a bunch of 10/14k speed 2,5' HDD that You have);

And You wrote that are “32 audio zones” in a building....

At the same time even You have so much space and zones in a living house, all of Yours imagination able to be realized by AppleTV / SONOS / RUSOUND / DENON media players/streamers + MESH WiFi (from any good brand, if You have no ability to drilling walls and cabling all inside house).

Are You sure this is room in Your house? Who are You, mr. President? :)

Sergei_Shablovsky

@flat4 said in Hardware recommendations for 40gb internet, 100gb lan:

@mercer2
Sir what do you do for a living?
15k House, 30 tons of ac,

Just as a joke: a lot of Russian oligarchs now escaping with A HUGE MONEY from Putin's dictatorship... I read from news half of a year ago one FSB-related person was arrested and in his 4-room flat in center if Moscow city 10 TON OF GOLDS was arrested! Another one time: 10 TON OF GOLD !!!! In a 4-room flat!!!!
And need to note this person was ordinary authority, even not top-level. Imagine how much top-level persons have?

Sergei_Shablovsky

@patch said in Hardware recommendations for 40gb internet, 100gb lan:

@mercer2 said in Hardware recommendations for 40gb internet, 100gb lan:

so would like something easy to setup and monitor.

A Lan with 40-100Gb capacity sounds generous but possibly useful.

To size your Wan, do not use what your ISP can provide and what Speedtest can measure (although at that speed you would probably have to run multiple concurrently).

Instead look at how many concurrent video channels you will want to access remotely. Or what is the actual high bandwidth concurrent tasks you will actually do.

The result of this calculation is likely to be within high end pfsense hardware capacity. If not someone is going to have to manage an enterprise system.

If you want it just for fun, then try playing with TNSR

ABSOLUTELY AGREE with Your opinion.

More than this, from our experience, the setup like this home (You may see building plan on a wall opposite to racks), need not more 5Gb for ALL NEEDS, even
SIMULTANEOUSLY each person (7 persons at all) in each separate room

see 4K stream from Netflix/Amazon Prime/Apple/YouTube
listen 32-bit ALAC/FLAC sound on external media player
playing a network Game in 4k
downloading 200-300 torrents
uploading streams from Apple iPhone/iPad to FaceBook, TikTok, YouTube at one time
making a group conversation on a stand-alone IP-telephone
surfing a 50 web sites with a huge graphics and video content
making “heavy” banking (online trading means)
have a separate personal web server (shopping site, blog, etc...) with 300 users online

Imagine? 5Gb for all of that mess!

Of course, in real life only 1-2Gb would be used.

Patch

@sergei_shablovsky
Not sure why you are trying to define adequate requirements. @mercer2 has already said he does not need the capacity but is doing it because he likes playing with technology and his ISP is offering it.

Sergei_Shablovsky

@patch said in Hardware recommendations for 40gb internet, 100gb lan:

@sergei_shablovsky
Not sure why you are trying to define adequate requirements. @mercer2 has already said he does not need the capacity but is doing it because he likes playing with technology and his ISP is offering it.

Just try to be reasonable.

Anyway, even this is like a toy, this is dangerous for himself and family members and neighbors, if they have no using fire-stop system and properly cooling for such amount UPS and equipment. Agree?