Networking testing tools

[Sergei_Shablovsky]

Dear pfSense Gurus!

When seeking for testing tools (beside Seagull, PacketSender..) I found this one thread on FreeBSD.org:

I found a nice site for some FreeBSD networking benchmarks.

https://github.com/ocochard/netbenches
Look for the results directory and scroll all the way to the bottom of the page.
Not a huge offering of platforms but some popular routing solutions.

Testlab setup here

What do You think about testing pfSense boxes by this tool ?

(I read a lot of opinions about not using hypertreading, locking to certain CPU cores, placing NICs on separate NUMA, - become not so helpful after FreeBSD 13.1 coming out...)

[stephenw10]

@sergei_shablovsky said in Networking testing tools:

https://github.com/ocochard/netbenches

All testing is good. As long as it's done right

Oliver already has some results there from an SG-4860.

Steve

[Sergei_Shablovsky]

@stephenw10 What about tweaking FreeBSD article on which many times pointing on in this forum ?

I mean https://calomel.org/freebsd_network_tuning.html

[stephenw10]

What about it?

[Sergei_Shablovsky]

@stephenw10
Sorry, I mean “is something changed since FreeBSD 13-based pfSense coming out?, May be some tweaks not actual yet?”

[stephenw10]

Not as far as I know. You'll have to wait for a build so we can start testing.

[Sergei_Shablovsky]

@stephenw10
Which tool You recommend for emulating real workload? (Of course this may be something complexing, scriptable for different protocols, etc...).

Thank You!

[stephenw10]

For network testing it's hard to beat T-Rex. However I like to use iperf3 also because almost anyone can replicate that in their own lab.

[Sergei_Shablovsky]

@stephenw10 said in Networking testing tools:

For network testing it's hard to beat T-Rex.

Interesting powerful tool...! I am impressed!

How You compare (Pro/Cons) with Seagull (Seagull fork on GitHub)? (or even TrafficGenerator)?

From official Seagull page

Seagull is a powerful traffic generator for functional, load, endurance, stress and performance/benchmark tests for almost any kind of protocol.

In addition, its openness allows to add the support of a brand new protocol in less than 2 hours - with no programming knowledge. For that, Seagull comes with several protocol families embedded in the source code:

Binary/TLV (Diameter, Radius and many 3GPP and IETF protocols)
External library (TCAP, SCTP)
Text (XCAP, HTTP, H248 ASCII)

Protocols are then implemented on top of those protocol families using user editable XML dictionaries. Those dictionaries describe how messages and parameters are encoded, allowing a great flexibility.

A Seagull scenario - written in XML - describes the messages that are sent and received. It also indicate the behavior to adopt in case a message is unexpected or a check on a parameter fails.

Entirely coded in C++, Seagull is optimized for performances.

Ready to install packages are available for HP-UX (PARisc and IPF/IA64), Linux and Win32 (Cygwin). Seagull can also be compiled from the source code.

Seagull supports currently the following protocols:

Diameter base ( RFC 3588) and any Diameter relating application - IMS Cx, Dx, Ro, Rf, Sh over TCP or SCTP or TLS over IPv4 or IPv6.
TCAP ITU and ANSI and any protocol over TCAP (Camel, GSM MAP, IS41, Win, ...) either over SS7 (E1/T1) or SIGTRAN. For that, it relies on HP OpenCall SS7.
XCAP over HTTP over IPv4
HTTP over IPv4
H248/Megaco ASCII form over UDP or TCP or SCTP/IPv4
Radius (subset) over IPv4.

Seagull has the following features:

Multi-protocol traffic generator
Command line tool with text interface
Protocols of the same family are described in an XML, user editable, dictionary (messages, parameters)
Existing protocol families: Binary/TLV (Type, Length, Value), Raw binary, Text, external API (first implementation: HP OpenCall SS7)
Support of IP (UDP/TCP), SCTP, SSL/TLS and SS7/TCAP transports
Portable programming (tested and supported on Linux x86, ia64, HPUX, SunOS and Windows)
Scenarios are described using XML files
Multi-threaded for performances and reliability
Dynamically adjustable scenario rate
Uniform, Poisson or Best-effort scenario arrival distribution
Remote-control (scenario-rate set, counter dump) through standard HTTP interface
Pause and restart of traffic
Support of automated traffic profile (varying scenario rate)
Smooth (no new scenarios then wait for ongoing scenarios to end) or brutal end
Scenario display with message counters
Scenarios have init (executed once), main (repeated for traffic) sections
Scenarios have default sections for defense in case of unexpected messages
A scenario can be mono (most cases) or multi-protocol
Message and parameters checking possible (disabled by default)
Support of parameter injection following a CSV like database
Multiple Seagull instances can be synchronized in the middle of scenario
Intra scenario synchronization using a synchronization protocol (example application provided in Java language)
Statistics: timer between two messages, scenario length, scenario rate, successful scenarios, failed scenarios (with reason)
Protocol decoding and hexadecimal dump
Trace files with or without timestamps (for performances and automation)
...

However I like to use iperf3 also because almost anyone can replicate that in their own lab.

But... iperf3 is simple tool just to test CPU/NIC/mem overall loading under 1-2 simple protocols.
By nature, It’s great for

• testing when tuning, BIOS, FreeBSD network stack, application settings in case sending one big file or its parts, and
• primary useful in local net because of modern networks with a lot of traffic control&mgmt applience in ISP make iperf3 unusable for testing bandwidth outside Your local net.

Am I wrong in this ?

[stephenw10]

No you're not wrong. It is a simple tool, it tests only large packets and single connections.
But that means most users can test with it and produce meaningful results that others can compare to. And it's better than Speedtest.net which is what everybody immediately tries.

Steve

[Sergei_Shablovsky]

@stephenw10 said in Networking testing tools:

No you're not wrong. It is a simple tool, it tests only large packets and single connections.

There are keys to push it using several connections:
iperf3 -c server -P streams (from iperf3 mans page https://www.mankier.com/1/iperf3)

But that means most users can test with it and produce meaningful results that others can compare to. And it's better than Speedtest.net which is what everybody immediately tries.

I using all of them:
speedtest
librespeed
fast

But anyway, speedtest/librespeed tools are not correct and are obsolete because most of all ISPs have own speedtest/librespeed servers deployed to demonstrate “Our Internet IS BETTER!”. In this case speed measurement made only between client “last mile” and ISP's “first hop server” (after 2-3 routers/switches and even before the core).

So I strongly recommend to end clients using Netflix fast.com for purpose to knowing real bandwidth for huge traffic, and using SmokePing for advanced users or geeks ;)

[Sergei_Shablovsky]

@stephenw10 said in Networking testing tools:

For network testing it's hard to beat T-Rex.

Could You be so please to give me .yaml profiles & .pcap files, that You using for pfSense CE testing?

(Right now reading https://github.com/cisco-system-traffic-generator/trex-profiles, looks very interesting..)

Could You be so please to explain in more details how You testing pfSense by TRex?
Thank You!

[Sergei_Shablovsky]

@stephenw10 said in Networking testing tools:

For network testing it's hard to beat T-Rex.

Dear Stephen, are You using TRex on separate Linux machine?

P.S. May be You find something interesting here https://forums.freebsd.org/threads/multi-protocol-traffic-generator-seagull-fork-for-freebsd.82769/

[stephenw10]

Yes, you have to run it on a dedicated machine b ut it can be both the source and sink in testing throughput.

Steve