@vah1280 Deterministic NAT is being deprecated in the next version. Dead end in tnsr.

See first warning here:

https://docs.netgate.com/tnsr/en/latest/nat/deterministic.html#deterministic-nat

So... I did end up opening a ticket.

Turns out:

dataplane nat max-translations-per-thread 1000000 dataplane cpu workers 1

Then restarting the dataplane got things working.

However, I starting thinking (perhaps wrongly) that in order to get the full throughput out of this XG-1537, I really should have more workers, otherwise most of the cores would just be sitting idle.

So, I changed "cpu workers" to 2, and restarted dataplane - it would not come up.... changing workers back to 1 worked fine...

Next, I figured maybe 2 million translations was somehow "maxing out" the box. (It has 16Gig of RAM). OK, I'll reduce to 200k translations-per-thread, and spin up 3 workers. That seemed to work fine, as the primary web server based on the LAN was processing transactions no problem.

However, the next morning, I found out that this "somehow" broke other simple static NAT rules being used for small services (remote SSH access to a couple hosts, etc.) Moved it back to 1 million max-translations-per-thread and a single worker. Fixed.

Very puzzing.... Can I process 10gig worth of traffic through the box, most of it NATed to an internal webserver with only the single worker? If so, That's fine I guess...

Anyone have any thoughts? Am I doing something "unusual" here? I know networking, but this is my first experience with VPP.

Thanks,
Dan

@derelict yes, it worked.
I changed gw to 192.168.96.5
Many thanks

@alan-jones That means nothing unless you are passing them through directly. Tnsr only sees virtio or vmxnet3. The underlying hardware is obfuscated.

@dbeyzade Can't promise, but I will certainly pass this along to our product manager.

@dbeyzade I assume you're capturing in the host shell on the capture interface presented there.

Your screenshot does not show any ping attempts.

Is it possible to capture your entire test and post the pcap file instead of a screen shot?

The same on the connected switch port would be nice as well.

Thank you.

@kiokoman This did the trick, thank you very much :)

@jamescr We have opened a ticket for you and assigned one of our engineers to see if we can work through these concerns. Thanks!

@derelict if I'm going to try, the problem I see is exactly the one you describe, even in mikrotik the neighbor is recognized and the changes that ospf announces.

@helmlein It is not intended to be this way and there is an open bug on the RA. DHCP6 features are simply not implemented yet.

@derelict Thanks.

@vesalius no they do not. RHEL can always be rebuilt from the source code that rhel MUST provide per the GPL. When centos was borged by RHEL the writing was on the wall for Centos. The other rebuilds will continue because the rhel sources will always be available. https://etc-md.com/2020/12/09/the-end-of-centos-and-my-moving-to-bsd/

@robing They need to be a type of NIC that dpdk supports. It should basically work with vmxnet3. Not sure what the capabilities of vmware player are.

@schnitzel_itdept Can do you do a packet capture on server B to double check all advertisements are received correctly?

December 10th

Performance results for December 10th 1 worker in routing mode only [ 4] 0.00-200.00 sec 225 GBytes 9.65 Gbits/sec 8405 sender [ 4] 0.00-200.00 sec 213 GBytes 9.14 Gbits/sec 7641 sender [ 4] 0.00-200.00 sec 228 GBytes 9.80 Gbits/sec 8849 sender [ 4] 0.00-200.00 sec 207 GBytes 8.89 Gbits/sec 7627 sender [ 4] 0.00-200.00 sec 206 GBytes 8.84 Gbits/sec 7547 sender [ 4] 0.00-200.00 sec 229 GBytes 9.83 Gbits/sec 8743 sender [ 4] 0.00-200.00 sec 203 GBytes 8.72 Gbits/sec 7888 sender [ 4] 0.00-200.00 sec 224 GBytes 9.64 Gbits/sec 8674 sender [ 4] 0.00-200.00 sec 198 GBytes 8.51 Gbits/sec 8194 sender [ 4] 0.00-200.00 sec 227 GBytes 9.75 Gbits/sec 8774 sender Which give 92.77 gbps 3 workers in routing mode only ipsec1 tnsr(config)# show dataplane cpu threads ID Name Type PID LCore Core Socket -- -------- ------- ---- ----- ---- ------ 0 vpp_main 4910 1 5 1 1 vpp_wk_0 workers 4914 2 5 0 2 vpp_wk_1 workers 4915 3 12 1 3 vpp_wk_2 workers 4916 4 2 0 ipsec2 tnsr(config)# show dataplane cpu threads ID Name Type PID LCore Core Socket -- -------- ------- ---- ----- ---- ------ 0 vpp_main 5009 1 5 1 1 vpp_wk_0 workers 5013 2 13 0 2 vpp_wk_1 workers 5014 3 13 1 3 vpp_wk_2 workers 5015 4 4 0 No changes related to performance 1 worker in encrypting mode IPSEC(aes256-sha512-modp4096) [ 4] 0.00-200.00 sec 77.3 GBytes 3.32 Gbits/sec 1594 sender [ 4] 0.00-200.00 sec 84.6 GBytes 3.63 Gbits/sec 1776 sender [ 4] 0.00-200.00 sec 79.6 GBytes 3.42 Gbits/sec 1609 sender [ 4] 0.00-200.00 sec 80.2 GBytes 3.45 Gbits/sec 1732 sender [ 4] 0.00-200.00 sec 82.4 GBytes 3.54 Gbits/sec 2110 sender [ 4] 0.00-200.00 sec 80.4 GBytes 3.46 Gbits/sec 1509 sender [ 4] 0.00-200.00 sec 78.3 GBytes 3.36 Gbits/sec 1642 sender [ 4] 0.00-200.00 sec 79.8 GBytes 3.43 Gbits/sec 1526 sender [ 4] 0.00-200.00 sec 83.1 GBytes 3.57 Gbits/sec 2211 sender [ 4] 0.00-200.00 sec 81.1 GBytes 3.48 Gbits/sec 2016 sender Which give 34.66 gbps 3 workers in encrypting mode IPSEC(aes256-sha512-modp4096) At this moment, it is not functional. Here is what I have after some analysis… Each client is able to ping is gateway which is the LAN interface in tnsr; Each client is able to ping is correspondent IPSEC interface ipip0 in tnsr; From one tnsr, unable to ping the IPSEC’s interface of the other end of IPSEC(the other tnsr); In the logs /var/log/messages of December 10th: VPP stopped and always been respawned Stgrongswan and charrond stopped and restarted (as VPP)

Still analyzing the logs at this moment...

@capitanblack

Thanks for your interest.

TNSR provides a RESTCONF API. I'm not aware of anyone that has attempted to integrate or test the API against Openstack/Ansible, but if those tools can make RESTCONFI requests it should be possible for them to configure TNSR.

The TNSR API docs can be found here: https://docs.netgate.com/tnsr/en/latest/api/

Feel free to send more questions or update us on your progress!

Time to put a CAPTCHA on logins? I dislike those things but sometimes you do what you gotta do...

The pricing from here is $500 per year for the Business Pro plan. There is no minimum/maximum speed, it's flat rate. It's not a replacement product for pfSense so your network doesn't need to be 'big enough' to benefit from it, you may use it even for the smallest networks if you wanted.

@prx said in TNSR on AMD EPYC ROME:

16.26.1040

i think it was specific for that firmware revision but for the network card you mentioned :

tested platform
https://doc.dpdk.org/guides-20.02/rel_notes/release_20_02.html#tested-platforms
Mellanox ConnectX-5 Ex EN 100G MCX516A-CDAT (2x100G)
Host interface: PCI Express 4.0 x16
Device ID: 15b3:1019
Firmware version: 16.27.1000 and above

also,
That is the kind of speed I have when one of my side it's not set with Mtu 9000, double-check that on all your machine.
I found this tuning useful for Ubuntu https://fasterdata.es.net/host-tuning/linux/

Hm, interesting! I haven't heard of anyone trying to install it on EVE-NG. Let us know how it goes.

Netgate provided an example on how to integrate Snort to create an IDS back in 2018, which needs an update as TNSR has continued to evolve. From a 2018 blog:

TNSR-IDS is written in the Go programming language, allowing it to be easily compiled for a large number of OS and architectures. Details, source code, and setup instructions (including TNSR, SNORT and ERSPAN) can be found at the TNSR-IDS Project GitHub Repository(https://github.com/Netgate/TNSR_IDS). A README file is included in the repository that provides a lot of detail about the process, as well as a TNSR-Snort setup file that gives detailed installation instructions.

I'd use that as a starting point, but there may well be some architectual or setting changes that need to be tweaked to get the spice flowing.

There was an issue with heap-size that will be corrected in version 20.10.1.

TNSR 20.10 will easily consume multiple BGP full tables, example:

vpp-test# show bgp sum IPv4 Unicast Summary: BGP router identifier 194.1.163.7, local AS number 50869 vrf-id 0 BGP table version 1844191 RIB entries 1521801, using 267 MiB of memory Peers 6, using 123 KiB of memory Peer groups 4, using 256 bytes of memory Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 194.1.163.0 4 50869 146492 110246 0 0 0 00:09:31 130439 194.1.163.1 4 50869 18905 110246 0 0 0 00:09:31 118135 194.1.163.2 4 50869 124916 110246 0 0 0 00:09:31 1527 194.1.163.3 4 50869 49283 110246 0 0 0 00:09:31 13 194.1.163.4 4 50869 49299 110246 0 0 0 00:09:31 11 45.129.224.233 4 58299 132111 11 0 0 0 00:07:51 836183 Total number of neighbors 6 IPv6 Unicast Summary: BGP router identifier 194.1.163.7, local AS number 50869 vrf-id 0 BGP table version 163503 RIB entries 186439, using 33 MiB of memory Peers 6, using 123 KiB of memory Peer groups 4, using 256 bytes of memory Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 2001:678:d78:: 4 50869 32391 19366 0 0 0 00:09:31 38666 2001:678:d78::1 4 50869 11392 19366 0 0 0 00:09:31 34712 2001:678:d78::2 4 50869 16616 19366 0 0 0 00:09:31 317 2001:678:d78::3 4 50869 3186 19366 0 0 0 00:09:31 3 2001:678:d78::4 4 50869 3234 19366 0 0 0 00:09:31 2 2a0e:5040:0:2::233 4 58299 32530 11 0 0 0 00:07:55 98031 Total number of neighbors 6