@jimp That would be my next step. I just figured it would be good information for the community to know about how the support works. I often feel there is a disconnect between the community and Netgate and so I thought it would be nice if that information was out there in case someone else ran into this situation.

You cannot add the SFP ports to the existing lagg and you need the lagg to the switch anyway. So, yes, I would create a new lagg (lagg1) with ix0 and ix1 in it and connect that to the Ubiquity Switch. You can then use LACP on that lagg which would be a far better choice there than loadbalance. Steve

There's little point using a mesh type setup when all the traffic is from the remote sites to the central site in my opinion. You might be able route traffic via another site if a link goes down providing some redundancy. It would be a far more complex setup though. You might check the TINC package for a mesh setup. Steve

@stephenw10 That's what I did, I just didn't want to specify how I've done the private subnetting. But that's a good point for anyone who stumbles on this looking for a leg up.

I wouldn't necessarily recommend doing this, but you could add (or create) an entry in /boot/loader.conf.local which sets pfsense.fsck.force=5 or so. Then on every boot it would perform that many iterations of fsck to check/repair potential problems, even when the filesystem is marked clean. It would drastically slow down the boot process and is typically unnecessary, but it might at least help with some of these situations. It's definitely not something we'd ever ship with set by default.

@stephenw10 Ok I think I may have figured it out. I had two switches Daisy chained and when I disconnected the second switch everything worked fine. I tried reconnecting and then the switches lost connection again. I tried several different cables with the same results. I ended up configuring the opt1 port and connected the second switch to that. I am not sure what made the switches stop working while Daisy chained but they showed they were still communicating. I am going to have to dig into this more and find out what caused the issue. Thank you for the help @stephenw10 .

Have you tried from multiple browsers, multiple remote clients?

Turns out, the SFP port needs to have speed set explicitly. Steve mentioned this earlier, but I ignored the advice. When I explicitly set the speed to 10G, Comcast said they were getting ARPs, so they disabled auto-negotiation on their side. When they did, the link came up. So, in the end, an easy fix. But Comcast had advised me to set the link at auto select. Learn from my mistake.

Not with LACP. As far as I know the switches need to be 'stacked' in order to have lagg links split across different physical units. You could connect everything to everything else and rely in spanning tree to prevent a loop. But.... That also means bridging the ix ports in pfSense. I would consider an LACP lagg from the 7100 to the first switch and a second lagg from that switch to the other switch. It won't help if the first switch fails entirely but you would have port/cable redundancy between everything. Steve

Yes the /32 mask was the problem... changed to /24, enabled DHCP, and good. Thank you all!

is there a fiber option? i could get a 1m premade if that would work

Just for reference I was able to see >300Mbps though a 3100 using iperf3 in local testing, so very low latency. That was using AES-CBC 128 and SHA1.

Hi, the ipsec speed is limited by upload speed at remote site due to ACKs. Having said that I have issue with SG-3100 - I cannot get greater than 45 Mbps on a physical 100Mbps fiber link to my home pfsense which is 940Mbps fiber, whereas I can 380Mbps+ to a SG-5100 on physical 500Mbps link, and 180Mbps to a SG-4860 on a 200Mbps physical link. My home is a home-built pfsense running i7. I will open another thread on this.

You (or verizon) will need to run a ethernet cable from your ont (box on side of building) to your router location, then call verizon and they will activate it. The other option is to put your G-1100 into bridge mode. There are instructions for this all over the internet. This is a reputable site here (https://www.dslreports.com/forum/r31057540-Networking-HOW-TO-Bridge-G1100-So-your-Router-becomes-Primary)

I was thinking about it overnight. If you want to save money, build your own. How about something like this (https://www.amazon.com/gp/product/B077Y8JR1R/ref=ox_sc_act_title_1?smid=A1JJPPPLBAC13R&psc=1) with a asrock combo board in it like this one (https://www.amazon.com/ASRock-J4005B-ITX-2-7GHz-Mini-ITX-Motherboard/dp/B079GFD84R/ref=sr_1_3?keywords=asrock+combo+board+mini+itx&qid=1581686651&sr=8-3) It will allow you to use a pcie intel 2 or 4 port nic

@stephenw10 said in Sg-3100 Trunk Port: You don't need to assign the parent interface (mvneta1) to use VLANs on top of it. Though you may want to if you want untagged traffic to pass through the switch completely. Setup the VLANs you need as interfaces and assign/enable them. Then in the switch config switch it to 802.1q mode and add the vlans you configured making sure they are set as tagged on the port(s) you want and tagged on port 5, the internal port. Steve stephenw10 Thank you for all the info

You mean like a 3g/4g modem? To connect to what? Internal/external?

Something like this: https://www.amazon.com/dp/B00G059G86/ or this one: https://www.amazon.com/AC-Infinity-MULTIFAN-Receiver-Playstation/dp/B00G05A2MU That last one "might" be a double fan kit, attached together with a single USB cable. Jeff

Thanks all, support was awesome and did indeed get this resolved.

This same thing has happened to me the last 3 installs over the course of a few years.