Yea, I've come to the conclusion short of getting new storage and putting it squarely in each VLAN that I need to do L3 routing at the switch for VLAN 20 and 70. I guess I was just more or less asking if there was any other architecture you can think of that may have worked better, but it really seems like it is that straight forward.

So to summarize my changes:

Disable Snort on VLAN 20 and 70

Create 172. something /30 subnet for transit

Create new VLAN tag 172 in pfSense

Create new interface tied to this VLAN

At both switches I will add a new VLAN for the transit network, and set that as the default route to 172.something.1

Add VLAN 20 and VLAN 30 at the core switch (sg300-10). I'll put in ACLs to block everything between the VLANs except the IP\Port combos I currently have in my
firewall relating to those two subnets.

Disable VLAN 20 and VLAN 70 interfaces in pfSense

Create new gateway with 172.something.1 as gateway

Create new static routes for 10.37.70.0/24 and 10.37.70.0/24 via the gateway created above

Enable Snort rules on new Transit interface.

Verify any needed VLAN 172 firewall rules that are needed (shouldn't be any as this will only be used for outbound requests, correct?)

Sound about right?

at Johnpoz

Thanks for that.

I'll give it a go (the worst fate is that guests have no access for a while, but then I don't get a million guests a day and I have an unlimited-data 4G modem if my occasional guests are "unhappy").

I am "greenfield" in a sense -- I have total and exclusive control over my networks and report only to myself in the event of a disaster (yeah, I might get some $#1t from Madame, but there is always the 4G modem to calm her down). I have elected to move to "my-net 3.0" -- my decision was unanimous :)

Why do I seek tagged admin?
Most VLAN attacks go for VLAN1, or failing which, go for native-VLAN. I ask myself WHY should I have VLAN1 or native-VLAN connected to anything at all ... let alone to the admin heart of the network -- just seems a silly choice!

Off to the mountains for skiing: you wont hear from me for a week or two, but I'll report back.

Appreciate all the feedback so far.

regards, Chris

RTFM: https://docs.netgate.com/pfsense/en/latest/book/interfaces/interfacetypes-lagg.html

After thinking about how MAC addresses work on a switch, I replicated the MAC across all bridge members and the bridge itself, and things began working!

I installed with success.
I set mikrotik like a Wisp AP - Bridge, I deleted DHCP rules and i create 1 vlan on eth5.
In PfSense I created 1 vlan, in Interfaces / Interface Assignments I associate WAN with PPPOE0(em0) created in PPPs like a Link Type - pppoe.
Now I have an Pfsense router with 1 single ethernet port for WAN and LAN.
Thanks!

https://forum.netgate.com/topic/139859/sg-1100-running-real-vlans/8 the SG-1100 is essentially a router-on-stick in one case. You can't simply change the MAC of just the WAN port, as this is a switch port.

You can assign the parent interface mvneta0 and then change it's MAC, which will affect all ports and create a conflict with the original device if it's still connected. So either get rid of the original device, put it on a different L2 if possible, register the SG-1100 MAC with your provider or return the SG-1100 and get a device with more dedicated interfaces.

It really pays to research the hardware before you buy it.

@johnpoz

How should I go about troubleshooting duplicate packets? I read the following link, as well as the link about Asymmetric Routing, but not sure if it applies.

https://docs.netgate.com/pfsense/en/latest/firewall/troubleshooting-blocked-log-entries-for-legitimate-connection-packets.html

@derelict said in Add NIC as extra LAN-port?:

Get a switch. Leave igb3 available for use as a router port.

If you insist on doing that look at bridging igb1 and igb3 into a bridge and use the bridge for LAN.

I got a 5-port switch today which is in LAN but it's full, and I want to buy a Ubiquiti PoE switch 8-60W (about $100) but I only need 1 port so therefore it's not really negotiable to buy one just for I need one extra port lol 😄 So I thought if it's easy to use it as a LAN-port in the meantime...

If you want all 4 ports to go to the same switch on 4 different broadcast domains then just connect all 4 ports to 4 untagged switch ports on 4 separate switch VLANS.

Pretty much nobody would do that because VLANs are much more flexible but if that's what you want, knock yourself out.

@johnpoz said in Moving device to new VLAN setup not working:

you don't have any vlan 2 setup on your switch from what you posted

You're right I don't have any vlan 2 setup now but from my readings for best practice if a vlan is created e.g.for port 3 for NAS then any other connection e.g. port 2 wlan interface 192.168.2.XXX should also have a vlan setup made. Correctly if I'm wrong.

If i want the NAS device to go outside the network (e.g. internet) how will this work if it no longer has a wifi connection since the wifi router in on subnet 192.168.2.2 Do I have to NAT the NAS device to the wifi router?

You have to create rules on those to allow what you want.

I have a all open rule setup on the NAS interface but still was't able to ping the device using pfsense box under diagnostics however the gateway 192.168.30.1 succeeded.

It may be my NAS device (wd mycloud ext2 ultra) giving me the problem. DHCP leases is displaying an ip for the device but it is showing it as offline. The unit itself is showing 3 blue lights indicating all is online.

@njanja said in Build router with pfSense:

What is the purpose of buying a network card with multiple ports.On almost every forum they recommend to me Use VLANs

Link aggregation, to increase available bandwidth. You might also want physically separate networks.

Not really.

It works fine.

The default configuration is DHCP WAN and 192.168.1.1/24 on LAN. All you have to do is edit Interfaces > OPT1, enable it, and number it with something other than what is on any other interface (like 192.168.2.1/24), add firewall rules to OPT1 to pass the traffic you want to allow, and enable a DHCP server on OPT1.

Thanks for the advice. I found some of it before I read your message but your message was right to the point. I had to read your message a few times to understand, the third rule was kicking my butt because I didn't include the DNS in the first rule. While this type of stuff is probably easy for many people, my goodness it's a lot to think about and keep track of. I mean, I do understand it but dang!

So it looks like I have a VLAN that is isolated from everything except the printer. I will still use that 8 port switch as I desire to run some VM's on my ESXi machine on a separate VLAN and I need the switch that handles VTAGS to go between the pfSense computer and ESXi computer, so there was a benefit to having purchased it.

[RESOLVED]

The problem was in Traffic Shaper ..

When I prioritized the traffic on the network using the wizard, I reported the speed of the WAN link, in this case, 20 mbps.

So, as I checked, pfsense limited the network bandwidth to 20 mb (2.5 MB), including LAN and WAN. As this information is not made explicit in pfsense, I only notice when I deleted Traffic Shaper and set it up again.

The solution I found was to report the maximum capacity of the circuit (1 Gb) to use the entire internal network bandwidth. Otherwise, the entire network is limited to the speed of the WAN link.

I had done this configuration for a long time, but since the network was not segmented, the internal traffic was not through pfsense, so when I created vlans, pfsense limited the band ..

Yes. The rules are the same whether they are on em3 or em3.2

its a good obfuscation then - looks like actual screen shot!

Better to mention you obfuscated or use documented example networks and mention that ;)

192.0.2/24
198.51.100/24
203.0.113/24

2001:DB8::/32

Just saying ;) You would be surprised at some of the stuff you see around here people doing for real ;)

Or just block out part of the actual address so its clear is obfuscation..