Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Best Network Topology with Current Hardware

    Scheduled Pinned Locked Moved General pfSense Questions
    36 Posts 4 Posters 2.0k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • ?
      A Former User @stevencavanagh
      last edited by

      @stevencavanagh

      Yes, you can use a so-called transit VLAN to link your L3 switch with pfSense, say VLAN99. However, I recommend using in it IP addresses from a different address space than 192.168.0.0/16, say 10.10.99.1 on the switch and 10.10.99.2 on pfSense. This will make it possible to have a summary static route on pfSense. You would need two static routes, one on the switch and the other on pfSense. On the switch, it needs to be a default route with pfSense as the gateway, so it would be something like Dest 0.0.0.0, Prefix 0, Next Hop 10.10.99.2, Interface VLAN99. On pfSense, you would need to create first a gateway with the IP address 10.10.99.1. To avoid some potential problems, disable the gateway monitoring for it and make sure the pfSense WAN is the default gateway. On this new LAN gateway, you would need to create a summary static route to 192.168.0.0/16. You will no longer have VLANs on pfSense so the port used to connect the switch with pfSense would need to be an access port to VLAN99, not a trunk.

      That's the easy part of the whole exercise. The difficult part will be to replace the firewall rules you might've created for your VLANs on pfSense with some ACL rules on the switch. ACL rules are not stateful and may be limited on a particular switch. That's the price to pay for faster inter-VLAN routing. Sorry to say that, but I stay away from help with it.

      S 2 Replies Last reply Reply Quote 0
      • S
        stevencavanagh @A Former User
        last edited by

        @kjk54 said in Best Network Topology with Current Hardware:

        Yes, you can use a so-called transit VLAN to link your L3 switch with pfSense, say VLAN99. However, I recommend using in it IP addresses from a different address space than 192.168.0.0/16, say 10.10.99.1 on the switch and 10.10.99.2 on pfSense. This will make it possible to have a summary static route on pfSense. You would need two static routes, one on the switch and the other on pfSense. On the switch, it needs to be a default route with pfSense as the gateway, so it would be something like Dest 0.0.0.0, Prefix 0, Next Hop 10.10.99.2, Interface VLAN99. On pfSense, you would need to create first a gateway with the IP address 10.10.99.1. To avoid some potential problems, disable the gateway monitoring for it and make sure the pfSense WAN is the default gateway. On this new LAN gateway, you would need to create a summary static route to 192.168.0.0/16. You will no longer have VLANs on pfSense so the port used to connect the switch with pfSense would need to be an access port to VLAN99, not a trunk.

        That's the easy part?? Will have a go and see where I get.

        Cheers
        Steve

        1 Reply Last reply Reply Quote 0
        • S
          stevencavanagh @A Former User
          last edited by

          @kjk54
          So, had a go at the L3 switch config, and.......

          I created a new VLAN99 (transit).......

          1c89923c-4548-4ab8-ba8e-f3fd69ae7838-image.png

          Then created a route.........

          4b86ac30-c141-4e09-b2b9-4bdee4f7d953-image.png

          If I understand it correctly, then this is all I need on the switch side for now and I now need to sort out the Pfsense end??

          ? 1 Reply Last reply Reply Quote 0
          • ?
            A Former User @stevencavanagh
            last edited by

            @stevencavanagh

            The images are "Not Found."

            S 1 Reply Last reply Reply Quote 0
            • S
              stevencavanagh @A Former User
              last edited by

              @kjk54

              Try these pics...........Pic 1.png Pic 2.png

              ? 1 Reply Last reply Reply Quote 0
              • V viragomann referenced this topic on
              • ?
                A Former User @stevencavanagh
                last edited by

                @stevencavanagh

                I'm not familiar with that switch, but the route looks suspicious to me. There is no interface in it. It should indicate that the VLAN99 interface is to be used for it.

                e47c0ceb-9c01-45ed-8b8f-74bfeaaa1d05-image.png

                Also, have you already tested IPv4 routing on the switch? Have you defined an access port for VLAN99?

                S 1 Reply Last reply Reply Quote 0
                • S
                  stevencavanagh @A Former User
                  last edited by

                  @kjk54

                  I followed a Draytek article that created a route for all traffic that wasn't in the VLANs by ticking the 'default' box above.

                  When I try and add it using the interface i.e. VLAN 99 I get an error.

                  I have not tested IPV4 routing on the switch as at the moment due to WFH, I simply have the two P2280Xs connected on the desk via the uplinks and a laptop so I don't kill off the main home network. Yes, I defined an access port for VLAN99.

                  ? 1 Reply Last reply Reply Quote 0
                  • ?
                    A Former User @stevencavanagh
                    last edited by

                    @stevencavanagh

                    "I followed a Draytek article that created a route for all traffic that wasn't in the VLANs by ticking the 'default' box above."

                    I don't have any idea what it is.

                    Have you enabled the IPv4 routing on your L3 switch? You need to get IPv4 routing up and running on the switch first. You can test it with two devices connected to the same switch. For the test to be valid you need to disconnect the uplinks. Note that in this configuration, the switch interfaces need be the gateways on your endpoint devices, not the pfSense interfaces.

                    S 1 Reply Last reply Reply Quote 0
                    • S
                      stevencavanagh @A Former User
                      last edited by

                      @kjk54

                      The default box was in the pic above and the article was...

                      https://www.draytek.co.uk/support/guides/kb-vigorswitch-vlanrouting

                      However, I have enabled IPv4 routing on the L3 switch. I can disconnect the uplinks but will struggle to test at the moment as I don't have 2 PCs near the switches to test with unfortunately.

                      ? Dobby_D 2 Replies Last reply Reply Quote 0
                      • ?
                        A Former User @stevencavanagh
                        last edited by

                        @stevencavanagh

                        It may be okay. Possibly they just do not bother to show the outgoing interface since it can be determined from the subnet of the next hop.

                        You do not necessarily need another PC. Almost anything with a RJ45 jack can be used.

                        S 1 Reply Last reply Reply Quote 0
                        • Dobby_D
                          Dobby_ @stevencavanagh
                          last edited by

                          @stevencavanagh

                          In normal you can use all switches as Layer2 switches and the pfSense
                          is routing the entire traffic and also the vlans or you may be set up a
                          transfer net between the pfSense and the or one Layer3 switch and the
                          Layer3 switch is routing then the vlans perhaps a bit faster, but you may
                          be setting up then ACLs on the switch for the vlan traffic (to secure or allow/deny)

                          #~. @Dobby

                          Turris Omnia - 4 Ports - 2 GB RAM / TurrisOS 7 Release (Btrfs)
                          PC Engines APU4D4 - 4 Ports - 4 GB RAM / pfSense CE 2.7.2 Release (ZFS)
                          PC Engines APU6B4 - 4 Ports - 4 GB RAM / pfSense+ (Plus) 24.03_1 Release (ZFS)

                          S 1 Reply Last reply Reply Quote 0
                          • S
                            stevencavanagh @A Former User
                            last edited by

                            @kjk54

                            Okay, I'm sure I can find something with a RJ45 connector!

                            1 Reply Last reply Reply Quote 0
                            • S
                              stevencavanagh @Dobby_
                              last edited by

                              @Dobby_ said in Best Network Topology with Current Hardware:

                              @stevencavanagh

                              In normal you can use all switches as Layer2 switches and the pfSense
                              is routing the entire traffic and also the vlans or you may be set up a
                              transfer net between the pfSense and the or one Layer3 switch and the
                              Layer3 switch is routing then the vlans perhaps a bit faster, but you may
                              be setting up then ACLs on the switch for the vlan traffic (to secure or allow/deny)

                              I am currently using the switches as Layer 2 and routing the VLANs via Pfsense but the inter VLAN speed is pathetic, much lower than it should be but I can't seem to get to the bottom of it. For example Iperf to NAS on 1G network.....

                              55f8f4aa-100c-4d93-8413-1733167e0d9a-image.png

                              Could put the NAS on the main LAN and get wire speed but still need to access it from the IOT network (firestick)! Hence the reason for going down the L3 route.

                              Same LAN speed is fine.......Between 2 PCs on same VLAN........

                              f173c05d-9e26-485d-b1eb-cce6744e9d91-image.png

                              ? 1 Reply Last reply Reply Quote 0
                              • Dobby_D
                                Dobby_
                                last edited by

                                The NAS speed can be also based on the HDD/SSD´s speed!

                                Perhaps there is also not the max. load on the line, so you could try out iPerf
                                with multiple streams to gain the load for getting better "numbers".

                                #~. @Dobby

                                Turris Omnia - 4 Ports - 2 GB RAM / TurrisOS 7 Release (Btrfs)
                                PC Engines APU4D4 - 4 Ports - 4 GB RAM / pfSense CE 2.7.2 Release (ZFS)
                                PC Engines APU6B4 - 4 Ports - 4 GB RAM / pfSense+ (Plus) 24.03_1 Release (ZFS)

                                S 1 Reply Last reply Reply Quote 0
                                • S
                                  stevencavanagh @Dobby_
                                  last edited by

                                  @Dobby_ said in Best Network Topology with Current Hardware:

                                  The NAS speed can be also based on the HDD/SSD´s speed!

                                  Perhaps there is also not the max. load on the line, so you could try out iPerf
                                  with multiple streams to gain the load for getting better "numbers".

                                  How do I run Iperf with multiple streams?

                                  Totally get that the NAS speed could be affected by HDD speed but they are ironwolfs in a Synology DS920+ and reading around people seem to be getting much better results than me!

                                  1 Reply Last reply Reply Quote 0
                                  • ?
                                    A Former User @stevencavanagh
                                    last edited by

                                    @stevencavanagh said in Best Network Topology with Current Hardware:

                                    For example Iperf to NAS on 1G network.....

                                    And it is ~50 Mbits/sec? You must have a link speed issue. More like 100M, not 1G.

                                    S 1 Reply Last reply Reply Quote 0
                                    • S
                                      stevencavanagh @A Former User
                                      last edited by

                                      @kjk54 said in Best Network Topology with Current Hardware:

                                      @stevencavanagh said in Best Network Topology with Current Hardware:

                                      For example Iperf to NAS on 1G network.....

                                      And it is ~50 Mbits/sec? You must have a link speed issue. More like 100M, not 1G.

                                      Totally agree but I cannot find the reason at all.

                                      Nothing is running at 100MB. NAS, both switches (P1280) & both PCs are showing as running as 1G. Can only assume it is either the LAG from switch 1 to switch 2 or LAG from switch 1 to Pfsense but they are indicating all is good at 1G

                                      1 Reply Last reply Reply Quote 0
                                      • stephenw10S
                                        stephenw10 Netgate Administrator
                                        last edited by

                                        Some asymmetric routing? Are you seeing a lot of retries/retransmissions?

                                        S 1 Reply Last reply Reply Quote 0
                                        • S
                                          stevencavanagh @stephenw10
                                          last edited by

                                          @stephenw10 said in Best Network Topology with Current Hardware:

                                          Some asymmetric routing? Are you seeing a lot of retries/retransmissions?

                                          Where is the best place to check for this? Would it be in the System Logs somewhere?

                                          1 Reply Last reply Reply Quote 0
                                          • stephenw10S
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            iperf normally reports retries at one end. Otherwise a pcap would show retransmissions.

                                            S 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.