Pfsense with 3 NICS

Jamerson

@Jamerson:

Dear Bryan,
the Goal is both not to reach each otherse, from each side there is a File server which is gonna be restricted. also LAN2 NIC will have different restriction than LAN1

Stephen
LAN 1 is 192.168.4.0/24
LAN2 is 192.168.6.0/24

John:
Automatic NAT is on
the firewall rules are exactly the same on both NICS, Allow all
but LAN1 works and LAN2 no,

the only different is LAN2 not attached to a Physique NIC and LAN1 it does so as it showen on the attached

Go to Esxi - firewall vm properties -> Add a third network card and place it on LAN2
Go to pFsense -> Interfaces -> Make sure the third interface is assigned and labeled
Go to pFsense -> Services -> DHCP Server -> Press the third interface tab -> Setup up DHCP
Go to pFsense -> Firewall -> Outbound -> Press "Add a new rule based on this one" For the "Auto created rule for LAN to WAN" and change the source to reflect your second LAN subnet

Bryan,
Go to pFsense -> Firewall -> Outbound -> Press "Add a new rule based on this one" For the "Auto created rule for LAN to WAN" and change the source to reflect your second LAN subnet
Do you mean create the same rule as LAN1 ? or Edit the Rule ?
on the firewall in dont have OUTBOUND option.
do i need some plug in ?

stephenw10

It's in Firewall: NAT: Outbound:
However by default the outbound NAT mode is set to 'Automatic outbound NAT rule generation'. That will work fine for your dual LAN setup. There should be need to add manual outbound rules as long as it's still set to automatic.

Please post your LAN2 firewall rules, a screenshot if possible.

Steve

Jamerson

@stephenw10:

It's in Firewall: NAT: Outbound:
However by default the outbound NAT mode is set to 'Automatic outbound NAT rule generation'. That will work fine for your dual LAN setup. There should be need to add manual outbound rules as long as it's still set to automatic.

Please post your LAN2 firewall rules, a screenshot if possible.

Steve

hi Stephen
there is only one rule to allow the whole traffic,
attached is the screenshot
how can i do this exactly ?

Go to pFsense -> Firewall -> Outbound -> Press "Add a new rule based on this one" For the "Auto created rule for LAN to WAN" and change the source to reflect your second LAN subnet

on the Outband there is no Manual rules yet. just Automatically.
i've configured a user with 192.168.6.0/24 on the same NIC but can't ping 192.168.6.1 the router
firewall screenshot is of the client behind the LAN2 when try to browse.
Much appreciate it !

LAN2.jpg_thumb

Firewall.jpg_thumb

bryan.paradis

@stephenw10:

It's in Firewall: NAT: Outbound:
However by default the outbound NAT mode is set to 'Automatic outbound NAT rule generation'. That will work fine for your dual LAN setup. There should be need to add manual outbound rules as long as it's still set to automatic.

Please post your LAN2 firewall rules, a screenshot if possible.

Steve

Yeah that makes sense. It should make it automatically

johnpoz

Why would you have a block rfc1918 rule on your LAN interface? You do understand your lan2 would be private, and pfsense own IPs on that lan would be rfc1918 as well

LAN 1 is 192.168.4.0/24
LAN2 is 192.168.6.0/24

And how is anything suppose to get to lan2 when there is nothing in your vswitch for lan 2 - what interface in pfsense is connected to lan2?

stephenw10

Yep that's your problem.
Go to Interfaces: LAN1: and uncheck 'block private networks' and 'block bogon networks'.
Bogons may not be necessary but there's no need to block them on LAN.

Steve

bryan.paradis

@johnpoz:

Why would you have a block rfc1918 rule on your LAN interface? You do understand your lan2 would be private, and pfsense own IPs on that lan would be rfc1918 as well

LAN 1 is 192.168.4.0/24
LAN2 is 192.168.6.0/24

And how is anything suppose to get to lan2 when there is nothing in your vswitch for lan 2 - what interface in pfsense is connected to lan2?

@stephenw10:

Yep that's your problem.
Go to Interfaces: LAN1: and uncheck 'block private networks' and 'block bogon networks'.
Bogons may not be necessary but there's no need to block them on LAN.

Steve

What is the default on those when configuring a new interface ?

johnpoz

OFF, it only defaults to be on WAN interface.

bryan.paradis

@johnpoz:

OFF, it only defaults to be on WAN interface.

This is what I had thought so I assume he must have changed options or maybe reversed his iunterfaces at some point and had to reassign?

Jamerson

Why would you have a block rfc1918 rule on your LAN interface? You do understand your lan2 would be private, and pfsense own IPs on that lan would be rfc1918 as well

LAN 1 is 192.168.4.0/24
LAN2 is 192.168.6.0/24

And how is anything suppose to get to lan2 when there is nothing in your vswitch for lan 2 - what interface in pfsense is connected to lan2?

hi Bryan,
i am using all my 3 NICS to use Teaming so all traffic will go thought the WAN.

 Yep that's your problem.
Go to Interfaces: LAN1: and uncheck 'block private networks' and 'block bogon networks'.
Bogons may not be necessary but there's no need to block them on LAN.

Steve

those rules were added automatically when i created the rules.
you mean i need to uncheck all private Network between the LANS ?

is it possible to have all traffic go out and in thought the WAN wihtout assign a physique NIC on the Vswitch ?

creating a Vlan with one Physique NIC could fix the issue ?

stephenw10

Uncheck the two boxes on the interface setup page.
It doesn't matter that all the traffic is going to the WAN. The source is the LAN subnet which is a private network. If you look at your firewall rules for that interface you'll see that the 'block private networks' rules is at the top of the list so it matches the incoming packets first and the 'allow all rules' are never reached. The 'block private networks' check box is only there for WAN interfaces where you would not expect traffic arriving with an RFC1918 source address unless it's been spoofed.

Steve

johnpoz

"those rules were added automatically when i created the rules."

No I highly doubt that, nnless you created a WAN interface, by putting a gateway on it?

"i am using all my 3 NICS to use Teaming so all traffic will go thought the WAN."

What – no you are not.. I suggest you look up the term "teaming" as it has to do with network interfaces - you are not doing that ;)

"is it possible to have all traffic go out and in thought the WAN wihtout assign a physique NIC on the Vswitch ?"

Sure - if your Other vm's have an interface in that vswitch - see example below on my dmz vswitch. There is an interface connected to my pfsense VM, and then the other vms that need to use this interface on pfsense are tied to that same vswitch. It does not need to be tied to the physical network. But it can access the physical network through pfsense that has its other lan and wlan in my case tied to physical networks. See attachment 2

nophysical.png_thumb

Jamerson

@johnpoz:

"those rules were added automatically when i created the rules."

No I highly doubt that, nnless you created a WAN interface, by putting a gateway on it?
"i am using all my 3 NICS to use Teaming so all traffic will go thought the WAN."

What -- no you are not.. I suggest you look up the term "teaming" as it has to do with network interfaces - you are not doing that ;)
this where i want to go to, use all my NIC for teaming

"is it possible to have all traffic go out and in thought the WAN wihtout assign a physique NIC on the Vswitch ?"

Sure - if your Other vm's have an interface in that vswitch - see example below on my dmz vswitch. There is an interface connected to my pfsense VM, and then the other vms that need to use this interface on pfsense are tied to that same vswitch. It does not need to be tied to the physical network. But it can access the physical network through pfsense that has its other lan and wlan in my case tied to physical networks. See attachment 2

the machine on your DMZ can reach the internet even there is no Physique interface?
this exactly what i am looking for.

johnpoz

Yes why would they not.. They are connected to a switch that is in connected to pfsense interface (be it all virtual), which has physical connection to the real network.

That is not TEAMING.. where did you come up that you think you want to do teaming?

Teaming in reference to network, call it link aggregation, port trunking, nic teaming, bonding has nothing to do with this.

But notice in your drawing - there is NOTHING connected to your switch, no vm's no interface in pfsense that your calling lan2, but where is that connected to a vswitch?

stephenw10

Are you saying that you would like to able to this without any physical NICs (which you can) because you want to use the NICs you have in an aggregated connection? You have setup you NICs as they are currently because you didn't realise you could do it all virtually without any NICs?

Steve

bryan.paradis

i am using all my 3 NICS to use Teaming so all traffic will go thought the WAN.

Guess it could be a lot of things. Jamerson you need to really clarify and perhaps draw a diagram of what you want.

johnpoz

Well clearly his NOT doing that, since he has a nic in his lan network and nic in his wan network - so how could he be "teaming" them. He clearly doesn't understand the term.

For that matter he doesn't even have 3 nics, be it physical or VM.. Look at his esxi host networking he posted

He doesn't even have a lan2 interface listed in his VM networking for pfsense, which I take is the firewall vm.. Yet he says he created firewall rules in pfsense for lan? So what did he connect to lan2? Mystery to be sure.

Jamerson

sorry guys for my late reply ( been sick ).
about the NIC teaming haven't configured it yet on the LAB and I know what is NIC teaming by the way, that why you don't see it on the Diagram.
what I need is when VM on the same LAN2 on ESXI that they need to reach the internet.
My PFSENSE has 3 NICS,
WAN and LAN1 and LAN2
LAN 1 and LAN 2 are not attached to Physical NIC,
attached is screenshot of the LAB
I didn't add the NIC yet of the PFSENSE to each group LAN 1 and LAN 2.
on LAN 1 will have PFSENSE NIC 2
on LAN 2 will have PFSENSE NIC 3
so when I move a VM to the LAN 1 0r LAN 2 it doesn't reach the internet. this is my problem.
when I attached a physical NIC to LAN 1 or LAN 2 on the ESXI it works

LAN1.jpg_thumb

johnpoz

Dude how is anything going to use pfsense if pfsense has no VIRTUAL nic in the switch??

How is any VM going to connect to pfsense lan1 or lan2 if there is NOTHING connected to the vswitch? Does not matter if the vswitch is not connected to the physical world or not..

See bottom of my virtual network from esxi

See how pfsense has a virtual interface connected into the vswitch, so that other virtual machines with a virtual interface connected to this vswitch can connect to pfsense and then access the real world (internet or lan or wlan) via the physical nics connected to those vswitches.

You have nothing connected to your lan1 or lan2 from your drawing - no pfsense interfaces, no vms – so NO how would anything use it.. What does pfsense show for its interfaces, since from your drawing you don't show any lan1 or lan2 interfaces for pfsense at all.

So for example


[2.1.1-PRERELEASE][admin@pfsense.local.lan]/root(1): ifconfig
em0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
        options=98 <vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:50:56:00:00:02
        inet 192.168.1.253 netmask 0xffffff00 broadcast 192.168.1.255
        inet6 fe80::250:56ff:fe00:2%em0 prefixlen 64 scopeid 0x1
        inet6 2001:470:1f11:9c4::1 prefixlen 64
        nd6 options=1 <performnud>media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
em1: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
        options=98 <vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:50:56:00:00:01
        inet 24.13.xx.xx netmask 0xfffff800 broadcast 255.255.255.255
        inet6 fe80::250:56ff:fe00:1%em1 prefixlen 64 scopeid 0x2
        nd6 options=1 <performnud>media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
em2: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
        options=98 <vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:0c:29:1e:18:90
        inet 192.168.2.253 netmask 0xffffff00 broadcast 192.168.2.255
        inet6 fe80::20c:29ff:fe1e:1890%em2 prefixlen 64 scopeid 0x3
        nd6 options=1 <performnud>media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
em3: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
        options=98 <vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:0c:29:1e:18:9a
        inet 192.168.3.253 netmask 0xffffff00 broadcast 192.168.3.255
        inet6 fe80::20c:29ff:fe1e:189a%em3 prefixlen 64 scopeid 0x4
        nd6 options=1 <performnud>media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active</full-duplex></performnud></vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast></full-duplex></performnud></vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast></full-duplex></performnud></vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast></full-duplex></performnud></vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast>

Those are the interfaces in my virtual pfsense.. Se how pfsense has 4 interfaces. One in each segment (wan, lan, wlan, dmz) notice how in my esxi network pfsense is listed as having a connection in each vswitch that is connected to each one of pfsense interfaces. If I connect a virtual machine to that vswitch, via a virtual interface. Like the W7 machine on the bottom above pfsense on the dmz vswitch.. It can talk to pfsense over the vswitch on its dmz virtual interface and then pfsense can route it to any of its other interfaces over virtual - which may or may not have access into the physical world via a physical adapter connected to that vswitch.

pfsensemissing.png_thumb

pfsenseconnected.png_thumb

Jamerson

@johnpoz:

Dude how is anything going to use pfsense if pfsense has no VIRTUAL nic in the switch??

How is any VM going to connect to pfsense lan1 or lan2 if there is NOTHING connected to the vswitch? Does not matter if the vswitch is not connected to the physical world or not..

See bottom of my virtual network from esxi

See how pfsense has a virtual interface connected into the vswitch, so that other virtual machines with a virtual interface connected to this vswitch can connect to pfsense and then access the real world (internet or lan or wlan) via the physical nics connected to those vswitches.

You have nothing connected to your lan1 or lan2 from your drawing - no pfsense interfaces, no vms – so NO how would anything use it.. What does pfsense show for its interfaces, since from your drawing you don't show any lan1 or lan2 interfaces for pfsense at all.

So for example
[2.1.1-PRERELEASE][admin@pfsense.local.lan]/root(1): ifconfig
em0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
        options=98 <vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:50:56:00:00:02
        inet 192.168.1.253 netmask 0xffffff00 broadcast 192.168.1.255
        inet6 fe80::250:56ff:fe00:2%em0 prefixlen 64 scopeid 0x1
        inet6 2001:470:1f11:9c4::1 prefixlen 64
        nd6 options=1 <performnud>media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
em1: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
        options=98 <vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:50:56:00:00:01
        inet 24.13.xx.xx netmask 0xfffff800 broadcast 255.255.255.255
        inet6 fe80::250:56ff:fe00:1%em1 prefixlen 64 scopeid 0x2
        nd6 options=1 <performnud>media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
em2: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
        options=98 <vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:0c:29:1e:18:90
        inet 192.168.2.253 netmask 0xffffff00 broadcast 192.168.2.255
        inet6 fe80::20c:29ff:fe1e:1890%em2 prefixlen 64 scopeid 0x3
        nd6 options=1 <performnud>media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
em3: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
        options=98 <vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:0c:29:1e:18:9a
        inet 192.168.3.253 netmask 0xffffff00 broadcast 192.168.3.255
        inet6 fe80::20c:29ff:fe1e:189a%em3 prefixlen 64 scopeid 0x4
        nd6 options=1 <performnud>media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active</full-duplex></performnud></vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast></full-duplex></performnud></vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast></full-duplex></performnud></vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast></full-duplex></performnud></vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast> 
Those are the interfaces in my virtual pfsense.. Se how pfsense has 4 interfaces. One in each segment (wan, lan, wlan, dmz) notice how in my esxi network pfsense is listed as having a connection in each vswitch that is connected to each one of pfsense interfaces. If I connect a virtual machine to that vswitch, via a virtual interface. Like the W7 machine on the bottom above pfsense on the dmz vswitch.. It can talk to pfsense over the vswitch on its dmz virtual interface and then pfsense can route it to any of its other interfaces over virtual - which may or may not have access into the physical world via a physical adapter connected to that vswitch.

hi John,
Let me thank you for your continue help really appreciate it.
apologies i've uploaded the wrong screenshot !
the attached is the correct screenshot

when a machine using LAN1 NIC as Gateway is not accessible the internet !

em0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
        options=9b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:50:56:a8:1a:4a
        inet 192.168.2.3 netmask 0xffffff00 broadcast 192.168.2.255
        inet6 fe80::250:56ff:fea8:1a4a%em0 prefixlen 64 scopeid 0x1
        nd6 options=1 <performnud>media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
em1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
        options=9b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:50:56:8c:7e:c8
        inet 192.168.4.1 netmask 0xffffff00 broadcast 192.168.4.255
        inet6 fe80::250:56ff:fe8c:7ec8%em1 prefixlen 64 scopeid 0x2
        nd6 options=1 <performnud>media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
em2: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
        options=9b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:50:56:8c:a6:3b
        inet 192.168.6.0 netmask 0xffffff00 broadcast 192.168.6.255
        inet6 fe80::250:56ff:fe8c:a63b%em2 prefixlen 64 scopeid 0x3
        nd6 options=1 <performnud>media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
plip0: flags=8810 <pointopoint,simplex,multicast>metric 0 mtu 1500
pflog0: flags=100 <promisc>metric 0 mtu 33144
lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
        options=3 <rxcsum,txcsum>inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
        nd6 options=3 <performnud,accept_rtadv>pfsync0: flags=0<> metric 0 mtu 1460
        syncpeer: 224.0.0.240 maxupd: 128 syncok: 1
enc0: flags=0<> metric 0 mtu 1536
bridge0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
        ether 02:1b:ed:00:ce:00
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: em2 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 3 priority 128 path cost 20000
        member: em1 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 2 priority 128 path cost 20000
ovpns1: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1500
        options=80000 <linkstate>inet6 fe80::250:56ff:fea8:1a4a%ovpns1 prefixlen 64 scopeid 0xa
        inet 192.168.200.1 --> 192.168.200.2 netmask 0xffffffff
        nd6 options=3 <performnud,accept_rtadv>Opened by PID 10306</performnud,accept_rtadv></linkstate></up,pointopoint,running,multicast></learning,discover,autoedge,autoptp></learning,discover,autoedge,autoptp></up,broadcast,running,simplex,multicast></performnud,accept_rtadv></rxcsum,txcsum></up,loopback,running,multicast></promisc></pointopoint,simplex,multicast></full-duplex></performnud></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,promisc,simplex,multicast></full-duplex></performnud></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,promisc,simplex,multicast></full-duplex></performnud></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast>

LAN1.jpg_thumb