Forward traffic on same interface / VEPA bridging

lars.lindstrom

Hi everyone!

I am operating a server hosting a set of services, each run in a separate Docker container. In addition, there is a KVM running pfSense acting as firewall. The firewall has a physical interface that is connected to the external network and a virtual network card that is connected to the internal container network, using MACVLAN Docker-side, so each container has its own IP address, but all of them are in the same subnet.

For security reasons, the containers need to be isolated and shall not be able to communicate with each other principally (just with the external network). For this, MACVLAN is configured in VEPA mode, which allows traffic from and to the parent device, but not to other addresses on the same parent device.

Now, I would like to allow specific traffic between specific containers, so pfSense must act as a VEPA bridge/router, considering the configured firewall rules (read, if incoming traffic on the internal interface matches a PASS rule it shall be forwarded to a host on the same interface / same subnet).

I can't seem to get that scenario working (no traffic between the hosts on the internal interface, traffic from and to the external network works as expected). Any ideas on how to proceed from here?

Thanks!

br Lars

lars.lindstrom

To be honest, I do not even known we this is not working in the first place.

If a packet arrives at the interface pfSense should forward the package according to the routing table, which will cause the package to go out on the same interface again.

Is there any configuration item the prevents such scenarios (like "filter traffic on own interface" or alike)?

lars.lindstrom

Interestingly enough, pfSense does not even reply to the ARP request:

[root@server ~]# ip r
default via 10.0.20.1 dev server proto static metric 410 
10.0.20.0/24 dev server proto kernel scope link src 10.0.20.2 metric 410

21:52:49.651286 ARP, Request who-has 10.0.20.4 tell 10.0.20.2, length 28
21:52:50.673895 ARP, Request who-has 10.0.20.4 tell 10.0.20.2, length 28
21:52:51.697860 ARP, Request who-has 10.0.20.4 tell 10.0.20.2, length 28
21:52:52.721992 ARP, Request who-has 10.0.20.4 tell 10.0.20.2, length 28

I would assume a response with the MAC of the 10.0.20.0/24 interface. The trace was made on the firewall on that interface (PING from the firewall to 10.0.20.4 works as expected).

When adding the entry manually I can see the ICMP echo request, but no reply:

[root@server ~]# arp -s 10.0.20.4 02:42:0a:00:14:04
10.0.20.4                ether   02:42:0a:00:14:04   CM                    server

22:00:21.403515 IP 10.0.20.2 > 10.0.20.4: ICMP echo request, id 5622, seq 1, length 64
22:00:22.450162 IP 10.0.20.2 > 10.0.20.4: ICMP echo request, id 5622, seq 2, length 64
22:00:23.473790 IP 10.0.20.2 > 10.0.20.4: ICMP echo request, id 5622, seq 3, length 64
22:00:24.497803 IP 10.0.20.2 > 10.0.20.4: ICMP echo request, id 5622, seq 4, length 64

The interface has a IPv4 ICMP PASS * * rule configured.

lars.lindstrom

Any idea?

bobymicroby

@lars-lindstrom did you managed to figure this one out ?