[SOLVED] em1 active when only VLANs are used within the interface (Virtualbox)
-
Your not showing vlan info in the tcpdump - so not sure what your asking about the 22-55020 traffic? Are you asking if its tagged or untagged?
I was wondering why I see this traffic with the command: "tcpdump -n -v -i em1". Not sure why my ssh towards em1.3 connexion is being seen on em1 (Pfsense 10.0.0.254 is configured on em1.3 and not em1).
I have only one ssh connection between 10.20.30.3 to 10.0.0.254 (and one vnc between 10.20.30.3 and 10.0.0.1)
[2.4.1-RELEASE][admin@MUR.localdomain]/root: tcpdump -n -i em1.3 -c 10 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on em1.3, link-type EN10MB (Ethernet), capture size 262144 bytes 19:56:02.735418 IP 10.20.30.3.62502 > 10.0.0.1.5900: Flags [.], ack 1559264236, win 3735, length 0 19:56:02.735674 IP 10.0.0.1.5900 > 10.20.30.3.62502: Flags [.], seq 68621:70081, ack 0, win 229, length 1460 19:56:02.735721 IP 10.0.0.1.5900 > 10.20.30.3.62502: Flags [.], seq 70081:71541, ack 0, win 229, length 1460 19:56:02.735888 IP 10.0.0.1.5900 > 10.20.30.3.62502: Flags [.], seq 71541:73001, ack 0, win 229, length 1460 19:56:02.735927 IP 10.0.0.1.5900 > 10.20.30.3.62502: Flags [.], seq 73001:74461, ack 0, win 229, length 1460 19:56:02.735960 IP 10.0.0.1.5900 > 10.20.30.3.62502: Flags [.], seq 74461:75921, ack 0, win 229, length 1460 19:56:02.735987 IP 10.0.0.1.5900 > 10.20.30.3.62502: Flags [.], seq 75921:77381, ack 0, win 229, length 1460 19:56:02.736017 IP 10.0.0.1.5900 > 10.20.30.3.62502: Flags [.], seq 77381:78841, ack 0, win 229, length 1460 19:56:02.736044 IP 10.0.0.1.5900 > 10.20.30.3.62502: Flags [.], seq 78841:80301, ack 0, win 229, length 1460 19:56:02.736081 IP 10.0.0.1.5900 > 10.20.30.3.62502: Flags [.], seq 80301:81761, ack 0, win 229, length 1460 10 packets captured 71 packets received by filter 0 packets dropped by kernel [2.4.1-RELEASE][admin@MUR.localdomain]/root: tcpdump -n -i em1 -c 10 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on em1, link-type EN10MB (Ethernet), capture size 262144 bytes 19:56:13.579747 IP 10.0.0.254.22 > 10.20.30.3.62669: Flags [P.], seq 3889019166:3889019406, ack 1239262628, win 513, length 240 19:56:13.580017 IP 10.0.0.254.22 > 10.20.30.3.62669: Flags [P.], seq 240:464, ack 1, win 513, length 224 19:56:13.580139 IP 10.0.0.254.22 > 10.20.30.3.62669: Flags [P.], seq 464:656, ack 1, win 513, length 192 19:56:13.580220 IP 10.0.0.254.22 > 10.20.30.3.62669: Flags [P.], seq 656:848, ack 1, win 513, length 192 19:56:13.580321 IP 10.0.0.254.22 > 10.20.30.3.62669: Flags [P.], seq 848:1040, ack 1, win 513, length 192 19:56:13.580413 IP 10.0.0.254.22 > 10.20.30.3.62669: Flags [P.], seq 1040:1232, ack 1, win 513, length 192 19:56:13.580505 IP 10.0.0.254.22 > 10.20.30.3.62669: Flags [P.], seq 1232:1424, ack 1, win 513, length 192 19:56:13.580598 IP 10.0.0.254.22 > 10.20.30.3.62669: Flags [P.], seq 1424:1616, ack 1, win 513, length 192 19:56:13.580690 IP 10.0.0.254.22 > 10.20.30.3.62669: Flags [P.], seq 1616:1808, ack 1, win 513, length 192 19:56:13.580781 IP 10.0.0.254.22 > 10.20.30.3.62669: Flags [P.], seq 1808:2000, ack 1, win 513, length 192 10 packets captured 10 packets received by filter 0 packets dropped by kernelKey here is broadcast traffic. Your switch/router/other is sending untagged broadcast traffic to the em1 port.
All the ports configured on my switch are with Tagging and none is with untag (others are excluded).
-
Regardless of how it is configured your switch is still sending untagged traffic on that port.
You might try changing the PVID on the port going to pfSense to some otherwise-unused VLAN ID.
-
Regardless of how it is configured your switch is still sending untagged traffic on that port.
You might try changing the PVID on the port going to pfSense to some otherwise-unused VLAN ID.
FYI Port 24 is Pfsense internal with this config. How can i change the PVID of the untag traffic which is the same as the 3 VLANS. Below some config screens.
em1.3: flags=8a43 <up,broadcast,running,allmulti,simplex,multicast>metric 0 mtu 1500 options=3 <rxcsum,txcsum>ether 08:00:27:86:b6:2d inet6 fe80::a00:27ff:fe86:b62d%em1.3 prefixlen 64 scopeid 0x7 inet 10.0.0.254 netmask 0xffffff00 broadcast 10.0.0.255 inet 10.7.4.1 netmask 0xffffffff broadcast 10.7.4.1 nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (1000baseT <full-duplex>) status: active vlan: 3 vlanpcp: 0 parent interface: em1 groups: vlan em1.5: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500 options=3 <rxcsum,txcsum>ether 08:00:27:86:b6:2d inet6 fe80::a00:27ff:fe86:b62d%em1.5 prefixlen 64 scopeid 0x8 inet 10.10.10.254 netmask 0xffffff00 broadcast 10.10.10.255 nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (1000baseT <full-duplex>) status: active vlan: 5 vlanpcp: 0 parent interface: em1 groups: vlan em1.4: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500 options=3 <rxcsum,txcsum>ether 08:00:27:86:b6:2d inet6 fe80::a00:27ff:fe86:b62d%em1.4 prefixlen 64 scopeid 0x9 inet 10.20.30.254 netmask 0xffffff00 broadcast 10.20.30.255 nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (1000baseT <full-duplex>) status: active vlan: 4 vlanpcp: 0 parent interface: em1 groups: vlan</full-duplex></performnud,auto_linklocal></rxcsum,txcsum></up,broadcast,running,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum></up,broadcast,running,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum></up,broadcast,running,allmulti,simplex,multicast>A0:36:9F:88:E4:72 is the MAC address of the physical port on the Host Virtualbox interface. Why is this being seen if the port is accepting only TAG traffic.
-
When you packet capture on em1 you have to look at the VLAN tags. A pcap there will include all tagged and untagged traffic arriving on that interface.
A packet capture on a VLAN interface such as em1.3 will not include dot1q tags and will only include traffic that was/is to be so tagged.
-
Pfsense is a VM… What other devices are on the same vswitch? On the esxi host?
-
Pfsense is a VM… What other devices are on the same vswitch? On the esxi host?
Tell me if thus helps as it took me some time to do. https://forum.pfsense.org/index.php?action=dlattach;topic=139245.0;attach=108551
I have 5 nics and 3 vms : pfsense, a router for my dsl accesses and a domotic one
Good sunday
-
I saw your pic already… It does not show how your vswitch setup on your host.. Or what VM software you using either..
Are these physical nics connected to the same vswitch and broken out into port groups, etc.
Example have multiple vswitches, tied to different physical host nics or not (see attached example of 1). They can then either besetup as say trunk port with 4095 as the vlan ID, or they can be setup as like dumb switches and strip all tags before pfsense would see them with vlan id 0... Or they could be setup with port groups and have specific vlan IDs set, etc..
You have to deal with your virtual networking switch environment as you do you physical network the nics on your host are just uplinks to another switch is all. And then is all handled slightly different depending on what your actually using for your VM host.. be it Xen, Hyper-V, Esxi or maybe your just using VirtualBox or KVM, etc.
edit: Just noticed your running Virtualbox.. Yeah that can be all messed up.. How are are you physical host nics tied to its virtual networking? You list 5 nics, but only 2 bridged networks?
-
Hello johnpoz
Thank you for your help and support.
All my NICs are physical ones. I bought a i350-t4 card which gave me a total of 5 physical gig ports which I wanted to not mix the traffic in virtual nics.
I am using VirtualBox 5.1.x and Pfsense public is an untag physical port to the switch vlan Wan and the private port is a tagged physical port with 3 vlans.
I don't have access to my home as just got a small lady and we are still in the hospital. All good and joy.
Buy will add them tonight if this is not clear enough. Note my signature too for the details of the port modes within VirtualBox.
Merci
-
Congrats on the small lady addition ;)
Been a while since played with virtualbox.. Isn't current 5.2? I do recall back in the day that virtualbox liked to strip tags.. So you could have issues if your sending tagged and untagged traffic to the same nic that is in bridge mode on a switch port that is sending tagged and untagged.. Like I said its been a bit since played with virtual box..
Why not just run a type 1 VM OS on this box?
-
Congrats on the small lady addition ;)
Then you!
Been a while since played with virtualbox.. Isn't current 5.2? I do recall back in the day that virtualbox liked to strip tags.. So you could have issues if your sending tagged and untagged traffic to the same nic that is in bridge mode on a switch port that is sending tagged and untagged.. Like I said its been a bit since played with virtual box..
Why not just run a type 1 VM OS on this box?
You are right it's 5.2 the latest. I started with an Ubuntu server and then added vms as I needed. It was not designed to be initially. VirtualBox is free and easy to use so didn't think of reinstalling it as a type 1 hyperversor. Not sure how much I will win and the free options may asked me to invest time in discovering new technology. I tried a while back Xen and it s was not that easy. Not sure if it was a true type 1.
Currently on VirtualBox one nic is untagged (public) and the other nic is tagged (private) so I am not mixing tagged and untagged in the same interface but I can try to tag the public one in Pfsense and in the switch.
Still my issue is more linked to the private interface where em1 traffic is being discarded on the firewall logs while this interface doesn't exist, only the van ones do. So it s me a display issue (as I can't not log them as I can't create a few rule on an unexisting interface). From a functionality I don't think it's affecting while I noticed web browsing slow with DNSBL and a vip floating ip address accessible and dans lookups quick).I was thinking of reinstalling but last time I exported imported I even having lost quite some configs like static dhcp, DNSBL aka PfBlocker etc… So not sure I want to redo it all as I have limited free time in the coming weeks lol.
I recall trying to create em1 then having to reboot as losing connectivity. Not sure why but I got some pré configured fw rules coming from my CAM interface/vlan. Therefore I tweaked them but felt strange to have a fake em1 created for that and worried than another issue could arise.
Are there while ssh cmd that would be worth double checking? Before attempting to redo a config? I will investigate if there is a better way to do a backup too
I would be back home in 5h so will add some screens fyi on the VirtualBox config which seem pretty standard to me.
Merci
-
From what I remember with virtualbox.. So you have these vlan interfaces setup in your host.. Ubuntu? When you want a VM to see traffic on a vlan interface vm network needs to be set to that vlan.. Not the interface itself.
This way ubuntu is handling the vlan tags and all your VM sees is untagged traffic.. So in pfsense you wouldn't be setting up any vlans at all.. To pfsense it would just be a native interface on that network.
example here is a ubuntu vm of mine that is using vlans..
ifconfig output, just showing a few of the vlan interfaces.
eth0.100 Link encap:Ethernet HWaddr 00:0c:29:f1:a5:4f
inet addr:192.168.5.20 Bcast:192.168.5.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fef1:a54f/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:19812 errors:0 dropped:0 overruns:0 frame:0
TX packets:1743092 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1023373 (1.0 MB) TX bytes:73253925 (73.2 MB)eth0.200 Link encap:Ethernet HWaddr 00:0c:29:f1:a5:4f
inet addr:192.168.4.20 Bcast:192.168.4.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fef1:a54f/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:206991 errors:0 dropped:0 overruns:0 frame:0
TX packets:1806062 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:12969379 (12.9 MB) TX bytes:76346840 (76.3 MB)eth0.300 Link encap:Ethernet HWaddr 00:0c:29:f1:a5:4f
inet addr:192.168.6.20 Bcast:192.168.6.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fef1:a54f/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:10371 errors:0 dropped:0 overruns:0 frame:0
TX packets:1754579 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:477066 (477.0 KB) TX bytes:73692670 (73.6 MB)See the vlans are setup in ubuntu itself.. You would then bridge these specific interfaces or subinterfaces vlan interfaces, different terms for the same thing.. You would then connect these to your vm via the bridged interface in virtualbox..
From what I remember you wouldn't do this with virtualbox
"em1 with VLANs for the LAN, DMZ and WIFI. "
You would just have the VM with em2, em3, em4 tied to the specific vlans in your virtualbox networking - pfsense would never see any tags, etc.
-
Again thank you for your active support.
Thanks to you I have solved 2 issues: one is getting better performances and the other to have the VLANs working.I will therefore move it all to Proxmox after having read a lot about hypervisor type 1 and VLAN tagging with Virtuabox. one of the post which gives this conclusion without much context is: https://community.ubnt.com/t5/UniFi-Routing-Switching/Solved-How-to-connect-Virtual-Machines-to-a-different-subnet/td-p/1840661 but that summarize my googling :)
if some people are interested,
http://www.aitek.ch/migrating-virtualbox-vdi-to-proxmox-ve-proxmox-support-forum/
https://rmoff.net/2016/06/07/importing-vmware-and-virtualbox-vms-to-proxmox/
https://pve.proxmox.com/wiki/Network_ModelI will put the thread as solved as the issue is clearly on Virtualbox and that should explain why I was finding Pfsense a little bit slow :)
