Categories

  • 449 Topics
    1k Posts
    S

    I have a 2 TNSR routers connected to a pair of MLAG connected switches. I also have my own IPV4 subnet that is being announcec by BGP via Interface 1 on the first TNSR device. I have no problems at all right now, all of the servers on my network can access the internet and be accessed via their public IP address.

    What I am struggling with now is segregating clients into VLANs. When I create an access VLAN (22) for my client, I can no longer access the internet. My understanding is that I must create a bridge so that the VLAN22 can access the LAN interface with the gateway IP assigned. Each VLAN client will have a public IP from the single /24 subnet.

    When I followed the instructions for TNSR VLAN, nothing seemed to be problematic, but when I created the bridge things went wonky. Not only do the VLANs not work, but I also lose access to the non-VLAN devices.

    interface bridge domain 10
    flood
    uu-flood
    forward
    learn
    exit

    int Interface1
    bridge domain 10
    enable
    exit
    int Interface1.22
    bridge domain 10
    enable
    exit
    interface loopback bridgeloop
    instance 1
    exit
    interface loop1
    ip address 10.25.254.1/24
    bridge domain 10 bvi
    enable
    exit

    I did try changing the loop1 IP to my gateway IP and removing it from Interface1 but that didn't help. Maybe I am going about this wrong, but I need some guidance if possible.

    Thanks,
    Shawn

    For background:
    On TNSR device1:
    Interface1 is connected to a switch that carries my upstream BGP using a 10.34.14.0/24 address for now.
    Interface2 is the interface that has my gateway IP 23.x.x.x/24 and is also the port connected to the first switch.
    Interface3 is connected to a second switch and has no IP address

    TNSR device2 :
    Interface1 is connected to the switch that carreies the BGP but has no IP address and for all practical purposes is doing nothing

    Interface 2 is connected to the 2nd switch and has no IP address

    Interface 3 is connected to the first switch and has no IP address

    As you can see, the 2nd TNSR device is mostly sitting around doing nothing but eventually should be integrated in via VRRP or whatever I can get working.

  • 120k Topics
    761k Posts
    nazar-pcN

    @wickeren I actually had it enabled with legacy version (but I didn't make a difference), while switching to modern I removed it.
    Probably should add back and see if there is a difference, however as mentioned in the links in the first post, I don't think pfSense has corresponding support enabled in the kernel anyway 😕

    There must be something equivalent in Proxmox as well, it probably designs PCIe architecture in a way that produces legacy devices just like it was in my case originally.
    I'm still puzzled as to why that was the case, but glad it is resolved.

    Here is the full QEMU command that libvirt generates for the VM in case it is helpful:

    Spoiler

    /usr/bin/qemu-system-x86_64 -name guest=pfSense,debug-threads=on -S -object {"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-26-pfSense/master-key.aes"} -blockdev {"driver":"file","filename":"/usr/share/OVMF/OVMF_CODE_4M.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"} -blockdev {"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/pfSense_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"} -machine pc-q35-8.2,usb=off,dump-guest-core=off,memory-backend=pc.ram,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format,hpet=off,acpi=on -accel kvm -cpu host,migratable=on -m size=2097152k -object {"qom-type":"memory-backend-ram","id":"pc.ram","size":2147483648} -overcommit mem-lock=off -smp 8,sockets=1,dies=1,cores=8,threads=1 -uuid REDACTED -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=38,server=on,wait=off -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-shutdown -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot menu=off,strict=on -device {"driver":"pcie-root-port","port":16,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x2"} -device {"driver":"pcie-root-port","port":17,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x2.0x1"} -device {"driver":"pcie-root-port","port":18,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x2.0x2"} -device {"driver":"pcie-root-port","port":19,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x2.0x3"} -device {"driver":"pcie-root-port","port":20,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x2.0x4"} -device {"driver":"pcie-root-port","port":21,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x2.0x5"} -device {"driver":"ich9-usb-ehci1","id":"usb","bus":"pcie.0","addr":"0x1d.0x7"} -device {"driver":"ich9-usb-uhci1","masterbus":"usb.0","firstport":0,"bus":"pcie.0","multifunction":true,"addr":"0x1d"} -device {"driver":"ich9-usb-uhci2","masterbus":"usb.0","firstport":2,"bus":"pcie.0","addr":"0x1d.0x1"} -device {"driver":"ich9-usb-uhci3","masterbus":"usb.0","firstport":4,"bus":"pcie.0","addr":"0x1d.0x2"} -blockdev {"driver":"file","filename":"/var/lib/libvirt/images/pfSense.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage","backing":null} -device {"driver":"virtio-blk-pci","bus":"pci.3","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1} -netdev {"type":"tap","fd":"39","vhost":true,"vhostfd":"44","id":"hostnet0"} -device {"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"REDACTED","bus":"pci.1","addr":"0x0"} -netdev {"type":"tap","fd":"45","vhost":true,"vhostfd":"46","id":"hostnet1"} -device {"driver":"virtio-net-pci","netdev":"hostnet1","id":"net1","mac":"REDACTED","bus":"pci.2","addr":"0x0"} -netdev {"type":"tap","fd":"47","vhost":true,"vhostfd":"48","id":"hostnet2"} -device {"driver":"virtio-net-pci","netdev":"hostnet2","id":"net2","mac":"REDACTED","bus":"pci.5","addr":"0x0"} -netdev {"type":"tap","fd":"49","vhost":true,"vhostfd":"50","id":"hostnet3"} -device {"driver":"virtio-net-pci","netdev":"hostnet3","id":"net3","mac":"REDACTED","bus":"pci.6","addr":"0x0"} -chardev pty,id=charserial0 -device {"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0} -audiodev {"id":"audio1","driver":"spice"} -spice port=5901,addr=127.0.0.1,disable-ticketing=on,seamless-migration=on -device {"driver":"qxl-vga","id":"video0","max_outputs":1,"ram_size":67108864,"vram_size":67108864,"vram64_size_mb":0,"vgamem_mb":16,"bus":"pcie.0","addr":"0x1"} -global ICH9-LPC.noreboot=off -watchdog-action reset -device {"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.4","addr":"0x0"} -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on

    And this is XML domain config it was generated from:

    Spoiler

    <domain type="kvm"> <name>pfSense</name> <uuid>REDACTED</uuid> <metadata> <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0"> <libosinfo:os id="http://freebsd.org/freebsd/14.0"/> </libosinfo:libosinfo> </metadata> <memory unit="KiB">2097152</memory> <currentMemory unit="KiB">2097152</currentMemory> <vcpu placement="static" cpuset="8-11,24-27">8</vcpu> <os firmware="efi"> <type arch="x86_64" machine="pc-q35-8.2">hvm</type> <firmware> <feature enabled="no" name="enrolled-keys"/> <feature enabled="no" name="secure-boot"/> </firmware> <loader readonly="yes" secure="no" type="pflash">/usr/share/OVMF/OVMF_CODE_4M.fd</loader> <nvram template="/usr/share/OVMF/OVMF_VARS_4M.fd">/var/lib/libvirt/qemu/nvram/pfSense_VARS.fd</nvram> <boot dev="hd"/> <bootmenu enable="no"/> </os> <features> <acpi/> <apic/> </features> <cpu mode="host-passthrough" check="none" migratable="on"> <topology sockets="1" dies="1" cores="8" threads="1"/> </cpu> <clock offset="utc"> <timer name="rtc" tickpolicy="catchup"/> <timer name="pit" tickpolicy="delay"/> <timer name="hpet" present="no"/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <pm> <suspend-to-mem enabled="no"/> <suspend-to-disk enabled="no"/> </pm> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type="file" device="disk"> <driver name="qemu" type="qcow2"/> <source file="/var/lib/libvirt/images/pfSense.qcow2"/> <target dev="vda" bus="virtio"/> <address type="pci" domain="0x0000" bus="0x03" slot="0x00" function="0x0"/> </disk> <controller type="sata" index="0"> <address type="pci" domain="0x0000" bus="0x00" slot="0x1f" function="0x2"/> </controller> <controller type="pci" index="0" model="pcie-root"/> <controller type="pci" index="1" model="pcie-root-port"> <model name="pcie-root-port"/> <target chassis="1" port="0x10"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0" multifunction="on"/> </controller> <controller type="pci" index="2" model="pcie-root-port"> <model name="pcie-root-port"/> <target chassis="2" port="0x11"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x1"/> </controller> <controller type="pci" index="3" model="pcie-root-port"> <model name="pcie-root-port"/> <target chassis="3" port="0x12"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x2"/> </controller> <controller type="pci" index="4" model="pcie-root-port"> <model name="pcie-root-port"/> <target chassis="4" port="0x13"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x3"/> </controller> <controller type="pci" index="5" model="pcie-root-port"> <model name="pcie-root-port"/> <target chassis="5" port="0x14"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x4"/> </controller> <controller type="pci" index="6" model="pcie-root-port"> <model name="pcie-root-port"/> <target chassis="6" port="0x15"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x5"/> </controller> <controller type="pci" index="7" model="pcie-root-port"> <model name="pcie-root-port"/> <target chassis="7" port="0x16"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x6"/> </controller> <controller type="usb" index="0" model="qemu-xhci" ports="15"> <address type="pci" domain="0x0000" bus="0x07" slot="0x00" function="0x0"/> </controller> <interface type="bridge"> <mac address="REDACTED"/> <source bridge="wan"/> <target dev="pfsense-wan"/> <model type="virtio"/> <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/> </interface> <interface type="bridge"> <mac address="REDACTED"/> <source bridge="wan2"/> <target dev="pfsense-wan2"/> <model type="virtio"/> <address type="pci" domain="0x0000" bus="0x02" slot="0x00" function="0x0"/> </interface> <interface type="bridge"> <mac address="REDACTED"/> <source bridge="lan"/> <target dev="pfsense-lan"/> <model type="virtio"/> <address type="pci" domain="0x0000" bus="0x05" slot="0x00" function="0x0"/> </interface> <interface type="bridge"> <mac address="REDACTED"/> <source bridge="guest"/> <target dev="pfsense-guest"/> <model type="virtio"/> <address type="pci" domain="0x0000" bus="0x06" slot="0x00" function="0x0"/> </interface> <serial type="pty"> <target type="isa-serial" port="0"> <model name="isa-serial"/> </target> </serial> <console type="pty"> <target type="serial" port="0"/> </console> <input type="mouse" bus="ps2"/> <input type="keyboard" bus="ps2"/> <graphics type="spice" autoport="yes"> <listen type="address"/> </graphics> <audio id="1" type="spice"/> <video> <model type="virtio" heads="1" primary="yes"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/> </video> <watchdog model="itco" action="reset"/> <memballoon model="virtio"> <address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/> </memballoon> </devices> </domain>

  • 20k Topics
    127k Posts
    G

    @pst Finally.... Got it. I used copilot to walk me through all of my errors. The end result,

    Boom—there’s the final piece of the puzzle. Telegraf is trying to write to the InfluxDB 2.x instance using the InfluxDB 1.x-style output plugin ([[outputs.influxdb]]), which expects a database name like "pfsense". But InfluxDB 2.x doesn’t use databases—it uses buckets, orgs, and tokens. That’s why you’re getting:

    401 Unauthorized and database creation failed

    So long story sort, the package from Pfsense still shows the database info, I pasted the following in the "Additional Config" section and removed the database info and boom...got data

    [[outputs.influxdb_v2]]
    urls = ["http://192.168.1.100:8086"]
    token = "your-super-secret-token"
    organization = "name"
    bucket = "pfsense"

    Holy Heck that was a lot. :) but thanks for stick with me I appreciate the support :)

  • 43k Topics
    267k Posts
    W

    toujours rien tourver les stencils pfesense pour la visio

  • Information about hardware available from Netgate

    2k Topics
    20k Posts
    GertjanG

    @Nightwolf said in Netgate 6100 LAN crashes:

    I had a crash yesterday that I was able to catch in time to retrieve the logs.
    The local IP address and local domain are truncated for security reasons.
    Everything else is unchanged; I intentionally left all packages running except ntopng.
    Disabling/uninstalling the packages doesn't change anything anyway.
    Hope this helps.

    Netgate_Logs.zip

    Wow ... that's a - sorry for the word : a bit messy.

    I'll start with this : I don't use it HA Proxy - have no usage for it as I lost all interest in hosting things myself decades ago. I got myself a small VPS server in some data centre, and that one took care of my web sites, mail stuff, powering, disk maintenance, cleaning the fans, paying the power bills, and all that stuff. @home and @work my pfSense is their to 'regulate' my ISP connection.

    Because 99++ % is TLS these days, this means that plain text data transfers, for mail, web sites etc etc doesn't exist anymore, stuff like Suricata (clam-av) or more general, IDS/IPS, has become pure rocket science.
    The latest example was shown last evening = see Not Nominal from Scott Manley. Rocket science is hard, hurts, and things will blow up "all the time".
    Those who master "IDS/IPS/Proxy" are not the ones** posting here on this forum. They are the TLS gods ....
    The good old days of 'traffic scanning' is gone now.

    ** well, not true, there is one person here in the forum : bmeeks.

    First things first : ok that your WAN disconnects ....
    But your log starts at the bottom with a LAN disconnect ! That's not good !!
    Your mission, if you want to have an easy admin live : stop that from happening.
    The 6100 LAN plugs, or actually, any plug, don't disconnect them. The devices connected to these : (switches, ISP boxes) : these are small power consumer, share the pfSense UPS with them.

    That said, it's 'ok' for interfaces to go down. In theory, this should break anything.
    But there is a but ....
    When an interface goes down, processes like nginx, the DHCP server (or client), the resolver, the gateway scanner 'dpinger' etc etc etc will restart. And here comes the issue : all these process restart nearly at the same moment, which opens the door for a the most dreaded situation : race conditions. Your mission, a an admin, is : never ever create situations where race conditions can bite you.
    The final goal is : Keep the logs dull, with no errors neither warnings messages. You'll be granted a very stable router, an admin's dream.

    Next issue : you use the "servicewatchdog" : that's another admin-self-inflicting-pain tool.
    Don't use it. Like never.
    Locate "error: bind: address already in use" phrase in your zipped log. That's "servicewatchdog" doing things it should not do. "servicewatchdog" wasn't needed in this case, and did make things worse.

    I use a 6100 (4100) myself, with UPS, and my 'unbound' (example) never gets restarted because interfaces went down.
    This means you and I have the same hardware, the same software. If you use the same default Netgate settings for the core settings, you have nearly the same settings like me : result, as it is meant to be : it works "forever" - and don"'t take my word for it, have a look yourself.

    Also repair this :
    "haproxy: startup error output!" (several)
    and
    "/status_logs.php: ERROR! ldap_get_groups() could not bind"

    Don't be ashamed if you can not make the error go away.
    You are allowed and I even advise you to apply the KIS rule : don't use stuff that produces errors in the logs.
    And if you do, accept them and with the consequences ^^

    I'm not saying you can't / shouldn't use "pfSense package X" and "pfSense package Y" **, they are all easy to install. Setting them up can go way beyond what is needed to operate a vanilla pfSense.

    ** exception : "servicewatchdog" which should be banned from the package list.
    Be assured : my goal is that you have a 6100 that 'never fails' on you, so you can go back to do other stuff like spending time in the garden ^^

  • Information about hardware available from Netgate

    44 Topics
    211 Posts
    AriKellyA

    It looks like unified web management could be coming soon. It would be great if it means easier control and management of all web services in one place. Let's see if any companies announce more details about it!

  • Feel free to talk about anything and everything here

    3k Topics
    19k Posts
    L

    @Wylbur Thank you!

Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.