Xen, pfSence, no web connections.



  • I feel dumb.
    I've reinstalled pfSense several times now and just can't get it working properly.

    Xenserver 7.4, pfSense attached to one of the physical networking ports and a virtual server-only port.

    Linux VM on same server attached to virtual net, can talk to pfSense ok.
    I can get a CLI to ping by name (shows the DNS is working), but the browser keeps timing out.

    Will eventually put a VPN on this, but first I need it to work.
    Why does a basic installation not seem to pass traffic properly?
    Set up as a basic router first, then add security would seem the optimal way to go.


  • Netgate Administrator

    Waaay more information needed.

    Missing default gateway/route?

    Conflicting subnets?

    Incorrect firewall rule?

    It could be a lot of things.

    Steve



  • Alright...
    Starting from scratch:
    Create VM on Xenserver 7.4
    Set up VM with 2G memory and a 2G drive with pfsense-CE-2.4.3-RELEASE-amd64.iso as boot media
    Networking port 0 = physical port 1
    Networking port 1 = single server private network
    default keymap
    no manual modifications
    No vlans
    WAN=xn0 - Xen port 0
    LAN=xn1 - Xen port 1
    Set LAN IP address to 192.168.2.1/24
    no gateway, no IPV6
    enable DHCP server, range 192.168.2.2 - 192.168.2.20
    don't revert weconfigurator to HTTP

    open Ubuntu 16.04 desktopVM
    Network assigned address 192.168.2.2
    open Firefox, direct to 192.168.2.1
    accept un-secure certificate
    start webconfigurator setup
    hostname and domain at defaults
    set primary DNS 8.8.8.8, secondary 1.1.1.1
    overide DNS checked
    set timezone
    leave WAN/LAN settings as is
    Set admin password
    open CLI on Ubuntu VM,
    ifconfig - 192.168.2.2
    ping Google.com - no reply.
    Firewall/NAT/Outbound - disable outbound NAT
    ping google.com - get reply.
    open webpage to google.com - "connecting to google.com" - connection times out.

    There's a notice on the webconfigurator
    Filter reload: There were errors loading the rules: /tmp/rules/debug:18: cannot define table bogonsv6: cannot allocate memory.

    So... why does the default configuration not allow for outbound communication?
    After disabling outbound NAT rules, PING works but the web browser doesn't.

    What else needs to be edited to allow for full outbound communication?


  • Netgate

    Probably the checksum stuff.

    The easiest thing to do is to install a new one fresh, reboot it, go into single-user mode and add this to /boot/loader.conf.local:

    hw.xen.disable_pv_nics=1

    You can probably also do this at the shell prompt if you open one after the install is done instead of rebooting to single user mode.

    Reboot again and run through the setup. You will notice you have reX NICs now instead of xnX NICs.

    See if that works closer to how you expect.

    You can disable checksumming on the VMs that have PV NICs but I have to dig it up. This is easier to try first.


  • Netgate

    @chaosmstr said in Xen, pfSence, no web connections.:

    Filter reload: There were errors loading the rules: /tmp/rules/debug:18: cannot define table bogonsv6: cannot allocate memory.

    Yeah. There's a reason that notice is there. It needs to be fixed.

    That should not be the case if you are using version 2.4.3_1. If you are not, why not?

    You can increase the Firewall Maximum Table Entries in System > Advanced, Firewall & NAT. Use 400000 there.



  • 2.4.3 is what is available as an ISO on the dl servers... from what I can see.

    Thank you, the disable_pv_nics worked. It also allowed me to run the update to 2.4.3_1.

    Now, off to bigger and better things!
    Thanks!


  • Netgate

    Right. you have to upgrade CE to 2.4.3-p1. I don't think this info has been refreshed on the new forum yet. There are other threads on it.

    Here are the basics for what you need to do for the PV NICs:

    Install it, shut it down. Add the NICs you want, then in XenServer:

    Get the VM's uuid
    # xe vm-list name-label="pfSense B" | grep "^uuid" | awk '{print $NF}'

    43fdd0da-73ca-22c0-97f6-0ac47ae82360
    

    Get the UUIDs for the NICs
    # xe vif-list vm-uuid="43fdd0da-73ca-22c0-97f6-0ac47ae82360" | grep "^uuid" | awk '{print $NF}'

    6c9cb724-705a-0449-2176-505dd332431d
    a4c4ec8f-de68-eab3-69c7-d5b6c8be7b53
    25e0d1b6-6d9a-6480-4612-e5aca876a922
    71919d5a-000c-b9b3-31ed-21fa1674ba4e
    1bf1eaf3-50fe-4a12-c3fa-1341766cee08
    7b50e7fd-d6ec-598d-8dd6-6068d5f2765b
    

    Turn off the checksum checking in the NICs. Run this for all of them:
    # xe vif-param-set uuid=6c9cb724-705a-0449-2176-505dd332431d other-config:ethtool-tx="off"

    Boot the VM and the traffic in should flow through fine on the PV NICs.

    The other major caveat is the HV NICs (reX) support altq shaping. The PV NICs (xnX) don't.