Xen, pfSence, no web connections.

chaosmstr

I feel dumb.
I've reinstalled pfSense several times now and just can't get it working properly.

Xenserver 7.4, pfSense attached to one of the physical networking ports and a virtual server-only port.

Linux VM on same server attached to virtual net, can talk to pfSense ok.
I can get a CLI to ping by name (shows the DNS is working), but the browser keeps timing out.

Will eventually put a VPN on this, but first I need it to work.
Why does a basic installation not seem to pass traffic properly?
Set up as a basic router first, then add security would seem the optimal way to go.

stephenw10

Waaay more information needed.

Missing default gateway/route?

Conflicting subnets?

Incorrect firewall rule?

It could be a lot of things.

Steve

chaosmstr

Alright...
Starting from scratch:
Create VM on Xenserver 7.4
Set up VM with 2G memory and a 2G drive with pfsense-CE-2.4.3-RELEASE-amd64.iso as boot media
Networking port 0 = physical port 1
Networking port 1 = single server private network
default keymap
no manual modifications
No vlans
WAN=xn0 - Xen port 0
LAN=xn1 - Xen port 1
Set LAN IP address to 192.168.2.1/24
no gateway, no IPV6
enable DHCP server, range 192.168.2.2 - 192.168.2.20
don't revert weconfigurator to HTTP

open Ubuntu 16.04 desktopVM
Network assigned address 192.168.2.2
open Firefox, direct to 192.168.2.1
accept un-secure certificate
start webconfigurator setup
hostname and domain at defaults
set primary DNS 8.8.8.8, secondary 1.1.1.1
overide DNS checked
set timezone
leave WAN/LAN settings as is
Set admin password
open CLI on Ubuntu VM,
ifconfig - 192.168.2.2
ping Google.com - no reply.
Firewall/NAT/Outbound - disable outbound NAT
ping google.com - get reply.
open webpage to google.com - "connecting to google.com" - connection times out.

There's a notice on the webconfigurator
Filter reload: There were errors loading the rules: /tmp/rules/debug:18: cannot define table bogonsv6: cannot allocate memory.

So... why does the default configuration not allow for outbound communication?
After disabling outbound NAT rules, PING works but the web browser doesn't.

What else needs to be edited to allow for full outbound communication?

Derelict

Probably the checksum stuff.

The easiest thing to do is to install a new one fresh, reboot it, go into single-user mode and add this to /boot/loader.conf.local:

hw.xen.disable_pv_nics=1

You can probably also do this at the shell prompt if you open one after the install is done instead of rebooting to single user mode.

Reboot again and run through the setup. You will notice you have reX NICs now instead of xnX NICs.

See if that works closer to how you expect.

You can disable checksumming on the VMs that have PV NICs but I have to dig it up. This is easier to try first.

Derelict

@chaosmstr said in Xen, pfSence, no web connections.:

Filter reload: There were errors loading the rules: /tmp/rules/debug:18: cannot define table bogonsv6: cannot allocate memory.

Yeah. There's a reason that notice is there. It needs to be fixed.

That should not be the case if you are using version 2.4.3_1. If you are not, why not?

You can increase the Firewall Maximum Table Entries in System > Advanced, Firewall & NAT. Use 400000 there.

chaosmstr

2.4.3 is what is available as an ISO on the dl servers... from what I can see.

Thank you, the disable_pv_nics worked. It also allowed me to run the update to 2.4.3_1.

Now, off to bigger and better things!
Thanks!

Derelict

Right. you have to upgrade CE to 2.4.3-p1. I don't think this info has been refreshed on the new forum yet. There are other threads on it.

Here are the basics for what you need to do for the PV NICs:

Install it, shut it down. Add the NICs you want, then in XenServer:

Get the VM's uuid
# xe vm-list name-label="pfSense B" | grep "^uuid" | awk '{print $NF}'

43fdd0da-73ca-22c0-97f6-0ac47ae82360

Get the UUIDs for the NICs
# xe vif-list vm-uuid="43fdd0da-73ca-22c0-97f6-0ac47ae82360" | grep "^uuid" | awk '{print $NF}'

6c9cb724-705a-0449-2176-505dd332431d
a4c4ec8f-de68-eab3-69c7-d5b6c8be7b53
25e0d1b6-6d9a-6480-4612-e5aca876a922
71919d5a-000c-b9b3-31ed-21fa1674ba4e
1bf1eaf3-50fe-4a12-c3fa-1341766cee08
7b50e7fd-d6ec-598d-8dd6-6068d5f2765b

Turn off the checksum checking in the NICs. Run this for all of them:
# xe vif-param-set uuid=6c9cb724-705a-0449-2176-505dd332431d other-config:ethtool-tx="off"

Boot the VM and the traffic in should flow through fine on the PV NICs.

The other major caveat is the HV NICs (reX) support altq shaping. The PV NICs (xnX) don't.