Trying to create filtered bridged but failing to (sigh)

vboxbsd

The problem is that I can't access the web admin from the OPT1 IP after completing the following steps from http://forum.pfsense.org/index.php?topic=50711.0 :
And of course I can no longer access it from the LAN interface.

1. Disable NAT (but not the firewall). See http://doc.pfsense.org/index.php/How_can_I_completely_disable_NAT%3F#Disable_NAT
2. VERY IMPORTANT: As mentioned at http://forum.pfsense.org/index.php?topic=30653.0, go to the 'System -> Advanced -> System Tunables' and set net.link.bridge.pfil_bridge from 'default' to '1'
3. Bridge WAN and LAN by going to 'Interfaces → Assign → Bridges'
4. Create OPT iface and assign the bridge to it by 'Interfaces → Assign → Network Port'
5. Add an IP address to the bridge interface; this IP is the one you will use to access the firewall long term
6. Add allow all rules to ALL firewall interfaces to avoid being locked out. Ifaces OPT, WAN, and LAN
7. Set WAN and LAN interface type to 'none'. (Under 'Interfaces' in GUI)
8. Disable DHCP server
9. The firewall should now be able to be accessed from all ifaces via the IP on the bridge from step 5
10. Carefully modify your firewall rules to be more restrictive. DNS, DHCP, etc.

I'm running pfsense in a virtualbox vm.
In the vm's network settings I have em0 (WAN) set as bridged adapter, em1 (LAN) as host-adapter vboxnet1 (so not the same as vboxnet0 which is same as the bridged adapter = hosts adapter), and em2 (OPT1) I'm confused about setting - I've tried host adapter vboxnet2 and as bridged.

The above steps are to create a

bridge with 2 interfaces

so perhaps I don't need to create a third network adapter in the pfsense vm?

However, even without it assigned, I can't access the web admin over OPT1 after completing the steps in the quoted post.

Can anyone suggest what I may have done wrong?

podilarius

The default rule is to block, did you create an allow rule so that then you bridge LAN and WAN, you can still access via opt1 (em2)? The reason for the "3rd wheel" (opt1 - em2) is so that you can manage the firewall "out of band".

vboxbsd

Thanks podilarius for your help…

Yes that is exactly why I'm wanting to use OPT1.

These are the rules I've setup:

WAN tab:
ID Proto Source Port Destination Port Gateway Queue Schedule

WAN net * * * * none

LAN tab:
ID Proto Source Port Destination Port Gateway Queue Schedule

LAN net * * * * none

OPT2 tab:
ID Proto Source Port Destination Port Gateway Queue Schedule

OPT1 net * * * * none

As I have setup 3 virtual adapters in pfsense's vm, I assume I only need to make set the LAN interface type to none?
Or do I need to set both interfaces, WAN and LAN, to type none? (The howtos I've found vary in this respect.)
Either way, I can still can't access OPT1 via 192.168.1.4. after completing all steps in the quoted steps.

Here are my settings in pfsense and in its vm:

WAN static 192.168.1.2 em0
and in the vm network setting using bridged adapter en0 cabled (same as hosts 192.168.1.1).

LAN static 192.168.1.3 em1,
and in the vm network setting using vboxnet1 (192.168.1.3 in virtualbox preferences Network > Host only networks).

OPT1 static 192.168.1.4 - bridged0 (WAN and LAN),
and in the vm network setting using vboxnet2 (192.168.1.4 in virtualbox preferences Network > Host only networks).

Is there anything obviously 'noobly' wrong with my rules?

podilarius

Did you set your vmnics to allow promiscuous mode for all? While technically it would work with your current setup, your wan LAN and opt net are the same. Also, if you change LAN and wan to none for ip address, there is no such thing as wan or LAN nets and the default of block will apply. I would on LAN and Wan set an allow all nets on all protocols to all source and destination ports.
I apologize for any spelling or grammar, and also for any shortness. I am on my iPhone. :)

Good luck.

vboxbsd

Haha, don't worry about it - your quick post from your iphone perfectly answered my question! :)
(I should be apologising for the noobness and verbosity of my explanations and questions! =)

Your suggestion re promiscuous mode worked - I can access the web admin via OPT!

THANK YOU!

I've set all vmnics to promiscuous mode Allow All, although I wonder if I should set some only to Allow VMs?
I've also set LAN's type to none and changed the Wan tab rule as you suggested (if I've understood you correctly) to:

WAN tab:
ID Proto Source Port Destination Port Gateway Queue Schedule
* * * * * * none

Where I'm stuck now are 3 very nub issues…

1. I still can't get traffic to my reverse proxy behind pfsense.
I think this is the case because:

a) the redirect rule for mydomain.com to www.mydomain.com in my reverse proxy isn't working,

b) when I try to access www.mydomain.com, the web admin page of my modem resolves instead, and over https.
If I then turn off pfsense and try to resolve my domain, my modem still redirects to its web admin page over https.
(I've telneted to the modem and disabled all WAN access to the web admin, ftp, telnet etc so this is confusing.)

c) I created a logged block rule in wan to any destination for http and https, but nothing is logged from local access to www.mydomain.com.
However, if I try to access the domain externally via a free proxy, while the http page won't resolve due to a generic 'couldn't connect to host error' reported by the proxy, the pfsense log shows the http rule has blocked the proxy's IP access to the wan on port 80.
But if I delete the wan block http rule in pfsense, once again http://www.mydomain.com resolves my modem's web admin page over https.

However, if I change the modem to forward port 80 to the reverse proxy and not pfsense, I can resolve www.mydomain.com from my webserver, both externally and locally (and over http as it should be).
And I can access email.mydomain.com/squirrelmail over https via my email server, both locally and externally, by port forwarding 443 from the modem to the reverse proxy and not pfsense.
I have no rule in my reverse proxy that redirects the www subdomain from http to https, nor one that catches all subdomains and redirects them to https.
And my modem port forward rules do no port translation as I've set this up to be handled by the reverse proxy - they directly forward ports 80 and 443, and I've deleted the latter to resolve this issue.

To sum...

I'm really confused as to whether the issue is caused by some setting(s) similar to NAT reflection (although I'm not using NAT) in pfsense that aren't enabled, or also partly caused by the modem.
It would appear to be caused by pfsense, as I can access the web/email servers over http/https if the modem port forwards to the reverse proxy and not pfsense.
And as my reverse proxy's domain.com to www.domain.com redirect rule isn't working with pfsense being forwarded to by the modem, it would seem that traffic isn't getting through the bridge to the reverse proxy.
I've tried putting the host-only adapters for the reverse proxy and webserver to be the same as that of pfsense - vboxnet2, but this didn't allow traffic to them.
(They normally are vboxnet1. If possible, my goal is to put them on an internal network connected to pfsense once the pfsense bridge is working, and ssh-ing to them through pfsense - so no local/bridged access at all.)

Any thoughts on how to fix this on pfsense or further test the cause?

2. Do I need to create any pfsense rules to direct traffic to my reverse proxy, similar to how I did with NAT rules prior to disabling NAT when setting up this filtered bridge?

3. Once I get traffic to the reverse proxy for http://www.mydomain.com, do I add rules to filter the bridge eg pass http traffic, and remove the allow any to any rule in wan (as above)?
If so, do I add them to the wan or lan part of the bridge?

Thanks again for your help podilarius... I'm learning a lot and can see my goal in sight now!

podilarius

okay … in a bridge, you don't use any address in the pfsense machine as a gateway for servers/computers or destination for any modem rules. So, your modem should be redirecting http and https traffic to the reverse proxy. The reverse proxy or your web servers needs to handle the http to https redirect. There is a lot there so in summary, the WAN of pfsense and the LAN of you modem should be in one network and the LAN of pfsense and all web servers and reverse proxy should be in another. The bridge in pfsense should forward packets so long as they pass filtering.
You are not using NAT in pfsense, so there are no rules to setup.
Once you know that your transparent FW is working, then you can go back and restrict access with FW rules.
As far as promiscuous mode, you only need to do that with the pfsense vmnics, at least that is how it works in ESXi. There just has to be a clear separation between the two networks. I don't remember the types of nics in virtual box (and I have one running, I just don't have access right now), but you want WAN on pfsense to be bridged with the host's adapter, and the LAN and the rest of the VMs to be in their own separate vswitch (yeah , I know, but I am used to ESXi).

vboxbsd

I understand most of that… the difference between transparent firewall (filtered bridge) and NAT...

where it appears I may be confused is understanding what they have in common.

In particular, regardless of whether I choose NAT or filtered bridge 'mode', the WAN and LAN of pfsense must be on separate subnets?

So the WAN could be 192.168.1.2 as it is in my setup, and the LAN could be 192.168.2.0, with reverse proxy 192.168.2.1, and backend servers on unique IPs in 192.168.2.0/24?

If so I misunderstood your previous post and thought I could use a filtered bridged to avoid putting the LAN, reverse proxy, and backend servers on a different subnet, which I'd have to do if using NAT 'mode' instead.

I chose the filtered bridge as I'm still unsure how to put any of the vms on a different subnet - I've tried changing host-only adapter addresses and updating IPs in the servers, but the vms stalled during boot.

Ideally I'd like the LAN of pfsense and all servers behind it to be on a different subnet to increase security, forcing me to ssh to them via pfsense - not directly from host as I do now, but this just seemed too hard to set up.

However, unless you correct my understand of what you are saying in your last post - I have to put pfsense's LAN and all backend servers on a different subnet from pfsense's WAN anyway?

podilarius

I find it hard to talk about pfsense when in use with virtualbox, as they call one of the network setups a bridge. So, basically for a routed or NATed solution, you have to have 2 unique subnets, with a tranparent FW, you will use same subnet. And yes, if you are nating, then you will need to setup LAN and all servers behind it on a different subnet than WAN. The gateway on the reverse proxy and all servers will be pfsense LAN address. And the gateway for WAN will be the existing router/modem. Do not set a GW on the pfsense LAN interface.

If you don't want to change IPs on servers or proxy, then you have to go for a transparent firewall.

vboxbsd

Ok, so it turns out I understood your earlier post correctly…
that is when trying to build a transparent firewall with pfsense I should have WAN, LAN, and all servers behind it on the same subnet.

I've set the WAN only to have a gateway, the LAN with none - and the LAN's interface type is set as none.
And I can access pfsense's web configurator via OPT.

However, as per my reply to your post from your iph, I can't seem to get traffic through the bridge to my reverse proxy.

Based on that post - any thoughts why?

Thanks again

podilarius

My guess is that you either have a rule blocking (incorrectly formed) or your promiscuous mode is not set correctly.

vboxbsd

I think to solve why the bridge isn't working, I first need to solve why pfsense is redirecting my domain to https.

If I…

1. port forward 80 from modem to reverse proxy (not pfsense)... everything works fine:
I can resolve http://www.mydomain.com.
And my reverse proxy rule redirecting from http://mydomain.com to http://www.mydomain.com works/resolves.

2. instead modem port forward 80 to pfsense in front of the reverse proxy... SOMETHING IS BROKEN (LOL):
I get redirected to non-resolvable https://www.mydomain.com.
(In a previous post, I wrote that in this setup my domain was resolving the modems web admin via https.
I've since disabled WAN and LAN modem web admin page access over https.)
The reverse proxy rule redirecting http://mydomain.com to http://www.mydomain.com doesn't work, but redirects to non-resolvable https://mydomain.com.

3. repeat 1 but with reverse proxy not listening (off)… everything works as it should:
http://www.mydomain.com doesn't resolve.
http://mydomain.com doesn't get redirected to http://www.mydomain.com, and doesn't resolve,
and neither is redirected to https://.

Clearly 1. and possibly 3. prove that the modem isn't causing this http to https redirection when I try to resolve www.mydomain.com with the modem port forwarding 80 to pfsense in front of the reverse proxy.

So as per 2. something in pfsense is causing http://www.mydomain.com to redirect to https://www.mydomain.com

And although the modem's web admin page is no longer resolving, the fact that this redirection is happening when pfsense is being forwarded to, makes me think this must be due to setting(s) similar to NAT reflection/loopback being disabled in pfsense, irrespective of the fact I'm not using NAT, that is, I have disabled automatic NAT rule creation. Should I be disabling it elsewhere also? The howto I originally quoted and others I read do not add extra steps in this regard.

Further, as I've written in previous posts - I have created no rules in pfsense nor in the reverse proxy that could be redirecting from http://mydomain.com to https://www.mydomain.com nor from http://mydomain.com to https://mydomain.com.

This is driving me nuts (and possibly you too… my apologies) as it would seem to be caused by some simple setting, and all I want to do is get pfsense working as a filtered bridge / transparent firewall and get my servers live ASAP.

Any ideas??

podilarius

No, what you have is a misunderstanding that you are supposed to be forwarding traffic to the pfsense machine.

If you are using a transparent proxy, then you still must be sending the traffic from the modem to the reverse proxy. The purpose of the transparent proxy is to be an invisible shield against unwanted traffic.

So…
1. This would be expected behavior.

2. pfSense is redirecting as that is what the web gui will do when it is running on https. Since you are forwarding the packets to pfsense, its web server responds. This is also expected behavior.

3. This is expected also as your reverse proxy is the "man in the middle". If you turn that one off, then no access to the web sites behind.

You are only going to direct web traffic to pfsense in a routed/NAT setup. This is NOT what you are doing.

As a test, remove all the rules on WAN and setup like you did in "1.", if pfsense is truely inline between the modem and the reverse proxy, then you will not have access to your sites. Then, put in a rule to allow 80 and 443 in. You should be able to get to websites and nothing else.

I do hope I cleared it up for you.