Pfsense on virtual box in ubuntu behind a router

[GeoffatMM]

I have searched the forum but cannot find anything that helps so am posting a new topic.  Please note that I know just about enough to be dangerous and not enough to really know what I am doing!

MY OBJECTIVE
I am trying to devise a structure that allows me to select a clear internet access for certain machines and a VPN network (airVPN) for others, mainly linked to television services.

I have followed many guides and tutorials but none of them seem to replicate my situation so at the end, I get stuck.  If anyone could be kind enough to help me resolve this I would greatly appreciate it.

My problem is entirely the management of NIC addresses and subnets which I have little experience of.

MY SYSTEM
I have an ADSL internet router supplied by SFR (France) that runs my current network.  I cannot find a way to bridge it or to change its base IP address range of 192.168.1.0 so will have to retain it for one of the subnets. It is an NBV6 box.

To it is connected my landline phone which is actually an SFR IP phone that is internet service dependant.  I could also connect French digital TV services to the box but do not.  There are a further 4 LAN ports on the back.  One is for a second IP telephone via a DECT adapter for my London number, one is for a femto cell for my SFR mobile phone service, one takes an ethernet link to my office hub and the fourth is used to set up a powerlink network over the house mains cables providing both ethernet ports and wifi in most rooms of the house.  All of these must be unaffected by the changes!

There are a variety of clients ranging from PC/Mac machines hardwired over powerlink, printers ditto and smart phones, laptops and iPads connected over the powerlink wifi system.

We use certain services that require VPN access and although the computers can be manipulated to use a VPN, less smart (TV boxes) cannot.  Also, multiple licenses for VPN can be expensive.  For this reason I want to have a network that enables both clear internet access on one subnet (the router) and VPN access on a second subnet (managed by pfsense) allowing users to select one (via DHCP I assume) or the other (Through manual static IP intervention).

MY CURRENT PROGRESS
I have selected an old (3 years!) Dell box which is pretty beefy and installed Ubuntu 14.04 LTS to it.  It has (currently) two NICs and a wifi card giving eth0, eth1 and wlan0.  On this I have installed Virtualbox and then generated a free BSD machine on which sits my pfsense OS.

All the above works fine. I can run DHCP over the LAN and access my web configurator for pfsense but only over the 192.168.1.0 network.  What I cannot do is set my pfsense WAN to DHCP at the same time as in order for LAN DHCP to work, I have to disable the DHCP service on the router and thus need to set a fixed IP for the WAN side.  When I do this, I cannot tell if the internet is being served by the router or pfsense (I am sure it is the router!)

If I set the WAN to DHCP and set a second private subnet on the LAN with DHCP (using 10.0.0.1 and DHCP range 10.0.0.100 to 10.0.0.200), the clients accept the router DHCP in preference.  If I change the WAN to a fixed address, the pfsense LAN gives out IP addresses with associated DNS and Gateway details but then nothing can access the internet.  Even the phones directly connected to the router stop working.

WHAT I NOW NEED
Can someone guide me on how to resolve these two subnets so they work side by side as I want them to?  What additional information or logs should I post to help identify what needs to be done?  Please remember I am not an expert and am struggling to make this work.

Ta,

Geoff

[codemaster]

Hi,

Would u mind drawing a simple network diagram for your setup pls. I'm sure it would be very helpful while trying to help and solve this network.

Regards

[GeoffatMM]

OK thanks code master, I will try.  I think I saw a link to some software that can used so will try that.

As an update, I now have things working although not precisely as I want them to.  Following pf_sense's tutorial for two NICs, I have got pfsense working on a single network (192.168.1.0) and have confirmed that I can access the services I want.  I have also got my Apple TV and my satellite box communicating so that we can catch up on programmes so effectively I am happy!

I am however, managing pfsense directed and non pfsense directed traffic by static addressing and using the different gateways and DNS routers via my router.  The base gateway, 192.168.1.1, for non pfsense traffic and the pfsense gateway 192.168.1.200 for all other (VPN) traffic.

I still would like to continue to get the three NIC solution properly working with different subnets as manually managing it all is a bit of a headache and I am probably using the VPN unnecessarily at times.

Will try to get a diagram up this weekend.

Geoff

[GeoffatMM]

Code master,

Managed to get it done today so here is a diagram of our "simple" home network.  (Apologies but it seems to have loaded the diagram twice and I cannot remove one of them).

The biggest problem I am facing on the current layout/arrangement is that the response time over the web has slowed down significantly.  Something in the routing seems to be slowing the connection time.

Not sure if this diagram is clear enough for you so if you need any clarification, please let me know.

Geoff

[codemaster]

Hi GeoffatMM,

I'm still analyzing your setup and my guts telling me that your pfSense setup is not where it's suppose to be. I mean from the diagram that you sketch out, pfSense NIC(s) are all INSIDE LAN and none is facing outward to WAN. It is expected that Modem DHCP (NBV6) will take precedence in terms of handing out IP(s) since pfSense is still figuring out it's role basically.

If i may suggest, put your pfSense AFTER the modem. Since the modem has it's own static IP and cannot change, then pfSense will need to use another class of IP (not Class C space) hint: Class B or Class A.

I'll try to put up my suggested diagram as soon as possible (due to being Saturday 7.40AM here and i need to do chores while having time with family) . I'll try my best to assist :)

Best regards.

EDIT_1: If it's not too much to ask, please screenshot your pfSense WAN interface page and LAN interface page (the whole from top to bottom). Usually cmb/jimp/ermal would make sure on WAN interface page, those two checkbox(s) on the BOTTOM of the page will be tick OFF (so that traffic from WAN would be flowing) since WAN traffic is considered Internal traffic and pfSense need to unblock those traffic.

1. If possible too, what type of connection on your modem? Does it uses PPPoE/Static/DHCP from ISP.

2. Do make sure that you are running only 1 instance of DHCP. If you decide that DHCP will be served from Modem, then you need to make sure that pfSense doesn't handout any IP(s) through DHCP

3. As for subnets, my suggestion still stands, you need to move to another class.

4. I'm still figuring out what kind of role pfSense would do in your setup. I know you already state that it's gonna run VPN and if that so, then pfSense will need to do the main router/routing role and your Modem would need to turn off any dialing (if it's PPPoE/Static/DHCP from ISP) and let pfSense take over Modem's role. This is to ensure that VPN traffic would be directly communicate to WAN address/WAN destination (although the possibility of your current setup still suggest that it can work).

[GeoffatMM]

Code master

Thanks for your continuing support.  I hope you do not regret it!  I am a bit at sea with all the networking but following your note I have researched classes etc. and think I have a little understanding now.  All my previous networks have been single LANs so based on 192.168.0.0 or 192.168.1.0 sets.  I can see now that this only give a 256 address set which has been more than adequate for any of my company needs but may not be suitable for this network applicaiton.

I cannot fully respond to your message as at the moment I appear to have lost access to the web configurator so cannot post the pages you have requested.  However I can confirm answers to your numbered questions.

1    It is using a PPP login so I assume PPPoE.  I do not pay for or receive a static ip but it also does not appear to change (at least it hasn't for the last six months).  However I am sure it is a dynamically assigned address from the ISP.

2    To get things going, DHCP was from the router with pfsense off.  once I had a WAN address I changed it to a fixed address and then moved the DHCP to pfsense, switching off the DHCP on the router and I think this is part of the problem.  When I have rebooted pfsense, I am not sure it is getting all the gateway etc. information properly as there is no WAN side DHCP and that is why I appear to have lost some connectivity.  I do not mond where the DHCP comes from, I set it from pfsense as i assumed it would be more flexible than the ISP provided box.  However, if I set the WAN and LAN to different address ranges, I am not sure how I should set up the DHCP to manage both?  As I say, I am not a networking specialist at all.  I have to leave the ISP router on 192.168.1.0 and assumed I would have to configure my LAN to say 192.168.0.0 and that is where I get suck as I can see that two DHCP servers is not acceptable but one cannopt serve the two address ranges (in a class C space?).  Is that why I have to move to a class A or B space?  (Sorry if I sound like an idiot in this area).

3    This is where I get lost.  What would be really helpful to me would be to know precisely how I need to set the addresses of the NICs and then the addressing of the WAN and LAN in pfsense.

4  What I want pfsense to do is act as the interpreter of requests for VPN and non VPN traffic but from a remote (ie not directly connected to the router) location, as the pfsense machine has to sit in another physical space to the router and incoming telephone line.  If all I set up is for pfsense to handle VPN and the non VPN traffic to be on static addresses with a gateway not to pfsense but to the router, I would be happy to set that up, it is after all, a small network so manually setting some of the addresses would not take too long.  Remember the main goal of all this is for the satellite box to go through the VPN and I had it working for a couple of days on my setup (but now have lost it).  I do not know how to turn the router/modem off so that pfsense did the PPoE dialling but if that is what is needed, I am willing to give it a go.

My answer to 4 above tries to explain the limitations of where I can have the pfsense server which , although it is not shown explicitly on my diagram, is connected to the router but over the Ethernet network.  I hope my responses give further insight to my arrangement and that you can help me further.  In the mean time I will continue to play with it.

Geoff

[GeoffatMM]

Codemaster

One other thing.  I have read that I should set the NICs on the pfsense server to 0.0.0.0 so that they are by passed in the network and address allocation is done by pfsense?  However, when I tried this, my machine would not start the network services as I assume it could not find a valid IP address as the pfsense server was not running, it only came online after the machine had booted and that was too late.

Also, please remember that I have two NICs and one wifi.  I will shortly have three NICs plus wifi but not sure when the new card will arrive.

Geoff

[GeoffatMM]

Codemaster

I am posting the interface pages you asked for including the AirVPN_WAN page.

Geoff


[GeoffatMM]

WAN


[GeoffatMM]

AirVPN_WAN


[codemaster]

Dear GeoffatMM,

From my experience, lets ensure your internet is really stable and ensure that basic surfing and other services would work as it should before proceed to adding more services such as VPN and such.

Here's my suggested network diagram layout to change as per need. Please take note of the following choices:

1st Choice
1. You will need to ensure/obtain username & password from your ISP (if it uses password authentication) the format might be like this
username@email_here OR username@ISP_here OR any other format (if it uses one). Like myself, i'm using PPPoE ADSL from my ISP (Telekom Malaysia) which uses the following format username@streamyx (streamyx is their package name) .
2. PROCEED ONLY IF YOU ALREADY OBTAIN PROPER INFORMATION ABOUT YOUR ADSL USERNAME & PASSWORD FROM ISP (just to be sure, the username and password from your ISP not your Modem Web Login Page (useful if you had the modem Web Login & Password since you might want to poke around inside the modem later on).
3. Navigate to your pfSense WAN page, set it to *PPPoE and fill in your username & password from ISP.
4. Navigate to your pfSense LAN page, set it to Static IP and use the following address 192.168.1.1
5. Navigate to page System -> General Setup and fill in your DNS accordingly.
6. Test your internet, ping/surf/nslookup

Please refer to my attachment named mm_network_layout_EDITED.png for this setup (both of this choices uses the same suggested network diagram as attached)

2nd Choice
1. We won't touch anything on modem side and we'll accept any IP that comes from Modem (this would cause uncertain effect since we're using double NAT setup and it's not advisable to proceed although it MAY work).
2. Set your pfSense WAN  to Static IP to 192.168.0.2
3. Untick 2 checkboxes that you see on the bottom of pfSense WAN page. (attention on this step. MUST BE DONE)
4. Navigate to page  pfSense LAN page, please set it to Static IP and use the IP Address 172.16.1.1 (you still can use 255.255.255.0 as your netmask but 255.255.0.0 is advisable) ******
5. Navigate to page System -> General Setup and fill in your DNS accordingly.
6. Please ensure that your LAN user(s) or the rest of LAN is using IP address the same range as 172.16.1.x .
7. Test your internet, ping/surf/nslookup

At this point, you want to ensure that on both choices of setup, DHCP is provided by pfSense itself and please navigate to Services -> DHCP Server to set the address range accordingly. Modem can turn off it's DHCP function for LAN since pfSense already doing it for Modem and even if it's turn on, DHCP traffic from Modem cannot reach to LAN user(s) since the diagram suggest that Modem is inside External zone and LAN user(s) is inside Internal zone. It's separated via 2 physical NIC (blue and red). Please refer to my suggested network diagram for references.

• = Need confirmation on from your ISP whether your package is PPPoE or DHCP Cable. Please do proceed and set accordingly to your package from ISP.
  ** = 255.255.255.0 has the same name for /24 and 255.255.0.0 has the same name as /16 . Please select from dropbox accordingly on that page.

Regards

[GeoffatMM]

I have just typed a long reply to you but the connection timed out and instead of posting it it has lost it.  It is too late to redo tonight.  Will try again in the morning.

[GeoffatMM]

Codemaster, first an apology for an error on the diagram of the network.  It shows the internal router address as 192.168.0.1 when it should be 192.168.1.1.

Although I would like to follow the first option so that pfsense manages the ADSL connection, I do not think it will be possible although I am wiling to try as I can always revert.  I have the login details for the ADSL but when I tried to allocate 192.168.1.1, it of course clashed with the existing router address.  Furthermore, I do not know how to stop the box managing the connection as the command interface is limited on the box and I cannot see anything to disable the connection or to bridge the router (which I assume I would have to do?).

In the diagram you have changed the link of the router to go direct to the pfsense box but although I can do this in a virtual sense (i.e., through address mapping) I cannot do it in a physical sense as I need to use the four ethernet connections in the box for the IP phones and other items.  Can you confirm that your revision is intended as a virtual not a physical break?

The IP phones and power link and connection to my office are essential and using this box is the only way for them to have a wired ethernet connection (i.e., not wifi or power link) without me rewiring the house which I do not want to do.  Also, the power link has to be connected to the router (pfsense or SFR) not passed ver wifi as your diagram suggests.  I would need to wire a second (physical) cable connection from the pfsense server back to the SFR router position with a switch if your diagram requires a physical break as shown and again, this would be very difficult to achieve.

What DNS settings should I use?  At the moment, AirVPN is set up and working on the pfsense router and the DNS is set to its requirements and should remain so I assume those are the ones I should use.

I am not really understanding why you want to amend the wiring (if that is what you are proposing) as I thought this could be managed through subnet addressing which is really all I am now trying to understand and achieve.  Whenever I set a separate subnet i cannot ping from one to the other and that is the bit I do not understand and there seems to be several ways to achieve it including iptables, NAT and routing, all of which seem to require adjusting the netmask?.  This is the part I need to understand better and apply to my network

Also, having looked at your instructions (and tried it) I could not see how to enable the pfsense box to manage the WAN service dial up.  Is it automatic?  Does it require a reboot?  Is there something else I need to do to start it if I use it?

I have manage to get it all working again on my single network but would still like to continue to try to set it up via subnets if possible.

Geoff