What is the right subnet mask in my situation?

torontob

Hi Everyone,

My provider gives me this:

Network: 65.65.65.152/29
Gateway: 65.65.65.153
Mask: 255.255.255.248
Usable: 65.65.65.156 to 65.65.65.158

I want to have TWO pfsense routers installed in this fashion:

ISP
|
Dumb Switch
| |__
| |
pfSense1 pfSense2
65.65.65.156/29 65.65.65.157/29

My question is should I setup /29 on each router for WAN port like above or do I have to do /31 because I am only picking one Public IP per router? I have tried /29 and it works but I am not 100% whether this is the right way to do it or not because if I pickup /29 doesn't that mean that I want to reserve all the three available address to one router? I have this thing in the back of my head that the packets sent from ISP will come to both routers in the above diagram because both request the /29.

Some feedback would be much appreciated.

Thanks

The WAN interface needs to be configured using the assigned subnet mask. If you change the subnet mask, you will make your gateway unreachable and break your network.

http://en.wikipedia.org/wiki/Subnetwork

torontob

Thanks for the input. So, you are approving my diagram then?

My problem is that since I installed the new router everything has got so slow to the point that it's annoying and SSH connection breaks every 5 seconds or so. I checked Stats on both router and I see multiples of connections for the same port making me believe that both routers are receiving packets regardless of which IP I try to query.

For example on pfSense-1 I have an Apache server running and on pfsense-2 I don't have an apache server. But I see stats on pfsense-2 for port 80 trying to reach a LAN client that doesn't exist.

However, I do have the same subnet scheme for both pfSense but since they are separate it shouldn't matter. Their only point of contact is the Dumbp-Switch where they pick their Public IP from my ISP.

Do you think my ISP is broadcasting whatever comes in to both pfSense public IPs in turn to both pfSense hence gives me the headache of dropped connections etc….or is it because I have both subnet's as 10.10.10.0/24 on both pfSenses?

Please let me know where I can look at to fix this really annoying problem.

Thanks

wallabybob

Suggestion: Do a packet capture on the WAN interfaces of both pfSense boxes at the same time. This would probably give you a better idea of what is happening; both pfSense boxes getting the same packets? the router sending packets to the wrong destination? etc.

Have you checked the firewall logs?

How are you checking the SSH? are the ssh attempts coming from the internet or from downstream of one pfSense to the other pfSense or a system downstream of it?

@torontob:

For example on pfSense-1 I have an Apache server running and on pfsense-2 I don't have an apache server. But I see stats on pfsense-2 for port 80 trying to reach a LAN client that doesn't exist.

Do you have some poorly configured load balancer? Where are these stats? Do they also report the source IP address? (Knowing where these attempts came from might help you determine why they were sent to pfSense-2.

@torontob:

However, I do have the same subnet scheme for both pfSense but since they are separate it shouldn't matter. Their only point of contact is the Dumbp-Switch where they pick their Public IP from my ISP.

Please explain what you mean by "pick their Public IP from my ISP"

It is not clear to me why you want that particular configuration. Load balancing? Link aggregation? fail over? something else?

torontob

Hello,

Thanks for the input but you got confused with a few points. Let's try again.

1- My network structure is just like I made the diagram. There is no traffic shaping. There is no any other equipment. My question still remains if I should pick /29 as the setting on BOTH routers when I try to obtain public IP from my provider. I am thinking maybe I should do something like this:

pfSense-1 WAN Port: 65.65.65.156/31
pfSense-2 WAN Port: 65.65.65.157/31

^^^^ because I want to only pick 1 of the three available IPs from my ISP then shouldn't I use a /31 as I think that /29 on both routers would mix all the data in the network but I could be wrong like you mentioned that I would be breaking my gateway connection if I use any other subnet. I need another re-confirmation on this.

2- Both routers LAN subnet is set to (10.100.100.0/24) but since they are on different routers it shouldn't make a difference right?

3- By SSH I meant when I was doing SSH into one of my clients behind one of the pfSense routers, it keeps disconnecting after few seconds. It's 100% not a fault of the client and it has to do with the installation of the second pfSense router that I did.

4- There are a few reasons why I do this, namely one of the pfSense router is leased to someone else and I want things to be separated. So, they can handle their own firewall/router. I can't change the equipment and network structure; I have to get this running with the right settings.

Q-1: Can you please detail how I can do packet capture? I can only connect to one router at a time through OpenVPN I think.
Q-2: What should I look for in firewall log?

Thanks

wallabybob

You should use a network mask of /29 in both pfSense boxes so that the WAN interface in each box is on the same subnet as the ISP gateway.

Your diagram puts the ISP gateway on the same LAN as the two pfSense routers. Therefore a packet with destination address pfsense1 (65.65.65.156) should get to the ISP gateway (65.65.65.153) which should see that this packet is not for the gateway and forward it to pfsense1 (65.65.65.156). The forwarding should be based on an ARP (Address Resolution Protocol) exchange which will go something like this: the gateway will broadcast a request WHO HAS 65.65.65.156 on the ethernet, pfsense1 will see it and respond I HAVE 65.65.65.156 and the gateway will thereby know the MAC address of the system with IP address 65.65.65.156. Thereafter (until the ARP entry expires) the gateway will know the MAC address to which it should packets with destination IP address 65.65.65.156. pfsense2 will ignore this ARP request since it doesn't have IP address 65.65.65.156. From what you have described its not clear how correctly functioning systems would "mix data".

@torontob:

For example on pfSense-1 I have an Apache server running and on pfsense-2 I don't have an apache server. But I see stats on pfsense-2 for port 80 trying to reach a LAN client that doesn't exist.

As described above, a TCP packet with destination address 65.65.65.156 will go to pfSense1 and a TCP packet with destination address 65.65.65.157 will go to pfSense2. Anyone on the internet is free to send TCP packets to port 80 at IP address 65.65.65.157. (It is possible you were "hit" by a curious hacker.) So your pfsense2 stats about 80 aren't of any particular interest unless they correlate with a particular known activity. Does the log record a source IP address? Is it known to you?

@torontob:

Q-1: Can you please detail how I can do packet capture? I can only connect to one router at a time through OpenVPN I think.
Q-2: What should I look for in firewall log?

Q1: Basic packet capture: pfsense shell command # tcpdump -i <interface name=""></interface>(e.g. tcpdump -i em0) or from the web GUI Diagnostics -> Packet capture To reduce the noise in apacket capture its better to avoid a capture on the interface over which you are accessing pfSense OR apply some filters to the capture so that it doesn't also display the decoded capture. See a tcpdump man page for information on the filter options.

Q2: Reports of blocked packets from the IP address from which you are attempting the access. Perhaps your firewall rule(s) or port forwarding for ssh from the internet is not correctly set up.

@torontob:

3- By SSH I meant when I was doing SSH into one of my clients behind one of the pfSense routers, it keeps disconnecting after few seconds. It's 100% not a fault of the client and it has to do with the installation of the second pfSense router that I did.

Please provide an example of what happens when you attempt ssh access. The problem description doesn't provide enough detail to distinguish between a number of possible problems.

Do you have any evidence that it has to do with the installation of the second pfSense router that I did? "It only happened after I installed the second pfSense" is a possibly useful observation but doesn't prove your claim. A packet capture on the initiating system might be informative. Is a VPN involved?

@torontob:

2- Both routers LAN subnet is set to (10.100.100.0/24) but since they are on different routers it shouldn't make a difference right?

If you don't want to allow access between the two LANs it shouldn't make a difference.