Amazing info. Thanks guys. -Bruce

You don't even need to do that. They can be done static, just use a CARP VIP hardcoded to each of your usable IPs. A CARP VIP will have a unique MAC address and make the R2 happy. It may all just work with the CARP VIPs, though you may need to send some traffic sourced from the CARP VIP before the R2 will see it on its IP Allocation page. Easy enough to do from Diagnostics > Command: ping -c1 -S <carp vip=""></carp>

1.2.3-RELEASE built on Sun Dec 6 23:38:21 EST 2009

http://forum.pfsense.org/index.php/topic,31112.0.html

You should use a network mask of /29 in both pfSense boxes so that the WAN interface in each box is on the same subnet as the ISP gateway. Your diagram puts the ISP gateway on the same LAN as the two pfSense routers. Therefore a packet with destination address pfsense1 (65.65.65.156) should get to the ISP gateway (65.65.65.153) which should see that this packet is not for the gateway and forward it to pfsense1 (65.65.65.156). The forwarding should be based on an ARP (Address Resolution Protocol) exchange which will go something like this: the gateway will broadcast a request WHO HAS 65.65.65.156 on the ethernet, pfsense1 will see it and respond I HAVE 65.65.65.156 and the gateway will thereby know the MAC address of the system with IP address 65.65.65.156. Thereafter (until the ARP entry expires) the gateway will know the MAC address to which it should packets with destination IP address 65.65.65.156. pfsense2 will ignore this ARP request since it doesn't have IP address 65.65.65.156. From what you have described its not clear how correctly functioning systems would "mix data". @torontob: For example on pfSense-1 I have an Apache server running and on pfsense-2 I don't have an apache server. But I see stats on pfsense-2 for port 80 trying to reach a LAN client that doesn't exist. As described above, a TCP packet with destination address 65.65.65.156 will go to pfSense1 and a TCP packet with destination address 65.65.65.157 will go to pfSense2.  Anyone on the internet is free to send TCP packets to port 80 at IP address 65.65.65.157. (It is possible you were "hit" by a curious hacker.) So your pfsense2 stats about 80 aren't of any particular interest unless they correlate with a particular known activity. Does the log record a source IP address? Is it known to you? @torontob: Q-1: Can you please detail how I can do packet capture? I can only connect to one router at a time through OpenVPN I think. Q-2: What should I look for in firewall log? Q1: Basic packet capture: pfsense shell command # tcpdump -i <interface name=""></interface>(e.g. tcpdump -i em0) or from the web GUI Diagnostics -> Packet capture To reduce the noise in apacket capture its better to avoid a capture on the interface over which you are accessing pfSense OR apply some filters to the capture so that it doesn't also display the decoded capture. See a tcpdump man page for information on the filter options. Q2: Reports of blocked packets from the IP address from which you are attempting the access. Perhaps your firewall rule(s) or port forwarding for ssh from the internet is not correctly set up. @torontob: 3- By SSH I meant when I was doing SSH into one of my clients behind one of the pfSense routers, it keeps disconnecting after few seconds. It's 100% not a fault of the client and it has to do with the installation of the second pfSense router that I did. Please provide an example of what happens when you attempt ssh access. The problem description doesn't provide enough detail to distinguish between a number of possible problems. Do you have any evidence that it has to do with the installation of the second pfSense router that I did? "It only happened after I installed the second pfSense" is a possibly useful observation but doesn't prove your claim. A packet capture on the initiating system might be informative. Is a VPN involved? @torontob: 2- Both routers LAN subnet is set to (10.100.100.0/24) but since they are on different routers it shouldn't make a difference right? If you don't want to allow access between the two LANs it shouldn't make a difference.

Thanks, jimp.  That's what I thought but wanted to be sure. Yakup

thanks for the quick reply. it's setup already from the freenas box so it would be easier yes. i was just wondering if there any any implications to storing keys on pfsense? i'm not going to do it as having them on the freenas box is an extra layer but it was just something i was toying with.

perhaps you could throttle bandwidth for EXE, CAB, MSI files. Further this high bandwidth usage is only as long as, till the proxy has them all in the cache. Perhaps you can do a windows update over night and at the next morning all files will be downloaded. Another option ist, that you lower the maxmum file size, so that only small updates gets cached. There are pros and cons for caching those files.

All Geode CPUs (ALIX, Soekris, etc) have the GLX Security Block device (glxsb) which will accelerate only AES-128. So for OpenVPN you need to set aes-128-cbc, and for IPsec, you set Rijndael (which is AES-128). Unless you have disabled the glxsb device under System > Advanced, it is loaded at boot time on supported platforms.

Yes I've disabled the block private ip addresses and the other one as well.  The only problem is when I change modes to the router mode I lose internet connectivity for clients on the network.  I'm looking for more information on this topic now.  I may try another stock router to see what happens as well.

I did a fresh install on another disk and identical system. Seems to work just fine, so I'm not sure what the issue is/was? Perhaps the disk has a problem. Update: For those who come across this thread … The cause of one PC not booting properly is still unknown but most likely a hardware issue with the MB. I determined that the issue was not the disk drive, cables, or network cards. I installed on two systems with identical MB (Asus p4c800e) and bios settings. On one board, the standard ich sata port would not boot properly. Same drives on alternate PC never showed a problem. Switchiing to the Promise controller sata port on the bad board does work. Other OS's have no problem booting (linux, windoze). I can only assume that the older MB may be failing in some way. I'm new to BSD and not familiar with how BSD determines device names, but on one it would find the disk as ad4s1a and on the other (non-working) ad8s1a with promise controller disabled.

Some more interesting info: This is the log from the other router that connects successfully 16:16:34 pppd pppd 2.4.4 started by root, uid 0 16:16:34 pppd Using interface ppp0 16:16:34 pppd Connect: ppp0 <--> /dev/ttyp0 16:16:35 pppoe PPP session is 13507 (0x34c3) 16:16:36 pppd PAP authentication succeeded 16:16:36 pppd kernel does not support PPP filtering I can see that it's using PAP authentication. At pfsense /var/etc/mpd.conf  the following statements are enabled: pppoe: new -i ng0 pppoe pppoe set iface route default set iface disable on-demand set iface idle 0 set iface up-script /usr/local/sbin/ppp-linkup set bundle disable multilink set bundle authname "xxxxx@xxxxxx.com"         set bundle password "xxx@xxxxx" set bundle no noretry set link keep-alive 10 60 set link max-redial 0 set link no acfcomp protocomp set link disable pap chap set link accept chap set link mtu 1492 set ipcp yes vjcomp set ipcp ranges 0.0.0.0/0 0.0.0.0/0 set ipcp enable req-pri-dns set ipcp enable req-sec-dns open iface I can see the "set link accept chap" So is the problem related to the type of the authentication used? Should i change to pap in pfsense?

CLOSED THANKS

This is awsome!  ;D In my opinion (as someone living in a supposedly Christian country!) it should be part of pfsense. If you reboot your box on Christmas day it should play jinglebells.  :D That opens up the possibility of other date related start up themes….... Steve

@dreamslacker: That will fall under:  Maximum number of established connections per host Just create a rule that catches all traffic from LAN then set the limits per host.  Of course, if you need to shape more then there's much more tweaking to be done. What is the recommended setting for this? I set it to 60 on both the WAN and the LAN side and after a few mins my connection just came to a crawl. I had to disable it to get back online.

Update to the question.  Initial issue resolved due to problems with cached mak address being seen by the firewall devices and our switches.

Because many people open the web interface or SSH from specific remote locations for management and want to do so without having to NAT. Changing that now would break thousands of upgraded systems. I agree it wouldn't be a bad idea to have an option to only bind to specific IPs. Patches welcome.