I'd start with a fresh install - it sounds like you either have some broken hardware, or you changed settings you don't understand.

It might work, but:

a) It isn't supported

b) Running your gateway and firewall as a virtual host isn't a good choice for security (see the Virtualisation forum)

** Quick update at the bottom **

Thanks for your reply mhab12.  I didn't intend make it more difficult.  Partly, my pfsense boxes are not the most robust machines and I've noticed that there are limits to what I can have running on one box at a time.  For instance I have had to limit what rules are running on snort using one box and compensate the snort on the other box, kind of splitting the load in a sense, so that one box covers certain rules and the other box the rest.  There are some rules in snort that cause my service to stop if I have too many selected.  I don't have the best boxes with the up to date components, but I wanted to make it still secure enough and not overwhelm one boxes resources.
Thanks for the heads up on the reverse proxy, I may give that a shot since my web box is not that great either and it would be less for it to deal with if one of the pfsense boxes to could handle a little of the load.  I think what I was finding is too much on one machine slows things down, but sharing resposibilities between boxes will lower the load on the computer and also give me more security on my home network as a perk.  I hope I didn't sound psycho about having two pfsense for security, I'm just better at visualizing things and this made sense for troubleshooting and, for some reason, give me a quick way to get the internet back up if one box goes down.

@mhab12:

Couple of thoughts:

1 - You mention binding squid to WAN.  This will not do what you're thinking and cache the outbound data from a 'slow' web server.  Doing this will require something called reverse proxy.  The squid package in pfSense will do it, yes, but it requires additional configuration beyond the included GUI.

2 - It sounds to me like what you're explaining could be accomplished by just adding an extra NIC to the first pfSense box.  By creating an OPT interface (likely OPT1), you can effectively have two LANs, LAN and OPT1, one will be 1.1 and one 2.1  You can setup firewall rules to prevent/limit access between them, setup bridges, anything you need.  If you do not trust the firewall rules well enough and chose to have two boxes for that reason, that's another issue.

** Update for my setup **

Just letting everyone know that I now have 1.1 running snort with rules split between it and 1.2 network pfsense boxes.  This is the main reason I wanted to set things up in this way, because I don't have the newest boxes and only 512mb ram in each.  I guess if I had a nice firewall box then it would be unnecessary for my setup, but I'm using what I've got…  my ram usage on 1.1 is at 62% with snort and squid running, and my ram usage on 1.2 is 68% with snort running 2 main rules and 2 empty rules.  I may end up swapping rules on the machines and see if I can balance them a little better, but for now I have backdoor and netbios running with the largest rules and then the two empty ones local and experimental.  The rest of the rules are running on the 1.1 pfsense box, but since it has a faster processor I may end up squeezing more out of these rules if I swap the rules between the two boxes.  We'll see how things go.

Just FYI.

It turns out that there was a configuration error upstream, so nothing was getting to the firewall from the outside at all. I'm sure there's some tuning to do, but I'm extremely happy with the job that pfSense is doing now.

Van

I would bind squid to any interface that is going to have users doing browsing.  I think for you that is all except WAN.  Make sure you've switched your GUI to run on HTTPS so there are no port conflicts on port 80.

As for caching windows update, there is nothing special to do.  Just make sure you set the 'Maximum Object Size' to something like 262144 (256Mb) if you want to grab items like windows update.  I've noticed this helps a lot across the board with any updates, not just MS (think AOL, AIM, P2P programs).  That said, I was having some issues with the most recent version of Squid not serving anything from cache, but that's another issue.

No I didn't actually because there was a time pressure and i had to put it again in production as soon as possible so didn't made more tests…but i will try again when i will have time ;)

thank you

:(

[SOLVED!]
right i've fixed it! I moved the virtual onto another PC that had dual onboard nics. One a Marvell tech nic and the other an Nvidia Nic (see where I'm going here?). I disabled the Nvidia Nic in the bios and slapped a spare  3Com card I had lying around into a PCI slot and presto worked first time!

Looks like the forcedeth driver doesn't play well will vmware on ubuntu 8.04.2, I'm kicking myself as I usually make sure the hardware is solid because I know how twitchy vmware is with network hardware…It worked fist time so I'll be switiching out the 3scom for 2x netgear at some point!!!!

Read/Write throughput isn't the end-all-be-all of performance.  Access time is VERY important when working with tiny bits of data and on that front an SLC SSD (be it Compact Flash, DoM, 2.5" SATA) will destroy a normal disk.  I'd say you'll be fine as long as your device supports DMA (PIO4 is still 20MB/s but it comes with high CPU usage).

To osopolis:  I'm not sure that a single core Pentium 4 will be able to deal with 400Mbit/s, though I'll admit that I've never tried to route that much traffic through anything but an actual hardware router (not to mention that that chip is going to run hot as hell, what is that, TDP of 120W?).  You'd probably be better off with something newer like a Intel E7400 or the Xeon equivalent.  Also, make sure you get Server network cards (or at least Intel Desktop cards) as cheap Realtek parts (or anything similar) aren't going to be able to keep up.

I was having a similar boot error on pfsense 1.2.2 after the full live CD ISO install:

hptrr: no controller found
ad4: FAILURE - SET_MULTI status=51 <ready,dsc,error>error=4 <aborted>My Hardware:

Motherboard  SuperServer 5015B-MRB
http://www.supermicro.com/products/system/1U/5015/SYS-5015B-MR.cfm

Processor    Xeon Dual Core 3065 2.33 4M 1333fsb Boxed

Memory    4GB (4 x 1GB) 667MHz DDRII Unbuffered ECC Memory

I installed pfsense with no swap on a 40 Pin IDE 1GB Dual embedded disk module (SLC).

http://www.innodisk.com/production.jsp?flashid=81

I ended up changing the BIOS setting for this IDE ad4 device by entering the BIOS ~~and selecting:

Ext. Primary Master [1048MB]

and changing the settings from [auto] to [user]

and changing the Transfer Mode to [FPIO 4 / DMA 2]

and changing the Ultra DMA Mode to [Mode 4]

After doing this, everything booted fine.

-Will http://www.tranquilnet.com~~</aborted></ready,dsc,error>

No.

You might try some of the suggestions from the Doc Wiki:

http://doc.pfsense.org/index.php/Boot_Troubleshooting

It works for me too as of the pfSense-1.2.3-20090224-1127.iso snapshot.

There was others before that which worked, but had other installer issues. (They tried to put gmirror support back into the installer, in addition to having the libraries on the CD, but it wasn't working properly.)

unfortunately, the update didn't fix the problem. Still needs to restart the firewall to get the flow going again. Any ideas?

What does your Xen config look like?

I have a setup almost like this working at the moment. booted up from the livecd and installed on a flash drive (have ata 2 CF adapter installed). This works great.

It seems the nic on the WAN was faulty.  I replaced it as a last resort, and everything worked after that.  :-)

Yeah it's a FreeBSD issue. I suggest trying 1.2.3 as it's based on a newer FreeBSD release.

@dA_vO:

Same problem here. :(

And you can tell that you're using exactly the same hardware and version of pfSense as akus despite him not posting enough to be able to tell - I'm impressed!

The symptoms may be similar, but the problem isn't likely to be the same.  If you're having problems you'll need to post more than "me too".  Details of the hardware you're using, version of pfSense you're using, what the problem is and where it happens.  I'd also suggest you search the forum and read the wiki.

Thanks for the quick response and the info.

I installed version 1.2.3 before the office filled up with people. It is running now for 2 hours without any issue. The boot process was normal.

I also suspected some hardware hassle. Typically I would assume timing or temperature being involved in such a case. I just do not have any explanation why this happens at the same point during the boot process and never during normal operation.

When the office gets empty I will perform more boot tests to see whether the issue is solved with the new FreeBSD or not.

Christian