Deployment issue

[jackryan]

Hi All,

I am deploying a pfsense 1.2.3 installation with two load balancers into our cage at our colo facility.  the mb is a tyan server mb w/ 2 amd cpus and 4 gigs of ram and three network interfaces.  one of the interfaces is setup as the pfsync interface and the other two are set as internal and external.  Our current config has a single linux based firewall with no nating setup, all our servers have internet facing addresses.  My plan was to move all of these internet facing addresses to virtual IPs on the pfSense boxes and nat them.  all of our servers are connected to 2 HP Pro curve switches.

We attempted a rollout last week with unsuccessful results.  All of the servers gave up there external IP addresses and were just running on the internal IP addresses, and I took offline the old firewall machine.  I could connect to them from the pfsense machines on the local IP addresses but no web traffic was hitting the IPs. I could also ping outside internet addresses.  I suspect that the switches were to blame.  Here is a sample of our setup:

Internet Feed
    |                                      |
    |                                      |      (2 x feeds in an HSRP config)
    |                                      |
Switch 1-cross connected -Switch 2
  |  |  |                              |  | 
  |  |  Firewall Server          |  |  (both interfaces on machine connected to the same switch)
  |  |                                  |  |
  |  Web server 1                |  Web Server1  (Webserver has two I/Fs on two different subnets 10.1.x.x and 10.0.x.x plus an external IP address on I/F 1)
  |                                    |
  Mail server 1                    Mail Server1  (same as web server)

Here is what we are hoping to achieve:
              Internet Feed
    |                                      |
    |                                      |      (2 x feeds in an HSRP config)
    |                                      |
Switch 1-cross connected -Switch 2
  |  |  |                              |  |  |
  |  |  Pfsense #1                |  |  Pfsense #2 (both interfaces on each machine Internal and External connected to the same switch)
  |  |                                  |  |
  |  Web server 1                |  Web Server1  (Webserver has two I/Fs on two different subnets 10.1.x.x and 10.0.x.x)
  |                                    |
  Mail server 1                    Mail Server1  (same as web server)

we have three internet IPs in one range, one of them has our second IP range of 8 IPs forwarded to it.  each of the pfsense servers has one of the three IPs and the third IP is set up as a CARP address.  the pfsense servers have the 8 IPs set up as straight virtual IPs with NATing set up to the internal IPs of mail server and web server (eventually we will move to load balancing) across both switches.

I think the issue is with the switches, since we are moving the IP Addresses from one port on a physical server to another virtual one on the same switch. is there a way to clear out the tables in the switch without rebooting the switch?

Thanks,
Sean

[cmb]

Most likely cause is your upstream router's ARP cache, you'll have to get your provider to clear that.