CARP, and multiple networks on a single interface.
-
Any chance there is an usb-nic involced in your setup?
-
None. This is the triple-gigabit hacom model. The only thing "different" is that on both of my boxes I've added a single 10/100 NIC to get a fourth interface. One was an rl and the other dc. (I think….I'm not at the data center right now).
You've seen this error before? Is it really required that you have an IP in the same subnet on a physical interface for a CARP virtual IP to work? Doesn't seem right to me for some reason.
EDIT: Yup, you've seen it before:
http://forum.pfsense.org/index.php?topic=1374.0
That behavior is pretty accurate, although there's a step here I guess I should have tossed in. Originally while I was waiting for my new hacoms to arrive, I have a wrap and a soekris(another thread with carp issues where the switch was to blame exists here...), and when the hacoms arrived, I exported the config.xml, then running beta4. I installed beta4 onto the hacoms, then imported config.xml for each, then ran the RC1 update. After I saw that was good, I shut each one down in sequence and installed an additional pci nic so that I would have another routable interface for the office. That started my troubles. I've tried three different driver sets, rl, dc, and de.
When pfSense comes back up, it wigs out a bit wondering what's up with the interfaces and runs the interface assignment code again. No big deal, I assign them, and as a result the interface I had labeled "pfSync" winds up losing it's identity, which was expected. I go back to that interface, enable it, and that's when I get a kernel panic regardless of which driver interface I add to the system. I also get the error above regarding CARP, again that doesn't make a lot of sense.
I hate giving actual IP's, but I feel like you need to see what I have going on here.
Originally we were only assigned a /28 worth of addresses, which we burned through in no time at all. We expected to get a permanent allocation from ARIN, but that STILL hasn't happened. We had this setup:
206.80.68.16/28, which we subnetted to a pair of /29's:
WAN side
.16 network
.17 upstream gateway
.18 pfsense1
.19 pfsense 2
.20 WAN CARP virtual IP for pfsense boxes
.21 and .22 vacant
.23 broadcastLAN side
.24 network
.25 pfsense1
.26 pfsense2
.27 LAN CARP virtual IP for pfsense boxes
.28-.30 servers
.31 broadcastThat worked just fine. Upstream gives me a pair of temporary /24 allocations. I want those on the LAN side, so I went into CARP and tried to add, as you see above 206.80.88.1/24 as a CARP on pfsense1. That worked on Beta4 without complaint. RC1 refuses with the above error. It is notable however that even though Beta4 allowed it, it didn't actually work so far as I could tell. Traffic wasn't actually getting passed from interface to interface. What is proper protocol for this?
-
None. This is the triple-gigabit hacom model. The only thing "different" is that on both of my boxes I've added a single 10/100 NIC to get a fourth interface. One was an rl and the other dc. (I think….I'm not at the data center right now).
You've seen this error before? Is it really required that you have an IP in the same subnet on a physical interface for a CARP virtual IP to work? Doesn't seem right to me for some reason.
Yes, it's really required.
–Bill
-
You beat me to the post by a few seconds. See my post above. How would you approach my situation?
-
Side note - I tried to alias 206.80.88.2 and .3 to the LAN interfaces on each box. pfSense's code does not recognize aliases and still states that an IP on a real interface must exist. Proxy-ARP on each box won't do it either.
-
Well, I went back to the data center, and I tell ya, I'm starting to wonder about the power supply of these Hacom boxes. I had to play musical NIC's, but I did finally get a combination of cards that seem to be stable, a single rl and de. Everything else was causing kernel panics. I'm concerned that the quad 10/100 soekris cards I just ordered will be just as unstable. :(
After having spent $700 a pop on these systems, I get the feeling I'm going to have to go with a home grown solution, which really sucks as the form factor on these is good, but if the PCI slots are useless, then the entire system is. :\
-
Contact bao from hacom concerning your (possible) powersupply issues. I'm sure he's willing to help you or give you some advice.
-
Is there anything that can be done with the network layout above? It really blows not being able to have more than one CARP-able network per interface.
-
Afaik this is a limitation of how CARP works but somebody might proof me wrong.
-
Well, a way to test it would be to not use the web interface at all, and use the console to set up a carp VIP. Then go back and try to use aliases and set up a second one. Don't have a console handy to try it at the moment, but it would be useful to know.
-
As others have already told you this will not work.
-
Sorry, wasn't trying to push my luck, was simply trying to figure out where the limitation was, whether it was with CARP or with pfSense.
That said, my "crashy"-ness appears to be part of a known bug. 2 phone calls with Bao Ha came to this:
http://www.freebsd.org/cgi/query-pr.cgi?pr=i386/88610
They're going to try to beef up the power supply from the current 60 Watt. Going to be next week before I have further news on that front. Shame you can't use 5 or more interfaces on FreeBSD 6.0. No movement on that bug since November either.
-
Update - Bao tried a better power supply, but that doesn't appear to be the problem, there seems to be something wrong with the PCI bus, and is taking the matter up with the manufacturer in Hong Kong.
I hope he gets it resolved soon. These are pretty expensive door stops!
(I guess this became a hardware thread on me, didn't it?)
-
Another update. Hacom has pulled their boxes from their website. They've confirmed a serious issue with the PCI bus and are working to resolve the problem. They've since refunded me for my systems. Hope they get it resolved soon!
:o