Making changes to haproxy package; how do I make them available to everyone?

marcelloc · Dec 13, 2011, 10:38 PM

test compiled version

i386
http://e-sac.siteseguro.ws/pfsense/8/All/haproxy-1.4.16.tbz

amd64
http://e-sac.siteseguro.ws/pfsense/8/amd64/All/haproxy-1.4.16.tbz

Briantist · Dec 13, 2011, 11:33 PM

Version should be 1.4.18.. any reason why it can't be? As I said we tried 1.4.16 before, even tried it on a linux VM to see it was something platform specific.

marcelloc · Dec 14, 2011, 12:19 AM

This is the freebsd ports version. not so easy to compile and build a package without ports. :-\

did you tried package modifications with haproxy devel version 1.5?

marcelloc · Dec 14, 2011, 12:33 AM

I've changed ports info to compile 1.4.18

amd64
http://e-sac.siteseguro.ws/pfsense/8/amd64/All/haproxy-1.4.18.tbz

i386
http://e-sac.siteseguro.ws/pfsense/8/All/haproxy-1.4.18.tbz

Briantist · Dec 14, 2011, 1:07 AM

I was going to say, I had no problem compiling 1.4.18, but you've already completed it. That's great. Is there anything else I need to do?

marcelloc · Dec 14, 2011, 1:36 AM

Install package 0.3 on a pfsense other then your production server and test it.

I'll check here too.

When all tests are done, I'll change version to 1.0 release.

marcelloc · Dec 14, 2011, 2:23 AM

@Briantist:

We did try 1.4.16 but there was something about it that didn't work correctly with RPC/MAPI (we're load balancing Exchange 2010). Whatever it was, it worked when we used 1.4.18 without any changes to the config.

The RPC/MAPI you use with 1.4.18 is for owa or all exchange services? Can I replace Micro$oft NLB with haproxy?

It could be very usefull to me. Exchange NLB freaks out my network everytime I enable it.

Briantist · Dec 14, 2011, 2:37 AM

All exchange services. We are using this for that on RPC/MAPI, OWA/EWS (both the HTTPS access and the HTTP listener which redirects to HTTPS), IMAP, POP3, SMTP (both internal and external).

I and a few co-workers have been eating our own dog food by running our own Outlook clients through our pfSense HAProxy setup for the better part of a month now and it's working great.

About 50% of our desktops are Mac and are running Outlook 2011, which uses EWS for all of its mail access, and in my limited testing so far it seems to work well that way too.

NLB is pretty crappy, so yeah we're definitely looking forward to replacing it. Once this package goes live, we're going to be doing some strict penetration testing since our pfSense cluster straddles our internal and DMZ (so that it can deal with external SMTP), and then we're going to get the rest of our group and department on it before rolling it out for the whole organization.

Once I get it all set I'll write up a post about it.

I'm doing some limited testing here at home of 0.3 version. Tomorrow when I get into work I will load it on there and try it out (it's not production yet, just the few people including myself who are using it live for ourselves) and I'll report back.

marcelloc · Dec 14, 2011, 3:13 AM

All exchange services. We are using this for that on RPC/MAPI, OWA/EWS

Great I`ll test too, any specific balance option to do this?

Once this package goes live, we're going to be doing some strict penetration testing since our pfSense cluster straddles our internal and DMZ (so that it can deal with external SMTP)

Try postfix forwarder package, it works really nice together with exchange. it keeps out more then 80% misconfigured/fake spam servers and protects your exchange servers from internet.

Briantist · Dec 14, 2011, 3:30 AM

The balance option depends on the service. For OWA, use source because you need to keep the same client on the same CAS server and since it's HTTPS you can't insert a tracking cookie. For SMTP/POP3/IMAP use round robin. RPC is a bit complex because by default Exchange uses three different services over RPC (the endpoint mapper, the address book, and MAPI) and a giant range of ports for RPC. You have to make changes so that the address book and MAPI use a single port. Then you need to create a separate frontend for each of those and add advanced options to keep the client connections on the same CAS servers (technically you don't need to create separate frontends to do this in HAProxy, but with the way it's implemented in pfSense you do). For the MAPI stuff I'm using the newly added leastconn balance option, but I was using round robin before that. I think that the advanced options end up overriding it anyway so I'm not sure it makes a difference.

It'll be clearer once I have time to put together a real write-up.

As for spam protection, we have that covered for now with a single appliance. Once this is in place, we plan making use of multiple spam gateways which we will also load balance with this setup.

Briantist · Dec 14, 2011, 5:20 AM

Also one quick question before I go to sleep, how big is your exchange environment?

marcelloc · Dec 14, 2011, 5:24 AM

There are 14 exchange servers distributed in some locations, but my problem is with 02 exchange servers in the main site.

Total mailboxes are 60k.

Briantist · Dec 14, 2011, 4:57 PM

Everything is looking good to me.

I noticed that you made the required version 2.0. Are we only updating this for 2.0? As far as I know it's only the binary for 7 that is needed to make this work on 1.2.3.

That's a large exchange environment you have! We've got around 1,100 mailboxes. Are you on Exchange 2010 as well?

marcelloc · Dec 14, 2011, 5:45 PM

Everything is looking good to me.

Good news. :) Did you tested all features?

I noticed that you made the required version 2.0. Are we only updating this for 2.0? As far as I know it's only the binary for 7 that is needed to make this work on 1.2.3.

I'll compile it to 1.2.3 too. The required version you see is just on 2.0 xml.

Are you on Exchange 2010 as well?

not yet.

Briantist · Dec 14, 2011, 5:53 PM

Yes, as far as I can tell all features are working. I am still running my own Outlook instance through it. All options in the package are there and appear to be working fine. The XMLRPC sync is good.

Any chance I can get access to this wiki page to update it once this goes totally live:
http://doc.pfsense.org/index.php/Haproxy_package

Are you on 2007 or 2003?

marcelloc · Dec 14, 2011, 8:50 PM

Are you on 2007 or 2003?

Both, some locations are not migrated yet.

Any chance I can get access to this wiki page to update it once this goes totally live:

Ask core developers to create an acount at docs.pfsense.org to you.

Briantist · Dec 15, 2011, 2:33 PM

Hey marcello, are we ready to finalize the package? Is there anything else you need from me?

What's the best way to contact a core developer for wiki access without annoying them? I know PMing is generally frowned upon..

marcelloc · Dec 17, 2011, 4:14 PM

Just changed package version to 1.4.18 pkg v 1.0

Since I finish 1.2.3 compiling and testing I'll change there too.

to create an account at docs.pfsense.com, just send an email to wikiadmin@pfsense.org asking it.

Briantist · Dec 19, 2011, 2:46 PM

Sounds good, though I do not see the version updated.

marcelloc · Dec 19, 2011, 5:01 PM

I've republished package version change.