Making changes to haproxy package; how do I make them available to everyone?

marcelloc · Dec 13, 2011, 8:18 PM

Using the gui, browse https://github.com/bsdperimeter/pfsense-packages and find file you want to change, click edit this file.

After this, github will clone pfsense-package and allow you edit the file and pull file change request.

Briantist · Dec 13, 2011, 8:26 PM

@marcelloc:

Using the gui, browse https://github.com/bsdperimeter/pfsense-packages and find file you want to change, click edit this file.

After this, github will clone pfsense-package and allow you edit the file and pull file change request.

When you say "the gui" do you just mean to browse that URL in a browser? When I go to one of the file that way, there is a button that says "fork and edit". Is that the one to click? Do I have to do this to every file individually?

Again, sorry for being so green. I really appreciate your patience.

marcelloc · Dec 13, 2011, 8:37 PM

yes, press fork and edit.
this way you get a clone from pfsense-packages.

Then you choose between edit each file individually or apply via git on your clone and then pull all file changes in a single request.

Briantist · Dec 13, 2011, 9:10 PM

Okay, I think I'm almost there:
https://github.com/briantist/pfsense-packages-1

I used to git to put all the changes up to this repo at once. I started to do the pull request but I got scared when it said that I was asking bsdperimeter to pull in 333 commits, most of which looked like they were from other people, so I thought maybe I was doing something wrong.

marcelloc · Dec 13, 2011, 9:38 PM

try to edit each file and see what happens.

Briantist · Dec 13, 2011, 9:45 PM

Should I be editing it on my fork, or in the main repo?

marcelloc · Dec 13, 2011, 9:47 PM

choose file on main repo. then fork and edit

Briantist · Dec 13, 2011, 10:07 PM

Okay I did it all one file at a time. There should be 6 new pull requests. I can't add the i386 folder under the binaries7 folder, nor the i386 and amd64 folders underneath binaries8 but they will need to be there.

If there's anything else I need to do please let me know. Thanks so much!

marcelloc · Dec 13, 2011, 10:38 PM

test compiled version

i386
http://e-sac.siteseguro.ws/pfsense/8/All/haproxy-1.4.16.tbz

amd64
http://e-sac.siteseguro.ws/pfsense/8/amd64/All/haproxy-1.4.16.tbz

Briantist · Dec 13, 2011, 11:33 PM

Version should be 1.4.18.. any reason why it can't be? As I said we tried 1.4.16 before, even tried it on a linux VM to see it was something platform specific.

marcelloc · Dec 14, 2011, 12:19 AM

This is the freebsd ports version. not so easy to compile and build a package without ports. :-\

did you tried package modifications with haproxy devel version 1.5?

marcelloc · Dec 14, 2011, 12:33 AM

I've changed ports info to compile 1.4.18

amd64
http://e-sac.siteseguro.ws/pfsense/8/amd64/All/haproxy-1.4.18.tbz

i386
http://e-sac.siteseguro.ws/pfsense/8/All/haproxy-1.4.18.tbz

Briantist · Dec 14, 2011, 1:07 AM

I was going to say, I had no problem compiling 1.4.18, but you've already completed it. That's great. Is there anything else I need to do?

marcelloc · Dec 14, 2011, 1:36 AM

Install package 0.3 on a pfsense other then your production server and test it.

I'll check here too.

When all tests are done, I'll change version to 1.0 release.

marcelloc · Dec 14, 2011, 2:23 AM

@Briantist:

We did try 1.4.16 but there was something about it that didn't work correctly with RPC/MAPI (we're load balancing Exchange 2010). Whatever it was, it worked when we used 1.4.18 without any changes to the config.

The RPC/MAPI you use with 1.4.18 is for owa or all exchange services? Can I replace Micro$oft NLB with haproxy?

It could be very usefull to me. Exchange NLB freaks out my network everytime I enable it.

Briantist · Dec 14, 2011, 2:37 AM

All exchange services. We are using this for that on RPC/MAPI, OWA/EWS (both the HTTPS access and the HTTP listener which redirects to HTTPS), IMAP, POP3, SMTP (both internal and external).

I and a few co-workers have been eating our own dog food by running our own Outlook clients through our pfSense HAProxy setup for the better part of a month now and it's working great.

About 50% of our desktops are Mac and are running Outlook 2011, which uses EWS for all of its mail access, and in my limited testing so far it seems to work well that way too.

NLB is pretty crappy, so yeah we're definitely looking forward to replacing it. Once this package goes live, we're going to be doing some strict penetration testing since our pfSense cluster straddles our internal and DMZ (so that it can deal with external SMTP), and then we're going to get the rest of our group and department on it before rolling it out for the whole organization.

Once I get it all set I'll write up a post about it.

I'm doing some limited testing here at home of 0.3 version. Tomorrow when I get into work I will load it on there and try it out (it's not production yet, just the few people including myself who are using it live for ourselves) and I'll report back.

marcelloc · Dec 14, 2011, 3:13 AM

All exchange services. We are using this for that on RPC/MAPI, OWA/EWS

Great I`ll test too, any specific balance option to do this?

Once this package goes live, we're going to be doing some strict penetration testing since our pfSense cluster straddles our internal and DMZ (so that it can deal with external SMTP)

Try postfix forwarder package, it works really nice together with exchange. it keeps out more then 80% misconfigured/fake spam servers and protects your exchange servers from internet.

Briantist · Dec 14, 2011, 3:30 AM

The balance option depends on the service. For OWA, use source because you need to keep the same client on the same CAS server and since it's HTTPS you can't insert a tracking cookie. For SMTP/POP3/IMAP use round robin. RPC is a bit complex because by default Exchange uses three different services over RPC (the endpoint mapper, the address book, and MAPI) and a giant range of ports for RPC. You have to make changes so that the address book and MAPI use a single port. Then you need to create a separate frontend for each of those and add advanced options to keep the client connections on the same CAS servers (technically you don't need to create separate frontends to do this in HAProxy, but with the way it's implemented in pfSense you do). For the MAPI stuff I'm using the newly added leastconn balance option, but I was using round robin before that. I think that the advanced options end up overriding it anyway so I'm not sure it makes a difference.

It'll be clearer once I have time to put together a real write-up.

As for spam protection, we have that covered for now with a single appliance. Once this is in place, we plan making use of multiple spam gateways which we will also load balance with this setup.

Briantist · Dec 14, 2011, 5:20 AM

Also one quick question before I go to sleep, how big is your exchange environment?

marcelloc · Dec 14, 2011, 5:24 AM

There are 14 exchange servers distributed in some locations, but my problem is with 02 exchange servers in the main site.

Total mailboxes are 60k.