After upgrading to HAProxy 0.59_2 nothing works anymore!!!!
-
@willywonka
Hi, I have the same issue, 6 sites offline! Backup file gives me a tag error. Do you know how to go back to haproxy 0.54_2?
Regards, LAV -
@lavenetz Nope, i do not know how to get back online ... i might need to review my upgrade policy
, because this update kicked me in the lulz hard. -
@willywonka thanks, anyway the contributor of the package is responsible to bring a new version with a patch. But I cannot wait until next year, by the way. I have an old pfSense with 2.3.5-RELEASE-p2 (i386), an there is the version 0.54_2 installed and running,but it's 32-bit version! What do you think?
Regards. LAV -
-
@willywonka yeah - downgrades for packages have never been supported, so I'd be cautious. - I get it though, I am also impacted, and wished I had looked extra hard at the changelog before hitting the button. Most of these upgrades are soo smooth.
-
@alexwitherspoon Perhaps, you don't believe it, but I have an extra test pfSense, exactly the same hardware. The problem is, althought I did a backup and the update log showed me successful
Installing pfSense-pkg-haproxy-devel...
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
The following 2 package(s) will be affected (of 0 checked):New packages to be INSTALLED:
pfSense-pkg-haproxy-devel: 0.59_2 [pfSense]
haproxy-devel: 1.8.12 [pfSense]Number of packages to be installed: 2
The process will require 2 MiB more space.
727 KiB to be downloaded.
[1/2] Fetching pfSense-pkg-haproxy-devel-0.59_2.txz: .......... done
[2/2] Fetching haproxy-devel-1.8.12.txz: .......... done
Checking integrity... done (2 conflicting)- pfSense-pkg-haproxy-devel-0.59_2 [pfSense] conflicts with pfSense-pkg-haproxy-0.59_2 [installed] on /usr/local/pkg/haproxy.xml
- haproxy-devel-1.8.12 [pfSense] conflicts with haproxy-1.7.11 [installed] on /usr/local/man/man1/haproxy.1.gz
Checking integrity... done (0 conflicting)
Conflicts with the existing packages have been found.
One more solver iteration is needed to resolve them.
The following 4 package(s) will be affected (of 0 checked):
Installed packages to be REMOVED:
haproxy-1.7.11
pfSense-pkg-haproxy-0.59_2New packages to be INSTALLED:
haproxy-devel: 1.8.12 [pfSense]
pfSense-pkg-haproxy-devel: 0.59_2 [pfSense]Number of packages to be removed: 2
Number of packages to be installed: 2
[1/4] Deinstalling pfSense-pkg-haproxy-0.59_2...
Removing haproxy components...
Menu items... done.
Services... done.
Loading package instructions...
Deinstall commands... done.
Syslog entries... done.
[1/4] Deleting files for pfSense-pkg-haproxy-0.59_2: .......... done
Removing haproxy components...
Syslog entries... done.
Configuration... done.
[2/4] Deinstalling haproxy-1.7.11...
[2/4] Deleting files for haproxy-1.7.11: ........ done
[3/4] Installing haproxy-devel-1.8.12...
[3/4] Extracting haproxy-devel-1.8.12: ........ done
[4/4] Installing pfSense-pkg-haproxy-devel-0.59_2...
[4/4] Extracting pfSense-pkg-haproxy-devel-0.59_2: .......... done
Saving updated package information...
done.
Loading package configuration... done.
Configuring package components...
Loading package instructions...
Custom commands...
Executing custom_php_install_command()...done.
Menu items... done.
Services... done.
Writing configuration... done.Cleaning up cache... done.
ErfolgreichI didn't see that the HAProxy showed me backend_server_ipvANY in red! So I executed the update on the productive pfSense as well. Interesting accidents!
By the way, I replaced on the test system the haproxy 0.59_ by haproxy-devel 0.59_2 but with no effect.
-
@lavenetz yeah I didn't actually have any issues running the upgrade, that went fine. My HAProxy shows all green status pages , and no issues, except that SNI isn't working. Only the primary certificate is issued, no other certificates are issued despite being in the crt_list.
That makes this one tricky to detect, though I could have tested ALL urls for proper 200 status and valid certs.
-
@alexwitherspoon Ok so i managed to revert to v0.54_2 successfully with again my ssl offloading (SNI) working, this is how:
- On pfSense console i insert 8 followed by enter (to choose Console).
- i type in pkg remove haproxy-0.59_2
- i got asked, are you sure? Insert yes
- then i type pkg add http://files00.netgate.com/pfSense_v2_4_2_amd64-pfSense_v2_4_2/All/pfSense-pkg-haproxy-0.54_2.txz
- after installation reboot pfSense and voila everything working again and package manager says: haProxy v0.54-2
- Party!
-
@willywonka I owe you a beer. That's magic, works here too!
[2.4.3-RELEASE][admin@edge.atwlab.com]/root: pkg remove pfSense-pkg-haproxy-0.59_2 Checking integrity... done (0 conflicting) Deinstallation has been requested for the following 1 packages (of 0 packages in the universe): Installed packages to be REMOVED: pfSense-pkg-haproxy-0.59_2 Number of packages to be removed: 1 Proceed with deinstalling packages? [y/N]: y [1/1] Deinstalling pfSense-pkg-haproxy-0.59_2... Removing haproxy components... Menu items... done. Services... done. Loading package instructions... Deinstall commands... done. Syslog entries... done. [1/1] Deleting files for pfSense-pkg-haproxy-0.59_2: 100% Removing haproxy components... Syslog entries... done. Configuration... done. [2.4.3-RELEASE][admin@edge.atwlab.com]/root: pkg add http://files00.netgate.com/pfSense_v2_4_2_amd64-pfSense_v2_4_2/All/pfSense-pkg-haproxy-0.54_2.txz Fetching pfSense-pkg-haproxy-0.54_2.txz: 100% 69 KiB 70.5kB/s 00:01 Installing pfSense-pkg-haproxy-0.54_2... Extracting pfSense-pkg-haproxy-0.54_2: 100% Saving updated package information... done. Loading package configuration... done. Configuring package components... Loading package instructions... Custom commands... Executing custom_php_install_command()...done. Menu items... done. Services... done. Writing configuration... done. [2.4.3-RELEASE][admin@edge.atwlab.com]/root: -
@alexwitherspoon said in After upgrading to HAProxy 0.59_2 nothing works anymore!!!!:
@willywonka I owe you a beer. That's magic, works here too!
No magic here, just pure desperation
-
As a workaround you can probably use the haproxy-devel package, it functions on the same configuration. and seems to work properly with SNI and offloading with multiple certificates..
For haproxy 'stable' ive send a preliminary 'quick fix'.., should be easy to apply the 2 changed lines manually for those who want need it 'now': https://github.com/pfsense/FreeBSD-ports/pull/542/files#diff-eb226b2eb58fc682fb444d554fb6bab8
That seems to fix the SNI behaviour.. but im not sure about the first report from @kdillen is actually a SNI issue.? Can you @kdillen try the patch?Sorry for the trouble guys..
-
@PiBa Correct in my case it is not SNI because I am using the ssl/https (TCP Mode ) . This is done because I needed the HTTP/2 support which was not yet in Haproxy at moment I first installed the Firewall.
If you want I can try the patch but that will be during the weekend. I actually was lucky to have also a backup for my standby firewall with the older Package version so I did a restore on that one also. Normally on Saturday morning 7:00 CET I make full image backup's of my firewalls so I can easy upgrade the standby node and apply the patch
Can you provide me with the instructions on how to do the patching ? Thanks in advance.
-
@kdillen hi I've checked also haproxy 0.59_4 on my main pfSense (normally I don't do this), but it also did not work, same as 0.59_2. I did the same workaround like Micha (many thnks, see https://forum.netgate.com/user/nonick):
- deinstallation of current version
- pkg add http://files00.netgate.com/pfSense_v2_4_2_amd64-pfSense_v2_4_2/All/lua53-5.3.4_1.txz
- pkg add http://files00.netgate.com/pfSense_v2_4_2_amd64-pfSense_v2_4_2/All/haproxy-1.7.10.txz
- pkg add http://files00.netgate.com/pfSense_v2_4_2_amd64-pfSense_v2_4_2/All/pfSense-pkg-haproxy-0.54_2.txz
- check box Encrypt (SSL) in HAProxy Frontend(s) and ev. reboot
- check HTTPS and/or HSTS with
6 a) https://www.ssllabs.com/ssltest/analyze.html?d=www.xxx.yy (should be at least a green A+)
6 b) https://www.sslshopper.com/ssl-checker.html#hostname=www.xxx.yy (all should be green)
Regards, LAV, sorry about my English!
-
Try adding SNI Filter in front end config "*.company.com" matching the following certificate. That's how I got mine to work again.
-
The package maintainer pushed several updates last night. They should become available very soon after the next snapshot builds. Watch for the updated versions in System->Package manager->Installed packages, or on the dashboard packages widget.
-
@kdillen
Can you check how the 'servers' are configured in the haproxy backend? I expect yours do want 'https' but dont need haproxy to do the encryption though do have the 'Encrypt(SSL)' checkbox checked while probably they shouldn't now?For others.:
Well 0.59_4 should be available for the 'haproxy' package (haproxy-devel does not need that particular change/fix..) this should have SNI certificate selection for people who are using ssl-offloading with haproxy, and fixed the files tab.. -
@lavenetz hi, am I doing this pkg add from shell or is there away to do it from the web GUI?
Thanks,
-
@piba said in After upgrading to HAProxy 0.59_2 nothing works anymore!!!!:
ckbox checked while probably they shouldn't now?
@PiBa I did the work around with going back to a previous version. Today I did the upgrade to the latest version of pfsense (2.4.4) and guess what the issue is back. I worked around it by using some extra public IP's and NAT and going back to the pfSense Loadbalancer.
Afterwards I tried to recreate a new TCP based loadbalancer. I played with all the settings for front en backend but or I get nothing passing through it or I get everything encrypted and not readable for my reverse proxies. I am not using any ACL's or anything just TCP forwarding towards 2 backends.
Anybody else has this problem ? Because it should not cause these kind of problems I guess ?
Thanks in advance.
-
@kdillen
Can you post your haproxy.cfg ? Tried enabling/disabling the ssl options on the backend server? -
Make sure there is nothing in "SSL Offloading - client certificates" in the Frontend. I noticed that the latest version had a change to the way this was handled.