Unable to add PPA to Ubuntu server when connected to HAProxy.

PiBa

@strongthany
I cannot tell whats wrong from this, perhaps its easier to copy the actual haproxy.cfg from bottom of settings tab.? more complete and easier to read (for me anyhow).

But if curl is getting a webgui certificate of pfSense, then its likely not connecting to haproxy. Can you move the pfSense-webgui to a different port.? i like usually configure :444 for my pfSense-webgui.. so if a browser or any other client is pointing to pfSense:443 i know for sure it will either connect to haproxy or not at all.. Perhaps check that no port-forward settings are redirecting the traffic..?

strongthany

I will goahead and try the webconfig port thing once I've got a bit more precautions in place in case I lock myself out In the meantime, Here's what I have for a raw config:

# Automaticaly generated, dont edit manually.
# Generated on: 2020-04-22 16:27
global
	maxconn			1000
	stats socket /tmp/haproxy.socket level admin  expose-fd listeners
	gid			80
	nbproc			1
	nbthread			1
	hard-stop-after		15m
	chroot				/tmp/haproxy_chroot
	daemon
	tune.ssl.default-dh-param	2048
	server-state-file /tmp/haproxy_server_state

listen HAProxyLocalStats
	bind 127.0.0.1:2200 name localstats
	mode http
	stats enable
	stats admin if TRUE
	stats show-legends
	stats uri /haproxy/haproxy_stats.php?haproxystats=1
	timeout client 5000
	timeout connect 5000
	timeout server 5000

frontend SharedFrontend-merged
	bind			My.Public.IP:443 name My.Public.IP:443   ssl crt-list /var/etc/haproxy/SharedFrontend.crt_list crt-ignore-err all  
	mode			http
	log			global
	option			http-keep-alive
	option			forwardfor
	acl https ssl_fc
	http-request set-header		X-Forwarded-Proto http if !https
	http-request set-header		X-Forwarded-Proto https if https
	timeout client		30000
	acl			aclcrt_SharedFrontend	var(txn.txnhost) -m reg -i ^([^\.]*)\.mydomain\.tld(:([0-9]){1,5})?$
	acl			ACL1	var(txn.txnhost) -m str -i chat.mydomain.tld
	acl			ACL2	var(txn.txnhost) -m str -i blog.mydomain.tld
	acl			ACL3	var(txn.txnhost) -m str -i cloud.mydomain.tld
	http-request set-var(txn.txnhost) hdr(host)
	use_backend chat.mydomain.tld_ipvANY  if  ACL1 
	use_backend blog.mydomain.tld_ipv4  if  ACL2 
	use_backend cloud.mydomain.tld_ipv4  if  ACL3 
	default_backend chat.mydomain.tld_ipvANY
	default_backend blog.mydomain.tld_ipv4
	default_backend cloud.mydomain.tld_ipv4

frontend http-to-https
	bind			My.Public.IP:80 name My.Public.IP:80   
	mode			http
	log			global
	option			http-keep-alive
	option			forwardfor
	acl https ssl_fc
	http-request set-header		X-Forwarded-Proto http if !https
	http-request set-header		X-Forwarded-Proto https if https
	timeout client		30000
	http-request redirect scheme https 

backend chat.mydomain.tld_ipvANY
	mode			http
	id			100
	log			global
	timeout connect		30000
	timeout server		30000
	retries			3
	option			httpchk OPTIONS / 
	server			chat.mydomain.tld 10.10.10.252:443 id 101 ssl check inter 1000  verify none 

backend blog.mydomain.tld_ipv4
	mode			http
	id			10102
	log			global
	option			log-health-checks
	timeout connect		30000
	timeout server		30000
	retries			3
	source ipv4@ usesrc clientip
	option			httpchk OPTIONS / HTTP/1.1\r\nHost:\ blog.mydomain.tld
	server			blog.mydomain.tld 10.10.10.251:443 id 10103 ssl check inter 1000  verify none 

backend cloud.mydomain.tld_ipv4
	mode			http
	id			10104
	log			global
	timeout connect		30000
	timeout server		30000
	retries			3
	source ipv4@ usesrc clientip
	option			httpchk OPTIONS / HTTP/1.1\r\nHost:\ cloud.mydomain.tld
	server			cloud.mydomain.tld 10.10.10.250:443 id 10105 ssl check inter 1000  verify none

Does this lead you anywhere?

PiBa

@strongthany
So i see 3 default_backends.. seems a bit much, 1 default should be enough, or none might be even better.. then at least you wont end up on the wrong webserver, but just get a 503 back..

None of the acl's match the ppa.launchpad.net domain..? How does that relate to chat/blog/cloud ?

strongthany

@PiBa the 3 backends are for the individual servers. I don't have a backend for the landscape server, that one isn't publicly reachable because it doesn't need to be.

the ppa server the one I need to be able to get through the reverse proxy to the landscape server. How do I have that whitelisted so that request doesn't go through the http to https redirect?

PiBa

@strongthany
Sorry but im not following..

Your sending a request to haproxy for a server it doesn't have configured.?? I'm missing something in my (mental) picture here.. Perhaps you can draw a little picture (doesnt need to be nice), about what ip's all the mentioned servers and pfSense use and from where to where you want the connection to work.?

Perhaps you can try and disable the 'transparent-client-ip' feature on any 'involved' server.?. if one is.. it causes reply packets to not get routed for that server:port combination..

strongthany

I have a server called Landscape that lives in my LAN. As shown in the config I posted, there is not a back-end for Landscape on the HAProxy config. I want to install Landscape on the server called Landscape. I cannot do so because HAProxy is filtering traffic coming into my network. HAProxy is filtering traffic coming into my network by routing things over http to https. When I try to install Landscape on the server called Landscape it fails to do so. It fails to do so because HAProxy moves the http traffic to https. I want to tell HAProxy not to filter that traffic Landscape.

How do I achieve this? Which of these options would be a better course of action, and how would I preform the one you deem better:

Exclude the server called Landscape from having it's traffic filtered

or

have the address for where I install Landscape from excluded form being filtered

Does this make sense?

PiBa

@strongthany
Either i'm totally not understanding your setup or the road we are on is totally unrelated to the problems your experiencing..

It seems to me that you have a new 'Landscape server' on which you want to install some software with 'apt', from a repository on 'the internet'..
This should have nothing to do with haproxy.. As the launchpad servers are not going to connect to your public ip where haproxy is listening, and you do not have the landscape server or dns records configured to point to your own wan-ip/haproxy-ip..
As such traffic should have absolutlely nothing to do with haproxy and also not with a pfSense-webgui certificate..
So it seems to me like your either running a other forward-proxy like squid, which could be filtering traffic, or something like snort/pfBlocker assuming the issue lies in a package... haproxy normally is not involved in traffic going from the LAN to the-internet.

But now i think its most likely that there is a wrong port-forward rule in the pfSense nat rules. Can you check/disable those.? Haproxy is listening on the wan-ip already, so doesnt need any nat rules.. only a firewallrule to allow traffic from clients on the internet to the wan-ip.

strongthany

I have no NAT rules. After spending some time talking to some people on the Ubuntu IRC forum I'm not so sure it's even HAProxy anymore.

When I run wget -qSO /dev/null http://ppa.launchpad.net/wireguard/wireguard/ubuntu 2>&1 | less from a device connected to my network I always get the following:

  HTTP/1.1 301 Moved Permanently
  Server: nginx
  Date: Fri, 24 Apr 2020 22:13:04 GMT
  Content-Type: text/html
  Content-Length: 162
  Connection: keep-alive
  Location: https://ppa.launchpad.net/wireguard/wireguard/ubuntu
  X-Frame-Options: SAMEORIGIN

Running the same command outside the network gets a 200 code from an apache server. Someone noted that the reason I'm seeing nginx is because of this note. However I have no idea why I'm not able to get out and to be quite honest I'm losing my mind over this.

PiBa

@strongthany
If that 'note' applies to your situation that would mean you have configured a services/captive-portal.? Do you have that.?

strongthany

@PiBa No captive portal, only thing that would be intercepting traffic like that would be the reverse proxy which I've shown by disabling it that is not the issue. Would there be something else in the way that would be breaking the connection?

PiBa

@strongthany
Can you show the output of these commands (maybe replace your public ip, if it shows up..)?:

pfctl -sn
ipfw show

strongthany

I have ran the command(doing remote cli, pfctl -sn didn't work but ipfw show did) and got the following:

00010    487555     70366491 fwd 127.0.0.1 tcp from 10.10.10.251 443 to any in recv re0
00011    651701    255737457 fwd 127.0.0.1 tcp from 10.10.10.250 443 to any in recv re0
65535 254477642 172839414961 allow ip from any to any

Not sure what this means..

PiBa

@strongthany
Traffic from the webserver running at '10.10.10.251 443' is redirected to pfSense's localhost, no matter where the client-return-address was, but this should not affect outgoing https requests from such a server. As they would target google:443 for example, but come from that server on a random client-port.. And certainly shouldn't affect traffic from a other new server..

I'm running out of ideas what to check without some 'hands on' checking.. But i think we are active in different time-zones.. 'CET-evenings'..

strongthany

@PiBa said in Unable to add PPA to Ubuntu server when connected to HAProxy.:

I'm running out of ideas what to check without some 'hands on' checking.. But i think we are active in different time-zones.. 'CET-evenings'..

That's okay, I understand. I think I'm going to make a new question in a different topic that might have more folks who can help. I really appreciate you helping me out here.