Varnish 3 can't get it to work without NAT
-
I accomplished what I want, WAN-IP => muliple subsites. It works by trial and error…
But if I read - especially MARCELLOC's - posts in the forum all should function WITHOUT making use of NAT. Due to security.
https://forum.pfsense.org/index.php?topic=47962.0
https://forum.pfsense.org/index.php?topic=60297.15My configuration now:
Pfsense 2.1.4 amd64 - Intel i3 530 - 12Gb RAM - 4 NIC's
WAN: PPPoE with subnet /29 (Have set Virtual IP's)
Packages: Snort and PfBlocker (switched off during testing)
Varnish widget: backends are "green"NAT_rules:
WAN_IP 80 = DMZ-B_IP 80
WAN_Virtual_IP 80 = DMZ-A_IP 80FW_rules:
- * DMZ-B_IP 80
- * DMZ-A_IP 80
- * WAN_IP 80
- * WAN_Virtual_IP 80
We are planning to downgrade in 2 weeks from now to single WAN IP instead of WAN with subnet IP's.
Does the fact that I use Virtual IP's make a difference?
In that case I might have to test it again with the new situation with single WAN IP.
But I don't want downtime, thats why I start testing it in old situation.Something else I've tried also with Squid reverse:
- listening on loopback
- added virt_ip's to virt_IP/CARP field
- FW_rule to ALL => WAN_address 80 or 127.0.0.1 80 (Think I used both)
- NAT to 127.0.0.1 80.
The problem this gives that I can't use FW filtering IP rules for one of the sites.
If I can get it to work with Squid-reverse or HAproxy is also fine. Maybe better because then I can use https for future site.
All suggestions are welcome.jmack
-
If i understand correctly, in the end you will only have 1 public ip. I don't see how virtual ip's would make a difference there. And you want to host multiple websites on different (sub-)domains.
That means you must allow incoming traffic on port 80 to that wan-ip. And you cannot have different firewall rules for different domains on that side.
Haven't used varnish myself, but can recommend haproxy(1.5) for 'routing' traffic from 1 public ip to multiple web servers.
-
Mmm, new situation will have 2 sites to be proxied:
Site A - public www - 80
Site B - extranet - 80 (filtered on IP-ranges)When all traffic goes to WAN 80, where to split the filtering on source_ip?
I guess this should be done then on proxy-level somehow? Is this possible with HAproxy? -
haproxy itself can accept/block traffic from a list of network subnets..
configuration would be something like this:tcp-request connection accept if { src -f /etc/haproxy/whitelist.lst }
or
acl goodguys src 10.0.0.0/24 acl badguys src 10.0.1.0/24 tcp-request content accept if goodguys tcp-request content reject if badguys WAIT_END tcp-request content reject
You can configure such settings in the 'advanced' textbox in the webgui. However the whitelist.lst file would need to created manually..
See http://cbonte.github.io/haproxy-dconv/configuration-1.5.html for the complete manual of options haproxy itself offers. (not everything is possible with the pfsense package webgui build in options..)
-
What I want to accomplish in HAproxy is following:
IF host matches www.mysite.com ==> tcp-request connection accept ALL IF host matches extranet.mysite.com ==> tcp-request connection accept if { src -f /etc/haproxy/whitelist.lst } ELSE deny access
Om monday they are about to migrate my connection to single IP…
(Above "language" probably maybe doesn make sense, hopefully it explains what I want...) -
here my new config, it does NOT block the IP's which are NOT whitelisted yet…:
global maxconn 1000 stats socket /tmp/haproxy.socket level admin gid 80 nbproc 1 chroot /tmp/haproxy_chroot daemon frontend mhi.nl-merged bind 37.111.111.111:80 mode http log global option http-keep-alive option forwardfor reqadd X-Forwarded-Proto:\ http maxconn 10000 timeout client 30000 acl 0_www.site.nl hdr(host) -i www.site.nl acl 1_extranet.site.nl hdr(host) -i extranet.site.nl acl 2_whitelist { src -f /var/etc/haproxy_whitelist.lst } use_backend www.site.nl_http if 0_www.site.nl use_backend extranet.site.nl_http if 2_whitelist 1_extranet.site.nl backend www.site.nl_http mode http balance roundrobin timeout connect 30000 timeout server 30000 retries 3 option httpchk OPTIONS / server www.site.nl 10.11.12.13:80 check inter 1000 weight 100 backend extranet.site.nl_http mode http balance roundrobin timeout connect 30000 timeout server 30000 retries 3 source 0.0.0.0 usesrc clientip option httpchk OPTIONS / server extranet.site.nl 10.14.15.16:80 check inter 1000 weight 100
-
the acl is invalid and haproxy should trow and fatal alert for that remove the accolades:
acl 2_whitelist src -f /var/etc/haproxy_whitelist.lstonly when writing the acl directly behind a 'if' it needs to be between { } characters.
-
It all works now, here is my configfile ==>
ALL what I needed could be done from the pfsense web-config-page, I did not read this sentence properly, thats why I started to edit cfg manually….:
"acl's with the same name wil be 'combined', acl's with different names will be evaluated seperately."
So do as written, and the particular backend uses BOTH acls combined as needed... ;-)Also, for other users, try to stick as long as possible with the web-config-page.
I was lost for at least an 2 hours, to be able to restart HAproxy properly... and maintain the right config file?!?
(it was when I began in /var/etc/haproxy.cfg, later in /var/etc/haproxy/haproxy.cfg of some reason)The custom error pages can be pasted in Pfsense config-page / Frontend / Advanced pass thru
PfSense 2.1.5, with haproxy-devel 1.5.3 pkg v 0.10
global maxconn 1000 stats socket /tmp/haproxy.socket level admin gid 80 nbproc 1 chroot /tmp/haproxy_chroot daemon listen HAProxyLocalStats bind 127.0.0.1:2200 mode http stats enable stats refresh 10 stats admin if TRUE stats uri /haproxy_stats.php?haproxystats=1 timeout client 5000 timeout connect 5000 timeout server 5000 frontend site.com-merged bind 80.90.100.110:80 errorfile 400 /home/jmack/haproxy400error.http errorfile 403 /home/jmack/haproxy403error.http errorfile 408 /home/jmack/haproxy408error.http errorfile 500 /home/jmack/haproxy500error.http errorfile 502 /home/jmack/haproxy502error.http errorfile 503 /home/jmack/haproxy503error.http errorfile 504 /home/jmack/haproxy504error.http mode http log global option log-separate-errors option httplog option http-keep-alive option forwardfor reqadd X-Forwarded-Proto:\ http maxconn 10000 timeout client 30000 acl 0_www.site.com hdr(host) -i www.site.com use_backend www.site.com_http if 0_www.site.com acl 1_extranet.site.com hdr(host) -i extranet.site.com acl 2_extranet.site.com src -f /home/jmack/haproxy_whitelist.1st use_backend extranet.site.com_http if 1_extranet.site.com 2_extranet.site.com backend www.site.com_http mode http balance roundrobin timeout connect 30000 timeout server 30000 retries 3 option httpchk OPTIONS / server www.site.com 10.22.22.22:80 check inter 1000 weight 100 backend extranet.site.com_http mode http balance roundrobin timeout connect 30000 timeout server 30000 retries 3 source 0.0.0.0 usesrc clientip option httpchk OPTIONS / server extranet.site.com 10.11.12.13:80 check inter 1000 weight 100
-
2 questions left:
A.)
I cannot put custom error files and whitelist file in /var/etc/ they get erased after restart.
When make readonly, they get deleted after server reboot.
I added files to /home/[user] is this place OK?B.)
How to add to the whitelist-file CIDR subnet IP's?
It seems only possible to add single IP's…. -
A) indeed /var and /tmp are deleted on every pfsense reboot
as for the proper location im not sure what would be..B) cidr ranges should be possible (seems to work for me), ill add some basic support for ip aliases in the webgui.. (changes only loaded when haproxy config is generated again..)
-
B) cidr ranges should be possible (seems to work for me),
To confirm:
I tested my whitelist "cidr-only", so when single IP's; they were /32 etc.
Haproxy refuses to start…To be able to use PFsense's aliasses would be a welcome function!
-
reinstall the package, and give the 'ip matches ip or alias' acl a try, you can just put the name of a ip-alias there and it should work..
remember the list is only reloaded when config is written again while starting haproxy.. changing an alias does not currently trigger that.