Pfsense HAPROXY Loadbalancer
- Brain Dump - not perfect or document tested yet (it has been tested and works, just not from this doc)
- After a FW reboot, stunnel must be restarted to work poperly. need to debug
- scrapped this and with with mod_balancer - which I might end up scrapping and going with nginx
Install
- Install if
- http://www.pfsense.org/
- You can find vmware images and other.
- LiveCD can be used a bare-metal install too
add route after install to hit web interface <source> route add defaults <gateway ip address> </source>
Failover
requirements - create a interface called SYNC. Allow SYNC subnet through the firewall rules. I may document this later...
Details
- Server 1 SYNC IP: 10.155.0.1
- Server 1 SYNC IP: 10.155.0.1
Setup
- Firewall -> Virtual IPs -> CARP settings
- PRIMARY ONLY!
- Check Synchronize States (both primary and secondary)
- Synchronize Interface: SYNC
- Synchronize Config to IP: 10.155.0.2
- remote username: same as admin
- remote password: sames as admin
- check what you want synced (I check most)
- Other services like HAproxy, have sync settings in their OWN tab. So far it's the one I have seen/used.
Load Balance
Load Balancer
- I have not used the BUILTIN load balancing. It looks like it's only useful for WAN -> LAN (internal servers .. behind firewall)
- HAproxy examples are for WAN -> External Website (PROXY!)
HAproxy Loadbalancer +SSL
This does not do SSL offloading. All your webservers must have SSL working.
I am using this IP as an example, please use your own
- External Web IP: 8.8.8.8
- Load Balanced IP 1: 4.4.4.4
- Load Balanced IP 2: 5.5.5.5
This will LOAD balanced requests
- 8.8.8.8:80 (HAproxy) -> External: [ 4.4.4.4:80 , 5.5.5.5:80 ]
- 8.8.8.8:443 (HAproxy) -> Extneral: [ 4.4.4.4:443 , 5.5.5.5:443 ]
Details
HTTP
- HAproxy is listens on 10.200.200.200:80 and will Loadbalance the connection between 4.4.4.4:80 and 5.5.5.5:80
HTTPS all real web servers handle SSL
- HAproxy is listens on 10.200.200.200:443 and will Loadbalance the connection between 4.4.4.4:443 and 5.5.5.5:443
- Make sure to use SOURCE load balancing for sticky sessions
- DO NOT enable any cookies - only HTTP mode
Virtual IP Addresses (VIPS)
- Firewall -> Virtual IPs
- Add 8.8.8.8
- IP Alias (or carp if failover is enable)
- Interface WAN
- Type: Network
- Address: 8.8.8.8/32
- Description: External LB
HAproxy
- System -> packages -> Available Packages: HAproxy
- Services -> HAproxy
Settings
- Check Enable HAproxy
- Maximum Connections: 1000 (or whatever you want)
- optional: Enable Sync if you have CARP enabled (only on primary)
- Password same as Admin Interface
- Host #1: IP address of Failover
Frontends
- This is for the normal HTTP
- Name: WEB_8.8.8.8_80
- Description: Public HTTP to 8.8.8.8:80
- Type: HTTP
- Balance: Least Connections
- Port: 80
- External IP: 8.8.8.8 (External LB)
- Check use forwardfor
- check httpclose
- Advanced Pass thru: cookie SERVERID insert nocache indirect
- This is for the HTTPS
- Name: SSL_8.8.8.8_443
- Description: Public SSL to 8.8.8.8:443
- Type: HTTPS
- Balance: SOURCE -- must be used for sticky sessions
- Port: 443
- External IP: 8.8.8.8 (Exernal LB)
- Check use forwardfor
- check httpclose
- Advanced Pass thru: cookie SERVERID insert nocache indirect
Servers
- add real server 4.4.4.4:80
- Name: 4.4.4.4-80
- Frontends: Add HTTP frontend we just created ( WEB_8.8.8.8_80 )
- IP Address: 4.4.4.4
- Cookie: server_4.4.4.4_80
- Weight: 1
- add real server 5.5.5.5:80
- Name: 5.5.5.5-80
- Frontends: Add HTTP frontend we just created ( WEB_8.8.8.8_80 )
- IP Address: 5.5.5.5
- Cookie: server_5.5.5.5_443
- Weight: 1
- add real server 4.4.4.4
- Name: 4.4.4.4-443
- Frontends: Add HTTPS frontend we just created ( WEB_8.8.8.8_443 )
- IP Address: 4.4.4.4
- Cookie: server_4.4.4.4_443
- Weight: 1
- add real server 5.5.5.5
- Name: 5.5.5.5-443
- Frontends: Add HTTPS frontend we just created ( WEB_8.8.8.8_443 )
- IP Address: 5.5.5.5
- Cookie: server_5.5.5.5_443
- Weight: 1
STunnel+HAproxy Loadbalancer
I am using this IP as an example, please use your own
- External Web IP: 8.8.8.8
- Load Balanced IP 1: 4.4.4.4
- Load Balanced IP 2: 5.5.5.5
This will LOAD balanced requests
- 8.8.8.8:80 (HAproxy) -> External:[ 4.4.4.4:80 , 5.5.5.5:80 ]
- 8.8.8.8:443 (STunnel) -> (HAproxy) -> Extneral: [ 4.4.4.4:80 , 5.5.5.5:80 ]
Details
HTTP
- HAproxy is listens on 10.200.200.200:80 and will Loadbalance the connection between 4.4.4.4 and 5.5.5.5
HTTPS - only if you want to utilize SSL offloading.
- STunnel with offload the SSL connections. This means you will be adding the SSL certificate to the firewall (STunnel service) which will handle all SSL requests for 8.8.8.8:443.
- Stunnel will then pass the connections to 10.200.200.200:80 ( an local IP alias ).
- HAproxy is listens on 10.200.200.200:80 and will Loadbalance the connection between 4.4.4.4 and 5.5.5.5
I am using 10.200.200.200 so I don't have to burn another real IP. I was hoping to just use 8.8.8.8:80 configured for HAproxy, however I use CARP ip for failover. Currently you cannot redirect STunnel to a CARP IP
Virtual IP Addresses (VIPS)
- Firewall -> Virtual IPs
- Add 10.200.200.200
- IP Alias (NO CARP)
- Interface WAN
- Type: Network
- Address: 10.200.200.200/32
- Description: LB test
- Add 8.8.8.8
- IP Alias (or carp if failover is enable)
- Interface WAN
- Type: Network
- Address: 8.8.8.8/32
- Description: External LB
STunnel
- Offloading SSL
- System -> packages -> Available Packages: STunnel
- Services -> STunnel
Certificates
- add new item
- Add New cert(s). Requires private key and certificate (chain)
Tunnels
- add new item
- Listen on IP: 8.8.8.8 - External IP address to listen on
- Listen on Port: 443 - self explanatory
- certificiate: choose your cert (drop down)
- Redirects to IP: 10.200.200.200 - This can be an internal IP or Public IP you want to encrypt
- Redirects to Port: 80 - self explanatory
- Outgoint Source: unused for now
HAproxy
- System -> packages -> Available Packages: HAproxy
- Services -> HAproxy
Settings
- Check Enable HAproxy
- Maximum Connections: 1000 (or whatever you want)
- optional: Enable Sync if you have CARP enabled (only on primary)
- Password same as Admin Interface
- Host #1: IP address of Failover
Frontends
- This is for the normal HTTP
- Name: WEB_8.8.8.8_80
- Description: Public HTTP to 8.8.8.8:80
- Type: HTTP
- Balance: Least Connections
- Port: 80
- External IP: 8.8.8.8 (External LB)
- Check use forwardfor
- check httpclose
- Advanced Pass thru: cookie SERVERID insert nocache indirect
- This is for the SSL+STunnel site
- Name: SSL_10.200.200.200_80
- Description: Public SSL to 10.200.200.200:80
- Type: HTTP
- Balance: Least Connections
- Port: 80
- External IP: 10.200.200.200 (LB test)
- Check use forwardfor
- check httpclose
- Advanced Pass thru: cookie SERVERID insert nocache indirect
Servers
Same backedn server will be used for both HTTP and HTTPS connections
- add real server 4.4.4.4
- Name: 4.4.4.4-80
- Frontends: Add BOTH frontends we just added ( WEB_8.8.8.8_80 and SSL_10.200.200.200_80 )
- IP Address: 4.4.4.4
- Cookie: server_4.4.4.4_80
- Weight: 1
- add real server 5.5.5.5
- Name: 5.5.5.5-80
- Frontends: Add BOTH frontends we just added ( WEB_8.8.8.8_80 and SSL_10.200.200.200_80 )
- IP Address: 5.5.5.5
- Cookie: server_5.5.5.5_80
- Weight: 1