Pfsense HAPROXY Loadbalancer

From RARForge
Revision as of 23:25, 19 April 2013 by Robertr (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search
  • Brain Dump - not perfect or document tested yet (it has been tested and works, just not from this doc)
  • After a FW reboot, stunnel must be restarted to work poperly. need to debug


scrapped this and with with mod_balancer - which I might end up scrapping and going with nginx

Install


Install if
http://www.pfsense.org/
  • You can find vmware images and other.
  • LiveCD can be used a bare-metal install too

add route after install to hit web interface <source> route add defaults <gateway ip address> </source>


Failover

requirements - create a interface called SYNC. Allow SYNC subnet through the firewall rules. I may document this later...

Details

  • Server 1 SYNC IP: 10.155.0.1
  • Server 1 SYNC IP: 10.155.0.1

Setup

  • Firewall -> Virtual IPs -> CARP settings
PRIMARY ONLY!
  1. Check Synchronize States (both primary and secondary)
  2. Synchronize Interface: SYNC
  3. Synchronize Config to IP: 10.155.0.2
  4. remote username: same as admin
  5. remote password: sames as admin
  6. check what you want synced (I check most)
  • Other services like HAproxy, have sync settings in their OWN tab. So far it's the one I have seen/used.



Load Balance

Load Balancer

  • I have not used the BUILTIN load balancing. It looks like it's only useful for WAN -> LAN (internal servers .. behind firewall)
  • HAproxy examples are for WAN -> External Website (PROXY!)



HAproxy Loadbalancer +SSL


This does not do SSL offloading. All your webservers must have SSL working.


I am using this IP as an example, please use your own

  • External Web IP: 8.8.8.8
  • Load Balanced IP 1: 4.4.4.4
  • Load Balanced IP 2: 5.5.5.5

This will LOAD balanced requests

  • 8.8.8.8:80 (HAproxy) -> External: [ 4.4.4.4:80 , 5.5.5.5:80 ]
  • 8.8.8.8:443 (HAproxy) -> Extneral: [ 4.4.4.4:443 , 5.5.5.5:443 ]


Details


HTTP

  1. HAproxy is listens on 10.200.200.200:80 and will Loadbalance the connection between 4.4.4.4:80 and 5.5.5.5:80

HTTPS all real web servers handle SSL

  1. HAproxy is listens on 10.200.200.200:443 and will Loadbalance the connection between 4.4.4.4:443 and 5.5.5.5:443
  • Make sure to use SOURCE load balancing for sticky sessions
  • DO NOT enable any cookies - only HTTP mode

Virtual IP Addresses (VIPS)


  1. Firewall -> Virtual IPs
  2. Add 8.8.8.8
    1. IP Alias (or carp if failover is enable)
    2. Interface WAN
    3. Type: Network
    4. Address: 8.8.8.8/32
    5. Description: External LB

HAproxy


  1. System -> packages -> Available Packages: HAproxy
  2. Services -> HAproxy
Settings
  1. Check Enable HAproxy
  2. Maximum Connections: 1000 (or whatever you want)
  3. optional: Enable Sync if you have CARP enabled (only on primary)
    1. Password same as Admin Interface
    2. Host #1: IP address of Failover
Frontends
  • This is for the normal HTTP
  1. Name: WEB_8.8.8.8_80
  2. Description: Public HTTP to 8.8.8.8:80
  3. Type: HTTP
  4. Balance: Least Connections
  5. Port: 80
  6. External IP: 8.8.8.8 (External LB)
  7. Check use forwardfor
  8. check httpclose
  9. Advanced Pass thru: cookie SERVERID insert nocache indirect
  • This is for the HTTPS
  1. Name: SSL_8.8.8.8_443
  2. Description: Public SSL to 8.8.8.8:443
  3. Type: HTTPS
  4. Balance: SOURCE -- must be used for sticky sessions
  5. Port: 443
  6. External IP: 8.8.8.8 (Exernal LB)
  7. Check use forwardfor
  8. check httpclose
  9. Advanced Pass thru: cookie SERVERID insert nocache indirect
Servers
  • add real server 4.4.4.4:80
  1. Name: 4.4.4.4-80
  2. Frontends: Add HTTP frontend we just created ( WEB_8.8.8.8_80 )
  3. IP Address: 4.4.4.4
  4. Cookie: server_4.4.4.4_80
  5. Weight: 1
  • add real server 5.5.5.5:80
  1. Name: 5.5.5.5-80
  2. Frontends: Add HTTP frontend we just created ( WEB_8.8.8.8_80 )
  3. IP Address: 5.5.5.5
  4. Cookie: server_5.5.5.5_443
  5. Weight: 1
  • add real server 4.4.4.4
  1. Name: 4.4.4.4-443
  2. Frontends: Add HTTPS frontend we just created ( WEB_8.8.8.8_443 )
  3. IP Address: 4.4.4.4
  4. Cookie: server_4.4.4.4_443
  5. Weight: 1
  • add real server 5.5.5.5
  1. Name: 5.5.5.5-443
  2. Frontends: Add HTTPS frontend we just created ( WEB_8.8.8.8_443 )
  3. IP Address: 5.5.5.5
  4. Cookie: server_5.5.5.5_443
  5. Weight: 1

STunnel+HAproxy Loadbalancer


I am using this IP as an example, please use your own

  • External Web IP: 8.8.8.8
  • Load Balanced IP 1: 4.4.4.4
  • Load Balanced IP 2: 5.5.5.5

This will LOAD balanced requests

  • 8.8.8.8:80 (HAproxy) -> External:[ 4.4.4.4:80 , 5.5.5.5:80 ]
  • 8.8.8.8:443 (STunnel) -> (HAproxy) -> Extneral: [ 4.4.4.4:80 , 5.5.5.5:80 ]


Details


HTTP

  1. HAproxy is listens on 10.200.200.200:80 and will Loadbalance the connection between 4.4.4.4 and 5.5.5.5


HTTPS - only if you want to utilize SSL offloading.

  1. STunnel with offload the SSL connections. This means you will be adding the SSL certificate to the firewall (STunnel service) which will handle all SSL requests for 8.8.8.8:443.
  2. Stunnel will then pass the connections to 10.200.200.200:80 ( an local IP alias ).
  3. HAproxy is listens on 10.200.200.200:80 and will Loadbalance the connection between 4.4.4.4 and 5.5.5.5

I am using 10.200.200.200 so I don't have to burn another real IP. I was hoping to just use 8.8.8.8:80 configured for HAproxy, however I use CARP ip for failover. Currently you cannot redirect STunnel to a CARP IP

Virtual IP Addresses (VIPS)


  1. Firewall -> Virtual IPs
  2. Add 10.200.200.200
    1. IP Alias (NO CARP)
    2. Interface WAN
    3. Type: Network
    4. Address: 10.200.200.200/32
    5. Description: LB test
  3. Add 8.8.8.8
    1. IP Alias (or carp if failover is enable)
    2. Interface WAN
    3. Type: Network
    4. Address: 8.8.8.8/32
    5. Description: External LB


STunnel


  • Offloading SSL
  1. System -> packages -> Available Packages: STunnel
  2. Services -> STunnel
Certificates
  • add new item
  1. Add New cert(s). Requires private key and certificate (chain)
Tunnels
  • add new item
  1. Listen on IP: 8.8.8.8 - External IP address to listen on
  2. Listen on Port: 443 - self explanatory
  3. certificiate: choose your cert (drop down)
  4. Redirects to IP: 10.200.200.200 - This can be an internal IP or Public IP you want to encrypt
  5. Redirects to Port: 80 - self explanatory
  6. Outgoint Source: unused for now

HAproxy


  1. System -> packages -> Available Packages: HAproxy
  2. Services -> HAproxy
Settings
  1. Check Enable HAproxy
  2. Maximum Connections: 1000 (or whatever you want)
  3. optional: Enable Sync if you have CARP enabled (only on primary)
    1. Password same as Admin Interface
    2. Host #1: IP address of Failover
Frontends
  • This is for the normal HTTP
  1. Name: WEB_8.8.8.8_80
  2. Description: Public HTTP to 8.8.8.8:80
  3. Type: HTTP
  4. Balance: Least Connections
  5. Port: 80
  6. External IP: 8.8.8.8 (External LB)
  7. Check use forwardfor
  8. check httpclose
  9. Advanced Pass thru: cookie SERVERID insert nocache indirect
  • This is for the SSL+STunnel site
  1. Name: SSL_10.200.200.200_80
  2. Description: Public SSL to 10.200.200.200:80
  3. Type: HTTP
  4. Balance: Least Connections
  5. Port: 80
  6. External IP: 10.200.200.200 (LB test)
  7. Check use forwardfor
  8. check httpclose
  9. Advanced Pass thru: cookie SERVERID insert nocache indirect
Servers

Same backedn server will be used for both HTTP and HTTPS connections

  • add real server 4.4.4.4
  1. Name: 4.4.4.4-80
  2. Frontends: Add BOTH frontends we just added ( WEB_8.8.8.8_80 and SSL_10.200.200.200_80 )
  3. IP Address: 4.4.4.4
  4. Cookie: server_4.4.4.4_80
  5. Weight: 1
  • add real server 5.5.5.5
  1. Name: 5.5.5.5-80
  2. Frontends: Add BOTH frontends we just added ( WEB_8.8.8.8_80 and SSL_10.200.200.200_80 )
  3. IP Address: 5.5.5.5
  4. Cookie: server_5.5.5.5_80
  5. Weight: 1