This is a text-only version of the following page on https://raymii.org: --- Title : Adding IPv6 to a keepalived and haproxy cluster Author : Remy van Elst Date : 24-09-2017 URL : https://raymii.org/s/articles/Adding_IPv6_to_a_keepalived_and_haproxy_cluster.html Format : Markdown/HTML --- At work I regularly build high-available clusters for customers, where the setup is distributed over multiple datacenters with failover software. If one component fails, the service doesn't experience issues or downtime due to the failure. Recently I was tasked with expanding a cluster setup to be also reachable via IPv6. This article goes over the settings and configuration required for haproxy and keepalived for IPv6. The internal cluster will only be IPv4, the loadbalancer terminates HTTP and HTTPS connections.

Recently I removed all Google Ads from this site due to their invasive tracking, as well as Google Analytics. Please, if you found this content useful, consider a small donation using any of the options below:

I'm developing an open source monitoring app called Leaf Node Monitoring, for windows, linux & android. Go check it out!

Consider sponsoring me on Github. It means the world to me if you show your appreciation and you'll help pay the server costs.

You can also sponsor me by getting a Digital Ocean VPS. With this referral link you'll get $200 credit for 60 days. Spend $25 after your credit expires and I'll get $25!

### Cluster setup ![cluster][2] This diagram gives a general idea of the clusters I often build at work. The CloudVPS network is fully redundant over multiple data centers, so I don't have to worry about that part. A setup often consist out of the following components per data center: a loadbalancer, multiple application servers (php, apache, rails, python, java), a database server (mysql/galera or postgresql). So you have three loadbalancer, three databaseservers and three or more application servers in total. Often there are extra components like DRBD/NFS for file storage, Redis as a key/value store, mongodb or elasticsearch. (All which can be clustered). Because we have three datacenters there is enough for a quorum. Sometimes customers choose just two datacenters for cost reasons, then we explain the issues without quorum and make them sign off the risks in the contract. The clusters are IPv4 internally, with keepalived or Corosync/Pacemaker handling the High Available IP addres (VIP). The loadbalancers all have their own IP and share one or more external VIP's via the cluster software. They also have a internal VIP because they function as a gateway for the internal servers. If one loadbalancer fails, VRRP detects that and the VIP becomes active on one of the other servers. For complex setups with depencencies and orders we use Corosync, for example with DRBD/NFS, so make sure the starting order is correct. First DRBD mount, then the VIP, then NFS. Most of the time keepalived is enough. Adding IPv6 is suprisingly easy, so this is a short article covering the following: * IPv6 on the OS * keepalived * haproxy The internal network stays the same, the load balancer terminates all traffic and sends it on over IPv4 to the application servers, which do not need to be configured with IPv6. In our case all serves come with a /64 IPv6 natively so there is no network configuration on switches or routers included in this guide. ### Operating System The clusters run Ubuntu (16.04) most of the time, so in `/etc/network/interfaces` there must be an IPv6 address: iface eth0 inet6 static address 2a02:123:45:67ab::1/48 netmask 48 gateway 2a02:123:45::1 This is not the IPv6 address you'll use as the VIP, but a local IPv6 address for the machine. You don't configure the VIP on the OS. We have ACL rules on the backend in our hypervisor environment so I added an extra IPv6 range to the cluster for use with high-availability: 2a02:123:45:67bb::1/48 (example range in this case, which will be used inside haproxy and keepalived as IPv6 VIP.) You also don't need to configure the following sysctl for ipv6: net.ipv4.ip_nonlocal_bind=1 We handle that inside of haproxy and keepalived. ### Keepalived This is tested with keepalived in ubuntu 16.04, version 1.2.19. Adding the IPv6 address to the `virtual_ipaddress` section and restarting keepalived is enough: vrrp_sync_group VG_1 { group { EXTERN INTERN } } vrrp_instance EXTERN { interface eth0 virtual_router_id 12 state EQUAL advert_int 1 smtp_alert notify /usr/local/bin/keepalived-extern.sh authentication { auth_type PASS auth_pass hunter2 } virtual_ipaddress { 1.2.3.4/32 2a02:123:45:67bb::1/32 } } ### haproxy haproxy is suprisingly easy with IPv6. Just add it to your `frontend` section as a `bind` option: frontend http-in mode http bind 1.2.3.4:80 bind 2a02:123:45:67bb::1:80 transparent option httplog option forwardfor option http-server-close option httpclose reqadd X-Forwarded-Proto:\ http http-request add-header X-Real-IP %[src] default_backend appserver You must add the `transparant` option. Otherwise, haproxy will not start if the VIP is not on the machine itself. (kind of like nonlocal.bind sysctl). haproxy is intelligent enough to understand the port number in the address. No need to screw around with brackets like `[2a02:123:45:67bb::1]:80` or special options. Restart haproxy and it's configured: netstat -tlpn | grep haproxy Output: tcp 0 0 1.2.3.4:80 0.0.0.0:* LISTEN 1163/haproxy tcp 0 0 1.2.3.4:443 0.0.0.0:* LISTEN 1163/haproxy tcp6 0 0 2a02:123:45:67bb::1:80 :::* LISTEN 1163/haproxy tcp6 0 0 2a02:124:45:67bb::1:443 :::* LISTEN 1163/haproxy The version of haproxy is the one from Ubuntu 16.04, 1.6.3. [1]: https://www.digitalocean.com/?refcode=7435ae6b8212 [2]: https://raymii.org/s/inc/img/cloudvps-cluster.png --- License: All the text on this website is free as in freedom unless stated otherwise. This means you can use it in any way you want, you can copy it, change it the way you like and republish it, as long as you release the (modified) content under the same license to give others the same freedoms you've got and place my name and a link to this site with the article as source. This site uses Google Analytics for statistics and Google Adwords for advertisements. You are tracked and Google knows everything about you. Use an adblocker like ublock-origin if you don't want it. All the code on this website is licensed under the GNU GPL v3 license unless already licensed under a license which does not allows this form of licensing or if another license is stated on that page / in that software: This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see . Just to be clear, the information on this website is for meant for educational purposes and you use it at your own risk. I do not take responsibility if you screw something up. Use common sense, do not 'rm -rf /' as root for example. If you have any questions then do not hesitate to contact me. See https://raymii.org/s/static/About.html for details.