GSLB and HA
kirbini
Member ✭
Posted by kirbini
Hi, All.
I have 2 working HA pairs (active/standby) of AX1000. Each pair is in a different datacenter and we would like to add GSLB between datacenters. A couple of questions come up and I was wondering if the voice of experience may be lurking about.
First, are there any gotchas to watch out for when using GSLB with two HA pairs? Is there a methodology I should use to chose the which unit GSLB controller? What happens if the site that I choose to be the GSLB controller goes offline; will the other site be able to take over the controller roll? Similarly, if the controller fails over to it's HA stadby, is the controller role moved to the now active box? Does it matter if the GSLB controller is behind a firewall and does not have access to the public IPs (that are mapped to the VIPs) in order to perform health checks? Also, since there are 4 units (2 HA pairs) in the GSLB grid, do I need to open the GSLB firewall port for all 4 units or just for one floating IP for each pair?
EDIT: Sorry for the double post. I swear I only hit submit once.
thx
Hi, All.
I have 2 working HA pairs (active/standby) of AX1000. Each pair is in a different datacenter and we would like to add GSLB between datacenters. A couple of questions come up and I was wondering if the voice of experience may be lurking about.
First, are there any gotchas to watch out for when using GSLB with two HA pairs? Is there a methodology I should use to chose the which unit GSLB controller? What happens if the site that I choose to be the GSLB controller goes offline; will the other site be able to take over the controller roll? Similarly, if the controller fails over to it's HA stadby, is the controller role moved to the now active box? Does it matter if the GSLB controller is behind a firewall and does not have access to the public IPs (that are mapped to the VIPs) in order to perform health checks? Also, since there are 4 units (2 HA pairs) in the GSLB grid, do I need to open the GSLB firewall port for all 4 units or just for one floating IP for each pair?
EDIT: Sorry for the double post. I swear I only hit submit once.
thx
0
Comments
Many questions here
Let's try to reply in the order:
Gotchas when using GSLB with two HA pairs?
No. That's actually a popular deployment. Doing so, you have actually HA on your GSLB service in each site.
Is there a methodology I should use to chose the which unit GSLB controller?
There are multiple ways to deploy your GSLB solution.
1. Your DSN can still be authoritative for AX your domain (example.com) and it forwards only specific DNS requests to your GSLB Controller for specific names (www.example.com for instance).
2. The AX is authoritative for your domain (example.com) and replies directly to specific requests (www.example.com for instance) but forwards the other requests to your DNS server.
The option1 is the most popular and I'll focus on that one now.
So what do you configure on your DNS server?
You delegate specific names (www) to your GSLB-Controller-Site1 + GSLB-Controller-Site2 so if one site goes down, clients will still be able to get a DNS resolution for www.example.com via the GSLB-Controller-Site2.
What happens if the site that I choose to be the GSLB controller goes offline; will the other site be able to take over the controller roll?
Since you have both GSLB-Controllers defined as delegate servers in your authoritative DNS server, if the site of GSLB-Controller1 goes down (so GSLB-Controller1 is no more reachable) the DNS will automatically request the GSLB-Controller2.
Similarly, if the controller fails over to it's HA stadby, is the controller role moved to the now active box?
If the whole site1 doesn't go down but only the active AX1-Site1 goes down, then the AX2-Site1 will become active => be the GSLB-Controller => it will automatically receive the DNS requests and reply to them.
Does it matter if the GSLB controller is behind a firewall and does not have access to the public IPs (that are mapped to the VIPs) in order to perform health checks?
If the GSLB-Controller can't access the "public IP@ used for the DNS resolution" but can access only the "private IP@", then you have to tell it to the GSLB-Controller. This is done under "Config >> Service >> GSLB >> Service IP" and the two options "IP Address" (private) and "External IP Address" (public). The private IP@ is the one tested. Then when the GSLB Controller decides to use that Service IP, then the external IP@ is sent in the DNS response.
Also, since there are 4 units (2 HA pairs) in the GSLB grid, do I need to open the GSLB firewall port for all 4 units or just for one floating IP for each pair?
Only the Active GSLB Controller in each location will receive DNS requests (on the GSLB Controller VIP@). But the Service IP healthchecks will be processed by both the Active and Standby AX (with their real IP@).
Lots of great information. I feel more comfortable about my deployment, thanks.
You say above that using the AX as the authoritative DNS for a delegated sub-domain is the most popular option (also confirmed by my DNS vendor) but I can't find any documentation or examples of how to set up that scenario. The manual gives a good example of setting up proxy DNS but nothing about setting up authoritative DNS other than to say that it is an option. Do you have any configuration examples of setting up GLSB with authoritative DNS in an HA configuration? (the manual also seems to be contradictory in a few places so I'm rather confused at the moment).
Thanks.
Here are examples for case 1: "AX as the authoritative DNS for a delegated sub-domain".
Authoritative DNS configuration (see bind_example_sub_domain_delegation.txt ).
AX configuration (see Example_Bryan.txt)
Now what I would suggest since that's the first time you configure GSLB:
. Configure 1 VIP on each site with dummy servers for "test-for-me.mycompany.com"
. Configure your AX GSLB Controllers in each site for "test-for-me.mycompany.com"
. Ask your DNS provider to delegate the sub-domain "test-for-me.mycompany.com" to your AX GSLB Controllers in each site
. Test it
Dimitri
Excellent. It all makes (almost) sense now. So much clearer, thanks for the documents. I have two remaining questions and one suggestion.
Q1: Does the DNS VIP (gslb_dns in the example) need to be in the same subnet as the site VIP (meaning I need to create a DNS VIP for each subnet) or can I use one DNS VIP for all sites and subnets that the A10 handles?
Q2: If I can use one VIP for DNS queries, can I also use that same VIP/IP as the port 4149 listener for GSLB communications or do I need to set up another IP (and hence need two public IPs at the upstream firewall; one for DNS and one for GSLB)?
Suggestion: On the DNS delegation, it is very important that the documentation always use the term "delegate" and not "forward" as in "the authoritative then forwards the query to the AX DNS". That should read "delegate" I know it's symantic but in the DNS world there is a very distinct difference. One that caught me off guard initially.
Q1: One DNS VIP to rule them all, one DNS VIP to bind them (pun intended). I put my DNS VIP into a different subnet and it happily answers with VIPs for any given subnet, nice one that.
Q2: The jury's still out.
New Q3: What's the purpose of a service port in the Zone configuration? Since DNS is purely layer-3 it has no concept of a service port, correct? But you must choose HTTP/HTTPS/FTP, etc and cant' assign the "www" to more than one. So if I set up my zone like this:
gslb zone mycompany.com
service http www
dns-a-record Site-VIP static
dns-a-record Remote-VIP static
When the virtual-service is bound to HTTP and HTTPS, what happens when a request comes in for
https://www.mycompany.comWill DNS still resolve properly?
Ok, I'm getting a little bit further. I've got both sites set up and answering DNS queries properly. However neither site can see the other via GSLB. I'm afraid it has something to do with the use of the management interface. Here's what I found:
I set up a virtual interface and set an IP in it's subnet it as the DNS virtual-server VIP so DNS queries will float among HA members during a failover. According to what you say in the second post of this thread, the IP used for GSLB communications will also float so my cross-site GSLB communications are always between active and active and that I only have to create one port 4149 rule through the firewall. Unfortunately this does not seem to be working out in practice.
While the VIP I created on the virtual interface will answer for DNS, it will not answer on port 4149. In reverse, my management port/IP will not answer for DNS but it will answer for port 4149/GSLB.
So, should I put my glsb_dns VIP on the management subnet? Is that even possible? Will the management interface answer for DNs queries? Can the documentation be updated with the fact that port 4149 only happens over the management interface?
thanks again
You do the questions and the answers now. Great
As you said:
Q1: Does the DNS VIP (gslb_dns in the example) need to be in the same subnet as the site VIP
The DNS VIP is the IP@ accessed by the DNS Clients with their DNS requests. It can be in any subnet.
Q2: If I can use one VIP for DNS queries, can I also use that same VIP/IP as the port 4149 listener for GSLB communications
Some theory first:
The GSLB controller is the AX that receives the DNS requests (CLI command: "gslb protocol enable controller").
The GSLB device is the AX that hosts VIPs used in DNS resolution done by the AX (CLI command: "gslb protocol enable device").
One AX can be simply controller (DNS-VIP) or simply device (Service-VIP) or both at the same time.
Now to reply to your question:
The GSLB device listens on TCP/UDP 4149 on all its AX data interfaces IP@ (not Management IP@).
The GSLB controller talks to the GSLB device IP@ you configure under "AX Site".
Q3: What's the purpose of a service port in the Zone configuration?
This I don't know either
I'm asking engineering (but the guy in charge of that specific piece is on vacation => I'll come back later to you on that one).
Dimitri, you've been a big help. I've got one last hurdle to get over and them I"m done.
DNS is working as expected. I created an SLB virtual-server, enabled port 53 and gslb and both sites properly answer DNS queries. The private VIP is mapped through the local site firewall to a public IP address. Our primary DNS servers properly delegate queries to that public address and the A10s answer. If I fail over the HA pair, the VIP moves to the secondary and DNS still works.
However, I'm still having problems with the port 4149/GSLB protocol. For each pair I have the DNS VIP mapped across the firewall. The firewall allows in port 53 for DNS and por 4149 for GSLB. The primary at each site is set to be the controller and a device. They are configured with 127.0.0.1 as the IP for the local site and as the public VIP as the other site. At each firewall I see local port 4149 requests going out and I see the remote port 4149 requests coming in. However, each site does not see the other when I go view GSLB status. One point: Although one site is sending port 4149 packets to the other site's VIP, it is sourcing those packets from its virtual ethernet interface IP address, not the IP address of the VIP. Does this mean I need to map a second public IP and port 4149 to the VE interface IP? If so, what happens if that HA pair fails over and that VE address does not float to the other box? Does that mean I need a 4-way mesh of public IPs and port 4149s to cover any and all HA failover situations?
It appears that 4149 packets coming into the (private) VIP address are not being accepted. Is there a way to get the AX to source 4149 packets from the VIP?
As you know now, the AX GSLB Controller talks to the AX GSLB Site via TCP/UDP 4149 to know its different statistics.
As said the AX GSLB Site can be reached on any of its data interfaces (not management).
The AX GSLB Controller contacts the AX GSLB Site via its data interface. Which one?
Look at AX GSLB Controller routing table (show ip route) and check what is the next hop to reach that AX GSLB Site. The AX GSLB Controller will use its IP@ in the same subnet as the next hop to reach the AX GSLB Site.
This behavior can't be changed.
So what do you need to open on your firewall?
You have to allow:
Note: I assume the route to reach the remote AX is via its ve1 interface.
For traffic from the "AX GSLB Controller" in the primary site to the "AX GSLB Site" in the secondary site:
. AX1-Primary-Site-ve1-IP@ => AX-Secondary-Site-Floating-IP@ (TCP/UDP 4149)
. AX2-Primary-Site-ve1-IP@ => AX-Secondary-Site-Floating-IP@ (TCP/UDP 4149)
For traffic from the "AX GSLB Controller" in the secondary site to the "AX GSLB Site" in the primary site:
. AX1-Secondary-Site-ve1-IP@ => AX-Primary-Site-Floating-IP@ (TCP/UDP 4149)
. AX2-Secondary-Site-ve1-IP@ => AX-Primary-Site-Floating-IP@ (TCP/UDP 4149)
Dimitri
Sorry Dimitri, that doesn't work. I see the port 4140/TCP SYN packet leave Site1-VE1-IP and arrive at Site2-Floating-IP. I never see a SYN/ACK packet returned from any interface on Site2. Likewise I see Site2-VE1-IP sending SYN packets to Site1-Floating-IP but Site1 never generates a return SYN/ACK packet so the TCP connections are never built.
Got any other ideas?
Here's the relevant config from Site2:
slb virtual-server gslb_dns 10.210.5.10
ha-group 1
port 53 udp
gslb-enable
source-nat pool GSLB_SNAT
use-rcv-hop-for-resp
gslb site DWG_int
slb-dev DWG_int 127.0.0.1
vip-server 60SecondPremier.com_DWG
vip-server PremierApplication.com_DWG
vip-server PremierCentral.com_DWG
!
gslb site DWS_ext
slb-dev DWS_ext 216.12.180.253
gateway 10.210.5.1
vip-server 60SecondPremier.com_DWS
vip-server PremierApplication.com_DWS
vip-server PremierCentral.com_DWS
gslb protocol enable device no-passive-rtt
gslb protocol enable controller
At various times I've tried these lines in the virtual-server config but nothing changes.
port 4149 udp
gslb-enable
source-nat pool GSLB_SNAT
use-rcv-hop-for-resp
port 4149 tcp
source-nat pool GSLB_SNAT
use-rcv-hop-for-resp