TCP-proxy Client-IP
flaneur
Member ✭
Hi.
We have SLB Virtual Service with type TCP-Proxy and we've also enabled Insert Client IP option in TCP-Proxy template but all traffic is forwarded from AX1030 internal IP .The question is how we can get source client IP ?
We have SLB Virtual Service with type TCP-Proxy and we've also enabled Insert Client IP option in TCP-Proxy template but all traffic is forwarded from AX1030 internal IP .The question is how we can get source client IP ?
0
Comments
port 25 tcp, then you need to look in the TCP header for the information. Similar to an HTTP Template which inserts the client ip in the HTTP Header as a Header Field, a TCP Template mucks with the Options in the TCP Header.In my experiment, the client IP address was inserted as a TCP option just after the Window Scale option (plus a
0x01pad).1c 07 01 64 40 01 64which corresponds with the IP address 100.64.1.1001c- option type (not recognized by my wireshark)07- length (7 bytes)01- pad64 40 01 64- Client-IP in hex.If your proxy can not read the inserted Client-IP in the TCP-Options, then you need to use another way.
If your proxy supports reading the X-Forwarded-For HTTP header, you need to make sure the AX1030 inserts it.
Normally you would set you Virtual Service to use HTTP/HTTPS and inserting the headers is very easily done in the HTTP Template.
If for some reason you can not use HTTP, you could look at using aFlex to insert the header in the raw TCP data stream.
Is there a particular reason why you are using TCP-Proxy and not HTTP?
The use of aFlex is going to be much more heavy on your system than using HTTP.
The aFlex would have look at all packets and buffer data for every connection that needs to have te header inserted.
Actually it will need to add the header multiple times, every time a new request is coming in, even if it using an existing connection.
What does the Memory and Data CPU usage look like on peak moment?
If your device can not handle the extra processing required for HTTP, you should not use aFLex.
You could change your network setup, so that the return traffic from your proxy goes back to the A10... so you can remove source-NAT.
What other services are you running on the AX 1030? If you are also using it for a VIP with SSL-Offload, I would strongly suggest upgrading to a newer system as the SSL chipset on the AX is extremely outdated.
Now when we use TCP type of our virtual service CPU and Memmory usage are very-very low.I can't find any benchmarks for AX1030 to see how many HTTP connections it can handle and I decided to use TCP to hedge.
And depending on the re-use of these connections well over 1 million HTTP Requests per second. I expect it should be able to handle over 6 million concurrent connections.
But again that all depends on your setup.
If I were you, I would set the port type to HTTP and use the template, and then see what the performance impact is.
And if you expect to grow more than the unit can possible handle, get the Thunder 1040S
The only way to remove the need for Source-Nat is by making sure the Proxy sends the return traffic back to the A10.
So if you know the client IP's, configure the proxy in such a way that for the client IP's as destination it uses de A10 as next hop. If the Proxy and A10 are not part of the same broadcast network, you also need to fix the routing on the devices between the A10 and Proxy.
insert-client-ipin the same way as the tcp template and vport: by inserting the client ip as a TCP Option in the TCP Header.As do how to read the TCP Option at your application server, I've heard of customers using firewall logging to record the value. I'm not sure what possibilities exist for the Application to read the value directly from the TCP socket (I took a seminar on UNIX tcp socket programming some 16 years ago but never used the knowledge).
The vport type HTTP with an HTTP template with
insert-client-ipsounds like it may be the way to go. It's been optimized pretty well over the past decade.I agree with diederik that Source-NAT is largely a solution to a routing problem: how does one get the server's reply packets back to the ADC in order to do whatever L4/L7 magic your need done without making the ADC the default gateway for all traffic from the server. There are other uses as well, but the return traffic challenge is the big one. If you can solve the problem with routing, then you don't need
insert-client-ipIf the information so far isn't enough to resolve the design/config challenge, you might be dealing with a situation perhaps too complex to handle in an online forum. Maybe contact A10 Professional Service? I would advise that especially if you wish to use the aFlex based solution.
I suppose the proxy is doing caching?
You can actually have the ADC bypass SSL traffic... but that means you still can not add the HTTP Header, as for HTTP Header insertion you need to alter the payload of the packets which are SSL encrypted.
Without knowing the total structure of your setup, I believe you only have 2 options.
Either remove the need for Source-NAT by possibly moving the location of the A10 ADC or the proxy, as the proxy is doing SSL Termination the proxy is the most logical device to do HTTP Header updates and Source-NAT.
Or use
insert-client-ipand find out how you can make your Proxy read the Client-IP's from the TCP Options.Like ericn suggested, you might want to get in touch with you A10 Sales/Systems Engineer so they kan have a look at your setup and possible help getting A10 Professional Services involved.
ADC with external IP address and Source-NAT to the internal IP address -> 2 Envoy proxies in Service Group that are balanced with the help of ADC by TCP type.
The only way that can help us is to remove the Source-NAT from this chain but how can we route traffic from external IP address to our internal IP addresses ?
To forward traffic from the ADC to the servioce group, the destination address is altered. (Destination NAT). And that happens automatically when you have your ADC point the traffic to the IP addresses of the Envoy Proxy servers that are in your Service Group.
Lets say we use a client from the internet which has a source in the 200.x.y.z range.
And you service has a public IP address of 100.0.0.100
The VIP on the ADC is 10.0.0.100 and the Envoy server the traffic gets forwarded to has 192.168.0.100 a private address.
What you will see is that the external ip addressing is used for the outside session on the ADC and from the ADC without the Source-NAT it will only change the destination IP address.
From client on the internet to the ADC
external scr: 200.x.y.z
external dst: 100.0.0.100
From the ADC to the Envoy Proxy
internal scr: 200.x.y.z
internal dst: 192.168.0.100
So when the traffic needs to go back from the proxy to the client om the internet, it will use the 200.x.y.z as destination address.
Now you just need to make sure that that traffic passed from the Envoy Proxy through the A10 ADC back to the internet... when traffic returns over the A10 ADC, it will recognise the session and will Source-NAT it automatically back to the VIP address.
From the Envoy Proxy to the ADC
internal src: 192.168.0.100
internal dst: 200.x.y.z
From the ADC to the client on the internet
external src: 100.0.0.100
external dst: 200.x.y.z
You need to take a look at the proxy and see where it sends the traffic that needs to go back to the internet. Check it's routing table.
If the Proxy and the A10 share the same broadcast domain, then you need to set the default gateway of the proxy to the A10 internal address. I assume you have VRRP-A setup, so set it to the floating address of your A10 cluster.
I assume this is the case as the A10 only has a default gateway set, and no specific routes for the internal network.
If there is one or more routers between the A10 and the proxy, you also need to make sure that those routers send the traffic in the right direction.
But, make sure that this does not break other traffic flows in your network.
Understand that if you do this, all traffic towards the internet from the proxy will go towards the A10.
So if there is a service on the proxy that needs to update things like signatures/URL Classification lists/or maybe needs to talk to other devices in your network that are not in a broadcast domain of one of the interfaces, you need to fix that as well. Maybe through specific routes or by enabling outbound NAT on the A10 for the proxies.
I would strongly suggest you contact the A10 Business partner or your A10 Systems Engineer/Account Team so they can assist you with finding out what the best approach is here.
The issue you are having getting the traffic back from the proxy to the A10, would be the same if you had any other equipment in place of the A10.
This is basic networking.
All they have to do is make sure the traffic from the Proxy is send back to the A10.
So the configuration of the proxy needs to change first... not the A10.
But of course you can always contact A10 to see who the contacts are in your region.
You could hire A10's Professional Services team to have a look.
slb server PROXY1 <some_private_ip> stuff slb server PROXY2 <other_private_ip> stuff slb service-group MY_Proxy member PROXY1 <port/s> member PROXY2 <port/s> slb virtual-server MYVIP <some_public_ip> port <port/s> TCP service-group MY_PROXYClient connects to <some_public_ip>, and the ADC does the NAT (Destination NAT, rather than SourceNAT) to the private IPs inside/behind the ACOS device.
So on the outside, you route to the public_IP, and on the inside, you set the ADC as the default gateway of the inside hosts.
For most topologies in use today (L3 Routed w/o SNAT, L3 Routed w/ SNAT, and L2 One Arm Mode), ACOS is doing DNAT.
2 interfaces - public - ip network 212.32.x.x and private - ip network 10.201.0.x.
routing table:
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 212.32.x.x 0.0.0.0 UG 0 0 0 eth4
10.0.0.0 0.0.0.0 255.128.0.0 U 0 0 0 br-bdfaffd0acf2
10.201.0.0 0.0.0.0 255.255.252.0 U 0 0 0 eth5
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
212.32.252.64 0.0.0.0 255.255.255.192 U 0 0 0 eth4
Configuration on the ADC side:
2 interfaces - ethernet1 and ethernet2
ethernet1 - 212.32.x.x/27 - public interface with static ip which I use to access ADC web ui
ethernet2 - 10.201.0.250/22 private interface to communicate with private network.
Routing map on ADC:
0.0.0.0 0.0.0.0 212.32.252.158 ethernet 1 Static
212.32.252.128 255.255.255.224 0.0.0.0 ethernet 1 Connected
ADC was setuped in gateway mode.
VRRP-A isn't used.
What else information do you need ?
0.0.0.0 212.32.x.x 0.0.0.0 UG 0 0 0 eth4Points towards the IP address of the A10 ADC then it should work.
Unless something on your network is doing Proxy-ARP
As the A10 does not have interfaces in the private IP range, you need to point the:
Towards the public IP of the Envoy servers.
Best is to do a packet trace on both the Envoy as well as on the A10 so you can exactly follow what is happening.
No this route isn't to ADC, it is to some device from our hoster,I'm now trying to get information from our hoster about what is it.So I assume this is our global issue.
I'm afraid this is not something that can easily be fixed using the forum.
Somebody needs to have a look at the total setup suggest how all devices should be configured.
I strongly suggest you to contact an A10 account team in your region so they can discuss the options to solve this.
You can find local contact details here:
https://www.a10networks.com/company/contact-us
I mean option Insert Client IP in Config Mode > SLB > Template > TCP Proxy.
It could possibly be build in aFlex, but then requires the port type HTTP.
The option Client IP in the template TCP Proxy, uses the TCP Options.
https://www.iana.org/assignments/tcp-parameters/tcp-parameters.xml