tencent cloud

Tencent Cloud EdgeOne

Release Notes and Announcements
Release Notes
Security Announcement
Announcements
Product Introduction
Overview
Strengths
Use Cases
Comparison Between EdgeOne and CDN Products
Use Limits
Purchase Guide
Description of Trial Plan Experience Benefits
Free Plan Guide
Billing Overview
Billing Items
Subscriptions
Renewals
Instructions for overdue and refunds
Comparison of EdgeOne Plans
About "clean traffic" billing instructions
DDoS Protection Capacity Description
Getting Started
Choose business scenario
Quick access to website security acceleration
Quick deploying a website with Pages
Domain Service&Origin Configuration
Domain Service
HTTPS Certificate
Origin Configuration
Site Acceleration
Overview
Access Control
Smart Acceleration
Cache Configuration
File Optimization
Network Optimization
URL Rewrite
Modifying Header
Modify the response content
Rule Engine
Image&Video Processing
Speed limit for single connection download
DDoS & Web Protection
Overview
DDoS Protection
Web Protection
Bot Management
API Discovery(Beta)
Edge Functions
Overview
Getting Started
Operation Guide
Runtime APIs
Sample Functions
Best Practices
Pages
L4 Proxy
Overview
Creating an L4 Proxy Instance
Modifying an L4 Proxy Instance
Disabling or Deleting an L4 Proxy Instance
Batch Configuring Forwarding Rules
Obtaining Real Client IPs
Data Analysis&Log Service
Log Service
Data Analysis
Alarm Service
Site and Billing Management
Billing Management
Site Management
Version Management
General Policy
General Reference
Configuration Syntax
Request and Response Actions
Country/region and Corresponding Codes
Terraform
Overview
Installing and Configuring Terraform
Practical Tutorial
EdgeOne Skill User Guide
Automatic Warm-up/Cache Purge
Resource Abuse/hotlinking Protection Practical
HTTPS Related Practices
Acceleration Optimization
Scheduling Traffic
Data Analysis and Alerting
Log Platform Integration Practices
Configuring Origin Servers for Cloud Object Storage (Such As COS)
CORS Response Configuration
API Documentation
History
Introduction
API Category
Making API Requests
Site APIs
Acceleration Domain Management APIs
Site Acceleration Configuration APIs
Edge Function APIs
Alias Domain APIs
Security Configuration APIs
Layer 4 Application Proxy APIs
Content Management APIs
Data Analysis APIs
Log Service APIs
Billing APIs
Certificate APIs
Origin Protection APIs
Load Balancing APIs
Diagnostic Tool APIs
Custom Response Page APIs
API Security APIs
DNS Record APIs
Content Identifier APIs
Legacy APIs
Ownership APIs
Image and Video Processing APIs
Multi-Channel Security Gateway APIs
Version Management APIs
Data Types
Error Codes
FAQs
Product Features FAQs
DNS Record FAQs
Domain Configuration FAQs
Site Acceleration FAQs
Data and Log FAQs
Security Protection-related Queries
Origin Configuration FAQs
Troubleshooting
Reference for Abnormal Status Codes
Troubleshooting Guide for EdgeOne 4XX/5XX Status Codes
520/524 Status Code Troubleshooting Guide
521/522 Status Code Troubleshooting Guide
Tool Guide
Agreements
Service Level Agreement
Origin Protection Enablement Conditions of Use
TEO Policy
Privacy Policy
Data Processing And Security Agreement
Contact Us
Glossary

Passing Real TCP Client IPs via TOA

PDF
Focus Mode
Font Size
Last updated: 2025-03-10 17:40:14
You can use this document to learn how to get the TCP client IP via TOA when using L4 proxy.

Use Cases

Using L4 acceleration for data packets will have the source IP and port modified, so the origin does not get the original information. To enable the origin to get the real client IP and port, you can pass the information using TOA when creating an acceleration channel. In this way, the real client IP and port are passed into the TCP option field. Meanwhile, you need to install TOA on the origin to get that information.
Note:
The L4 proxy is only available with the Enterprise Edition package.

Directions

Step 1: Pass the client IP via TOA

To get the TCP client IP via TOA, set Pass client IP to TOA in L4 proxy forwarding rules in the console. For details on how to modify rules, see: Modifying an L4 Proxy Instance.


Step 2: Load TOA on backend server

You can load the TOA module using either of the following methods:
Method 1 (recommended): Based on the Linux version the origin uses, download the complied toa.ko file and load it directly.
Method 2: If you cannot find the appropriate Linux version, download the TOA source code file and compile and load it yourself. The source code only supports the x86_64 version. If you need support for the arm64 version, please contact us.
Note:
Due to the differences in installation environments, if you encounter issues during the loading process using Method 1, please try Method 2 and install the compilation environment manually.
Method 1: Download and load the compiled TOA module
Method 2: Compile and load the TOA module
1. Download and decompress the TOA package corresponding to the version of Linux OS on Tencent Cloud.
centos
TencentOS
debian
suse linux
ubuntu

2. After decompression is complete, run the cd command to access the decompressed folder. Then load the module as follows:
Load TOA with a script
Manually load TOA
/bin/bash -c "$(curl -fsSL https://edgeone-document-file-1258344699.cos.ap-guangzhou.myqcloud.com/TOA/install_toa.sh)"
When it is loaded successfully, you will see the following information:

# Decompress the tar package.
tar -zxvf CentOS-7.2-x86_64.tar.gz
# Enter the directory of the decompressed package.
cd CentOS-7.2-x86_64
# Load the TOA module.
insmod toa.ko
# Copy the TOA module to the kernel module directory.
cp toa.ko /lib/modules/`uname -r`/kernel/net/netfilter/ipvs/toa.ko
# Configure the TOA module to load automatically at system startup.
echo "insmod /lib/modules/`uname -r`/kernel/net/netfilter/ipvs/toa.ko" >> /etc/rc.local
Run the following command to check whether the loading is successful:
lsmod | grep toa
If you see "TOA" in the message, the module is loaded successfully:


1. Install the compilation environment.
1.1 Make sure kernel-devel and kernel-headers are installed and consistent with the kernel version.
1.2 Make sure the gcc and make dependencies are installed.
1.3 If these environmental dependencies are not installed, run the installation command:
CentOS
Ubuntu/Debian
yum install -y gcc
yum install -y make
yum install -y kernel-headers kernel-devel
apt-get install -y gcc
apt-get install -y make
apt-get install -y linux-headers-$(uname -r)

2. After the compilation environment is installed, download, compile and load the source code.
Compile and load TOA with a script
Manually compile and load TOA
/bin/bash -c "$(curl -fsSL https://edgeone-document-file-1258344699.cos.ap-guangzhou.myqcloud.com/TOA/compile_install_toa.sh)"
# Create a compilation directory and enter it.
mkdir toa_compile && cd toa_compile
# Download the source code (tar.gz)
curl -o toa.tar.gz https://edgeone-document-file-1258344699.cos.ap-guangzhou.myqcloud.com/TOA/toa.tar.gz
# Decompress the tar package
tar -zxvf toa.tar.gz
# Compile the toa.ko file. After the compilation is successful, the file will be generated in the current directory.
make
# Load the TOA module.
insmod toa.ko
# Copy the TOA module to the kernel module directory.
cp toa.ko /lib/modules/`uname -r`/kernel/net/netfilter/ipvs/toa.ko
# Configure the TOA module to load automatically at system startup
echo "insmod /lib/modules/`uname -r`/kernel/net/netfilter/ipvs/toa.ko" >> /etc/rc.local
3. Run the following command to check whether the loading is successful:
lsmod | grep toa
If you see "TOA" in the message, the module is loaded successfully:


Step 3: Verify the configuration

You can verify the configuration by building a TCP server to receive client requests from another server. See the sample:
1. On the current server, create an HTTP server in Python to act as a TCP server:
# Use python2
python2 -m SimpleHTTPServer 10000

# Use python3
python3 -m http.server 10000
2. Make another server work as a client to send requests:
# Use curl to initiate an HTTP request, where the hostname and forwarding port of the L4 proxy is used.
curl -i "http://a8b7f59fc8d7e6c9.example.com.edgeonedy1.com:10000/"
3. If TOA is loaded, the real client address can be seen on the server:

You can also get either the IPv4 or IPv6 address of the client by following the steps above.
For origin IPv4 addresses, get the client IPv4 address.
For origin IPv6 addresses, get the client IPv6 address.
If you need to get both IPv4 and IPv6 addresses, modify the origin's business code while loading the TOA module as instructed here.

Getting Both IPv4 and IPv6 Addresses

Note:
This section provide guidance on how to get both IPv4 and IPv6 addresses by modifying the business code of the origin.
The origin can listen on requests in either of the following methods:
1. Use the structure struct sockaddr_in to listen on IPv4 addresses.
2. Use the structure struct sockaddr_in6 to listen on IPv6 addresses.

Sample code

Listen on IPv4 addresses
Listen on IPv6 addresses
C
Java
#include <sys/socket.h>
#include <stdio.h>
#include <unistd.h>
#include <netinet/in.h>
#include <memory.h>
#include <arpa/inet.h>

int main(int argc, char** argv) {
int l_sockfd;
// The server address is an IPv4 address.
struct sockaddr_in serveraddr;
// In this case, the client address must adopt the IPv6 structure.
struct sockaddr_in6 clientAddr;
int server_port = 10000;

memset(&serveraddr, 0, sizeof(serveraddr));
// Create a socket.
l_sockfd = socket(AF_INET, SOCK_STREAM, 0);
if (l_sockfd == -1){
printf("Failed to create socket.\\n");
return -1;
}
// Initialize the server.
memset(&serveraddr, 0, sizeof(struct sockaddr_in));
serveraddr.sin_family = AF_INET;
serveraddr.sin_port = htons(server_port);
serveraddr.sin_addr.s_addr = htonl(INADDR_ANY);

int isReuse = 1;
setsockopt(l_sockfd, SOL_SOCKET,SO_REUSEADDR,(const char*)&isReuse,sizeof(isReuse));
// Associate the socket and server address.
int nRet = bind(l_sockfd,(struct sockaddr*)&serveraddr, sizeof(serveraddr));
if(-1 == nRet)
{
printf("bind error\\n");
return -1;
}
// Listen on the socket.
listen(l_sockfd, 5);

int clientAddrLen = sizeof(clientAddr);
memset(&clientAddr, 0, sizeof(clientAddr));
// Accept connections from the client.
int linkFd = accept(l_sockfd, (struct sockaddr*)&clientAddr, &clientAddrLen);
if(-1 == linkFd)
{
printf("accept error\\n");
return -1;
}
// Modifications to make: Decide whether the client is an IPv4 or IPv6 address based on sin6_family.
// AF_INET indicates that the client adopts IPv4. In this case, convert the client address pointer to struct sockaddr_in* and get the IPv4 address.
// AF_INET6 indicates that the client adopts IPv6. In this case, use struct sockaddr_in6* to get the IPv6 address.
if (clientAddr.sin6_family == AF_INET) {
printf("AF_INET accept getpeername %s : %d successful\\n",
inet_ntoa(((struct sockaddr_in*)&clientAddr)->sin_addr),
ntohs(((struct sockaddr_in*)&clientAddr)->sin_port));
}else if (clientAddr.sin6_family == AF_INET6){
char addr_p[128] = {0};
inet_ntop(AF_INET6, (void *)&((struct sockaddr_in6*)&clientAddr)->sin6_addr, addr_p, (socklen_t )sizeof(addr_p));
printf("AF_INET6 accept getpeername %s : %d successful\\n",
addr_p,
ntohs(((struct sockaddr_in6*)&clientAddr)->sin6_port));
}else{
printf("unknow sin_family:%d \\n", clientAddr.sin6_family);
}
close(l_sockfd);
return 0;
}
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.ServerSocket;
import java.net.Socket;
import java.net.SocketAddress;


public class ServerDemo {


/** If using the IPv4 address structure to build the service, use IPV4_HOST */
public static final String IPV4_HOST = "0.0.0.0";


/** If using the IPv6 address structure to build the service, use IPV6_HOST */
public static final String IPV6_HOST = "::";


public static void main(String[] args) {
int serverPort = 10000;
try (ServerSocket serverSocket = new ServerSocket()) {
// Setting address reuse
serverSocket.setReuseAddress(true);
// Bound server address and port, using IPv4 here
serverSocket.bind(new InetSocketAddress(InetAddress.getByName(IPV4_HOST), serverPort));
System.out.println("Server is listening on port " + serverPort);


while (true) {
// Accepting Client connections
Socket clientSocket = serverSocket.accept();
System.out.println("New client connected: " + clientSocket.getRemoteSocketAddress());


// Processing Client requests
handleClientRequest(clientSocket);
}
} catch (IOException e) {
System.err.println("Failed to create server socket: " + e.getMessage());
}
}


/**
* Processing Function, site business implement, here is just an example
* The purpose of this Function is to Return the Client's input verbatim to the Client
*/
private static void handleClientRequest(Socket clientSocket) {
try (InputStream inputStream = clientSocket.getInputStream();
OutputStream outputStream = clientSocket.getOutputStream()) {


// Reading the Data received from the Client
byte[] buffer = new byte[1024];
int bytesRead;
while ((bytesRead = inputStream.read(buffer)) != -1) {
// Reply the received Data to the Client as it is
outputStream.write(buffer, 0, bytesRead);
}


} catch (IOException e) {
// When the Client disconnects
System.err.println("Failed to handle client request: " + e.getMessage());
} finally {
try {
clientSocket.close();
} catch (IOException e) {
System.err.println("Failed to close client socket: " + e.getMessage());
}
}
}
}
C
Java
#include <sys/socket.h>
#include <stdio.h>
#include <unistd.h>
#include <netinet/in.h>
#include <memory.h>
#include <arpa/inet.h>

int main(int argc, char **argv)
{
int l_sockfd;
// The server address is an IPv6 address.
struct sockaddr_in6 serveraddr;
// The client address is an IPv6 address.
struct sockaddr_in6 clientAddr;
int server_port = 10000;

memset(&serveraddr, 0, sizeof(serveraddr));
// Create a socket.
l_sockfd = socket(AF_INET6, SOCK_STREAM, 0);
if (l_sockfd == -1){
printf("Failed to create socket.\\n");
return -1;
}
// Set the server address.
memset(&serveraddr, 0, sizeof(struct sockaddr_in6));
serveraddr.sin6_family = AF_INET6;
serveraddr.sin6_port = htons(server_port);
serveraddr.sin6_addr = in6addr_any;

int isReuse = 1;
setsockopt(l_sockfd, SOL_SOCKET,SO_REUSEADDR,(const char*)&isReuse,sizeof(isReuse));
// Associate the socket and server address.
int nRet = bind(l_sockfd,(struct sockaddr*)&serveraddr, sizeof(serveraddr));
if(-1 == nRet)
{
printf("bind error\\n");
return -1;
}
// Listen on the socket.
listen(l_sockfd, 5);

int clientAddrLen = sizeof(clientAddr);
memset(&clientAddr, 0, sizeof(clientAddr));
// Accept connection requests from the client.
int linkFd = accept(l_sockfd, (struct sockaddr*)&clientAddr, &clientAddrLen);
if(-1 == linkFd)
{
printf("accept error\\n");
return -1;
}
// The client addresses received here are all stored in the IPv6 structure.
// The IPv4 addresses are mapped to IPv6 addresses, for example, "::ffff:119.29.1.1".
char addr_p[128] = {0};
inet_ntop(AF_INET6, (void *)&clientAddr.sin6_addr, addr_p, (socklen_t )sizeof(addr_p));
printf("accept %s : %d successful\\n", addr_p, ntohs(clientAddr.sin6_port));
// Modifications to make: Use the macro definition IN6_IS_ADDR_V4MAPPED to decide whether the client IP is an IPv4-mapped IPv6 address.
if(IN6_IS_ADDR_V4MAPPED(&clientAddr.sin6_addr)) {
struct sockaddr_in real_v4_sin;
memset (&real_v4_sin, 0, sizeof (struct sockaddr_in));
real_v4_sin.sin_family = AF_INET;
real_v4_sin.sin_port = clientAddr.sin6_port;
// The last four bytes represent the IPv4 address of the client.
memcpy (&real_v4_sin.sin_addr, ((char *)&clientAddr.sin6_addr) + 12, 4);
printf("connect %s successful\\n", inet_ntoa(real_v4_sin.sin_addr));
}
close(l_sockfd);
return 0;
}
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.ServerSocket;
import java.net.Socket;
import java.net.SocketAddress;

public class ServerDemo {

/** If using the IPv4 address structure to build the service, use IPV4_HOST */
public static final String IPV4_HOST = "0.0.0.0";

/** If using the IPv6 address structure to build the service, use IPV6_HOST */
public static final String IPV6_HOST = "::";

public static void main(String[] args) {
int serverPort = 10000;
try (ServerSocket serverSocket = new ServerSocket()) {
// Setting address reuse
serverSocket.setReuseAddress(true);
// Bound server address and port, using IPv4 here
serverSocket.bind(new InetSocketAddress(InetAddress.getByName(IPV6_HOST), serverPort));
System.out.println("Server is listening on port " + serverPort);

while (true) {
// Accepting Client connections
Socket clientSocket = serverSocket.accept();
System.out.println("New client connected: " + clientSocket.getRemoteSocketAddress());

// Processing Client requests
handleClientRequest(clientSocket);
}
} catch (IOException e) {
System.err.println("Failed to create server socket: " + e.getMessage());
}
}

/**
* Processing Function, site business implement, here is just an example
* The purpose of this Function is to Return the Client's input verbatim to the Client
*/
private static void handleClientRequest(Socket clientSocket) {
try (InputStream inputStream = clientSocket.getInputStream();
OutputStream outputStream = clientSocket.getOutputStream()) {

// Reading the Data received from the Client
byte[] buffer = new byte[1024];
int bytesRead;
while ((bytesRead = inputStream.read(buffer)) != -1) {
// Reply the received Data to the Client as it is
outputStream.write(buffer, 0, bytesRead);
}

} catch (IOException e) {
// When the Client disconnects
System.err.println("Failed to handle client request: " + e.getMessage());
} finally {
try {
clientSocket.close();
} catch (IOException e) {
System.err.println("Failed to close client socket: " + e.getMessage());
}
}
}
}

console output result

Server is listening on port 10000
New client connected: /127.0.0.1:50680
New client connected: /0:0:0:0:0:0:0:1:51124
New client connected: /127.0.0.1:51136

References

Monitoring TOA Running Status

To ensure execution stability, this kernel module allows you to monitor status. After inserting the toa.ko kernel module, you can monitor the TOA working status in either of the following ways.
cat /proc/net/toa_stats
This figure shows the TOA running status:

The monitoring metrics are described as follows:
Metric
Description
syn_recv_sock_toa
Receives connections with TOA information.
syn_recv_sock_no_toa
Receives connections without TOA information.
getname_toa_ok
This count increases when you call getsockopt and obtain the source IP address successfully or when you call accept to receive client requests.
getname_toa_mismatch
This count increases when you call getsockopt and obtain a source IP address that does not match the required type. For example, if a client connection contains a source IPv4 address whereas you obtain an IPv6 address, the count will increase.
getname_toa_empty
This count increases when the getsockopt function is called in a client file descriptor that does not contain TOA.
ip6_address_alloc audio/video proxy
Allocates space to store the information when TOA obtains the source IP address and source port saved in the TCP data packet.
ip6_address_free audio/video proxy
When the connection is released, TOA will release the memory previously used to save the source IP and source port. If all connections are closed, the total count of ip6_address_alloc for each CPU should be equal to the count of this metric.


Help and Support

Was this page helpful?

Help us improve! Rate your documentation experience in 5 mins.

Feedback