小鹿记录

菜单

Administrator
发布于 2023-06-15 / 50 阅读
0
0

Accepting the PROXY Protocol

refer to : https://docs.nginx.com/nginx/admin-guide/load-balancer/using-proxy-protocol/

This article explains how to configure NGINX and NGINX Plus to accept the PROXY protocol, rewrite the IP address of a load balancer or proxy to the one received in the PROXY protocol header, configure simple logging of a client’s IP address, and enable the PROXY protocol between NGINX and a TCP upstream server.

Introduction

The PROXY protocol enables NGINX and NGINX Plus to receive client connection information passed through proxy servers and load balancers such as HAproxy and Amazon Elastic Load Balancer (ELB).

With the PROXY protocol, NGINX can learn the originating IP address from HTTP, SSL, HTTP/2, SPDY, WebSocket, and TCP. Knowing the originating IP address of a client may be useful for setting a particular language for a website, keeping a denylist of IP addresses, or simply for logging and statistics purposes.

The information passed via the PROXY protocol is the client IP address, the proxy server IP address, and both port numbers.

Using this data, NGINX can get the originating IP address of the client in several ways:

  • With the $proxy_protocol_addr and $proxy_protocol_port variables which capture the original client IP address and port. The $remote_addr and $remote_port variables capture the IP address and port of the load balancer.

  • With the RealIP module which rewrites the values in the $remote_addr and $remote_port variables, replacing the IP address and port of the load balancer with the original client IP address and port. The $realip_remote_addr and $realip_remote_port variables retain the address and port of the load balancer, and the $proxy_protocol_addr and $proxy_protocol_port variables retain the original client IP address and port anyway.

Configuring NGINX to Accept the PROXY Protocol

To configure NGINX to accept PROXY protocol headers, add the proxy_protocol parameter to the listen directive in a server block in the http {} or stream {} block.

http {
    #...
    server {
        listen 80   proxy_protocol;
        listen 443  ssl proxy_protocol;
        #...
    }
}
   
stream {
    #...
    server {
        listen 12345 proxy_protocol;
        #...
    }
}

Now you can use the $proxy_protocol_addr and $proxy_protocol_port variables for the client IP address and port and additionally configure the HTTP and stream RealIP modules to replace the IP address of the load balancer in the $remote_addr and $remote_port variables with the IP address and port of the client.

Changing the Load Balancer’s IP Address To the Client IP Address

You can replace the address of the load balancer or TCP proxy with the client IP address received from the PROXY protocol. This can be done with the HTTP and stream RealIP modules. With these modules, the $remote_addr and $remote_port variables retain the real IP address and port of the client, while the $realip_remote_addr and $realip_remote_port variables retain the IP address and port of the load balancer.

To change the IP address from the load balancer’s IP address to the client’s IP address:

Make sure you’ve configured NGINX to accept the PROXY protocol headers. See Configuring NGINX to Accept the PROXY Protocol.

Make sure that your NGINX installation includes the HTTP and Stream Real‑IP modules:

nginx -V 2>&1 | grep -- 'http_realip_module'
nginx -V 2>&1 | grep -- 'stream_realip_module'

If not, recompile NGINX with these modules. See Installing NGINX Open Source for details. No extra steps are required for NGINX Plus.

In the set_real_ip_from directive for HTTP, Stream, or both, specify the IP address or the CIDR range of addresses of the TCP proxy or load balancer:

server {
    #...
    set_real_ip_from 192.168.1.0/24;
   #...
}

In the http {} context, change the IP address of the load balancer to the IP address of the client received from the PROXY protocol header, by specifying the proxy_protocol parameter to the real_ip_header directive:

http {
    server {
        #...
        real_ip_header proxy_protocol;
      }
}

real_ip_header表示，从哪个 header中，检索出需要的IP地址。

其中proxy_protocol这个header中，可能会包含多个ip地址，比如221.181.222.135, 75.27.154.65. 此时，怎么在多个ip地址中，找到实际的客户端ip呢？

这里，采用排除法，从右到左排除，排除那些set_real_ip_from里面出现的ip，因为这些ip是proxy server的IP，剩下的iP地址，就是实际的客户端ip。

Logging the Original IP Address

When you know the original IP address of the client, you can configure the correct logging:

For HTTP, configure NGINX to pass the client IP address to upstream servers using the $proxy_protocol_addr variable with the proxy_set_header directive:

http {
    proxy_set_header X-Real-IP       $proxy_protocol_addr;
    proxy_set_header X-Forwarded-For $proxy_protocol_addr;
}

Add the $proxy_protocol_addr variable to the log_format directive (HTTP or Stream):

In the http block:

http {
    #...
    log_format combined '$proxy_protocol_addr - $remote_user [$time_local] '
                        '"$request" $status $body_bytes_sent '
                        '"$http_referer" "$http_user_agent"';
}

In the stream block:

stream {
    #...
    log_format basic '$proxy_protocol_addr - $remote_user [$time_local] '
                      '$protocol $status $bytes_sent $bytes_received '
                      '$session_time';
}

PROXY Protocol for a TCP Connection to an Upstream

For a TCP stream, the PROXY protocol can be enabled for connections between NGINX and an upstream server. To enable the PROXY protocol, include the proxy_protocol directive in a server block at the stream {} level:

stream {
    server {
        listen 12345;
        proxy_pass example.com:12345;
        proxy_protocol on;
    }
}

Example

http {
    log_format combined '$proxy_protocol_addr - $remote_user [$time_local] '
                        '"$request" $status $body_bytes_sent '
                        '"$http_referer" "$http_user_agent"';
    #...

    server {
        server_name localhost;

        listen 80   proxy_protocol;
        listen 443  ssl proxy_protocol;

        ssl_certificate      /etc/nginx/ssl/public.example.com.pem;
        ssl_certificate_key  /etc/nginx/ssl/public.example.com.key;

        location /app/ {
            proxy_pass       http://backend1;
            proxy_set_header Host            $host;
            proxy_set_header X-Real-IP       $proxy_protocol_addr;
            proxy_set_header X-Forwarded-For $proxy_protocol_addr;
        }
    }
} 

stream {
    log_format basic '$proxy_protocol_addr - $remote_user [$time_local] '
                     '$protocol $status $bytes_sent $bytes_received '
                     '$session_time';
    #...
    server {
        listen              12345 ssl proxy_protocol;

        ssl_certificate     /etc/nginx/ssl/cert.pem;
        ssl_certificate_key /etc/nginx/ssl/cert.key;

        proxy_pass          backend.example.com:12345;
        proxy_protocol      on;
    }
}

The example assumes that there is a load balancer in front of NGINX to handle all incoming HTTPS traffic, for example Amazon ELB. NGINX accepts HTTPS traffic on port 443 (listen 443 ssl;), TCP traffic on port 12345, and accepts the client’s IP address passed from the load balancer via the PROXY protocol as well (the proxy_protocol parameter to the listen directive in both the http {} and stream {} blocks.

NGINX terminates HTTPS traffic (the ssl_certificate and ssl_certificate_key directives) and proxies the decrypted data to a backend server:

It includes the client IP address and port with the proxy_set_header directives.

The $proxy_protocol_addr variable specified in the log_format directive also passes the client’s IP address to the log for both HTTP and TCP.

Additionally, a TCP server (the stream {} block) sends its own PROXY protocol data to its backend servers (the proxy_protocol on directive).

总结

可以看到，上面的样例，指的都是Nginx的上一层，已经实现了Proxy ptotocol，此时，我们能直接从$proxy_protocol_addr这个variables中，获取到客户端实际ip；

但是如果，Nginx上一层，如果没有实现Proxy ptotocol，那么$proxy_protocol_addr 这个variables就没有值了，如果此时，我们仍然想，得到客户端实际ip，那么应该怎么做呢？

思路：
1.我们先确定客户端实际ip 和 Nginx上一层服务器的ip，存在哪个header中？一般是在X-Forwarded-For中，所以，我们需要使用real_ip_header来指定 header: real_ip_header X-Forwarded-For;

2.从X-Forwarded-For这个header中，取到的值，有很多个ip，那么哪一个是客户端实际ip呢？这里采用排除法，这里就需要使用到set_real_ip_from这个directive了，这个directive表示Nginx的上一层服务的ip地址，这个ip地址肯定不是客户端实际ip，所以排除掉，剩下的就是客户端实际ip。
排除的时候，可能还需要结合real_ip_recursive这个directive来实现。

real_ip_header

real_ip_header这个指令，常用的值，包含了X-Real-IP | X-Forwarded-For | proxy_protocol，其中如果设置为proxy_protocol，那么要求nginx的上一层，必须实现代理协议；如果上一层没有实现代理协议，一般还是设置为X-Forwarded-For。

Syntax:	real_ip_header field | X-Real-IP | X-Forwarded-For | proxy_protocol;
Default:	
real_ip_header X-Real-IP;
Context:	http, server, location

nginx 获取客户端真实 IP 详细解析实践

refer to : https://zhuanlan.zhihu.com/p/391215425

nginx 是通过 ngx_http_realip_module 模块来实现获取客户端真实 IP 的；

一般一个请求在网络传输时会经过多个代理层，当请求到达真实应用服务器时如何获取客户端真实 IP 就是一个问题了；

以下是介绍 nginx 如何处理来获取到客户端真实 IP，及 nginx 同时作为反向代理时如何传递客户端真实 IP 给应用服务器；

个人理解如下：

假如客户端 C 访问应用服务器 S 中间经过三层代理，分别是 Proxy1 / Proxy2 / Proxy3，X-Forwarded-For 处理流程如下：

客户端 C 访问 Proxy1，Proxy1 会记录客户端地址 IP0，Proxy1 把请求转发给 Proxy2 的时候会在请求头部加上 X-Forwarded-For : IP0；
Proxy2 接收到 Proxy1 的请求后也会记录 Proxy1 的地址 IP1，发现请求头中有头部字段 X-Forwarded-For : IP0，Proxy2 把请求转发给 Proxy3 的时候会拼接 Proxy1 的地址，即 X-Forwarded-For : IP0, IP1；
同理，Proxy3 接收到 Proxy2 的请求后，把请求转发给应用服务器的时候会拼接 Proxy2 的地址，即 X-Forwarded-For : IP0, IP1, IP2；
应用服务器接收到 Proxy3 的请求，头部字段X-Forwarded-For : IP0, IP1, IP2，没有 Proxy3 的IP地址，nginx 可以通过 $remote_addr 变量获取，web 应用服务可以通过 request.getRemoteAddr() 方法获取；

模块指令

ngx_http_realip_module 模块有如下三个指令；

1、set_real_ip_from
该指令值一般是前几层代理的 IP ；

2、real_ip_header
该指令用于告知 nginx 从每个客户端请求中的哪个头字段来获取客户端真实的 IP；
该指令默认值是 X-Real-IP，不过现在主流的都是通过 X-Forwarded-For 字段来获取客户端真实 IP，X-Forwarded-For 目前已经是主流运用的字段了；
我们也可以在 nginx 配置时自定义一个新的字段；

3.real_ip_recursive

nginx 从 real_ip_header 指令指定的头字段中获取 IP，可能会有多个 IP 值；

当 real_ip_recursive 指令值为 off，nginx 从获取到 IP 值中从右往左（也即从后往前）的顺序，以最后一个 IP 值作为客户端的真实 IP ，此时不会排除授信 IP；
当 real_ip_recursive 指令值为 on，nginx 从获取到 IP 值中从右往左（也即从后往前）的顺序，排除 set_real_ip_from 指令指定的授信 IP，以最后一个非授信 IP 值作为客户端的真实 IP ；

实验

【实验一】

客户端真实 IP 192.168.135.1，我们设置两个头字段，分别如下

X-Forwarded-For 192.168.135.12, 192.168.135.22
mfz 192.168.135.13, 192.168.135.23
nginx 不做任何相关的 IP 指令配置，设置如下

# Proxy
upstream backend {        
    server 192.168.135.128:8906;
}

server {
    listen 8093;
    server_name _;

    underscores_in_headers on;
    location /test {
        proxy_pass http://backend;
    }
}

我们从 nginx 的日志文件 access.log 中看效果，默认日志格式如下

image-20230615110015059

img

发现 nginx 拿到客户端的地址是 192.168.135.1 ，即 $remote_addr 变量值；

【实验二】

客户端真实 IP 192.168.135.1，我们设置两个头字段，分别如下

X-Forwarded-For  192.168.135.12, 192.168.135.22
mfz  192.168.135.13, 192.168.135.23

设置 real_ip_header X-Forwarded-For，如下

# Proxy
upstream backend {        
    server 192.168.135.128:8906;
}

server {
    listen 8093;
    server_name _;

    underscores_in_headers on;
    location /test {
        proxy_pass http://backend;
        real_ip_header X-Forwarded-For;
    }
}

我们从 nginx 的日志文件 access.log 中看效果
img

发现 nginx 拿到客户端的地址还是 192.168.135.1 ，即 $remote_addr 变量值；


$remote_addr 的值，是怎么来的呢？
nginx 是通过 TCP 连接拿到最近一层代理的 IP 并赋值给 $remote_addr 变量，因为请求是由代理服务器转发过来的，是上一层代理服务和 nginx 服务产生了 TCP 连接；

如果客户端和 nginx 服务器之间没有代理层，那么 $remote_addr 就是客户端的地址，因为客户端直接和 nginx 服务进行 TCP 连接；

这里，是客户端直接和nginx进行了TCP 连接，所以$remote_addr 的值就是客户端的ip—192.168.135.1

【实验三】

客户端真实 IP 192.168.135.1，我们设置两个头字段，分别如下

X-Forwarded-For  192.168.135.12, 192.168.135.22
mfz  192.168.135.13, 192.168.135.23

把客户端真实 IP 作为授信 IP，设置如下

# Proxy
upstream backend {        
    server 192.168.135.128:8906;
}

server {
    listen 8093;
    server_name _;

    underscores_in_headers on;
    location /test {
        proxy_pass http://backend;
        real_ip_header X-Forwarded-For;
        set_real_ip_from 192.168.135.1;
    }
}

发现 nginx 拿到客户端的地址是 192.168.135.22 ，即 X-Forwarded-For 头字段的最后一个 IP；

也就是说变量 remoteaddr的值变成192.168.135.22了，此时setrealipfrom指令管用，remote_addr 的值变成 192.168.135.22 了，此时 set_real_ip_from 指令管用，remote_addr 值会排除掉授信 IP，拿到 real_ip_header 指令指定的头字段携带的 IP最后一个作为真实的客户端IP，赋值给变量 $remote_addr；

【实验四】

客户端真实 IP 192.168.135.1，我们去掉 X-Forwarded-For 字段，只设置一个头字段，分别如下

mfz  192.168.135.13, 192.168.135.23

把客户端真实 IP 作为授信 IP，设置如下

# Proxy
upstream backend {        
    server 192.168.135.128:8906;
}

server {
    listen 8093;
    server_name _;

    underscores_in_headers on;
    location /test {
        proxy_pass http://backend;
        real_ip_header X-Forwarded-For;
        set_real_ip_from 192.168.135.1;
    }
}

我们从 nginx 的日志文件 access.log 中看效果

发现 nginx 拿到客户端的地址变为 192.168.135.1 了，是客户端真实 IP；

也就是说变量 $remote_addr 的值变成 192.168.135.1 了，此时 set_real_ip_from 指令不管用了，因为过来的请求头中没有 real_ip_header 指定的字段；

Full Example Conf...
HTTP Health Checks

评论