Странное поведение при нагрузке

Добрый день. Хочу использовать angie в качестве reverse proxy.
Столкнулся со следующей проблемой:
После небольшой нагрузки сервер перестает отвечать на новые запросы с моего ip. Просто отваливается по таймауту, без ошибок. В логах вижу следующее:

 [debug] 96834#96834: accept on 0.0.0.0:80, ready: 0
 [debug] 96861#96861: accept() not ready (11: Resource temporarily unavailable)

При этом на запросы с других ip сервер отвечает. Бэкенд не падает. Конфиг angie базовый. Пробовал применить директивы worker_connections worker_rlimit_nofile, но это не дало результатов.

Добрый день. Для начала, давайте убедимся что конфиг применился. Выполните команду ps aux | grep angie, затем, сделайте релоад angie с помощью команды sudo angie -t && sudo service angie reload и снова выполните ps aux | grep angie. Пришлите выводы этих команд нам. Воспроизведите проблему и пришлите полный конфиг (вывод команды sudo angie -T), а так же полные access и error логи, записаные во время воспроизведения проблемы. Поясните, пожалуйста, что конкретно “не дало результатов”, не удалось применить директивы или применение директив не изменило поведение?

Может не хватает лимитов?

sysctl net.core.somaxconn
sysctl net.core.netdev_max_backlog

Вы уверены, что у вас порты на клиентской машине не закончились из-за большого количества соединений в состоянии TIME_WAIT?

Благодарю за ответ. Рестарт сервиса я выполняю после каждого внесения изменений. Начал с базового, на текущий момент расширил возможные узкие места в конфигурации, но положительных изменений в моей проблеме не произошло.

worker_processes  auto;

timer_resolution 100ms;
worker_priority -15;
worker_rlimit_nofile 200000;

error_log  /opt/angie/error.log debug;

events {
    worker_connections  100000;
    use epoll;
    multi_accept on; 
}

http {

    include       mime.types;
    default_type  application/octet-stream;

    sendfile            on;
    tcp_nodelay         off;
    tcp_nopush          on;

    keepalive_timeout 30 15;
    reset_timedout_connection on;
    send_timeout 120;

    server_tokens off;

    server_names_hash_bucket_size 256;
    server_names_hash_max_size 1024;

    proxy_next_upstream error invalid_header timeout http_500 http_502 http_504;

    gzip on;
    gzip_http_version 1.1;
    gzip_comp_level 5;
    gzip_min_length 1100;
    gzip_buffers 256 32k;
    gzip_types
        # text/html is always compressed by HttpGzipModule
        text/css
        text/javascript
        text/xml
        text/plain
        text/x-component
        application/javascript
        application/x-javascript
        application/json
        application/xml
        application/rss+xml
        application/atom+xml
        font/truetype
        font/opentype
        application/vnd.ms-fontobject
        image/svg+xml;
    gzip_static on;
    gzip_proxied any;
    gzip_proxied            expired no-cache no-store private auth;
    gzip_disable "msie6";
    gzip_vary on;


        proxy_cache_path /tmp/ram/ levels=1:2 use_temp_path=off keys_zone=level-1:20m max_size=26000m inactive=1440m;
        proxy_temp_path /etc/angie/proxy_temp;
        proxy_cache_key $server_name$request_uri;

    proxy_hide_header X-Powered-By;
    add_header X-Penguin "Penguin";
    client_body_buffer_size 128k;
    large_client_header_buffers 8 128k;

    proxy_redirect   off;
    proxy_set_header Range "";
    proxy_set_header Connection "";
    proxy_connect_timeout 15;
    proxy_send_timeout    40;
    proxy_read_timeout    66;
    proxy_headers_hash_max_size 1024;
    proxy_headers_hash_bucket_size 128;
    proxy_http_version 1.1;
    proxy_buffer_size   128k;
    proxy_buffers   8 256k;
    proxy_busy_buffers_size   256k;
    proxy_max_temp_file_size 0;

    open_file_cache max=200000 inactive=20s;
    open_file_cache_valid 30s;
    open_file_cache_min_uses 2;
    open_file_cache_errors on;


}

Текущие параметры ядра такие:

net.core.bpf_jit_enable = 1
net.core.bpf_jit_harden = 1
net.core.bpf_jit_kallsyms = 0
net.core.busy_poll = 0
net.core.busy_read = 0
net.core.default_qdisc = pfifo_fast
net.core.dev_weight = 64
net.core.dev_weight_rx_bias = 1
net.core.dev_weight_tx_bias = 1
net.core.message_burst = 10
net.core.message_cost = 5
net.core.netdev_budget = 600
net.core.netdev_max_backlog = 500000
net.core.netdev_rss_key = 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
net.core.netdev_tstamp_prequeue = 1
net.core.optmem_max = 40960
net.core.rmem_default = 8388608
net.core.rmem_max = 33554432
net.core.rps_sock_flow_entries = 0
net.core.somaxconn = 65535
net.core.warnings = 1
net.core.wmem_default = 4194394
net.core.wmem_max = 33554432
net.core.xfrm_acq_expires = 30
net.core.xfrm_aevent_etime = 10
net.core.xfrm_aevent_rseqth = 2
net.core.xfrm_larval_drop = 1

Не сказал бы что соединений очень много. Сейчас в рамках теста генерирую их с помощью siege.

# netstat -n | awk '/^tcp/ {++S[$NF]} END {for(a in S) print a, S[a]}'
SYN_RECV 2
ESTABLISHED 4
FIN_WAIT1 45
TIME_WAIT 170

Основной error log выглядит следующим образом:

2023/12/11 15:26:44 [debug] 24910#24910: worker cycle
2023/12/11 15:26:44 [debug] 24920#24920: epoll timer: -1
2023/12/11 15:26:44 [debug] 24882#24882: epoll timer: -1
2023/12/11 15:26:44 [debug] 24894#24894: timer delta: 0
2023/12/11 15:26:44 [debug] 24916#24916: worker cycle
2023/12/11 15:26:44 [debug] 24890#24890: epoll timer: -1
2023/12/11 15:26:44 [debug] 24921#24921: epoll: fd:7 ev:0001 d:00007F1D1B10C010
2023/12/11 15:26:44 [debug] 24902#24902: worker cycle
2023/12/11 15:26:44 [debug] 24910#24910: epoll timer: -1
2023/12/11 15:26:44 [debug] 24894#24894: worker cycle
2023/12/11 15:26:44 [debug] 24916#24916: epoll timer: -1
2023/12/11 15:26:44 [debug] 24902#24902: epoll timer: -1
2023/12/11 15:26:44 [debug] 24921#24921: timer delta: 0
2023/12/11 15:26:44 [debug] 24894#24894: epoll timer: -1
2023/12/11 15:26:44 [debug] 24921#24921: worker cycle
2023/12/11 15:26:44 [debug] 24921#24921: epoll timer: -1
2023/12/11 15:26:44 [debug] 24915#24915: epoll: fd:7 ev:0001 d:00007F1D1B10C010
2023/12/11 15:26:44 [debug] 24915#24915: timer delta: 0
2023/12/11 15:26:44 [debug] 24915#24915: worker cycle
2023/12/11 15:26:44 [debug] 24915#24915: epoll timer: -1
2023/12/11 15:26:44 [debug] 24914#24914: epoll: fd:7 ev:0001 d:00007F1D1B10C010
2023/12/11 15:26:44 [debug] 24912#24912: epoll: fd:7 ev:0001 d:00007F1D1B10C010
2023/12/11 15:26:44 [debug] 24906#24906: epoll: fd:7 ev:0001 d:00007F1D1B10C010
2023/12/11 15:26:44 [debug] 24914#24914: timer delta: 0
2023/12/11 15:26:44 [debug] 24912#24912: timer delta: 0
2023/12/11 15:26:44 [debug] 24912#24912: worker cycle
2023/12/11 15:26:44 [debug] 24906#24906: timer delta: 0
2023/12/11 15:26:44 [debug] 24914#24914: worker cycle
2023/12/11 15:26:44 [debug] 24912#24912: epoll timer: -1
2023/12/11 15:26:44 [debug] 24906#24906: worker cycle
2023/12/11 15:26:44 [debug] 24914#24914: epoll timer: -1
2023/12/11 15:26:44 [debug] 24906#24906: epoll timer: -1
2023/12/11 15:26:44 [debug] 24911#24911: epoll: fd:7 ev:0001 d:00007F1D1B10C010
2023/12/11 15:26:44 [debug] 24909#24909: epoll: fd:7 ev:0001 d:00007F1D1B10C010
2023/12/11 15:26:44 [debug] 24911#24911: timer delta: 0
2023/12/11 15:26:44 [debug] 24911#24911: worker cycle
2023/12/11 15:26:44 [debug] 24911#24911: epoll timer: -1
2023/12/11 15:26:44 [debug] 24905#24905: epoll: fd:7 ev:0001 d:00007F1D1B10C010
2023/12/11 15:26:44 [debug] 24909#24909: timer delta: 0
2023/12/11 15:26:44 [debug] 24909#24909: worker cycle
2023/12/11 15:26:44 [debug] 24909#24909: epoll timer: -1
2023/12/11 15:26:44 [debug] 24907#24907: epoll: fd:7 ev:0001 d:00007F1D1B10C010
2023/12/11 15:26:44 [debug] 24905#24905: timer delta: 0
2023/12/11 15:26:44 [debug] 24905#24905: worker cycle
2023/12/11 15:26:44 [debug] 24905#24905: epoll timer: -1
2023/12/11 15:26:44 [debug] 24908#24908: epoll: fd:7 ev:0001 d:00007F1D1B10C010

error log виртхоста:

2023/12/11 15:27:51 [debug] 24918#24918: *46773 http cl:-1 max:629145600
2023/12/11 15:27:51 [debug] 24918#24918: *46773 rewrite phase: 3
2023/12/11 15:27:51 [debug] 24918#24918: *46773 post rewrite phase: 4
2023/12/11 15:27:51 [debug] 24918#24918: *46773 generic phase: 5
2023/12/11 15:27:51 [debug] 24918#24918: *46773 generic phase: 6
2023/12/11 15:27:51 [debug] 24918#24918: *46773 generic phase: 7
2023/12/11 15:27:51 [debug] 24918#24918: *46773 access phase: 8
2023/12/11 15:27:51 [debug] 24918#24918: *46773 access phase: 9
2023/12/11 15:27:51 [debug] 24918#24918: *46773 access phase: 10
2023/12/11 15:27:51 [debug] 24918#24918: *46773 post access phase: 11
2023/12/11 15:27:51 [debug] 24918#24918: *46773 generic phase: 12
2023/12/11 15:27:51 [debug] 24918#24918: *46773 generic phase: 13
2023/12/11 15:27:51 [debug] 24907#24907: accept on 0.0.0.0:80, ready: 1
2023/12/11 15:27:51 [debug] 24904#24904: accept on 0.0.0.0:80, ready: 1
2023/12/11 15:27:51 [debug] 24904#24904: accept() not ready (11: Resource temporarily unavailable)
2023/12/11 15:27:51 [debug] 24916#24916: accept on 0.0.0.0:80, ready: 1
2023/12/11 15:27:51 [debug] 24916#24916: accept() not ready (11: Resource temporarily unavailable)
2023/12/11 15:27:51 [debug] 24899#24899: accept on 0.0.0.0:80, ready: 1
2023/12/11 15:27:51 [debug] 24899#24899: accept() not ready (11: Resource temporarily unavailable)
2023/12/11 15:27:51 [debug] 24907#24907: accept() not ready (11: Resource temporarily unavailable)
2023/12/11 15:27:51 [debug] 24894#24894: accept on 0.0.0.0:80, ready: 1
2023/12/11 15:27:51 [debug] 24894#24894: accept() not ready (11: Resource temporarily unavailable)
2023/12/11 15:27:51 [debug] 24903#24903: accept on 0.0.0.0:80, ready: 1
2023/12/11 15:27:51 [debug] 24900#24900: accept on 0.0.0.0:80, ready: 1
2023/12/11 15:27:51 [debug] 24903#24903: accept() not ready (11: Resource temporarily unavailable)
2023/12/11 15:27:51 [debug] 24896#24896: accept on 0.0.0.0:80, ready: 1
2023/12/11 15:27:51 [debug] 24900#24900: accept() not ready (11: Resource temporarily unavailable)
2023/12/11 15:27:51 [debug] 24896#24896: accept() not ready (11: Resource temporarily unavailable)
2023/12/11 15:27:51 [debug] 24905#24905: accept on 0.0.0.0:80, ready: 1
2023/12/11 15:27:51 [debug] 24882#24882: accept on 0.0.0.0:80, ready: 1
2023/12/11 15:27:51 [debug] 24905#24905: accept() not ready (11: Resource temporarily unavailable)
2023/12/11 15:27:51 [debug] 24882#24882: accept() not ready (11: Resource temporarily unavailable)
2023/12/11 15:27:51 [debug] 24897#24897: accept on 0.0.0.0:80, ready: 1
2023/12/11 15:27:51 [debug] 24897#24897: accept() not ready (11: Resource temporarily unavailable)
2023/12/11 15:27:51 [debug] 24909#24909: *46774 posix_memalign: 0000000002058620:4096 @16
2023/12/11 15:27:51 [debug] 24909#24909: *46774 http process request header line
2023/12/11 15:27:51 [debug] 24912#24912: accept on 0.0.0.0:80, ready: 1
2023/12/11 15:27:51 [debug] 24909#24909: *46774 http header: "Host: example.com"
2023/12/11 15:27:51 [debug] 24909#24909: *46774 http header: "Cookie: 5a6a0e72b480etqabe2f48740cb13116=cb7535766f1537eec505b88c3497b569"
2023/12/11 15:27:51 [debug] 24909#24909: *46774 http header: "Accept: */*"
2023/12/11 15:27:51 [debug] 24912#24912: accept() not ready (11: Resource temporarily unavailable)
2023/12/11 15:27:51 [debug] 24909#24909: *46774 http header: "Accept-Encoding: gzip, deflate"
2023/12/11 15:27:51 [debug] 24909#24909: *46774 http header: "User-Agent: Mozilla/5.0 (redhat-x86_64-linux-gnu) Siege/4.1.1"
2023/12/11 15:27:51 [debug] 24909#24909: *46774 http header: "Connection: close"
2023/12/11 15:27:51 [debug] 24909#24909: *46774 http header done
2023/12/11 15:27:51 [debug] 24909#24909: *46774 event timer del: 3: 45886583424
2023/12/11 15:27:51 [debug] 24909#24909: *46774 generic phase: 0
2023/12/11 15:27:51 [debug] 24909#24909: *46774 rewrite phase: 1

Вывод netstat это с сервера или с клиентской машины? Нам нужно увидеть вывод netstat с машины с Angie.
Можете оставить в конфиге только основной error log, а остальные закомментировать? Воспроизвести и прислать полный лог.
Было бы хорошо, как вас просили, все же увидеть полный конфиг (sudo angie -T).

Прошу прощения за длительный ответ. Вывод netstat выше был с сервера на котором установлен Angie.

angie -T
angie: the configuration file /etc/angie/angie.conf syntax is ok
angie: configuration file /etc/angie/angie.conf test is successful
# configuration file /etc/angie/angie.conf:
worker_processes  auto;

error_log  /opt/angie/error.log debug;


events {

}

http {

    server {
        listen 80;
        server_name  example.com;

       error_log /opt/angie/example_error.log debug;


        location / {
                proxy_pass      http://10.1.85.14;
        }


}


}

Основной лог:

2023/12/12 10:03:47 [debug] 87620#87620: worker cycle
2023/12/12 10:03:47 [debug] 87609#87609: channel command: 1
2023/12/12 10:03:47 [debug] 87620#87620: epoll timer: -1
2023/12/12 10:03:47 [debug] 87609#87609: get channel s:39 pid:87627 fd:38
2023/12/12 10:03:47 [debug] 87609#87609: channel: -2
2023/12/12 10:03:47 [debug] 87609#87609: timer delta: 1
2023/12/12 10:03:47 [debug] 87609#87609: worker cycle
2023/12/12 10:03:47 [debug] 87609#87609: epoll timer: -1
2023/12/12 10:03:47 [debug] 87613#87613: worker cycle
2023/12/12 10:03:47 [debug] 87613#87613: epoll timer: -1
2023/12/12 10:04:08 [debug] 87625#87625: epoll: fd:7 ev:0001 d:00000000027AE8F0
2023/12/12 10:04:08 [debug] 87619#87619: epoll: fd:7 ev:0001 d:00000000027AE8F0
2023/12/12 10:04:08 [debug] 87621#87621: epoll: fd:7 ev:0001 d:00000000027AE8F0
2023/12/12 10:04:08 [debug] 87620#87620: epoll: fd:7 ev:0001 d:00000000027AE8F0
2023/12/12 10:04:08 [debug] 87612#87612: epoll: fd:7 ev:0001 d:00000000027AE8F0
2023/12/12 10:04:08 [debug] 87626#87626: epoll: fd:7 ev:0001 d:00000000027AE8F0
2023/12/12 10:04:08 [debug] 87618#87618: epoll: fd:7 ev:0001 d:00000000027AE8F0
2023/12/12 10:04:08 [debug] 87613#87613: epoll: fd:7 ev:0001 d:00000000027AE8F0
2023/12/12 10:04:08 [debug] 87617#87617: epoll: fd:7 ev:0001 d:00000000027AE8F0
2023/12/12 10:04:08 [debug] 87615#87615: epoll: fd:7 ev:0001 d:00000000027AE8F0
2023/12/12 10:04:08 [debug] 87608#87608: epoll: fd:7 ev:0001 d:00000000027AE8F0
2023/12/12 10:04:08 [debug] 87605#87605: epoll: fd:7 ev:0001 d:00000000027AE8F0
2023/12/12 10:04:08 [debug] 87601#87601: epoll: fd:7 ev:0001 d:00000000027AE8F0
2023/12/12 10:04:08 [debug] 87597#87597: epoll: fd:7 ev:0001 d:00000000027AE8F0
2023/12/12 10:04:08 [debug] 87596#87596: epoll: fd:7 ev:0001 d:00000000027AE8F0
2023/12/12 10:04:08 [debug] 87626#87626: timer delta: 21111
2023/12/12 10:04:08 [debug] 87618#87618: timer delta: 21111
2023/12/12 10:04:08 [debug] 87586#87586: epoll: fd:7 ev:0001 d:00000000027AE8F0
2023/12/12 10:04:08 [debug] 87613#87613: timer delta: 21112
2023/12/12 10:04:08 [debug] 87612#87612: timer delta: 21112
2023/12/12 10:04:08 [debug] 87617#87617: timer delta: 21111
2023/12/12 10:04:08 [debug] 87627#87627: epoll: fd:7 ev:0001 d:00000000027AE8F0
2023/12/12 10:04:08 [debug] 87626#87626: worker cycle
2023/12/12 10:04:08 [debug] 87618#87618: worker cycle
2023/12/12 10:04:08 [debug] 87613#87613: worker cycle
2023/12/12 10:04:08 [debug] 87608#87608: timer delta: 21112
2023/12/12 10:04:08 [debug] 87615#87615: timer delta: 21112
2023/12/12 10:04:08 [debug] 87612#87612: worker cycle
2023/12/12 10:04:08 [debug] 87617#87617: worker cycle
2023/12/12 10:04:08 [debug] 87621#87621: timer delta: 21111
2023/12/12 10:04:08 [debug] 87605#87605: timer delta: 21112
2023/12/12 10:04:08 [debug] 87601#87601: timer delta: 21111
2023/12/12 10:04:08 [debug] 87597#87597: timer delta: 21111
2023/12/12 10:04:08 [debug] 87620#87620: timer delta: 21111
2023/12/12 10:04:08 [debug] 87596#87596: timer delta: 21112
2023/12/12 10:04:08 [debug] 87586#87586: timer delta: 21112
2023/12/12 10:04:08 [debug] 87626#87626: epoll timer: -1
2023/12/12 10:04:08 [debug] 87618#87618: epoll timer: -1
2023/12/12 10:04:08 [debug] 87613#87613: epoll timer: -1
2023/12/12 10:04:08 [debug] 87608#87608: worker cycle
2023/12/12 10:04:08 [debug] 87615#87615: worker cycle
2023/12/12 10:04:08 [debug] 87612#87612: epoll timer: -1
2023/12/12 10:04:08 [debug] 87617#87617: epoll timer: -1
2023/12/12 10:04:08 [debug] 87621#87621: worker cycle
2023/12/12 10:04:08 [debug] 87605#87605: worker cycle
2023/12/12 10:04:08 [debug] 87601#87601: worker cycle
2023/12/12 10:04:08 [debug] 87597#87597: worker cycle
2023/12/12 10:04:08 [debug] 87620#87620: worker cycle
2023/12/12 10:04:08 [debug] 87596#87596: worker cycle
2023/12/12 10:04:08 [debug] 87586#87586: worker cycle
2023/12/12 10:04:08 [debug] 87627#87627: timer delta: 21130
2023/12/12 10:04:08 [debug] 87608#87608: epoll timer: -1
2023/12/12 10:04:08 [debug] 87615#87615: epoll timer: -1
2023/12/12 10:04:08 [debug] 87621#87621: epoll timer: 60000
2023/12/12 10:04:08 [debug] 87605#87605: epoll timer: -1
2023/12/12 10:04:08 [debug] 87601#87601: epoll timer: -1
2023/12/12 10:04:08 [debug] 87597#87597: epoll timer: -1
2023/12/12 10:04:08 [debug] 87620#87620: epoll timer: 60000
2023/12/12 10:04:08 [debug] 87623#87623: epoll: fd:7 ev:0001 d:00000000027AE8F0
2023/12/12 10:04:08 [debug] 87596#87596: epoll timer: -1
2023/12/12 10:04:08 [debug] 87586#87586: epoll timer: -1
2023/12/12 10:04:08 [debug] 87627#87627: worker cycle
2023/12/12 10:04:08 [debug] 87621#87621: epoll: fd:16 ev:0001 d:00000000027AEAD0
2023/12/12 10:04:08 [debug] 87620#87620: epoll: fd:18 ev:0001 d:00000000027AEAD0
2023/12/12 10:04:08 [debug] 87625#87625: timer delta: 21111
2023/12/12 10:04:08 [debug] 87619#87619: timer delta: 21111
2023/12/12 10:04:08 [debug] 87623#87623: timer delta: 21111
2023/12/12 10:04:08 [debug] 87627#87627: epoll timer: -1
2023/12/12 10:04:08 [debug] 87625#87625: worker cycle
2023/12/12 10:04:08 [debug] 87619#87619: worker cycle
2023/12/12 10:04:08 [debug] 87623#87623: worker cycle
2023/12/12 10:04:08 [debug] 87625#87625: epoll timer: -1
2023/12/12 10:04:08 [debug] 87619#87619: epoll timer: -1
2023/12/12 10:04:08 [debug] 87623#87623: epoll timer: -1
2023/12/12 10:04:08 [debug] 87621#87621: timer delta: 0
2023/12/12 10:04:08 [debug] 87620#87620: timer delta: 0

Лог с виртхоста:

2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write buf ls:1 00000000027E9970 4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write buf ls:1 00000000027E7950 4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write: out:000000000278DCE0, f:1
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http output filter "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http copy filter: "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http postpone filter "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88" 000000000278F7C0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 write new buf t:1 f:0 000000000278DD10, pos 000000000278DD10, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 write new buf t:1 f:0 00000000027E9970, pos 00000000027E9970, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter: l:0 f:1 s:8192
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter limit 2097152
2023/12/12 10:05:55 [debug] 87617#87617: *1340 writev: 8192 of 8192
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter 0000000000000000
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http copy filter: 0 "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write busy: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write buf ls:1 00000000027E7950 4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write: out:000000000278F2D0, f:0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http output filter "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http copy filter: "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http postpone filter "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88" 000000000278F2D0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 write new buf t:1 f:0 00000000027E7950, pos 00000000027E7950, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter: l:0 f:1 s:4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter limit 2097152
2023/12/12 10:05:55 [debug] 87617#87617: *1340 writev: 4096 of 4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter 0000000000000000
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http copy filter: 0 "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write busy: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write: out:0000000000000000, f:0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe read upstream: 1
2023/12/12 10:05:55 [debug] 87617#87617: *1340 readv: eof:1, avail:131981
2023/12/12 10:05:55 [debug] 87617#87617: *1340 readv: 9, last:4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 readv: avail:95117
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe recv chain: 36864
2023/12/12 10:05:55 [debug] 87617#87617: *1340 input buf #130
2023/12/12 10:05:55 [debug] 87617#87617: *1340 input buf #131
2023/12/12 10:05:55 [debug] 87617#87617: *1340 input buf #132
2023/12/12 10:05:55 [debug] 87617#87617: *1340 input buf #133
2023/12/12 10:05:55 [debug] 87617#87617: *1340 input buf #134
2023/12/12 10:05:55 [debug] 87617#87617: *1340 input buf #135
2023/12/12 10:05:55 [debug] 87617#87617: *1340 input buf #136
2023/12/12 10:05:55 [debug] 87617#87617: *1340 input buf #137
2023/12/12 10:05:55 [debug] 87617#87617: *1340 input buf #138
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe downstream ready
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe buf in   s:1 t:1 f:0 00000000027E7950, pos 00000000027E7950, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe buf in   s:1 t:1 f:0 000000000278DD10, pos 000000000278DD10, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe buf in   s:1 t:1 f:0 00000000027E9970, pos 00000000027E9970, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe buf in   s:1 t:1 f:0 00000000027E5930, pos 00000000027E5930, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe buf in   s:1 t:1 f:0 00000000027EB990, pos 00000000027EB990, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe buf in   s:1 t:1 f:0 00000000027E6940, pos 00000000027E6940, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe buf in   s:1 t:1 f:0 00000000027EA980, pos 00000000027EA980, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe buf in   s:1 t:1 f:0 00000000027E4920, pos 00000000027E4920, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe buf in   s:1 t:1 f:0 00000000027E8960, pos 00000000027E8960, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe length: 95117
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write downstream: 1
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write busy: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write buf ls:1 00000000027E7950 4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write buf ls:1 000000000278DD10 4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write buf ls:1 00000000027E9970 4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write: out:000000000278F2D0, f:1
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http output filter "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http copy filter: "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http postpone filter "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88" 000000000278F3B0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 write new buf t:1 f:0 00000000027E7950, pos 00000000027E7950, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 write new buf t:1 f:0 000000000278DD10, pos 000000000278DD10, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter: l:0 f:1 s:8192
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter limit 2097152
2023/12/12 10:05:55 [debug] 87617#87617: *1340 writev: 8192 of 8192
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter 0000000000000000
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http copy filter: 0 "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write busy: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write buf ls:1 00000000027E9970 4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write buf ls:1 00000000027E5930 4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write buf ls:1 00000000027EB990 4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write: out:000000000278DCE0, f:1
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http output filter "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http copy filter: "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http postpone filter "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88" 000000000278F780
2023/12/12 10:05:55 [debug] 87617#87617: *1340 write new buf t:1 f:0 00000000027E9970, pos 00000000027E9970, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 write new buf t:1 f:0 00000000027E5930, pos 00000000027E5930, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter: l:0 f:1 s:8192
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter limit 2097152
2023/12/12 10:05:55 [debug] 87617#87617: *1340 writev: 8192 of 8192
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter 0000000000000000
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http copy filter: 0 "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write busy: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write buf ls:1 00000000027EB990 4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write buf ls:1 00000000027E6940 4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write buf ls:1 00000000027EA980 4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write: out:000000000278DC10, f:1
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http output filter "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http copy filter: "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http postpone filter "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88" 000000000278F7A0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 write new buf t:1 f:0 00000000027EB990, pos 00000000027EB990, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 write new buf t:1 f:0 00000000027E6940, pos 00000000027E6940, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter: l:0 f:1 s:8192
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter limit 2097152
2023/12/12 10:05:55 [debug] 87617#87617: *1340 writev: 8192 of 8192
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter 0000000000000000
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http copy filter: 0 "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write busy: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write buf ls:1 00000000027EA980 4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write buf ls:1 00000000027E4920 4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write buf ls:1 00000000027E8960 4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write: out:000000000278F3D0, f:1
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http output filter "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http copy filter: "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http postpone filter "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88" 000000000278F7C0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 write new buf t:1 f:0 00000000027EA980, pos 00000000027EA980, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 write new buf t:1 f:0 00000000027E4920, pos 00000000027E4920, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter: l:0 f:1 s:8192
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter limit 2097152
2023/12/12 10:05:55 [debug] 87617#87617: *1340 writev: 8192 of 8192
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter 0000000000000000
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http copy filter: 0 "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write busy: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write buf ls:1 00000000027E8960 4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write: out:000000000278D7D8, f:0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http output filter "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http copy filter: "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http postpone filter "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88" 000000000278D7D8
2023/12/12 10:05:55 [debug] 87617#87617: *1340 write new buf t:1 f:0 00000000027E8960, pos 00000000027E8960, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter: l:0 f:1 s:4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter limit 2097152
2023/12/12 10:05:55 [debug] 87617#87617: *1340 writev: 4096 of 4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter 0000000000000000
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http copy filter: 0 "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write busy: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write: out:0000000000000000, f:0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe read upstream: 1
2023/12/12 10:05:55 [debug] 87617#87617: *1340 readv: eof:1, avail:95117
2023/12/12 10:05:55 [debug] 87617#87617: *1340 readv: 9, last:4096
2023/12/12 10:05:55 [debug] 87617#87617: *1340 readv: avail:58253
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe recv chain: 36864
2023/12/12 10:05:55 [debug] 87617#87617: *1340 input buf #139
2023/12/12 10:05:55 [debug] 87617#87617: *1340 input buf #140
2023/12/12 10:05:55 [debug] 87617#87617: *1340 input buf #141
2023/12/12 10:05:55 [debug] 87617#87617: *1340 input buf #142
2023/12/12 10:05:55 [debug] 87617#87617: *1340 input buf #143
2023/12/12 10:05:55 [debug] 87617#87617: *1340 input buf #144
2023/12/12 10:05:55 [debug] 87617#87617: *1340 input buf #145
2023/12/12 10:05:55 [debug] 87617#87617: *1340 input buf #146
2023/12/12 10:05:55 [debug] 87627#87627: accept on 0.0.0.0:80, ready: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 input buf #147
2023/12/12 10:05:55 [debug] 87623#87623: accept on 0.0.0.0:80, ready: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe downstream ready
2023/12/12 10:05:55 [debug] 87625#87625: accept on 0.0.0.0:80, ready: 0
2023/12/12 10:05:55 [debug] 87621#87621: accept on 0.0.0.0:80, ready: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe buf in   s:1 t:1 f:0 00000000027E8960, pos 00000000027E8960, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe buf in   s:1 t:1 f:0 00000000027EA980, pos 00000000027EA980, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe buf in   s:1 t:1 f:0 00000000027E4920, pos 00000000027E4920, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe buf in   s:1 t:1 f:0 00000000027EB990, pos 00000000027EB990, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe buf in   s:1 t:1 f:0 00000000027E6940, pos 00000000027E6940, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe buf in   s:1 t:1 f:0 00000000027E9970, pos 00000000027E9970, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe buf in   s:1 t:1 f:0 00000000027E5930, pos 00000000027E5930, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe buf in   s:1 t:1 f:0 00000000027E7950, pos 00000000027E7950, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe buf in   s:1 t:1 f:0 000000000278DD10, pos 000000000278DD10, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe length: 58253
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write downstream: 1
2023/12/12 10:05:55 [debug] 87612#87612: accept on 0.0.0.0:80, ready: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write busy: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write buf ls:1 00000000027E8960 4096
2023/12/12 10:05:55 [debug] 87610#87610: accept on 0.0.0.0:80, ready: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write buf ls:1 00000000027EA980 4096
2023/12/12 10:05:55 [debug] 87619#87619: accept on 0.0.0.0:80, ready: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write buf ls:1 00000000027E4920 4096
2023/12/12 10:05:55 [debug] 87598#87598: accept on 0.0.0.0:80, ready: 0
2023/12/12 10:05:55 [debug] 87595#87595: accept on 0.0.0.0:80, ready: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 pipe write: out:000000000278D7D8, f:1
2023/12/12 10:05:55 [debug] 87598#87598: accept() not ready (11: Resource temporarily unavailable)
2023/12/12 10:05:55 [debug] 87621#87621: posix_memalign: 0000000002798960:512 @16
2023/12/12 10:05:55 [debug] 87619#87619: accept() not ready (11: Resource temporarily unavailable)
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http output filter "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87595#87595: accept() not ready (11: Resource temporarily unavailable)
2023/12/12 10:05:55 [debug] 87625#87625: accept() not ready (11: Resource temporarily unavailable)
2023/12/12 10:05:55 [debug] 87621#87621: *1342 accept: 100.100.100.100:50748 fd:20
2023/12/12 10:05:55 [debug] 87610#87610: accept() not ready (11: Resource temporarily unavailable)
2023/12/12 10:05:55 [debug] 87623#87623: accept() not ready (11: Resource temporarily unavailable)
2023/12/12 10:05:55 [debug] 87627#87627: accept() not ready (11: Resource temporarily unavailable)
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http copy filter: "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88"
2023/12/12 10:05:55 [debug] 87621#87621: *1342 event timer add: 20: 60000:45953687402
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http postpone filter "/public/lib/base.min.js?v=787094a1bb494b3032871e41939315fb65589a88" 000000000278F760
2023/12/12 10:05:55 [debug] 87612#87612: accept() not ready (11: Resource temporarily unavailable)
2023/12/12 10:05:55 [debug] 87621#87621: *1342 reusable connection: 1
2023/12/12 10:05:55 [debug] 87617#87617: *1340 write new buf t:1 f:0 00000000027E8960, pos 00000000027E8960, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87621#87621: *1342 epoll add event: fd:20 op:1 ev:80002001
2023/12/12 10:05:55 [debug] 87617#87617: *1340 write new buf t:1 f:0 00000000027EA980, pos 00000000027EA980, size: 4096 file: 0, size: 0
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter: l:0 f:1 s:8192
2023/12/12 10:05:55 [debug] 87617#87617: *1340 http write filter limit 2097152
2023/12/12 10:05:55 [debug] 87617#87617: *1340 writev: 8192 of 8192

По ощущениям как будь то упираюсь в какие то буферы. Перестают отвечать все виртхосты только для моего ip адреса.

Давайте попробуем проверить версию с исчерпанием возможных соединений со стороны клиента. Покажите, пожалуйста такой же netstat с клиентской машины во время воспроизведения проблемы. Так же, хорошо бы проверить лимиты на клиенте.

Не думаю что проблема на стороне клиента. Поскольку при воспроизведении проблемы она проявляется на всех устройствах за роутером(его кстати тоже меняли). Интересная ситуация получается на виртуалке, с которой запускается siege. Ей сервер отвечает даже под нагрузкой(можно попутно с siege кидать курлы на которое мгновенно приходит ответ). Другие устройства словно игнорируются.

Другими словами, Вы хотите сказать что проблема может быть на стороне сети? Как вариант, вполне вероятно. Возможно стоит тогда посмотреть в роутер.

Хочу заверить что проблем в сети нет. Тест проводился на разных провайдерах и с роутерами различных вендоров. Каналы при этом не сильно нагружены. Уверен что проблема на стороне сервера с Angie. Косвенно это подтверждает дамп запущенный на этом сервере. При достижении определенного порога с одного ip, сервер перестает отвечать на новые запросы. При этом, как и говорил ранее запросы с других внешних ip обрабатываются в момент нагрузки.
Подозревал наличие лимита, но там все ок

# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 119819
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 99999
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 99999
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

А повторяется проблема с nginx-mainline или nginx-stable?

В представленных логах в принципе нет никаких ошибок связанных с соединениями. Судя по всему, соединения до Angie даже не доходят. Т.е. в самом Angie проблему искать бесполезно, ядро не сообщает о каких-либо новых соединениях. Проблему стоит поискать где-то в сетевой части между Angie и клиентом.

Эти сообщения являются частью нормального функционирования сервера. Они сообщают о том, что новых соединений на слушающем сокете в текущий момент времени нет, сервер принял все поступившие ранее соединения.

Проверил работу с nginx. Проблема воспроизводится. Со стороны сети там обычный инет, без какой либо защиты от ддоса. Белый ip висит на самом сервере. Наличие/отсутствие тюна ядра не проблему не влияет. В iptables лимитов нет. Странно, tcpdump показывает что клиентские запросы приходят на сервер, но ответа на них не поступает. Мог я что то упустить и не учесть со стороны сервера?