Skip to content

Conversation

@u5surf
Copy link
Contributor

@u5surf u5surf commented Nov 1, 2022

  • the location which includes %le sequence only skip %
  • thus the case of se%lect just becomes select.

The sequence has reached`%`, it skips and breaks in L.1668 and it scans next character.

The next character is l, it unstatisfies 0-9a-f, then the state has become sw_usual and the l has stored the dst in L.1691-L.1693.
Finally, se%lect becomes simply select.

tengine/src/core/ngx_string.c

Lines 1657 to 1695 in c2e166a

switch (state) {
case sw_usual:
if (ch == '?'
&& (type & (NGX_UNESCAPE_URI|NGX_UNESCAPE_REDIRECT)))
{
*d++ = ch;
goto done;
}
if (ch == '%') {
state = sw_quoted;
break;
}
*d++ = ch;
break;
case sw_quoted:
if (ch >= '0' && ch <= '9') {
decoded = (u_char) (ch - '0');
state = sw_quoted_second;
break;
}
c = (u_char) (ch | 0x20);
if (c >= 'a' && c <= 'f') {
decoded = (u_char) (c - 'a' + 10);
state = sw_quoted_second;
break;
}
/* the invalid quoted character */
state = sw_usual;
*d++ = ch;
break;

Signed-off-by: u5surf [email protected]

@chobits
Copy link
Member

chobits commented Nov 3, 2022

Hi, I am also looking at this case. Then you can remove this case directly because it is deprecated. See following:

(link commit ca58fa3)

This test cases was used to test old tengine feature, as following:

```
    commit ca58fa3a038cd932e8152d9307be07f7918b5260
    Author: Weibin Yao <[email protected]>
    Date:   Wed Apr 17 12:10:42 2013 +0800
    fixed the missing with invalid character for ngx_unescape_uri
```

After we removed this feature when we upgraded tengine core to
nginx-1.8.x, the case string.t is deprecated.

@chobits
Copy link
Member

chobits commented Nov 3, 2022

also note that , if you cannot pass tests/nginx-tests/tengine-tests/ngx_dtls.t for some reason, you can just skip it in ci.yml.
It is a standalone project for DTLS https://github.com/Tongsuo-Project/Tongsuo, which should be tested by their team

I think there should be a standalone CI/CD workflow for dtls.

@u5surf u5surf changed the title test: Fix invalid unescaped test. test: Remove string.t Nov 3, 2022
@u5surf
Copy link
Contributor Author

u5surf commented Nov 3, 2022

@chobits I removed string.t in this PR. Can you check again please?

@chobits chobits merged commit 0523f9a into alibaba:master Nov 3, 2022
@chobits
Copy link
Member

chobits commented Nov 3, 2022

Thanks, ack

@chobits chobits added this to the 2.3.5 milestone Nov 5, 2022
@chobits chobits modified the milestones: 2.3.5, 2.4.0 Feb 8, 2023
Homqyy added a commit to Homqyy/hengine that referenced this pull request Jul 8, 2023
* updated ngx_http_upstream_check_module_cn.md 

fixed configure command

* bugfix:  Filter status is down, check_status/API returns JSON error.

curl -s "http://127.0.0.1/?format=json&status=down" 

returns JSON error
{
"servers": {
  "total": 2,
  "generation": 1,
  "server": [
    {"index": 35, "upstream": "www", "name": "127.0.0.1:8080", "status": "down", "rise": 0, "fall": 57, "type": "http", "port": 0},
    {"index": 123, "upstream": "task", "name": "127.0.0.1:8081", "status": "down", "rise": 0, "fall": 56, "type": "http", "port": 0},
  ]
}}

* add build feature test: compiler structure-packing pragma

* Add vnswrr algorithm for upstream module.

* add http dubbo docs

* feature: http to dubbo backend support

* bugfix: ensured 'init_worker_by_lua*' does not mutate another NGINX module's main_conf. (#1326)

* Fix typo in README

Signed-off-by: 林博仁(Buo-ren Lin) <[email protected]>

* bugfix: repaired compilation error of upstream_check_module.

* fixed dyups module build error when compiled with a higher version of OpenSSL.

* sync changes and fix for dubbo

* add dubbo quick start

* Updated core file to Nginx-1.17.3.

* fix build error on some os

* updated ngx_http_proxy_connect_module to v0.0.2.

* Modify some files for release 2.3.2 version.

* Bugfix: maybe caused traffic uneven when some peers are unavailable.

* fix build on alpine

* fix build on freebsd

* update docs for dubbo

* fix some for dubbo dynamic module

* Bugfix: avoid limit_req_zone directive in multiple variables problems.

* Add dingtalk user group

* Bugfix: vnswrr adaptated to dynamic_resolve module.

* Bugfix: a segmentation fault might occur in master process.

* bugfix: POST request with keepalive peer cause 400 response

When the POST data sent is incomplete, the peer cannot be reused and should be discarded, otherwise the upstream server will discard some data from next request which use the same peer

* Delete unused variable

Delete unused variable.

* Add the total number of up/down nodes to json format

* Add support for Prometheus format

* Tolerate '\0' in URI when mapping URI to path.

If a rewritten URI has the null character, only a part of URI was
copied to a memory buffer allocated for path.  In some setups this
could be exploited to expose uninitialized memory via the Location
header.

* when dyups and health check module together use, ngx_shmtx_lock block too long time and cpu full load, cause health check timeout, 502

* Feature: Support DTLS offload

* Feature: Add dtls test

* Bugfix: remove redundant upstream helath check.

* Bugfix: modify upstream_check_module, fix upstream share memory bug.

* bugfix: memory leak of debug log

* Chore: remove duplicate c->log init

The same code repeats in
https://github.com/alibaba/tengine/blob/27cac3a9d84666885c066945de3612b006e467ba/modules/ngx_multi_upstream_module/ngx_http_multi_upstream_module.c#L485

* Fix(ngx_multi_upstream_module): fake request isn't freed

* Fix(ngx_multi_upstream_module): use a new way to free fake_r

* Feat(dubbo): allow using variable inside dubbo_pass

Now we can configure `dubbo_pass` like:

dubbo_pass $dubbo_service_name $dubbo_service_version $dubbo_method dubbo_backend;

* Docs: allow using variable inside dubbo_pass

* Doc: tweak dubbo module documentation

* fix: don't crash when upstream returns integer status code

* fix: correct the hessian2 decode for int32

The tag is in [0, 255]. Correct it according to the rule described
in https://zhuanlan.zhihu.com/p/44787200

* Updated core file to Nginx-1.18.0.

* Update some files for release tengine-2.3.3.

* Support dynamic build for ngx_http_upstream_session_sticky_module

* Fix outdated http_concat module installation documentation

The `--with-http_concat_module` configure option is no longer
available since Tengine 2.3.0.

Fixes #1558.

Signed-off-by: 林博仁(Buo-ren Lin) <[email protected]>

* fix: build error on ubuntu 20.04 (#1566)

* Fix(ngx_http_upstream_check_module): ngx_feature_libs need to be cleared when compiled with some 3rdparty modules

* fix death lock when use directive "zone" in upstream block

* fix death lock when use directive "zone" in upstream block (#1587).

* Support NTLS (TLCP and GM/T 0024-2014), based on BabaSSL

Add ngx_openssl_ntls module.
Support NTLS for http_ssl and stream_ssl modules.

* Upgrade ngx_openssl_ntls module from BabaSSL to Tongsuo.

* Bugfix: fix ngx_debug_pool module clang compile err

* Fix(ngx_http_upstream_dyups_module): If a domain name contains multiple IP addresses, call them

* Bugfix: r->limit_rate does't work if limit_rate_set variable is not set
nginx-patch: https://hg.nginx.org/nginx/rev/c19ca381b2e6\?revcount\=480

* fixed CVE-2021-23017

* import patch from: https://nginx.org/download/patch.2021.resolver.txt
* updated resolver test cases from official nginx-tests lib

* Modify some files for release 2.3.4 version.

* fixed wrong resetting upstream flags in #T_NGX_HTTP_UPSTREAM_RANDOM (#1660)

* dyups: fixed compilation error without upstream check module

* dyups: fix unlocking behavior

Fix alibaba/tengine#1661

* ci: Support Github Actions

* Modernized the continuous integration of this repository.
* Also checks it enables to build in ubuntu 20.04

Signed-off-by: u5surf <[email protected]>

* test case concat.t: fixed compilation error

* test case consistent_hash.t: port() return random port

If "127.0.0.1:8080" is not included in nginx.conf in test case,
function port(8080) in Test::Nginx will fail to test listening on
"127.0.0.1:8080".

* remove deprecated test case and doc for error_page default

* test case: fixed tengine-tests/resolver.t

* resolver_file: new test case tengine-tests/resolver_file.t

* The old test case tengine-tests/resolver.t can only test whether
  the "resolver_file" directive is loaded successfully.
* The new test case can check whether the resolver server loaded from
  "resolver_file" works.

* clear deprecated tengine DSO logic in test cases

* README.markdown: remove deprecated function of error_page default

* VNSWRR reduces memory usage with GCD

* test: Remove string.t

* alibaba/tengine#1673 (comment)

Signed-off-by: u5surf <[email protected]>

* fix: dyups module doc obsoleted

* tengine test case: fixed resolver_file.t

* fixed dynamic_resolver_port.t: cannot listen on dns server port

Use "resolver 127.0.0.01:8xxx" instead of "resolver 127.0.0.1:8xxxx".
Otherwise, port() in Nginx.pm will listen on that port firstly.

* ci: Add tengine tests.

Signed-off-by: u5surf <[email protected]>

* test: Fix unintialized value in test

Signed-off-by: u5surf <[email protected]>

* ci: separate test cases

Signed-off-by: u5surf <[email protected]>

* dyups: fix typo in docs

* tengine test case: fixed ngx_dtls.t

1. fix error: Ssl_ctx_use_certificate key too small
2. make it case insensitive for checking "Verification: OK"

* fixed segfault: conf->ssl.log is NULL for ssl_certificate directive in stream block

* ci.yml: enable ssl module to run more test cases

* ssl proxy module support ntls.

* stream sni: fixed segfault that addr_conf->default_server is NULL

If INET6 is enabled, ngx_stream_add_addrs6 is runned instead of
ngx_stream_add_addrs. The stream sni logic forgot to initialize
addr_conf->default_server in ngx_stream_add_addrs6().

* ci.yml: run tengine test cases using test-nginx lib

1. modified ci.yml: run tengine test cases using test-nginx lib
2. remove deprecated syslog.t and limit_upstream_tries.t
   (this feature is replaced by similar one from nginx
3. fixed user_agent.t: remove ngx_echo dependence via using "return"
   directive

* fixed fastcgi_check.t: cannot listen on dns server port

Use "127.0.0.01:8xxx" instead of "127.0.0.1:8xxx".
Otherwise, port() in Nginx.pm will listen on that port firstly,
which makes fastcgi_daemon() fail to open port.

* fixed reqstat.t: reduce the checking accuracy of ups_rt time

* nginx security advisory (CVE-2022-41741, CVE-2022-41742)

* Rename NGX_STREAM_SNI macro to T_NGX_STREAM_SNI

* Tests: fixed test cases in tests/nginx-tests/nginx-tests/

* dyups: each processes starts read_msg_timer separately at random timeout (#1691)

If every worker process starts timer at the same time, most processes may hang with ngx_shmtx_lock in handler of read_msg_timer.

* SSL: disable T_NGX_HAVE_DTLS when build with boringssl
boringssl missing SSL_CTX_set_cookie_generate_cb

* move stream_sni.t to tengine-tests/

* tengine-tests/resolver_file.t: skipped without root

* [tengine] Resolver: fixed compliation failure when missing /etc/resolv.conf file

* [tengine] StreamSNI: fixed segfault with multi stream server blocks
(issue #1697).

A configuration like

stream {
    server {
        listen      unix:/tmp/nginx-test-YIXVQOSR0a/unix.sock proxy_protocol;
        return      $remote_addr;
    }

    server {
        listen      unix:/tmp/nginx-test-YIXVQOSR0a/unix2.sock proxy_protocol;
        return      $remote_addr;
    }
}

 resulted in a segmentation fault.

* updated ssl_certificates.t to latest version of nginx-tests

* openssl async: fix test case

1. added configuration check for test case
2. move case to tengine-tests/
3. added description of help usage of --with-openssl-async

* ci.yml: enable openssl-async to test this feature

* Add CI workflow for Linux ARM64

* Give unique names to the CI workflows (#1705)

* fixed test fastcgi_check.t

* fixed test case of dynamic resolve: reduce address usage

* ci.yml: modify ci name

* dyups: fixed memory leak of ssl session reuse

* dyups.t: fixed empty response from deleted upstream

* dyups.t: added test cases for ssl session reuse

* session sticky: fixed memory leak of ssl session reuse

It reuses ssl session reuse of round robin module now, and
removes duplicated codes in session sticky module.

* session_sticky.t: added test cases for ssl session reuse

* nginx-tests: sync official uwsgi test cases

uwsgi_body.t and uwsgi_ssl_certificate_vars.t are not synchronized,
which is not supported by currenty Tengine 2.3.x (nginx 1.8.x).

* dynamic resolve: fixed multiple retries for one server (#1712)

If upstream one server has multiple static addresses, dynamic resolve module
will resolve hostname multiple times wrongly with some dns error.

* VNSWRR limit the number of virtual peers when init (#1717)

* added test-nginx-core.yml: test nginx core with nginx-tests cases (#1721)

* docs: VNSWRR limit the number of virtual peers per initialization (#1722)

* VNSWRR limit the number of virtual peers per initialization
* VNSWRR limit the number of virtual peers per initialization
* Describe the rules of `max_init`

* docs: improve grammar (#1726)

* Update to nginx 1.22.1 (#1719)

Update to nginx 1.22.1

---------

Co-authored-by: cuijiuzhou <[email protected]>
Co-authored-by: Xiaochen Wang <[email protected]>

* Fix build error about ngx_ssl_certificate. (#1731)

alibaba/tengine#1730

* Release 2.4.0 version. (#1733)

* ingress module: add ingress module

* add upstream timeout

* updated proxy_connect module to v0.0.4 (#1735)

* updated proxy_connect module to v0.0.4

For more details, see https://github.com/chobits/ngx_http_proxy_connect_module/releases/tag/v0.0.4

* ci.yml: added proxy_connect test cases

* ingress module: add ingress pb-c

* stream log module: add pipe log

* Configure: fixed compilation error in stream_set module (#1749)

It had not imported complete patch of auto/options file
as tengine was updated to nginx-1.22.1.
nginx-1.22.1.

* Bugfix: fix NTLS cert check, move sign/enc certficate to upstream

Update to nginx 1.22.1 (#1719) add reject_handshake, but not fully
consider the NTLS. That's okay if no ssl_certificate is
configured but ssl_sign_certificate and ssl_enc_certificate are
configured when NTLS is enabled.

Move sign_certificate, enc_certificate to upstream to adapt to
nginx 1.22.1.

Rename ngx_openssl_ntls to ngx_tongsuo_ntls.

Add test-nginx-ntls CI.

* tengine:ingress+xquic+xdup

* Create Makefile

* modify makefile for the xquic-xdp module

* xquich and xudp macro

* http xquic NGX_QUIC_CID_ROUTE_FIRST_OCTER

* convert ngx_sockaddr_t to sockaddr for debug mode of xquic-recv

* uninitialized warn about xquic filter

* uninitialized warn about xquic filter

* check ret code for geting chain certificate failed

* add ngx_http_find_virtual_server_inner with alibaba comments

* test-nginx-core.yml: fixed the failure of test case image_filter_finalize.t

For more details, see alibaba/tengine#1767.

* bugfix async ssl_early_data

* HTTP/3: long connection for QUIC request.

* Fixed some test cases that failed to start the DNS daemon (#1783)

1. resolver_file.t: reduced dependency on DNS server listening. 
   * In new github workflow environment, we cannot bind 53 port sometimes. In order to make the case pass, we should remove the dependency on 53 port listening, using debug log to check the logic.
2. proxy_connect_*.t: refactored the logic of the DNS daemon test cases to follow the style of the nginx-tests repository.
   * When attempting to bind UDP ports fails, retry other random ports

* Release 2.4.1 version.

* Version bump for 2.4.1

* HTTP/3: the $xquic_ssl_protocol, $xquic_ssl_cipher and $xquic_ssl_session_reused variables.

* fix tengine coredump bug with ssl_async on. issues#1793

* HTTP/3: get ssl handler in callback of connection establishment

* HTTP/3: only generate random cid without qch currently

* join branch update/* to ci.yml

* delete abandoned testcase script

* delete abandoned testcase script

* delete abandoned modules

* add branche update/* to actions

---------

Signed-off-by: 林博仁(Buo-ren Lin) <[email protected]>
Signed-off-by: u5surf <[email protected]>
Co-authored-by: zhangqx2010 <[email protected]>
Co-authored-by: IYism <[email protected]>
Co-authored-by: MengqiWu <[email protected]>
Co-authored-by: wangfakang <[email protected]>
Co-authored-by: 林博仁(Buo-ren Lin) <[email protected]>
Co-authored-by: Xiaochen Wang <[email protected]>
Co-authored-by: Jony Wang <[email protected]>
Co-authored-by: Jiayuan Chen <[email protected]>
Co-authored-by: zhifeng hu <[email protected]>
Co-authored-by: 董宇 <[email protected]>
Co-authored-by: XiaoLong Hong <[email protected]>
Co-authored-by: Weiliang Li <[email protected]>
Co-authored-by: David Krutsko <[email protected]>
Co-authored-by: Jin Jiu <[email protected]>
Co-authored-by: Zhang Jinde <[email protected]>
Co-authored-by: scriptkids <[email protected]>
Co-authored-by: Hawker <[email protected]>
Co-authored-by: spacewander <[email protected]>
Co-authored-by: Mengqi Wu <[email protected]>
Co-authored-by: lianglli <[email protected]>
Co-authored-by: lianglli <[email protected]>
Co-authored-by: Harry Lin <[email protected]>
Co-authored-by: RocFang <[email protected]>
Co-authored-by: Weibin Yao(姚伟斌) <[email protected]>
Co-authored-by: K1 <[email protected]>
Co-authored-by: taoyuanyuan <[email protected]>
Co-authored-by: Zhang Jinde <[email protected]>
Co-authored-by: zhusy <[email protected]>
Co-authored-by: yefei.dyf <[email protected]>
Co-authored-by: u5surf <[email protected]>
Co-authored-by: jizhuozhi <[email protected]>
Co-authored-by: Ruochen Xu <[email protected]>
Co-authored-by: jun1.huang <[email protected]>
Co-authored-by: harry-xm <[email protected]>
Co-authored-by: litao <[email protected]>
Co-authored-by: 黎叔 <[email protected]>
Co-authored-by: nandsky <[email protected]>
Co-authored-by: Martin Grigorov <[email protected]>
Co-authored-by: jiuzhou <[email protected]>
Co-authored-by: cuijiuzhou <[email protected]>
Co-authored-by: drawing <[email protected]>
Co-authored-by: K1 <[email protected]>
Co-authored-by: ElvaLiu <[email protected]>
Co-authored-by: foxriver1025 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants