stream: Allow sending data in the preread phase

[shankerwangmiao]

There are cases when some interactions should happen between the client
and the server before a successful preread. This patch enables sending
data to the client in the preread phase. To avoid possible conflict
between the preread callback and filtering callbacks, the data sent
bypasses js_body_filter, but not other filters, if any.

This patch introduces a new field in ngx_stream_js_ctx_t named preread
to indicate if the session is now in the preread phase.

[shankerwangmiao]

@xeioex any idea on this patch?

[xeioex]

Hi @shankerwangmiao,

We do not think it is good idea to allow any sends in preread phase. Preread phase is designed to have no side effects on the stream (think of idempotent request as analogy).

[shankerwangmiao]

But sometimes it is needed to do a simple handshake to read meaningful information out before deciding which upstream backend can be used. That's why I'm suggesting this feature.

[xeioex]

@shankerwangmiao

But sometimes it is needed to do a simple handshake to read meaningful information out before deciding which upstream backend can be used. That's why I'm suggesting this feature.

Can you elaborate on this? Please provide real world example. The whole approach seems brittle to me.

[shankerwangmiao]

A real example is rsync protocol. I need to route rsync requests for different modules to different backend servers, because files are stored separately. Although rsync protocol is rather complex, the handshaking process is simple. When connected, the client and server send their version to each other. When the client receives the version number, it will send the name of the module it want to operate on. The first work the njs script needs to do is to emulate a rsync server and send out a version number in the preread phase. When received, the module name will be put into a variable, which will then be mapped to a certain backend. The second work is to filter the version number sent by the backend rsync server and prevent it from being sent to the client. After that, the njs script will detach the event handler and let the connection go.

[shankerwangmiao]

Here is an implementation of such rsync proxy using njs:

https://gist.github.com/shankerwangmiao/ffe48d51eef5b7178a442ba4c32568a2

In the above configuration, for example, when the client execute rsync rsync://ip.addr:12345/debian, the connection will be forwarded to mirrors.bfsu.edu.cn:873.

The njs script is now actually running on our server.

[shankerwangmiao]

Hi,

According to readme of njs

Please ... send patches ... via Github:

I wonder if this is the right place to send patches to njs?

[thresheek]

Sorry @shankerwangmiao was an error in our automation, reopening again.

[shankerwangmiao]

ping @xeioex

[shankerwangmiao]

ping @xeioex

[shankerwangmiao]

ping @xeioex

[fuweichin]

It's interesting that Chrome 103 supports HTTP status code 103 (Early Hints).

Please also consider sending status 1xx (more specifically, status 103) and headers (like 'Link') in the preread phase in ngx_http_js_module, for HTTP/1.1 and/or HTTP/2.

And I just get the 103 Early Hints working in Node.js Express and Java Tomcat, but noway in Nginx njs. Here are the code screenshots, waiting for njs to launch such feature: