Using the mirror module

[Kenny Meyer]

Hi,

I’m having trouble using the new mirror module. I want to mirror incoming requests from Nginx to other upstream servers. 1) a production server 2) a staging server

This is my config:

server {
listen 80 default_server;
listen [::]:80 default_server;

location / {
mirror /mirror;
proxy_pass http://www.example.com;
}

location /mirror {
internal;
proxy_pass http://staging.example.com$request_uri;
}
}

So, I request http://myserver.com (where Nginx is hosted) and it successfully redirects me to www.example.com, however I don’t see any requests hitting staging.example.com.

What could be the error?

[Kenny Meyer]

Hi Roman,

Are there any errors in error.log?

No errors…

And what happens if you switch www.example.com and staging.example.com?

Then I get redirected to staging.example.com and I don’t see any requests being logged on example.com

Hi Kenny,

Post by Kenny Meyer
Hi,
I’m having trouble using the new mirror module. I want to mirror incoming requests from Nginx to other upstream servers. 1) a production server 2) a staging server
server {
listen 80 default_server;
listen [::]:80 default_server;
location / {
mirror /mirror;
proxy_pass http://www.example.com;
}
location /mirror {
internal;
proxy_pass http://staging.example.com$request_uri;
}
}
So, I request http://myserver.com (where Nginx is hosted) and it successfully redirects me to www.example.com, however I don’t see any requests hitting staging.example.com.
What could be the error?

The configuration looks fine.
Are there any errors in error.log?
And what happens if you switch www.example.com and staging.example.com?
--
Roman Arutyunyan
_______________________________________________
nginx mailing list
http://mailman.nginx.org/mailman/listinfo/nginx

Kenny Meyer
www.kennymeyer.net

[Peter Booth]

This is the point where I would jump to using the debug log. You need to build you nginx binary
with —with-debug switch and change the log level to debug innginx.conf. Debug generates a *huge*
amount of logs but it really is invaluable.

I would also want to double check what is actually happening and use ss or tcpdump
to confirm that no request is sent to your staging destination.

I’m assuming that both ww.example.com and staging.example.com are hosted
on different hosts, different IPs and are both functional.

Peter

Post by Kenny Meyer
Hi Roman,

Are there any errors in error.log?

No errors…

And what happens if you switch www.example.com and staging.example.com?

Then I get redirected to staging.example.com and I don’t see any requests being logged on example.com

Hi Kenny,

Post by Kenny Meyer
Hi,
I’m having trouble using the new mirror module. I want to mirror incoming requests from Nginx to other upstream servers. 1) a production server 2) a staging server
server {
listen 80 default_server;
listen [::]:80 default_server;
location / {
mirror /mirror;
proxy_pass http://www.example.com;
}
location /mirror {
internal;
proxy_pass http://staging.example.com$request_uri;
}
}
So, I request http://myserver.com (where Nginx is hosted) and it successfully redirects me to www.example.com, however I don’t see any requests hitting staging.example.com.
What could be the error?

The configuration looks fine.
Are there any errors in error.log?
And what happens if you switch www.example.com and staging.example.com?
--
Roman Arutyunyan
_______________________________________________
nginx mailing list
http://mailman.nginx.org/mailman/listinfo/nginx

Kenny Meyer
www.kennymeyer.net
_______________________________________________
nginx mailing list
http://mailman.nginx.org/mailman/listinfo/nginx

[Kenny Meyer]

How do you define a resolver?

Post by Kenny Meyer
Hi Roman,

Are there any errors in error.log?

No errors…

And what happens if you switch www.example.com and staging.example.com?

Then I get redirected to staging.example.com and I don’t see any requests being logged on example.com

Do you have a resolver defined in the http{} block?
Proxying to "http://staging.example.com$request_uri" requires a resolver.
Normally, you would have "no resolver defined to resolve ..."
error logged if resolver is missing, but you may not see it if, for example,
your log level is too high.

Post by Kenny Meyer

Hi Kenny,

Post by Kenny Meyer
Hi,
I’m having trouble using the new mirror module. I want to mirror incoming requests from Nginx to other upstream servers. 1) a production server 2) a staging server
server {
listen 80 default_server;
listen [::]:80 default_server;
location / {
mirror /mirror;
proxy_pass http://www.example.com;
}
location /mirror {
internal;
proxy_pass http://staging.example.com$request_uri;
}
}
So, I request http://myserver.com (where Nginx is hosted) and it successfully redirects me to www.example.com, however I don’t see any requests hitting staging.example.com.
What could be the error?

The configuration looks fine.
Are there any errors in error.log?
And what happens if you switch www.example.com and staging.example.com?
--
Roman Arutyunyan
_______________________________________________
nginx mailing list
http://mailman.nginx.org/mailman/listinfo/nginx

Kenny Meyer
www.kennymeyer.net
_______________________________________________
nginx mailing list
http://mailman.nginx.org/mailman/listinfo/nginx

--
Roman Arutyunyan
_______________________________________________
nginx mailing list
http://mailman.nginx.org/mailman/listinfo/nginx

Kenny Meyer
www.kennymeyer.net

[Peter Booth]

Suggestion:

Define two more locations - one that proxies www.example.com and another that proxies staging.example.com. If both locations work then your problem is probably mirroring. If one doesn’t work then the issue is your configuration and not mirroring. Either way you have reduced the size of your problem space.

Peter

Sent from my iPhone

Post by Kenny Meyer
Hi Roman,

Are there any errors in error.log?

No errors…

And what happens if you switch www.example.com and staging.example.com?

Then I get redirected to staging.example.com and I don’t see any requests being logged on example.com

Hi Kenny,

Post by Kenny Meyer
Hi,
I’m having trouble using the new mirror module. I want to mirror incoming requests from Nginx to other upstream servers. 1) a production server 2) a staging server
server {
listen 80 default_server;
listen [::]:80 default_server;
location / {
mirror /mirror;
proxy_pass http://www.example.com;
}
location /mirror {
internal;
proxy_pass http://staging.example.com$request_uri;
}
}
So, I request http://myserver.com (where Nginx is hosted) and it successfully redirects me to www.example.com, however I don’t see any requests hitting staging.example.com.
What could be the error?

The configuration looks fine.
Are there any errors in error.log?
And what happens if you switch www.example.com and staging.example.com?
--
Roman Arutyunyan
_______________________________________________
nginx mailing list
http://mailman.nginx.org/mailman/listinfo/nginx

Kenny Meyer
www.kennymeyer.net
_______________________________________________
nginx mailing list
http://mailman.nginx.org/mailman/listinfo/nginx

[jlangr]

Thanks for the posts Kenny & pbooth.

I have this problem as well. I am running nginx 1.15 under MacOS and also
experience this under Windows.

Requests successfully go to the server specified under the location but are
not received at the proxy_pass defined in the mirror location's proxy_pass.
Switching between the two servers (swapping the mirror for the proxy_pass in
the primary location) exhibits the same problem.

My config is:
```
http {
server {
listen 3333;
server_name localhost;

location / {
mirror /other;
proxy_pass http://localhost:3001;
}

location = /other {
internal;
proxy_pass http://localhost:3002;
}
}
}

events {
}
```
Based on your other post, it does not look like I need a resolver.

I've also tried with two servers--one localhost and one established server
on another machine.

Any help greatly appreciated!

Jeff

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,279036,280828#msg-280828

[jlangr]

hmm looks like the mirror kills the url. I need to understand how to do URL
rewrites, probably, but the original request's path isn't coming across to
the mirror location.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,279036,280829#msg-280829

[Francis Daly]

On Thu, Aug 09, 2018 at 05:08:25PM -0400, jlangr wrote:

Hi there,

Post by jlangr
hmm looks like the mirror kills the url. I need to understand how to do URL
rewrites, probably, but the original request's path isn't coming across to
the mirror location.

Does it work for you if you use the Example Configuration in the
documentation?

http://nginx.org/en/docs/http/ngx_http_mirror_module.html

f

[jlangr]

Hi Francis--

We got this working to wrap up the day. My ignorance of nginx meant that I
was interpreting the example literally--I viewed the "/mirror" as the name
for the internal route, but that actually becomes part of the URL the mirror
location proxy forwards to.

http {
server {
listen 3333;
server_name localhost;

location / {
mirror /;
resolver 8.8.8.8; # I shouldn't need this, will test tomorrow
proxy_pass http://1.2.3.4:3001;
}

location = / {
internal;
resolver 8.8.8.8;
set $upstream_endpoint http://1.2.3.4:3002;
proxy_pass $upstream_endpoint$request_uri;
}
}
}

That works like a champ. Initially I was trying to use localhost as the
proxy_pass destination, and had turned on port forwarding at my router, but
for whatever reason that was having troubles.

Many thanks in any case!

Jeff

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,279036,280832#msg-280832

[jlangr]

ps how do you format code in these posts? (It's not markdown)

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,279036,280833#msg-280833

[Francis Daly]

On Thu, Aug 09, 2018 at 07:04:50PM -0400, jlangr wrote:

Hi there,

Post by jlangr
We got this working to wrap up the day.

Good that you have something working for you.

Post by jlangr
My ignorance of nginx meant that I
was interpreting the example literally--I viewed the "/mirror" as the name
for the internal route, but that actually becomes part of the URL the mirror
location proxy forwards to.

It's actually a bit more subtle than that.

The argument to the "mirror" directive is the local url of the subrequest;
it should start with "/" or "@".

Then, in the matching location{}, if you choose to proxy_pass, then the
proxy_pass rules come in to play. And it is proxy_pass that decides what
request to make to the upstream server.

http://nginx.org/r/proxy_pass

Post by jlangr
mirror /;

That means that the subrequest will be exactly "/"...

Post by jlangr
location = / {

...which will be handled in this location...

Post by jlangr
set $upstream_endpoint http://1.2.3.4:3002;
proxy_pass $upstream_endpoint$request_uri;

...and the proxy_pass argument includes something after the host:port,
so that is what the request to upstream will be.

An alternative could have been to use a named location -- "mirror
@mirror;", with "location @mirror { proxy_pass http://1.2.3.4:3002; }"
-- which would avoid the need for some of the extra variables.


But, once you have something that works, there is no need to change it :-)

Cheers,

f

[jlangr]

thanks Francis for the additional info, very helpful!

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,279036,280872#msg-280872