Problems with cache by mime/type

[Jorge Pereira]

Hi,

I am using the nginx/1.12.0 and I am trying to use the below config.
but, the below "map" by "$upstream_http_content_type" is always
matching with default value "1". but, if I remove "proxy_cache_bypass"
then the map it works. therefore, I need the "proxy_cache_bypass "
capability.

http {
include /etc/nginx/mime.types;
default_type application/octet-stream;

map $sent_http_content_type $no_cache {
default 1;

application/zip 0; # *.zip files
application/octet-stream 0; # /artifactory/api/pypi and any
other binary files.
application/java-archive 0; # *.jar
application/x-nupkg 0; # /artifactory/api/nuget/.*/Download/.*
}

proxy_cache_path $CACHE_FOLDER
levels=1:2
keys_zone=artifactory_cache:$CACHE_KEYS_SIZE
max_size=$CACHE_SIZE
inactive=$CACHE_DURATION
use_temp_path=off;

server {
listen 80;
server_name mywww.lan;

set $upstream https://172.16.0.1/artifactory;

proxy_read_timeout 10;
proxy_pass_header Server;
proxy_cookie_path ~*^/.* /;
proxy_next_upstream http_503 non_idempotent;
proxy_set_header Host $http_host;

proxy_cache artifactory_cache;
proxy_cache_bypass $no_cache;
proxy_no_cache $no_cache;

add_header X-Proxy-Cache $upstream_cache_status;

proxy_cache_key $scheme$proxy_host$request_uri$auth_cache_key;
proxy_cache_valid 200 24h;
proxy_ignore_headers "Set-Cookie" "Cache-control";

location ~/mypath(?<relative_uri>.*)$ {
proxy_pass $upstream${relative_uri}${is_args}${args};
}
}
}

--
Jorge Pereira

[Francis Daly]

On Thu, Nov 29, 2018 at 09:51:13PM -0200, Jorge Pereira wrote:

Hi there,

Post by Jorge Pereira
I am using the nginx/1.12.0 and I am trying to use the below config.
but, the below "map" by "$upstream_http_content_type" is always
matching with default value "1". but, if I remove "proxy_cache_bypass"
then the map it works. therefore, I need the "proxy_cache_bypass "
capability.

It looks like you want to bypass the cache (and therefore always serve
these requests from upstream), for certain requests.

Is that correct?

If so -- you must set the "$no_cache" variable based on something in
the request, not on something in the response like a $sent_http_ variable.

Post by Jorge Pereira
map $sent_http_content_type $no_cache {
default 1;
application/zip 0; # *.zip files
application/octet-stream 0; # /artifactory/api/pypi and any
other binary files.
application/java-archive 0; # *.jar
application/x-nupkg 0; # /artifactory/api/nuget/.*/Download/.*
}

Perhaps try setting $no_cache based on $request_uri or $document_uri? That
might fit three of the four above.

(Alternatively: set $no_cache to 0 by default, and to 1 based on requests
that you are happy to potentially have served from the cache.)

Post by Jorge Pereira
proxy_cache artifactory_cache;
proxy_cache_bypass $no_cache;
proxy_no_cache $no_cache;

Good luck with it,

f

[Jorge Pereira]

Hi Francis,

I sent the wrong snip. the correct is using
$upstream_http_content_type as can be seen below. basically, always
when I use "proxy_cache_bypass $no_cache;" that impact the value of
"map $upstream_http_content_type $no_cache".... I didn't understand
what is the reason. thanks for any suggestions.

http {
include /etc/nginx/mime.types;
default_type application/octet-stream;

map $upstream_http_content_type $no_cache {
default 1;
application/zip 0; # *.zip files
application/octet-stream 0; # /artifactory/api/pypi and any
other binary files.
application/java-archive 0; # *.jar
application/x-nupkg 0; # /artifactory/api/nuget/.*/Download/.*
}

proxy_cache_path $CACHE_FOLDER
levels=1:2
keys_zone=artifactory_cache:$CACHE_KEYS_SIZE
max_size=$CACHE_SIZE
inactive=$CACHE_DURATION
use_temp_path=off;

server {
listen 80;
server_name mywww.lan;

set $upstream https://172.16.0.1/artifactory;

proxy_read_timeout 10;
proxy_pass_header Server;
proxy_cookie_path ~*^/.* /;
proxy_next_upstream http_503 non_idempotent;
proxy_set_header Host $http_host;

proxy_cache artifactory_cache;
proxy_cache_bypass $no_cache;
proxy_no_cache $no_cache;

add_header X-Proxy-Cache $upstream_cache_status;

proxy_cache_key $scheme$proxy_host$request_uri$auth_cache_key;
proxy_cache_valid 200 24h;
proxy_ignore_headers "Set-Cookie" "Cache-control";

location ~/mypath(?<relative_uri>.*)$ {
proxy_pass $upstream${relative_uri}${is_args}${args};
}
}
}


--
Jorge Pereira

Post by Francis Daly
Hi there,

Post by Jorge Pereira
I am using the nginx/1.12.0 and I am trying to use the below config.
but, the below "map" by "$upstream_http_content_type" is always
matching with default value "1". but, if I remove "proxy_cache_bypass"
then the map it works. therefore, I need the "proxy_cache_bypass "
capability.

It looks like you want to bypass the cache (and therefore always serve
these requests from upstream), for certain requests.
Is that correct?
If so -- you must set the "$no_cache" variable based on something in
the request, not on something in the response like a $sent_http_ variable.

Post by Jorge Pereira
map $sent_http_content_type $no_cache {
default 1;
application/zip 0; # *.zip files
application/octet-stream 0; # /artifactory/api/pypi and any
other binary files.
application/java-archive 0; # *.jar
application/x-nupkg 0; # /artifactory/api/nuget/.*/Download/.*
}

Perhaps try setting $no_cache based on $request_uri or $document_uri? That
might fit three of the four above.
(Alternatively: set $no_cache to 0 by default, and to 1 based on requests
that you are happy to potentially have served from the cache.)

Post by Jorge Pereira
proxy_cache artifactory_cache;
proxy_cache_bypass $no_cache;
proxy_no_cache $no_cache;

Good luck with it,
f
--
_______________________________________________
nginx mailing list
http://mailman.nginx.org/mailman/listinfo/nginx

[Francis Daly]

On Fri, Nov 30, 2018 at 11:32:39AM -0200, Jorge Pereira wrote:

Hi there,

Post by Jorge Pereira
I sent the wrong snip. the correct is using
$upstream_http_content_type as can be seen below. basically, always

$upstream_http_content_type is the http Content-Type header sent by
upstream in response to the request from nginx to upstream.

http://nginx.org/r/$upstream_http_

At the time that nginx is deciding "should this request be served from
cache or sent to upstream?", $upstream_http_content_type cannot have
a value.

Your $no_cache variable that is used in the proxy_cache_bypass directive
can only usefully be made up of things that are available in the request
from the client to nginx.

Post by Jorge Pereira
when I use "proxy_cache_bypass $no_cache;" that impact the value of
"map $upstream_http_content_type $no_cache".... I didn't understand
what is the reason. thanks for any suggestions.

When "proxy_cache_bypass" is used, it must find the value of the variable
$no_cache. At that time, $upstream_http_content_type is empty, so $no_cache
maps to 1.

When "proxy_no_cache" is used, it must find the value of the variable
$no_cache. If $no_cache was set to 1 previously, it will keep that
value. If $no_cache was not set previously, nginx will check the map,
and now potentially set $no_cache to 0.

That is why your "proxy_no_cache" sees a different value depending on
whether "proxy_cache_bypass" is commented out or not.

f

[Jorge Pereira]

ahh!

Post by Francis Daly
At the time that nginx is deciding "should this request be served from
cache or sent to upstream?", $upstream_http_content_type cannot have
a value.

now I figured out the reason. thank you Francis.