As you’re probably aware, the speed at which your site loads can affect how well it ranks on search engines. It has an impact on sales, too: a one second delay can reduce conversions by as much as 7%. One of the best ways to improve the speed of WordPress websites is to use caching. …
Caching
Varnish Caching with Joomla
Hello There!
One of the exciting new technologies to come out in the last few years is a tremendously efficient and dynamic caching system called Varnish (see : http://www.varnish-cache.org).
We have been employing the use of Varnish for high traffic websites for the purposes of user experience improvements as well as for redundancy and load balancing purposes.
Varnish can do it all – complex load balancing and polling based on many different weighting methodologies for fail over, as well as holding on to a “stale” cache in the event of a back end web server outage, or perhaps for geographic redundancy (holding a stale cache in a secondary data center).
One of the challenges we have faced in the many different implementations of varnish into web stacks, is dealing with dynamic and user session (i.e. “logged in”) content.
If the Internet was full of only static (see 1995) html files, varnish would work beautifully out of the box. Unfortunately the web is a complicated mess of session based authentication, POSTS, GETS and query strings among a few things.
One of our recent accomplishments was getting the Joomla 1.5 content management system to work with Varnish 2.1.
The biggest challenge for Joomla was that it creates a session cookie for all users. This means the session is created and established for any guest visiting the site, and if they decide to log in , that same session is used to establish a logged in session through authentication. This is an apparent effort to deter or avoid session hijacking.
The problem with this is that Varnish ends up caching all the logged in session content, as well as the anonymous front page content.
I spent a significant amount of time fine tuning my VCL (varnish configuration language) to play nice with Joomla. Unfortunately it became apparent that some minor modifications to the Joomla code was necessary in order for it to communicate properly with Varnish.
Step 1 : Move the login form off the front page
I realize this might be a hard decision. I cant offer an alternative. If you have an integrated login form on the front page of your site, and you wish to cache that page with varnish, you will likely have to chose one or the other. It would probably be ideal to replace that login form with a button to bring the user to a secondary page off the main page.
For the sake of argument, lets call our site “example.com” and the login page url within Joomla should look like the following :
http://www.example.com/index.php?option=com_user&view=login
Take note of login URI in this string.
The reason we need the login form on a secondary page is because we need an almost “sandboxed” section of the site where the anonymous session cookie can be established, and passed through the authentication process to a logged in session. We will tell varnish to essentially ignore this page.
Step 2 : Modify Joomla to send HTTP headers for user/guest sessions
This isn’t that hard. In the Joomla code, there is a section where it defines the HTTP headers it sends to the browser for cache variables such as expire times and whatnot. I’m going to assume you have turned off the built-in Joomla caching system.
What you need to do is tell Joomla to send a special HTTP header that will give either a True or False value if the user is logged in or not. This is useful information. It will allow varnish to not cache any logged in content such as “Welcome back, USERNAME” after the user is passed back to the front page from logging in.
In my joomla installation, I modified the following file :
The parent folder being the public_html / root folder for your Joomla installation. In this file, please find the line that determines if the Joomla caching system is disabled :
After this line, you will see about 5 HTTP header declarations (expires, last-modified, cache-control, cache-control again and pragma). Above those declarations , add the following 6 lines of code :
if (!$ user->guest) {
JResponse::setHeader( ‘X-Logged-In’, ‘True’, true);
} else {
JResponse::setHeader( ‘X-Logged-In’, ‘False’, true );
}
If you read the above code, its fairly straight forward. I do a check to see if the user is a guest (aka anonymous) or not. If they are logged in I send an HTTP header called “X-Logged-In”, and assign a “True” value to it. If the user is not logged in, it sets it to “False”.
Pretty easy, right?
This will allow varnish to avoid caching a logged in user’s page.
Step 3 : Configure Varnish
This is the part that took the most time during this entire process. Mind you patching the Joomla code and whatnot took some time as well, this process took a lot of experimentation and long hours examining session cookies and host headers.
What I will do is break down the generalized configuration directives into two groups : VCL_RECV and VCL_FETCH.
VCL_RECV
In here, I set a bunch of IF statement directives to tell varnish what it should look up in the cache and what it should pipe to the backend and what it should pass. This could probably be optimized and improved upon, but it works for me :
if (req.request == "POST") {
set req.backend = iamloggedin;
return(pipe);
}
# http authenticated sessions are piped
if (req.http.Authenticate || req.http.Authorization) {
set req.backend = iamloggedin;
return(pipe);
}
# if the user is coming FROM the login page, pipe to backend
if (req.http.referer ~ "(?i)(com_user|login)") {
set req.backend = iamloggedin;
return(pipe);
}
VCL_FETCH
The fetch section is a little bit easier. I only have about 5 directives. The first one is the most important one you want to look at. It “unsets” the cookie from any page on the site, EXCEPT the login page. This allows varnish to properly establish the logged in session. The subsequent rules determine what to deliver and what to pass based on URI or HTTP header checks :
if (!req.url ~ "(?i)(login|com_user|user|logout)") {
unset beresp.http.Set-Cookie;
}
if (req.http.referer ~ "(?i)(com_user|login|logout)") {
set req.backend = iamloggedin;
return(pass);
}
if (beresp.http.x-logged-in ~ "False"){
set req.backend = webfarm;
return(deliver);
}
if (beresp.http.x-logged-in ~ "True"){
set req.backend = iamloggedin;
return(pass);
}
if (req.http.Authenticate || req.http.Authorization) {
set req.backend = iamloggedin;
return(pass);
}
Thats it! I just saved you many sleepless nights (I hope!). Hopefully your headers will look something like this after you implement varnish in front of Joomla :
P3P CP="NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM"
X-Logged-In False
Expires Mon, 1 Jan 2001 00:00:00 GMT
Last-Modified Mon, 08 Aug 2011 20:49:37 GMT
Cache-Control post-check=0, pre-check=0
Pragma no-cache
Content-Type text/html; charset=utf-8
Content-Length 85898
Date Mon, 08 Aug 2011 21:01:52 GMT
X-Varnish 761778669 761751685
Age 735
Via 1.1 varnish
Connection keep-alive
X-Cache-Svr cache.example.com
X-Cache HIT
X-Cache-Hits 121
UPDATE : 12/08/2011
I realize I made a mistake and have corrected this post. In vcl_fetch, i had the following :
if (!req.url ~ "(?i)(login|com_user|user|logout)") {
unset req.http.Set-Cookie;
}
Well I realize I should be unsetting the response cookie, not the set cookie. For some reason, the above (erroneous) directive works only right after you login. If you start clicking around the site, your logged in session disappears. I suspect this is because either joomla or varnish is mistakenly unsetting a logged in session.
This is the correct entry (I have fixed it in my original post as well) :
if (!req.url ~ "(?i)(login|com_user|user|logout)") {
unset beresp.http.Set-Cookie;
}
After making the above change, I can login and browse the site and my session stays intact. Mind you, the Joomla site I am testing with is definitely not a vanilla Joomla installation.
I’d love to hear from anyone who has accomplished the above scenario either way!
Varnish Caching with Joomla
Hello There!
One of the exciting new technologies to come out in the last few years is a tremendously efficient and dynamic caching system called Varnish (see : http://www.varnish-cache.org).
We have been employing the use of Varnish for high traffic websites for the purposes of user experience improvements as well as for redundancy and load balancing purposes.
Varnish can do it all – complex load balancing and polling based on many different weighting methodologies for fail over, as well as holding on to a “stale” cache in the event of a back end web server outage, or perhaps for geographic redundancy (holding a stale cache in a secondary data center).
One of the challenges we have faced in the many different implementations of varnish into web stacks, is dealing with dynamic and user session (i.e. “logged in”) content.
If the Internet was full of only static (see 1995) html files, varnish would work beautifully out of the box. Unfortunately the web is a complicated mess of session based authentication, POSTS, GETS and query strings among a few things.
One of our recent accomplishments was getting the Joomla 1.5 content management system to work with Varnish 2.1.
The biggest challenge for Joomla was that it creates a session cookie for all users. This means the session is created and established for any guest visiting the site, and if they decide to log in , that same session is used to establish a logged in session through authentication. This is an apparent effort to deter or avoid session hijacking.
The problem with this is that Varnish ends up caching all the logged in session content, as well as the anonymous front page content.
I spent a significant amount of time fine tuning my VCL (varnish configuration language) to play nice with Joomla. Unfortunately it became apparent that some minor modifications to the Joomla code was necessary in order for it to communicate properly with Varnish.
Step 1 : Move the login form off the front page
I realize this might be a hard decision. I cant offer an alternative. If you have an integrated login form on the front page of your site, and you wish to cache that page with varnish, you will likely have to chose one or the other. It would probably be ideal to replace that login form with a button to bring the user to a secondary page off the main page.
For the sake of argument, lets call our site “example.com” and the login page url within Joomla should look like the following :
http://www.example.com/index.php?option=com_user&view=login
Take note of login URI in this string.
The reason we need the login form on a secondary page is because we need an almost “sandboxed” section of the site where the anonymous session cookie can be established, and passed through the authentication process to a logged in session. We will tell varnish to essentially ignore this page.
Step 2 : Modify Joomla to send HTTP headers for user/guest sessions
This isn’t that hard. In the Joomla code, there is a section where it defines the HTTP headers it sends to the browser for cache variables such as expire times and whatnot. I’m going to assume you have turned off the built-in Joomla caching system.
What you need to do is tell Joomla to send a special HTTP header that will give either a True or False value if the user is logged in or not. This is useful information. It will allow varnish to not cache any logged in content such as “Welcome back, USERNAME” after the user is passed back to the front page from logging in.
In my joomla installation, I modified the following file :
The parent folder being the public_html / root folder for your Joomla installation. In this file, please find the line that determines if the Joomla caching system is disabled :
After this line, you will see about 5 HTTP header declarations (expires, last-modified, cache-control, cache-control again and pragma). Above those declarations , add the following 6 lines of code :
if (!$ user->guest) {
JResponse::setHeader( ‘X-Logged-In’, ‘True’, true);
} else {
JResponse::setHeader( ‘X-Logged-In’, ‘False’, true );
}
If you read the above code, its fairly straight forward. I do a check to see if the user is a guest (aka anonymous) or not. If they are logged in I send an HTTP header called “X-Logged-In”, and assign a “True” value to it. If the user is not logged in, it sets it to “False”.
Pretty easy, right?
This will allow varnish to avoid caching a logged in user’s page.
Step 3 : Configure Varnish
This is the part that took the most time during this entire process. Mind you patching the Joomla code and whatnot took some time as well, this process took a lot of experimentation and long hours examining session cookies and host headers.
What I will do is break down the generalized configuration directives into two groups : VCL_RECV and VCL_FETCH.
VCL_RECV
In here, I set a bunch of IF statement directives to tell varnish what it should look up in the cache and what it should pipe to the backend and what it should pass. This could probably be optimized and improved upon, but it works for me :
if (req.request == "POST") {
set req.backend = iamloggedin;
return(pipe);
}
# http authenticated sessions are piped
if (req.http.Authenticate || req.http.Authorization) {
set req.backend = iamloggedin;
return(pipe);
}
# if the user is coming FROM the login page, pipe to backend
if (req.http.referer ~ "(?i)(com_user|login)") {
set req.backend = iamloggedin;
return(pipe);
}
VCL_FETCH
The fetch section is a little bit easier. I only have about 5 directives. The first one is the most important one you want to look at. It “unsets” the cookie from any page on the site, EXCEPT the login page. This allows varnish to properly establish the logged in session. The subsequent rules determine what to deliver and what to pass based on URI or HTTP header checks :
if (!req.url ~ "(?i)(login|com_user|user|logout)") {
unset beresp.http.Set-Cookie;
}
if (req.http.referer ~ "(?i)(com_user|login|logout)") {
set req.backend = iamloggedin;
return(pass);
}
if (beresp.http.x-logged-in ~ "False"){
set req.backend = webfarm;
return(deliver);
}
if (beresp.http.x-logged-in ~ "True"){
set req.backend = iamloggedin;
return(pass);
}
if (req.http.Authenticate || req.http.Authorization) {
set req.backend = iamloggedin;
return(pass);
}
Thats it! I just saved you many sleepless nights (I hope!). Hopefully your headers will look something like this after you implement varnish in front of Joomla :
P3P CP="NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM"
X-Logged-In False
Expires Mon, 1 Jan 2001 00:00:00 GMT
Last-Modified Mon, 08 Aug 2011 20:49:37 GMT
Cache-Control post-check=0, pre-check=0
Pragma no-cache
Content-Type text/html; charset=utf-8
Content-Length 85898
Date Mon, 08 Aug 2011 21:01:52 GMT
X-Varnish 761778669 761751685
Age 735
Via 1.1 varnish
Connection keep-alive
X-Cache-Svr cache.example.com
X-Cache HIT
X-Cache-Hits 121
UPDATE : 12/08/2011
I realize I made a mistake and have corrected this post. In vcl_fetch, i had the following :
if (!req.url ~ "(?i)(login|com_user|user|logout)") {
unset req.http.Set-Cookie;
}
Well I realize I should be unsetting the response cookie, not the set cookie. For some reason, the above (erroneous) directive works only right after you login. If you start clicking around the site, your logged in session disappears. I suspect this is because either joomla or varnish is mistakenly unsetting a logged in session.
This is the correct entry (I have fixed it in my original post as well) :
if (!req.url ~ "(?i)(login|com_user|user|logout)") {
unset beresp.http.Set-Cookie;
}
After making the above change, I can login and browse the site and my session stays intact. Mind you, the Joomla site I am testing with is definitely not a vanilla Joomla installation.
I’d love to hear from anyone who has accomplished the above scenario either way!
Varnish Caching with Joomla
Hello There!
One of the exciting new technologies to come out in the last few years is a tremendously efficient and dynamic caching system called Varnish (see : http://www.varnish-cache.org).
We have been employing the use of Varnish for high traffic websites for the purposes of user experience improvements as well as for redundancy and load balancing purposes.
Varnish can do it all – complex load balancing and polling based on many different weighting methodologies for fail over, as well as holding on to a “stale” cache in the event of a back end web server outage, or perhaps for geographic redundancy (holding a stale cache in a secondary data center).
One of the challenges we have faced in the many different implementations of varnish into web stacks, is dealing with dynamic and user session (i.e. “logged in”) content.
If the Internet was full of only static (see 1995) html files, varnish would work beautifully out of the box. Unfortunately the web is a complicated mess of session based authentication, POSTS, GETS and query strings among a few things.
One of our recent accomplishments was getting the Joomla 1.5 content management system to work with Varnish 2.1.
The biggest challenge for Joomla was that it creates a session cookie for all users. This means the session is created and established for any guest visiting the site, and if they decide to log in , that same session is used to establish a logged in session through authentication. This is an apparent effort to deter or avoid session hijacking.
The problem with this is that Varnish ends up caching all the logged in session content, as well as the anonymous front page content.
I spent a significant amount of time fine tuning my VCL (varnish configuration language) to play nice with Joomla. Unfortunately it became apparent that some minor modifications to the Joomla code was necessary in order for it to communicate properly with Varnish.
Step 1 : Move the login form off the front page
I realize this might be a hard decision. I cant offer an alternative. If you have an integrated login form on the front page of your site, that you also with to have Varnish cache, it would probably be ideal to replace that login form with a button to bring the user to a secondary page. For the sake of argument, lets call our site “example.com” and the login page url within Joomla should look like the following :
http://www.example.com/index.php?option=com_user&view=login
Take note of login URI in this string.
The reason we need the login form on a secondary page is because we need an almost “sandboxed” section of the site where the anonymous session cookie can be established, and passed through the authentication process to a logged in session. We will tell varnish to essentially ignore this page.
Step 2 : Modify Joomla to send HTTP headers for user/guest sessions
This isn’t that hard. In the Joomla code, there is a section where it defines the HTTP headers it sends to the browser for cache variables such as expire times and whatnot. I’m going to assume you have turned off the built-in Joomla caching system.
What you need to do is tell Joomla to send a special HTTP header that will give either a True or False value if the user is logged in or not. This is useful information. It will allow varnish to not cache any logged in content such as “Welcome back, USERNAME” after the user is passed back to the front page from logging in.
In my joomla installation, I modified the following file :
libraries/joomla/environment/response.php
The parent folder being the public_html / root folder for your Joomla installation. In this file, please find the line that determines if the Joomla caching system is disabled :
if (JResponse::allowCache() === false)
After this line, you will see about 5 HTTP header declarations (expires, last-modified, cache-control, cache-control again and pragma). Above those declarations , add the following 6 lines of code :
$ user =& JFactory::getUser(); if (!$ user->guest) { JResponse::setHeader( 'X-Logged-In', 'True', true); } else { JResponse::setHeader( 'X-Logged-In', 'False', true ); }
If you read the above code, its fairly straight forward. I do a check to see if the user is a guest (aka anonymous) or not. If they are logged in I send an HTTP header called “X-Logged-In”, and assign a “True” value to it. If the user is not logged in, it sets it to “False”.
Pretty easy, right?
This will allow varnish to avoid caching a logged in user’s page.
Step 3 : Configure Varnish
This is the part that took the most time during this entire process. Mind you establishing the Joomla code and whatnot took some time as well, this process took a lot of experimentation and long hours examining session cookies and host headers.
What I will do is break down the generalized configuration directives into two groups : VCL_RECV and VCL_FETCH.
VCL_RECV
In here, I set a bunch of IF statement directives to tell varnish what it should look up in the cache and what it should pipe to the backend and what it should pass. This could probably be optimized and improved upon, but it works for me :
if (req.http.x-forwarded-for) { set client.identity = req.http.x-forwarded-for; } # If user is not logged in, look in the cache for a response if (req.http.x-logged-in ~ "False"){ set req.backend = webfarm; return(lookup); } # If user is logged in , pipe to backend if (req.http.x-logged-in ~ "True"){ set req.backend = iamloggedin; return(pipe); } # If user sends an http POST, pipe to backend if (req.request == "POST") { set req.backend = iamloggedin; return(pipe); } # http authenticated sessions are piped if (req.http.Authenticate || req.http.Authorization) { set req.backend = iamloggedin; return(pipe); } # if the user is coming FROM the login page, pipe to backend if (req.http.referer ~ "(?i)(com_user|login)") { set req.backend = iamloggedin; return(pipe); }
VCL_FETCH
The fetch section is a little bit easier. I only have about 5 directives. The first one is the most important one you want to look at. It “unsets” the cookie from any page on the site, EXCEPT the login page. This allows varnish to properly establish the logged in session. The subsequent rules determine what to deliver and what to pass based on URI or HTTP header checks :
# discard backent setcookie unless it equals the following if (!req.url ~ "(?i)(login|com_user|user|logout)") { unset req.http.Set-Cookie; } if (req.http.referer ~ "(?i)(com_user|login)") { set req.backend = iamloggedin; return(pass); } if (req.http.x-logged-in ~ "False"){ set req.backend = webfarm; return(deliver); } if (req.http.x-logged-in ~ "True"){ set req.backend = iamloggedin; return(pass); } if (req.http.Authenticate || req.http.Authorization) { set req.backend = iamloggedin; return(pass); }
Thats it! I just saved you many sleepless nights (I hope!). Hopefully your headers will look something like this after you implement varnish in front of Joomla :
Set-Cookie example_auth_129bf15asdfasdf52f3afaafawef; path=/ P3P CP="NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM" X-Logged-In False Expires Mon, 1 Jan 2001 00:00:00 GMT Last-Modified Mon, 08 Aug 2011 20:49:37 GMT Cache-Control post-check=0, pre-check=0 Pragma no-cache Content-Type text/html; charset=utf-8 Content-Length 85898 Date Mon, 08 Aug 2011 21:01:52 GMT X-Varnish 761778669 761751685 Age 735 Via 1.1 varnish Connection keep-alive X-Cache-Svr cache.example.com X-Cache HIT X-Cache-Hits 121
Secure64 Releases New Version of DNS Caching Software
Secure64 Releases New Version of DNS Caching Software
Secure64 Software Corporation today announced the latest release of its popular caching software appliance — Secure64 DNS Cache v2.5 — the most secure caching name server software available.
Read more on PR Newswire via Yahoo! Finance
Simple clear advice in plain English
Altering the DNS settings on your PC can speed up your internet. We explain how When browsing the web it is unlikely you give a second thought to how the pages end up on your computer’s screen.
Read more on Computer Active
Nominum Provides Virgin Media with Improved Website Navigation
February 8, 2011 — DNS solutions provider Nominum announced on Tuesday that Virgin Media has used its NavAssist and Vantio NXR for its navigation assistance service. The terms of the contract was not disclosed.
Read more on Web Host Industry Review