XalokNext Documentation

Appearance

Setting up Varnish ​

Intro ​

The XalokNext editor saves Page entities (both articles and boards are instances of this class and are handled in the same way by XalokNext) through /vendor/wfcms/cms-base-admin-bundle/Wf/Bundle/CmsBaseAdminBundle/Controller/PageApiController. createAction is invoked for new page entities and updateAction is invoked for edited page entities.

In order to avoid having the editor wait for the backend to clear the cache when publishing (send request to all Varnishes), the CMS integrates BCCResqueBundle for two types of jobs: publishing pages, both now and in the future, and clearing the cache.

Enqueing the jobs is handled by the wf_cms_admin.listeners.page_publish doctrine listener that checks if the created/edited page is set up to be published.

When the editor publishes a page instantly, a Wf\Bundle\CmsBaseBundle\Job\PublishJob is created, its job is to clear the cache from Varnish. When the editor schedules the publishing, a FuturePublishJob is scheduled for that time. BCCResqueBundle takes care of running that job at the scheduled time.

The background jobs are ran by the two standard commands that come with BCCResqueBundle (for more details, read the documentation of that bundle): bcc:resque:worker-start - the CMS uses the default queue for <=v3.0, or, in v3.1+, a queue that has the same name as the name of the public bundle of the project name (e.g. ProjectCmsBundle) - and bcc:resque:scheduledworker-start .

Resque by defaults waits for 5 seconds until it checks the queue again. To change this interval, add the interval option to the worker-start command:

./app/admin/console --env=prod bcc:resque:worker-start __QUEUE_NAME__ -f --interval 1

Check the resque:queue:default v3 or resque:queue:ProjectCmsBundle v3.1+ key in Redis if you want to make sure the doctrine listener worked correctly and added the job to the queue.

When a job is being ran, its run method is invoked. The FuturePublishJob job is simply updating the page at the date of its publication (updates the published_at column with the value from next_published_at, since the date stored in next_published_at already arrived), letting the doctrine listener to enque a PublishJob for that page.

The PublishJob enques a ClearCacheJob, which, when being ran, invokes the purgePageCache of the wf_cms.frontend_cache.manager service. This service uses the liip_cache_control.varnish service for XalokNext <=v3.0, which was renamed to fos_http_cache.cache_manager in v3.1+.

Purged cache objects ​

<=v3.0 ​

For v3.0 and previous, the ClearCacheJob for a page sends a PURGE request to the Varnish servers with the URL /page and the header purgethis set to obj.http.x-url ~ concatenated with the list of URLs generated by the following routes, separated by |:

  • wf_article_show - the route showing the article HTML - if the cleared page is an article
  • wf_page_show - the route showing the content of the page (just the content part, not the entire HTML)
  • wf_cms_rss_article and wf_cms_rss_latest if the RSS bundle is enabled
  • wf_cms_rss_category for all the categories assigned to the page

IMPORTANT: The above routes are sent as part of the header, not individual PURGE requests. Please configure Varnish to invalidate objects containing that header.

A second PURGE request is sent to the Varnish servers to purge all the objects that embed the given page (boards that include the article). This request has purgethis (some projects use purgeboards) header set to obj.http.X-Embedded-Pages: ^X\D|\DX\D|\DX$ - where X is the ID of the page. The X-Embedded-Pages header contains a list of IDs separated by | (e.g.: 1|2|3). Basically, this regular expression purges all the objects that have X-Embedded-Pages header either starting with the ID of the page (^X\D|), that have the ID of the page in between other IDs (|\DX\D|) or that end in the ID of the page (|\DX$).

Example .vcl configuration (for Varnish 4):

vcl_recv {
    ...
    if (req.method == "PURGE") {
        if (req.http.purgethis) {
            ban(req.http.purgethis);
        }

        if (req.http.purgeboards) {
            ban(req.http.purgeboards);
        }

        return(purge);
    }
}

v3.1 ​

The purging process in v3.1 is sending one PURGE request to the wf_page_show route for the purged page, and another PURGE request with the X-Cache-Tags header containing a few tags to be purged: page-__ID__ for the given page, category-__ID__ for all the categories that the page belongs to, and the rss-latest tag.

Example .vcl configuration (for Varnish 4) (in vcl_recv):

vcl_recv {
    ...
    if (req.method == "PURGE") {
        if (req.http.X-Cache-Tags) {
            ban("obj.http.X-Cache-Tags ~ " + req.http.X-Cache-Tags);
        }

        return(purge);
    }
    if (req.method == "BAN") {
        if (req.http.X-Cache-Tags) {
            ban("obj.http.X-Cache-Tags ~ " + req.http.X-Cache-Tags);
        }

        return(synth(200, "Banned '" + req.http.X-Cache-Tags + "'");
    }
}

Example .vcl configuration (for Varnish 3):

vcl_recv {
    //...
    if (req.request == "PURGE") {
        return (lookup);
    }

    if (req.request == "BAN") {
        ban("obj.http.X-Cache-Tags ~ " + req.http.X-Cache-Tags);
        error 200 "Banned '" + req.http.X-Cache-Tags + "'";
        return (error);
    }
    //...
}

sub vcl_hit {
    if (req.request == "PURGE") {
        purge;
        error 200 "Purged " + req.url + " - " + req.http.host;
        return (error);
    }
}

sub vcl_miss {
    if (req.request == "PURGE") {
        purge;
        error 200 "Purged alternative " + req.url + " - " + req.http.host;
        return (error);
    }
}

Configure the bcc.resque.redis client ​

Clearing the Varnish cache is done through a queue manager called bcc.resque. Since this client doesn't use a prefix for its redis keys, it's important to have a Redis instance for this purpose that is separated from other possible projects running in the same infrastructure. Once the separate instance of Redis is running, set up its connection details by adding the following parameters to the app/config/parameters/local.yml:

yaml
parameters:
    bcc_resque.resque.redis.host: __REDIS_HOST__
    bcc_resque.resque.redis.port: __REDIS_PORT__

Configure the IPs of Varnish instances ​

On the backend, configure the IPs of the Varnish:

XalokNext <=v3.0 ​

yaml
liip_cache_control:
    varnish:
        host: http://__PUBLIC_DOMAIN_OF_THE_PROJECT__.com
        ips: 192.168.0.2,192.168.0.3
        port: 8100

XalokNext v3.1+ ​

yaml
fos_http_cache:
    invalidation:
        enabled: true
    proxy_client:
        varnish:
            servers:
                - 192.168.0.2:8080
                - 182.168.0.3:8080
            base_url: http://__PUBLIC_DOMAIN_OF_THE_PROJECT__.com

Configure trusted_proxies on middles ​

Check the trusted proxies configuration on middles.

varnishlog ​

You can monitor the PURGE requests by checking the varnishlog on the frontends

varnishlog -i "ReqURL,ReqHeader" -q 'ReqMethod eq "PURGE"'

VCL configuration ​

Whitelist IPs allowed to purge ​

It's a good idea to limit PURGE requests only to IPs of the internal network:

acl purge {
        "localhost";
        "10.0.0.0"/16;
}

vcl_recv {
    if (req.method == "PURGE") {
        if (!client.ip ~ purge) {
            return (synth(405, "This IP is not allowed to send PURGE requests."));
        } else {
            //...add purge logic here
        }
    }
}