Couchbase Labs Expiry Callbacks

TTL’s and Expiry Notification

Couchbase includes a TTL (Time To Live) as part of the meta data fields for each document.  All of the client SDK’s includes support for setting/querying/touching this value.   The value is set as the “number of seconds a document should live” for documents that are to expire in 30 days or less.  For values longer than 30 days a unix epoch time value must be passed.  TTL’s are often used in high performance session stores and caching applications.   What happens to a session when a document expires can be problematic and cumbersome to manage at the application tier.   What if the high performance and flexibility of Couchbase Server were leveraged to provide callbacks for document expiration?    Introducing “expiry-notifier” from Couchbase Labs.   The application, source code, and instructions can be downloaded from the couchbaselabs repo on github.**

Couchbase Labs expiry-notifier

The application runs a small application instance of node.js and express.   It is configured and accessed through a REST API.  Why Node.js?   Node.js is a well supported application platform for Couchbase, is incredibly lightweight, manages its footprint efficiently and is well suited for RESTful applications.  Expiry-notifier can be called directly or can be configured to automatically perform expiry callbacks within a given time interval. The expiry callback functionality returns meta id’s and expiration time (or seconds until expiration) for documents that will expire within a specified time window.

For example, to receive an expiry callback from an application that will directly poll the service, a document is defined in Couchbase with a 300 second expiration as:

Key (meta.id): triumph

{
“value”: “1966 Triumph Spitfire Mark II 4 speed with overdrive and twin Weber carburetors.”
}

If expiry-notifier is running, relative time has been set to true (more on this in a second), and after 10 seconds the REST endpoint is called to poll for expiry callbacks:

http://servername:3000/poll/300

A JSON object will be returned:

{triumph:290}

How expiry-notifier Works

Expiry-notifier can be run from any server that has node.js installed.   After expiry-notifier is installed (instructions can be found at the couchbaselabs repo on github), the application needs to be started.  From the root directory where the application is installed, run

“npm start”

By default the application will run using port 3000.   To test if the application is running, call the following REST command:

“http://:3000/status”??????

To configure the service to communicate with a couchbase cluster, call the following REST command:

“http://:3000/setup/server/192.168.60.101/8091/beer-sample”

The above REST command is defined as: /setup/server/:server/:port/:bucket [RETURNS: {JSON OBJECT} with status change].  Calling this REST command performs the following:

• Instantiates the smart client to communicate to the Couchbase cluster
• Sets up a lightweight index in couchbase that holds meta data for expiry time only.

To setup an endpoint for the service to send expiry callbacks to an application server use the following REST command:

“http://:3000/setup/endpoint//3000/receive_callback”

The above REST command is defined as: /setup/endpoint/:hostname/:port/:path/ [RETURNS: {JSON OBJECT} with status change] Note: the “https” flag is only used if expiry-notifier is to send callbacks over the https protocol.  Refer to the documentation for more specific information.

To begin sending callbacks to the above defined endpoint for documents that are set to expire within the next 30 seconds, and each 30 seconds thereafter use the following REST command :

“http://:3000/setup/poll/30/loop”

The above REST command is defined as:/setup/poll/:interval/:loop [RETURNS: {JSON OBJECT} with status change] This command will send a JSON object of meta id’s and their expiration in UNIX epoch time (number of seconds after 1/1/1970 that the document is set to expire).  If the number of seconds until the document expires are preferred, use the command below.

To return the number of seconds until a document expires instead of the UNIX epoch time use the following command:

“http://:3000/setup/relative/true”

The above REST command is defined as: /setup/relative/:bool [RETURNS: {JSON OBJECT} with status change]

** Disclaimer: Couchbase Labs provides experimental code for research and development purposes only.   Code and applications from Couchbase Labs are not supported under any Couchbase Support Agreement and are provided as is with no warranty of any kind.