Products

In Part 5 of this series, we mentioned that Ehcache could be configured at runtime via ColdFusion code as well as by using an XML configuration file called ehcache.xml.

On a Java EE install of ColdFusion 9, you can find the ehcache.xml file located here:

/JRun4/servers/servername/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib

There are two main sections of the file you need to be concerned with for basic configuration. The first section you should take a look at is the DiskStore configuration:

This tag tells Ehcache where it should write cache files if you have the cache configured to overflow to disk or to persist to disk. We'll get into the specifics of those options, but for now it's important to know that by default Ehcache will use your Java temp directory to store cache files if it is configured to do so. On Windows, the Java temp directory is located in c:/windows/temp. You can change the value here to any drive/directory on your system if you wish to use a location other than the Java temp directory.

The next section to take a look at comes at the end of the ehcache.xml file. Skip on down to the very end where you should see a block of XML that looks like this:

This block of XML tells ColdFusion how to configure all of the Object and Template caches that are automatically created for your application. When a cache is automatically created, the name for the cache is also automatically created using the convention appnameOBJECT for object caches and appnameTEMPLATE for template caches. Each cache has a number configurable parameters:

• maxElementsInMemory: Sets the max number of objects that will be created in memory. Once this limit is reached, the cache will either overflow to disk (if overflowToDisk is set to true), or the appropriate eviction policy will be executed against the cache to make enough room for the new item(s) being added.
• eternal: Sets whether elements are eternal. If eternal is set to true, timeouts are ignored and the element is never expired.
• timeToIdleSeconds: Sets the time to idle for an element before it expires.
• timeToLiveSeconds: Sets the time to live for an element before it expires.
• overflowToDisk: Sets whether elements can overflow to disk when the memory store has reached the maxElementsInMemory limit.
• diskSpoolBufferSizeMB: Specifies the spool buffer size for the DiskStore, if enabled. Writes are made to the spool buffer before they are asynchronously written to the DiskStore.
• maxElementsOnDisk: Sets the max number of objects that will be maintained in the DiskStore
• diskPersistent: Whether the disk store persists between restarts of the Virtual Machine.
• diskExpiryThreadIntervalSeconds: Specifies the interval (in seconds) between runs of the disk expiry thread.
• memoryEvictionPolicy: Policy to enforce upon reaching the maxElementsInMemory limit (LRU, LFU, FIFO).

If you make changes to any of these parameters, ColdFusion will apply them to any new caches that it automatically creates. If you have disk persistence or overflow to disk turned on, two files will be written to your file system per cache, an index file and a data file. For an object cache you would get appnameOBJECT.index and appnameOBJECT.data.

If you want to see what properties have been set for your application's cache, you can do so using the cacheGetProperties() function. The function takes a single optional parameter that specifies the type of cache to return the properties for. Options are Template or Object. If you don't specify the cache type to return properties for, ColdFusion returns them for both cache types. Here's an example that dumps the properties for both the default Object and Template caches:

This will result in output that looks like this:

As you can see from the screen shot, the structure keys correlate to parameters form the ehcache.xml file with two notable exceptions. Both diskSpoolBufferSizeMB and diskExpiryThreadIntervalSeconds are not reported on as properties that can be changed programmatically at runtime.

If you wish to change any of these properties programmatically, you can do so using the cacheSetProperties() function. This function takes a single argument – a structure containing all of the properties that should be configured. You can configure any of the following parameters:

• objectType: Specifies the cache type: Object, Template, or All
• diskStore: Supposed to specify the location of the DiskStore for disk based caching but this is currently not working as of ColdFusion 9.0.
• diskPersistent: Whether the disk store persists between restarts of the Virtual Machine. True|False
• eternal: Sets whether elements are eternal. If eternal, timeouts are ignored and the element is never expired. True|False
• maxElementsInMemory: Sets the max number of objects that will be created in memory. Integer
• maxElementsOnDisk: Sets the max number of objects that will be maintained in the DiskStore. Integer
• memoryEvictionPolicy: Policy to enforce upon reaching the maxElementsInMemory limit: LRU, LFU, FIFO
• overflowToDisk: Sets whether elements can overflow to disk when the memory store has reached the maxElementsInMemory limit: True|False
• timeToIdleSeconds: Sets the time to idle for an element before it expires: Integer number of seconds
• timeToLiveSeconds: Sets the time to live for an element before it expires: Integer number of seconds

Remember, this only applies to caches automatically created by ColdFusion. If a cache doesn't yet exist and you call cacheSetProperties(), ColdFusion will automatically create the cache for you based on whether you are setting properties for an Object cache, a Template cache, or both.

The following code shows how to build the structure of parameters necessary to configure a cache and set the cache properties using the cacheSetProperties() function:

It's also possible to create more caches than just the default template and object caches that ColdFusion creates automatically for you. This can be achieved by defining them in your ehcache.xml file or at runtime using the cfcache tag. To configure a new cache region in your ehcache.xml file, you would do so like this (place this before or after the defaultCache block in your ehcache.xml file):

As you can see, the only difference between this code and the code for the default cache is that you give the cache region a name using the name parameter.

You should know that neither cacheGetProperties() nor cacheSetProperties() can be used to configure the properties for a custom cache in ColdFusion 9.0. Hopefully this is a feature that will be added in a future version of ColdFusion.

If you want to read from or write to a custom cache, you can only do so using the cfcache tag. Here's an example:

This code is almost identical to the code we wrote in Part 5 where we introduced the object cache (don't worry about the extra metadata we're pulling using cfcache. We'll cover that later). The only real difference is that here we specify a name for our cache in both cfcache tags by defining it in the key attribute. Key allows us to specify a custom name for our cache. If a cache by that name hasn't been configured in your ehcache.xnl file, ColdFusion will automatically create it using the parameters set in the default cache settings. Remember, you can only work with custom caches using the cfcache tag. None of the cache functions in ColdFusion 9.0 allow you to specify the cache name you want to apply the function to. Go ahead and run the code a few times to verify it's pulling from the custom cache.

There's a lot more advanced stuff that can be configured in the ehcache.xml file such as cache clustering. These topics deserve their own posts, which we'll get to soon. For now, thanks for sticking with the series and I hope you've learned a little more about the ins and outs of cache configuration in ColdFusion 9.

In previous versions of ColdFusion (before ColdFusion 9 that is), there were three built-in mechanisms you could for caching – persistent variable scopes, query caching and the cfcache tag. As I mentioned in Part 2 of the series, each of these methods has inherent limitations and generally requires a good deal of additional programming to gain any semblance of control over the actual cache – if that is even possible. In many cases it really isn't practical or even possible to see how much information is stored in one of the aforementioned caching mechanisms. All of this changes with the introduction of Ehcache as the underlying cache provider in ColdFusion 9. Five parts into this series, and I just realized I've mentioned Ehcache several times but I never really took the time to talk about what exactly it is and how it's been implemented in ColdFusion 9.

The Ehcache project was started by a gentleman named Greg Luck. Ehcache can best be described as "... a widely used Java distributed cache for general purpose caching, Java EE, and lightweight containers." Caches are implemented as key-value stores, much like a ColdFusion structure. One important feature of an Ehcache cache is that it can be persisted to memory, disk, or both. Optionally, memory caches can be configured to survive JVM restarts (or in our case, ColdFusion server restarts). Caches can be configured via an XML configuration file named ehcache.xml or programmatically at runtime. Ehcache ships as a JAR file and runs in-process within an application server's JVM. In the case of ColdFusion 9, the JAR file and the XML file used to configure it (ehcache.xml) are both located in:

/JRun4/servers/server_name/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib

ColdFusion 9 implements three types of caches using the Ehcache engine: Template Caches, Object caches and Hibernate Caches. Each of these cache types has their own use cases and we'll cover them in depth in future blog posts. For now, here's a quick summary of what each cache type is and generally what it's used for.

The template cache is designed for caching entire web pages as well as page fragments (sections of web pages). You shouldn't confuse this with the template cache mentioned in the ColdFusion Administrator. That template cache is concerned with compiled ColdFusion templates stored in memory. It's unfortunate that this dual use of the term "template cache" exists in ColdFusion so you need to be aware of which type of cache is being referred to when you see template cache mentioned. In terms of the Ehcache template cache implementation in ColdFusion 9, it's a fairly automatic process with ColdFusion handling the bulk of the work managing keys, cache gets/puts and expiry/eviction from the cache. Working with the template cache is done exclusively using the cfcache tag. Here's a very basic example:

In this code, the first section displays the current date/time. The second section uses the cfcache tag with action="serverCache". This action specifies that we want to use server side caching (Ehcache) and is now the default action in ColdFusion 9. To cache a fragment of code in a ColdFusion page we simply have to wrap it in a cfcache block. The next section of code is again entirely dynamic. The fourth and final section is wrapped in another set of cfcache tags. When you execute this code for the first time, the output will show the same date/time for all four sections of code, like this:

Running the code a few seconds later has different results:

In this case, the output contains both dynamic data (the first and third lines) as well as cached data (the second and fourth lines). Using the template cache it's easy to see how you can mix both dynamic data and multiple fragments of data that needs to be cached on the same page. There's really not a whole lot for you to do. You simply wrap the content you want to cache in a cfcache block and ColdFusion handles the rest. There are a few things you can control like cache timeouts, but we'll cover that in a full blog post dedicated to the template cache and all of it's features.

The object cache gives you much more granular control over what you cache than the template cache does. Using the object cache, you can pretty much cache anything you want – simple values, complex variables, objects, files, and just about anything else you want to throw at it. The advantage to using the object cache is that you have complete control over key names, get/put/remove operations, and cache expiry/eviction. This takes a little more work on your part but what you give up in convenience you easily gain back in flexibility and control. You can work with the object cache using both the cfcache tag as well as the new caching functions introduced in ColdFusion 9. Here's a very basic example of how you can cache the results of a query in an object cache using the cfcache tag:

In this code, the first thing we do is try to pull our query out of the cache using the cfcache tag with action="get". The key that identifies our cached query in the object cache is set using id="artistQuery". Of course the first time we run this the query won't be in the cache, so the result variable we specified (name="getArtists") will return Null. After we attempt to pull the query from the cache, we then use the isNull function to see if anything was returned from our get operation. If getArtists is Null, we know that the query data isn't in the cache so we need to run our query, retrieve the data and then place it in the cache using the cfcache tag with action="put". When we do our cache put, we set id="artistQuery", which is the name of our cache key. The value to store in the cache is our ColdFusion query (value="getArtists") and we can specify how long the data should be cached using the timespan argument. The first time you execute the page (with debugging turned on), you'll notice that there's debugging information for your executed SQL Queries – because the query data didn't exist in the cache yet and you told ColdFusion to go off and get it from the database and put it into the cache. If you execute the page again, you won't see any debug info for your SQL Queries because ColdFusion is pulling the data from the cache and not from the database. It's important to note that in the dump of the query data from this example, Cached will always be False, regardless of whether you are accessing cached data or not. This is because Cached refers to whether or not the result set is in the ColdFusion Query Cache, which it is not because we are using Coldfusion's Ehcache implementation here and not the built in query cache. There's a lot more you can do with the object cache which we'll cover in another blog post.

If you're using ColdFusion 9's new Hibernate ORM functionality you can configure the ORM such that it uses Hibernate as its second level cache. This is a more complicated topic that we have time to discuss now, so let's table it for another blog post.

When you create a ColdFusion application and your application uses an Application.cfc or Application.cfm file, ColdFusion automatically creates a template cache and an object cache for you. These caches are bound to your application name (defined in your cfapplication tag for an Application.cfm file or this.name if you are using Application.cfc). If you have an unnamed application, ColdFusion will create a cache that is shared and accessible among all unnamed applications. It's important to remember that caches are not tied to ColdFusion scopes. If your application times out, this does not affect the cache(s) you create with Ehcache.

That's about it for getting to know the basics of Ehcache in ColdFusion 9. In Part 6 we'll start looking at the ehcache.xml file and how it can be used to configure the behavior of the caches you create in ColdFusion 9.

So far in this series, we've covered why you would want to cache, what to cache and when, and basic caching architectures. In part 4 of this series, we're going to talk about caching strategies and eviction policies.

A caching strategy is nothing more than an architectural decision on how you're going to manage putting data in and retrieving data from your cache and the corresponding relationship between the cache and your backend data source. There are two main caching strategies you need to be aware of, deterministic and non-deterministic.

A non-deterministic caching strategy involves first looking in the cache for the object or data you want to retrieve. If it's there, your application uses the cached copy. If it's not there, you must then query the backend system for the object or data you want to retrieve. This is by far the most popular caching strategy as it's relatively simple to implement and is very flexible.

A deterministic caching strategy is one in which you always go to the cache for the object or data that you need. It's assumed that if it's not in the cache, then it doesn't exist. This strategy requires that your cache be pre-populated with data as there's no mechanism for a cache miss to query the backend system for the missing object or data.

Both deterministic and non-deterministic caching strategies have their pros and cons. For non-deterministic caching, the upside is that it's simple to implement in code and you have a lot of flexibility in how you do this. The downside to this caching strategy is an issue called stampeding requests, otherwise known as the dog pile. This occurs, usually under load, when a cache miss results in multiple threads simultaneously querying the backend system for the missing cache data. Under this scenario, it's very easy to overwhelm the backend system with requests as the database struggles to fetch the data and repopulate the cache. There are various ways that you can code around this, which we'll discuss later on in another blog post. For now, it's just important to realize that it can happen.

For the purposes of the rest of this discussion as well as the rest of the series, we'll be focusing on non-deterministic caching. That said; let's now turn our attention to cache eviction algorithms. Think of a cache like a box. A box has a limit on how much stuff it can hold before things start falling out when you try to pile on more. A cache is the same way when it comes to the objects and data you store in it – eventually it runs out of room.

Cache eviction policies can be broken down in to two categories: time based and cost based. Time based policies let you associate a time period or an expiration date for individual cache items. This lets you do things like keep an item in the cache for 6 hours, or 30 days, or until December 15, 2040 at 10:00pm. When a request is made to a cache that contains items with time based expirations, the cache first checks to see if the item is expired. If it is, the item is evicted from the cache and is not returned to the operation that called it (most caches simply return null).

Cost based eviction policies work a little differently. A cost based eviction policy doesn't kick in until a cache is full and needs to kick some items out (evict) before allowing new ones in. Most caches give you several cost based eviction policies to choose from. In this scenario, when you attempt to put a new item in the cache, the cache first looks to see if it's full. If it is, it runs whatever cost based eviction policy has been set for the cache and evicts the appropriate item(s). The following are some of the most common cost based eviction policies you'll encounter:

First In First Out (FIFO): The first item that was placed in the cache is the first item to be evicted when it becomes full. It's essential to remember that the first item in the cache is not necessarily the least important. If the first item in your cache is also the most frequently accessed item you might want to think twice about implementing an eviction policy that would result in evicting it from the cache first in the event the cache fills up.

Least Recently Used (LRU): This policy implements an algorithm to track which items in the cache are the least frequently accessed. Various cache providers implement this algorithm in different ways but the result is that the items in the cache that haven't been used in a while are evicted first.

Less Frequently Used (LFU): This algorithm is unique to Ehcahe. It uses a random sampling of items in the cache and picks the item with the lowest number of hits to evict. The Ehcache documentation claims that an element in the lowest quartile of use is evicted 99.99% of the time with this algorithm. In a cache that follows a Pareto distribution (20% of the items in the cache account for 80% of the requests) this algorithm may offer better performance than LRU. For more detailed discussion of various cache eviction algorithms, see the cache algorithms page on Wikipedia.

That's about it for this post on caching strategies and eviction policies. In Part 5 of this series, we'll finally start to take a look at caching in ColdFusion including what's always been there and what's new in ColdFusion 9.

A quick little plug: If you're heading to Adobe MAX 2009 in LA this October and want to know more about caching in ColdFusion 9, check out my session on Advanced ColdFusion Caching Strategies where I'll be covering a lot of what's already been discussed on my blog as well as a whole bunch of new material. I hope to see you there!

Welcome to Part 3 in my series on Caching Enhancements in ColdFusion 9. In Part 2, we talked about caching granularity. This time around, were going to spend some time discussing caching architectures. When talking about caching architectures, it's important to understand the type of cache being referred to. Basically, caches come in two flavors: in-process and out-of-process.

An in-process cache operates in the same process as its host application server. As I mentioned in Part 1 of this series, the new caching functionality in ColdFusion 9 is based on an implementation of Ehcache. Because Ehcache is an in-process caching provider that means that the cache operates in the same JVM as the ColdFusion server. The biggest advantage to an in-process cache is that it's lightning fast as data/object serialization is generally not required when writing to or reading from the cache. On the other side of the coin, in-process caches have limitations that you need to be aware of when it comes to system memory - particularly if you're on a 32-bit platform or a system that's light on RAM. On 32-bit systems, the JVM is typically limited to between 1.2GB and 2GB of RAM, depending on platform (although some 32-bit JVM's running on 64-bit systems may be able to use up to 4GB of RAM). Because you have to share this with your application server, that leaves considerably less RAM available to your cache.

In-process caches can be scaled up by adding more RAM, but not out by adding more servers as each cache is local to the application server's JVM it's deployed with. We'll discuss this in more depth when we talk about clustered caching. When using an in-process cache you always need to be aware of the number of items you'll be caching and how much RAM they take up to avoid a sudden spike in cache evictions if the available memory to both your application server and cache tops out. Fortunately for ColdFusion, Ehcache can be configured so that it fails over from RAM based storage to disk in the event that the cache fills up.

Out-of-process caches, like their name suggests, run outside of the same process as the application server. In the Java world, they run inside their own JVM. Out-of-process caches tend to be highly scalable on both 32-bit and 64-bit platforms as they scale both out and up. If you need to scale an out-of-process cache, you simply install more instances of the cache on any machines with spare RAM on your network. The main drawback to out-of-process caches is speed. Data and objects being written to and read from an out-of-process cache must be serialized and deserialized. Although the overhead for doing so is relatively small, it's still considerable enough to have an impact on performance.

Although Ehacahe itself is not an out-of-process cache, it does come with something called Ehcache Server which is available as a WAR file that can be run with most popular web containers or standalone. The Ehcache server has both SOAP and REST based web services API's for cache reads/writes. Another example of an out-of-process cache is the ever popular Memcached.

Now that we've covered the basics of in-process and out-of-process caches, it's time to make things a little more complicated by adding distributed caching and cache clustering to the mix. My experience over the last few years with caching has been that the term distributed tends to be a catch-all for what most would consider a true distributed cache as well as for a clustered cache. Confused yet? Let me attempt to clarify. Most of you are probably already familiar with how clustering works. In the application server world, you take an application server such as ColdFusion and you deploy it on two or more identically configured machines (or you can deploy multiple instances to one or more machines) which you then tie together through hardware and/or software. The result is that you are able to distribute load to your application across multiple servers which allows you to scale your application out. Need to be able to support more users? Add more servers to the cluster. It's the same for caching. If you have an in-process cache, you can't make the cache hold more items

When it comes to cache clustering, the primary reason for doing so is usually that you already have or are planning to deploy your application on a cluster. If you have a clustered application that needs to make use of caching, the first problem you face is that each application server has its own in-process cache which is local to the server. If Server A writes a piece of data to its in-process cache, that data is not available to Server B. This might not be a big deal for some clustered applications that implement sticky sessions, have light load or have data that doesn't necessarily need to be synchronized, but it becomes a serious problem for clusters that are configured for failover, have heavier load, or have cached data that needs to be in synch across every server in the cluster. In these instances, standalone in-process caching doesn't work well. The solution is to cluster your in-process caches as well as your application server. In the case of ColdFusion 9, the underlying Ehcache implementation fully supports caching. When configured, each local cache automatically replicates its content via RMI, JMS, JGroups, TerraCotta, or other plugable mechanisms to all other caches specified in the configuration. There's a small amount of latency while the data replicates but it's negligible in all but the most extreme use cases. I have set this up, tested, and verified it works with the ColdFusion 9 implementation. I'll put up a detailed post of exactly how to do this in a future blog post. The important thing to understand here is that clustering of in-process caches gets you redundancy, but the limit on the size of a single cache is still the limiting factor on scalability (e.g. if the cache you want to cluster has a limit of 500MB of data, clustering the cache between two servers means you are still limited to that 500MB of data in the cache, only now it's stored on two different servers).

Distributed caching differs from clustered caching in that a distributed cache is essentially one gigantic out-of-process cache spread across multiple machines. If you think of a clustered cache as comparable to a clustered application server then a distributed cache is much like a computing grid. Whereas a clustered cache gets you redundancy, a distributed cache gets you horizontal scalability with respect to how much data or how many objects can be put in the cache. Different distributed caching providers handle the exact caching mechanics differently, but the basics remain the same. If you need redundancy in a distributed cache, many distributed caching providers, including Ehcache Server let you cluster distributed cache nodes. The following diagram illustrates how a distributed, out-of-process cache cluster using Ehcache Server might look.

You should note that this is just one of many possible configurations. Using a combination of hardware and software it's possible to build out some pretty sophisticated caching architectures depending on your performance, scalability and redundancy requirements. It's even possible to create hybrid in-process/out-of-process architectures using solutions such as Terracotta.

That's about it for caching architectures. If you want to learn more, a fantastic resource is the website High Scalability. I hope you continue to find this series helpful. In Part 4 we'll cover our last foundation topic - the basics of caching strategies, before moving into ColdFusion 9's specific Ehcache implementation.