I would like to share some experiences of working with Microsoft AppFabric Cache for Windows Server.

AppFabricCache is distributed cache solution from Microsoft. It has very simple API and would take you 10-20 minutes to start playing with. As I worked with it for about one month I would say that product itself is very good, but probably not enough mature. There are couple of common problems and I would like to share those. But before let’s get started!

If distributed cache concept is something new for you, don’t be scary. It is quite simple. For example, you’ve got some data in your database, and you have web service on top of it which is frequently accessed. Under high load you would decide to run many web instances of your project to scale up. But at some point of time database will become your bottleneck, so as solution you will add caching mechanism on the back-end of those services. You would want same cached objects to be available on each of the web instances for that you might want to copy them to different servers for short latency & high availability. So that’s it, you came up with distributed cache solution. Instead of writing your own you can leverage one of the existing.

Install

You can easily download AppFabricCache from here. Or install it with Web Platform Installer.

Installation process is straight forward. If you installing it to just try, I wouldn’t even go for SQL server provider, but rather use XML provider and choose some local shared folder for it. (Provider is underlying persistent storage as far as I understand it.)

After installation you should get additional PowerShell console called “Caching Administration Windows PowerShell”.

So you can start your cache using: “Start-CacheCluster” command.

Alternatively you can install AppFabric Caching Amin Tool from CodePlex, which would allow you easily do lot of things though the UI. It will show PowerShell output, so you can learn commands from there as well.

Usually you would want to create named cache. I created NamedCacheForBlog, as can be seen above.

Simple Application

Let’s now create simple application. You would need to add couple of references:

Add some configuration to your app/web.config

  <section name="dataCacheClient" type="Microsoft.ApplicationServer.Caching.DataCacheClientSection, Microsoft.ApplicationServer.Caching.Core" allowLocation="true" allowDefinition="Everywhere"/>
  
<!-- and then somewhere in configuration... -->
  
  <dataCacheClient requestTimeout="5000" channelOpenTimeout="10000" maxConnectionsToServer="20">
    <localCache isEnabled="true" sync="TimeoutBased" ttlValue="300" objectCount="10000"/>
    <hosts>
      <!--Local app fabric cache-->
      <host name="localhost" cachePort="22233"/>
      <!-- In real world it could be something like this:
      <host name="service1" cachePort="22233"/>
      <host name="service2" cachePort="22233"/>
      <host name="service3" cachePort="22233"/>
      -->
    </hosts>
    <transportProperties connectionBufferSize="131072" maxBufferPoolSize="268435456"
                       maxBufferSize="134217728" maxOutputDelay="2" channelInitializationTimeout="60000"
                       receiveTimeout="600000"/>
  </dataCacheClient>

Note, that above configuration is not the minimal one, but rather more realistic and sensible. If you are about to use AppFabric Cache in production I definitely recommend you to read this MSDN page carefully.

Now you need to get DataCache object and use it. Minimalistic, but wrong, way of doing it would be:

public DataCache GetDataCacheMinimalistic()
{
    var factory = new DataCacheFactory();
    return factory.GetCache("NamedCacheForBlog");
}

Above code would read configuration from config and return you DataCache object.

Using DataCache is extremely easy:

object blogPostGoesToCache;
string blogPostId;
dataCache.Add(blogPostId, blogPostGoesToCache);
var blogPostFromCache = dataCache.Get(blogPostId);
object updatedBlogPost;
dataCache.Put(blogPostId, updatedBlogPost);

DataCache Wrapper/Utility

In real world you would probably write some wrapper over DataCache or create some Utility class. There are couple of reasons for this. First of all DataCacheFactory instance creation is very expensive, so it is better to keep one. Another obvious reason is much more flexibility over what you can do in case of failures and in general. And this is very important. Turns out that AppFabricCache is not extremely stable and can be easily impacted. One of the workarounds is to write some “re-try” mechanism, so if your wrapping method fails you retry (immediately or after X ms).

Here is how I would write initialization code:

private DataCacheFactory _dataCacheFactory;
private DataCache _dataCache;
private DataCache DataCache
{
    get
    {
        if (_dataCache == null)
        {
            InitDataCache();
        }
        return _dataCache;
    }
    set
    {
        _dataCache = value;
    }
}

private bool InitDataCache()
{
    try
    {
        // We try to avoid creating many DataCacheFactory-ies
        if (_dataCacheFactory == null)
        {
            // Disable tracing to avoid informational/verbose messages
            DataCacheClientLogManager.ChangeLogLevel(TraceLevel.Off);
            // Use configuration from the application configuration file
            _dataCacheFactory = new DataCacheFactory();
        }

        DataCache = _dataCacheFactory.GetCache("NamedCacheForBlog");

        return true;
    }
    catch (DataCacheException)
    {
        _dataCache = null;
        throw;
    }
}

DataCache property is not exposed, instead it is used in wrapping methods:

public void Put(string key, object value, TimeSpan ttl)
{
    try
    {
        DataCache.Put(key, value, ttl);
    }
    catch (DataCacheException ex)
    {
        ReTryDataCacheOperation(() => DataCache.Put(key, value, ttl), ex);
    }
}

ReTryDataCacheOperation performs retry logic I mentioned before:

private object ReTryDataCacheOperation(Func<object> dataCacheOperation, DataCacheException prevException)
{
    try
    {
        // We add retry, as it may happen,
        // that AppFabric cache is temporary unavailable:
        // See: http://msdn.microsoft.com/en-us/library/ff637716.aspx
        // Maybe adding more checks like: prevException.ErrorCode == DataCacheErrorCode.RetryLater

        // This ensures that once we access prop DataCache, new client will be generated
        _dataCache = null;

        Thread.Sleep(100);
        var result = dataCacheOperation.Invoke();

        //We can add some logging here, notifying that retry succeeded
        return result;
    }
    catch (DataCacheException)
    {
        _dataCache = null;
        throw;
    }
}

You can go further and improve retry logic to allow for many retries and different intervals between retries and then put all that stuff into configuration.

RetryLater

So, why the hell all this retry logic is needed?

Well, when you open MSDN page for AppFabric Common Exceptions be sure RetryLater is the most common one. To know what exactly happened you need to verify ErrorCode.

So far I’ve see this sub-errors of the RetryLater:

There was a contention on the store. – This one is quite frequent one. Could happen when someone is playing some concurrent mess with cache. Problem is that any client can affect the whole cluster.

The connection was terminated, possibly due to server or network problems or serialized Object size is greater than MaxBufferSize on server. Result of the request is unknown. – This usually has nothing to do with object size. Even if configuration is correct and you save small objects you can still get this error. Retry mechanism is good for this one.

One or more specified cache servers are unavailable, which could be caused by busy network or servers. – Have no idea how frequent this one could be, but it can happen.

No specific SubStatus. – Amazing one!

Conclusion

AppFabricCache is very nice distributed cache solution from Microsoft. It has a lot of features. Of course not described here, as you can read it elsewhere, say here. But to be able to go live with it you should be ready for AppFabricCache not being extremely stable & reliable, so you better put some retry mechanisms in place.

To be honest if I was one to make decision if to use this dcache, I would go for another one. But who knows, maybe other are not much better… I’ve never tried other distributed caches.

Links

• Introduction white paper (pdf direct link)
• Slides from MIX10
• AppFabric on MSDN

Thank you, and hope this is of some help.