Caching big data in application using MemoryCache

Possibly save 2 hours of your time: I have only encountered this more recently in an application that deals with big dataset.
In this scenario, the application needs to provide a feature to download full datasets. And this dataset is huge (~75MB to ~125MB) in size. And there are many datasets.
It took awhile to query this data and do some calculation and send it back to the client app. So we decide to cache this data at the application level using System.Runtime.Caching.MemoryCache for subsequent faster performance. As you can see, the problem can become bigger as more datasets are downloaded and cached.
There are few options in no particular order:
  1. More expensive but easiest solution may be to increase memory on physical server to account for the size of the cache that can grow to.
  2. Recycle the application pool after reaching maximum virtual memory usage. See here for specifics.
  3. Set the CacheMemoryLimit on the MemoryCache object. This sets the amount of memory on the computer, in bytes, that can be used by the cache.

 

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s