What is MemCache?
Some of you have probably just stumbled along this post without actually knowing what MemCache does, so here is a bit of technical context before I dive into the implementation methods, problems and of course solutions.
In it’s most basic form MemCache is a normal program that runs on top of your operating system, typically you run this on something like your web server as a separate service, just like you would Apache, NTP, MySQL etc.
The service has a very simple aim, you provide it with a key and some corresponding data, this is then saved in memory via the MemCache process and you can access it at a later date. You leverage the benefit in two ways, firstly because the data is held in RAM which is much faster than other storage options, and secondly because the data is typically already in a usable format and therefore requires little extra processing by your application, saving time and physical resources.
How do I use it?
Well I did start of by writing an explanation, however here is an example (with some inline comments):
// We will start by making a new MemCache instance, and assign it to the variable $mc
$mc = new Memcache;
// Using the $mc instance we will connect to our server, normally this will be on port 11211 (localhost)
$mc->connect('127.0.0.1', 11211) or die ("Could not connect");
// Memcache will save data to a specific unique key, you can set this to almost anything
$thekey = "PostCount";
// Now we are going to define the data, this will be saved to our $thekey
$thedata = "12340";
// Now to actually pass the key and data to memcache, note '0' means we are not going to have this expire after a specified amount of time.
$mc->set($thekey, $thedata, false, 0);
// Using $thekey we will get the memcache data back, this will be saved into our variable $result
$result = $mc->get($thekey);
echo $result; //will return 12340
Problems?
The majority of todays applications are built on top of relational databases such as MySQL, therefore adding a basic key/data technology on top of them for cacheing can be hell for programmers. With Key based data you have to know the exact key otherwise you will get nothing returned, MySQL however allows you to search by almost any method you can think of using the saved data.
Another problem is building a solution to define keys that’s going to be accurate, scalable and easy to implement. For example i could save web based user information to memcache using a key based on their numeric user-id (Eg: Key = 412, UserName = Bob), however I then need to go through all of my code and before performing a SQL lookup, perform a memcache lookup, that’s going to be a headache. Plus some things such as the login progress might want to convert ‘bob’ into the userid ’412′, that’s not possible as ‘bob’ is my memcache data, which you can’t search by, you can only request data by using the Key.
Solution, Hash Key
The solution is fairly simple to implement, you leave almost all of your code as it is and go to your database class (I am assuming that you have one, most developers use one), below is an example extract of what yours could look like, before we get started changing it.
// Function: pass some SQL and it will return the result
// (Insecure, just shown as an example)
function sql_search($sql){
// Connect to my database
mysql_connect("localhost", "root", "password") or die(mysql_error());
mysql_select_db("testdb") or die(mysql_error());
// Run the SQL query
$result = mysql_query($sql) or die(mysql_error());
return mysql_fetch_array( $result );
}
Solution: So what you do is accept the $sql string, run it through an md5() hash function, and check if there is any memcache result using the hash as your key, if there is skip the SQL, if there isnt run the SQL but then add the result to memcache.
// Function: pass some SQL and it will return the result
// (Insecure, just shown as an example)
function sql_search($sql){
// Create the hash key
$thekey = md5($sql);
// Connect to memcache server
$mc = new Memcache;
$mc->connect('127.0.0.1', 11211) or die ("Could not connect");
$result = $mc->get($thekey);
// If there was a memcache hit, return the cache result
if(!empty($result)) return $result;
// Connect to my database, there was no cache hit
mysql_connect("localhost", "root", "password") or die(mysql_error());
mysql_select_db("testdb") or die(mysql_error());
// Run the SQL query
$result = mysql_query($sql) or die(mysql_error());
// Save the result to memcache
$mc->set($thekey, $result, false, 0);
return mysql_fetch_array( $result );
}
Overview
As you can see it’s a simple solution, no matter what application your using if there is a SQL backend and also a SQL class/function memcache can be added in with very minimal work, here are a few last tips:
- Build a variable into your SQL function to disable/enable cache, I have a few applications that only use it in high server load times.
- Consider building a function to clear the cache for some of your variables, for example if you add a blog comment you might want to clear blog comment cache.
- Sometimes just use memcache code if the data isn’t valuable, i have done this with stat’s information before, and just poll/clear memcache every few hours.
- Don’t forget cache will be deleted after some time, so make sure your application doesn’t error if this happens.
- Build some checks in to returned memcache data, don’t just assume that its worked
