On 15th March ‘14, we at SDSLabs and PAG organized the international ACM-ICPC style programming challenge, Insomnia as a part of annual tech fest of IIT Roorkee, Cognizance. It was a 12 hour long overnight contest hosted on our online coding portal, CodeVillage. We saw participation from over 170 teams and over 1000 submissions. Such response was overwhelming for us. Let me tell you how we prepared CodeVillage for a contest on this scale.

CodeVillage was also used for Insomnia 2013. At that time we could not properly estimate the load before-hand and optimize our platform accordingly. Lots of performance issues were faced by the users. In spite of enormous efforts put in by the problem setters, the event turned out to be a disappointment. This made us realize the need to tackle this issue before we could ever hope to host Insomnia on CodeVillage or take the platform online in future.

So, we started preparing early this time. We refactored a lot of code to optimize database queries, removed unneccessary static resources, minified others and used browser caching. Basically we did everything that meant even a tiniest bit improvement in page load speed. All this helped but what worked like magic for us was memcaching of database load.

Memcached

Memcached is a distributed memory object caching system that speeds up your application by alleviating database load.

Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.

If you are the curious kind, go read this story and it should remove your doubts.

So, we grabbed this great tool, installed it on our server and started playing with it. Its implementation is a fairly simple task. All you have to do is wrap your code for SELECT query using an instance of Memcache() class. Here is how:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
<?php
    
    // instance of Memcache()
    $meminstance = new Memcache();

    // connect to your server        
    $meminstance->pconnect( 'localhost', 11211 );    

    $query = "SELECT * FROM table";

    // create key for your query
    $querykey = "KEY" . md5( $query ); 

    // first try to get data from cache
    $data = $meminstance->get( $querykey );    

    if ( !$data ) {
        // if the data was not present in cache, fetch it from the database
        $result = mysql_query( $query );
        $data = mysql_fetch_assoc( $result );

        // and save data corresponding to the key in cache for a specified time
        $meminstance->set( $querykey, $pdata, 0, 43200 );    
    }
?>

Voilà! No more slow CodeVillage. We cached heavy and constant database queries like competition and problem data for the duration of the contest. But this meant that memcaching had to be removed before editing data during the contest or the changes won’t be reflected, so we made sure all problem statements were proof read several times in advance to avoid any editing.

We also carried out proper testing of CodeVillage before Insomnia through dummy contests, apache benchmarking and another ACM-ICPC style online contest, Algophobic during Srishti, the annual techno-hobby exhibition organised by the Hobbies Club, IITR.

After all this, Insomnia ran pretty smoothly and was a success. So, that’s how memcached saved the day, or rather night!