Presence Protocol implementation - Redis/Nodejs

For the recent enterprise collaboration product which we have created similar to Yammer/Asana , i had to implement an Instant Messenger .  I have built a contact list using a proprietary technique  but i could not find a solution to implement presence protocol without using XMPP protocol.  I explored the XMPP setup with Nodejs and figured out that it was little cumbersome and costly solution . I was struck for 3-4 days before i could find a better solution as given below.

While stumbling upon , i found a simple solution for presence protocol for Instant Messenger without using XMPP protocol.
I have come up with an great article http://www.lukemelia.com/blog/archives/2010/01/17/redis-in-practice-whos-online/ . This solution uses Redis Sadd,Sunion,Sintersect techniques

The idea is to have one active set per minute. During each request that comes in from a logged-in user, we’ll add a user ID to the active set. When we want to know which user IDs are online, we can union the last 5 sets to get a collection of user IDs who have made a request in the last 5 minutes. 

Though i got a clue from above algorithm , we had lot of problems to tackle.

I need to enhance the solution to make it suitable for my application with more real-time presence.

I need to maintain 3 state in my application . 1. Online 2. Offline 3. Idle.
The real issue is in implementing similar technique in Nodejs and detecting each request and update the Active set. I need to modify the existing Redis library(connect-redis) as it does not support sadd,sunionstore and sinterstore.

I added couple of additional steps to this solution to implement complete presence protocol for my application .

1. Whenever the New user Logs-in , Online status is broadcasted to the people who had this user as a member list and the information is broadcasted using socket.io and update the state field in corresponding collection in Mongodb and/or add the member from the active set .
2. Whenever the user logs-out (Tracked using Socket-disconnect) , Offline status is published to the people who had this user as a member list  and update the state in corresponding collection in Mongodb/and or remove the member from the active set .
3. Maintain Friend's List for each user in Redis Database. Whenever the new friend is added/removed the list will get updated on per user basis.
4. Every 3 minutes ,the client polls the server to get the latest presence information for checking the state. I called the Redis function and  publishes back to the client. This offloaded the CPU usage at the server end in tracking "Idle" subscribers.

 4.1 The solution is to find the Online Users and Intersect with the Friend's list maintained in Step-3 . The resultant set is the Online contacts of given user

 4.2 Do the difference of the resultant set from the User's Friends list - That gives the list of users that are either Offline or Idle. Since we already publish in step-2 about the offline members, the remaining members are IDLE. Great we got the idle members too .

Steps involved :
===========
1. Create Cron Job every one minute and the set should expire in 10 minutes. This makes sures that redis is not overloaded with data and any given time only 10 active set is maintained .
2. Modified Connect-Redis Library to include sinterstore,sunionstore,sdiffstore etc.
3. The online-members set created every minute 
4. Included additional code for additional steps mentioned in 1,2,3 above .

1. Cron-job that runs every minute in Nodejs

new cronJob('0 */1 * * * *', function(){
        var time =  new Date().getTime();
        var Hour = new Date().getHours();
        var minute = Math.round(time/60000) ;
        var key =  "presence"+":"+minute;
        logger.info('You will see this message every minute',time,minute,Hour,key);
        sessionStore.set_online_users("xxxxxxx",function(res){ 

                if(res)

               {
                logger.debug("presence info stored in redis ");

               }
});
}, null, true, "America/Los_Angeles");

2. Modified connect-redis Library with additional functionality 

/* Active set created every minute that stores the list of Online Users */
  RedisStore.prototype.set_presence_info = function( user_id,fn){

        var time =  new Date().getTime();
        var minute = Math.round(time/60000) ;
        var key =  "presence"+":"+minute;
        try {
        // debug('SETEX "%s" ttl:%s %s', sid, sess);
        this.client.sadd( key, user_id,function(err){
                err || debug('online users added ');             
                fn && fn.apply(this, arguments);
        });
        this.client.expire(key,600,function(err){
                logger.info("expiration set");
        });
        } catch (err) {
                fn && fn(err);
                } 
};

/* Online Users tracked every 5 minutes */ 

RedisStore.prototype.get_presence_info = function(number_of_minutes, fn){
        logger.info("Presence Information entered");
        var time =  new Date().getTime();
        var minute = Math.round(time/60000) ;   
        logger.log('info','Presence Information entered minute', { is : minute });
        var key2 = minute-1 , key3 = minute-2 , key4 = minute-3, key5 = minute-4;
        var key1 = "presence"+":"+minute;
        var key2 = "presence"+":"+ key2;
        var key3 = "presence"+":"+ key3;
        var key4 = "presence"+":"+ key4;
        var key5 = "presence"+":"+ key5;
        try{
        this.client.sunionstore('online:users',key1,key2,key3,key4,key5, function(err, data){
//      this.client.sunionstore('online:users',keys, function(err, data){
                if (err) return fn(err);
                if (!data) return fn();
        return fn(null, data);
        });
        } catch (err) {
                fn && fn(err);
                }
};

/* List of Friends of a particular User */

RedisStore.prototype.set_friendlist_info = function( user_id, new_user, fn){
        var key = "friendslist"+":"+user_id ;
        try {
                this.client.sadd( key, new_user,function(err){
                        err || debug('friend included');       
                        fn && fn.apply(this, arguments);
                });

                this.client.expire(key,600,function(err){
                        logger.info("expiration set");
                });
        } catch (err) {
                fn && fn(err);
        } 
};

/*  Online Friend's list of a particular User */

RedisStore.prototype.set_friends_info = function(user_id,fn){
        var key = "friends"+":"+user_id;
        var friendslist = "friendslist"+":"+user_id ;
        logger.log('info','Friends', {  Key : key});
        try{
        this.client.sinterstore(key,friendslist,'online:users', function(err, data){
                if (err) return fn(err);
                if (!data) return fn();
                return fn(null, data);
        });
        } catch (err) {
                fn && fn(err);
                } 
};

We can also use srem function to remove particular member in a set.

this.client.srem(key1,user_id, function(err, data){
                if (err) return fn(err);
                if (!data) return fn();
                return fn(null, data);
        });

 Hope this helps and if needed more information let me know .