Posted by & filed under Development.

During implementation of HutGrip’s dashboard and especially the feeds part, we had to find a proper way of taking the latest data of each feed in a feed group. In HutGrip, the user will be able to define a logical group of feeds, where each feed is used to store data that comes from widgets used in a scenario. For a simple scenario where you have a digital input form a motion sensor and digital output for switching the lights, you can create a feed that will store information each time the lights are switched on or off. Of course in one group you can combine different feeds: for the lights, the heating, the temperature, etc.

In order to show the current status, we had to find a way to take the latest data form the feeds in a single feed group. We are using NodeJS for the back-end and Couchbase to storing the data.

We decided to make a view with a simple reduce function. Here is the code for the view:

1
2
3
4
5
6
function (doc, meta) {
  if (doc.type == "feedData")
  {
    emit(doc.feedId, {date: doc.date, sysDate: doc.sysDate, value: doc.value});
  }
}

 

The code of the reduce function:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
function(key, values, rereduce) {
  var result = null;
  for (var i = 0; i < values.length; i++) 
  {
    if(result == null)
      result = values[i];
    else     
      if(result.date < values[i].date)
        result = values[i];
  }
  return result
}

Of course, we set the group_level to 1 in order to take the data for each feed:

“TestFeedId_1″
undefined
{ "date": "2013-02-01T08:00:02.000Z", "sysDate": "2013-02-19T12:30:12.138Z", "value": 3 }
“TestFeedId_2″
undefined
{ "date": "2013-02-01T08:00:02.000Z", "sysDate": "2013-02-19T12:40:26.899Z", "value": 3 }
“TestFeedId_3″
undefined
{ "date": "2013-02-01T08:00:02.000Z", "sysDate": "2013-02-19T12:58:07.894Z", "value": 3 }

 

To test the view, we added for each feed three feed data values with different datetime. As you can see the view works just fine by giving us the latest value for all the three test feeds. Now having working view, lets see how we did it in Node (the code is in coffee script):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
getLatestFeedData = (feedId, callback = null) ->
  return unless callback?

  cb.connect config, (err, bucket) ->    
    return callback null, err if err      

    options = {
      key: feedId,
      group_level: 1,
      stale : "false"}

    bucket.view "feedData","vFeedLatestData", options, (err, resp, view) ->
      return callback(null, err) if err        
      return callback(null, null) if not resp? || resp.length == 0

      callback {
        feedId: resp[0].key, 
        date: new Date(resp[0].value.date), 
        sysDate: new Date(resp[0].value.sysDate), 
        value: resp[0].value.value}, null

 

The function above returns the latest value of a single feedId. To take the values for several feeds you can change the options you pass to the bucket.view:

1
2
3
4
5
options = {
  keys: feedIds,
  group_level: 1,
  group: true,
  stale : "false"}

 

Here is another good read by Tugdual Grall on getting the latest document by datetime.

Leave a Reply

  • (will not be published)