Tuesday, June 14, 2016

How to send data to the cloud service Ubidots from a Sky mote using Contiki-OS?

Objective

I have a sensor node CM5000, which is compatible with a sky mote, and want to use it to send temperature data to the cloud service Ubidots. Currently, the node is working with the operating system Contiki-OS, specifically i am working with Instant-contiki-3.0.

I did some research and the web page below gives you the steps to send temperature data to Ubidots using a Z1 mote. I follow the steps of the web page, but since the sky mote is not compatible with a Z1 mote then it does not work.

http://wireless.ictp.it/school_2015/book/#_sending_data_to_ubidots

Here, i will show you how to send data to Ubidots using Sky motes modifying the code from the web page above.

Step 1: Create an account in Ubidots

Create a Data Source by clicking
```
Sources -> Add Data Source -> Generic -> Create
```
Now, you must have a data source named My Datasource.

Create a variable by clicking

Sources -> My Datasource -> Add Variable -> Name = Temperature -> Create

Now, you must have a variable named Temperature.

Keep in mind the variable's ID: Click in
```
Sources -> My Datasource 
```
Each variable at the top right-hand corner have an i, and if you click it, then it will show you the variable's ID. My variable's ID is:
```
567a05d07625424d60d32a31 
```
Keep in mind your API Key: Click in
```
My Profile -> API Keys
```
There you will find your API Key. My API Key is:
```
2eed47ae4097bbbfd26ae57c8eb799cc450d0416 
```

Create a Dashboard by clicking

Dashboard -> + -> Line Chart -> My Datasource -> Temperature -> Finish

Step 2: Create the serial-ubidots App

Go to the folder /contiki-3.0/apps
```
cd /home/user/contiki-3.0/apps
```
Create a folder named serial-ubidots
Within that folder, you must create a Makefile named Makefile.serial-ubidots with the following code in it:
```
serial-ubidots_src = serial-ubidots.c
```

Within that folder, you must create a file named serial-ubidots.c with the following code in it:

#include 
#include "serial-ubidots.h"
void
send_to_ubidots(const char *api, const char *id, uint16_t val) // api = API KEY, ID = Variable's ID, Val = Value of temperature
{
    unsigned char buf[6];
    snprintf(buf, 6, "%d", val); 
    printf("***%s\t", api); // The *** indicate the start of the data sent to Ubidots
    printf("%s\t", id);
    printf("%s***\n", buf); // The *** indicate the end of the data sent to Ubidots
 }

Within that folder, you must create a file named serial-ubidots.h with the following code in it:

#define VAR_LEN 24
 #define UBIDOTS_MSG_LEN (VAR_LEN + 2)
 struct ubidots_msg_t {
   char var_key[VAR_LEN];
   uint8_t value[2];
 };
 void send_to_ubidots(const char *api, const char *id, uint16_t val);

Include the serial-ubidots App in the makefile of sky motes: Go to the folder /home/user/contiki-3.0/examples/sky

cd /home/user/contiki-3.0/examples/sky

Open the Makefile and add the following line

APPS=serial-ubidots

Now, the Makefile should look like this

CONTIKI = ../..
ifndef TARGET
TARGET=sky
endif

all: blink sky-collect #rt-leds test-button test-cfs tcprudolph0

APPS=serial-ubidots

%.tgz: %.ihex
    mkdir $(basename $<) ; \
    mv $< $(basename $<) ; \
    echo $(basename $<)/$(basename $<).ihex 600 > $(basename $<)/runfile ; \
    tar czf $@ $(basename $<)

CONTIKI_WITH_IPV4 = 1
CONTIKI_WITH_RIME = 1
include $(CONTIKI)/Makefile.include

Step 3: Create a contiki C file that reads the temperature and send it to the serial port via the serial-ubidots app previously created

Go to the folder /home/user/contiki-3.0/examples/sky

cd /home/user/contiki-3.0/examples/sky

Create a file named temperature.c with the following code in it


#include "contiki.h"
 #include "dev/sht11/sht11-sensor.h"
 #include "dev/light-sensor.h"
 #include "dev/leds.h"
 #include 
 #include "serial-ubidots.h"

 //Declare the process
 PROCESS(send_sensor_info_process, "Print the Sensors Information");

//Make the process start when the module is loaded
AUTOSTART_PROCESSES(&send_sensor_info_process);

static uint16_t 
get_temp(void)
{
  return ((sht11_sensor.value(SHT11_SENSOR_TEMP) / 10) - 396) / 10;
}
/*---------------------------------------------------------------------------*/

//Define the process code
PROCESS_THREAD(send_sensor_info_process, ev, data)
{

  static struct etimer et;
  static const char api_key[] = "2eed47ae4097bbbfd26ae57c8eb799cc450d0416"; // This is your API Key previously copied from Ubidots
  static const char var_key[] = "567a05d07625424d60d32a31"; // This is your variable's ID previously copied from Ubidots

  PROCESS_BEGIN();

  uint16_t  raw;

  while(1){
    etimer_set(&et, CLOCK_SECOND * 15);
    SENSORS_ACTIVATE(sht11_sensor);
    PROCESS_WAIT_EVENT_UNTIL(etimer_expired(&et));
    raw = get_temp() ;
    send_to_ubidots(api_key, var_key, raw);
    etimer_reset(&et);
    SENSORS_DEACTIVATE(sht11_sensor);
 }
  PROCESS_END();
}

Upload the temperature.c code into the sky mote and login with the following command:

make TARGET=sky temperature.upload && make login

Then, you should be able to see this in the terminal every 15 seconds

***2eed47ae4097bbbfd26ae57c8eb799cc450d0416 567a05d07625424d60d32a31    29***

That means that the sky mote is sending over the /dev/ttyUSB0/ the following information

        *** = Indicates start of frame
        2eed47ae4097bbbfd26ae57c8eb799cc450d0416 = Api Key
        \t = Tab key to separate data 
        567a05d07625424d60d32a31 = Variable's ID
        \t = Tab key to separate data
        29 = Temperature (Celsius) 
        *** = Indicates end of frame

Step 4: Create a Python Script to read the data from the serial port and send it to Ubidots

Install Python in your computer

sudo apt-get install build-essential checkinstall
 sudo apt-get install libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev
 cd ~/Downloads/
 wget http://python.org/ftp/python/2.7.5/Python-2.7.5.tgz
 tar -xvf Python-2.7.5.tgz
 cd Python-2.7.5
 ./configure
 make

Install pip in Linux
```
 sudo easy_install pip
```
Install the Python library Ubidots for python
```
 sudo pip install ubidots==1.6.1
```
Go to the folder /home/user/contiki-3.0/examples/sky
```
 cd /home/user/contiki-3.0/examples/sky
```

Create a file named UbidotsPython.py with the following code in it:

# -*- coding: utf-8 -*-
 # ----------------------------------------------------------------------------#
 # Simple application to relay data to Ubidots from a Contiki serial-based conn
 # ----------------------------------------------------------------------------#


 import serial
 from time import sleep
 from ubidots import ApiClient

 # Use as default
 PORT = "/dev/ttyUSB0"

 # ----------------------------------------------------------------------------#
 # Create a serial object and connect to mote over USB
 # ----------------------------------------------------------------------------#
 def connectMote(port):
   try:
     ser = serial.Serial(port, 115200,timeout=0, parity=serial.PARITY_NONE,
               stopbits=serial.STOPBITS_ONE, bytesize=serial.EIGHTBITS)
   except:
     sys.exit("Error connecting to the USB device, aborting")
   return ser

   # ----------------------------------------------------------------------------#
   # Parse the serial data and publish to Ubidots, this assumes the following:
   # \r\n API_KEY \t VAR_KEY \t VALUE \r\n
   # ----------------------------------------------------------------------------#
   def process_data(raw):

   # Search for start and end of frame and slice, discard incomplete

   if "***" in raw:
    raw = raw[(raw.index("***") + 3):]

    if "***" in raw:
      raw = raw[:raw.index("***")]

      # We should have a full frame, parse based on tab and create a list
      ubidots_raw = raw.split("\t")
      # Create a Ubidots client and get a pointer to the variable
      client = ApiClient(ubidots_raw[0])
      try:
        my_variable = client.get_variable(ubidots_raw[1])
      except Exception, e:
        print "Ubidots error: %s" % e
        return
      # Update the variable
      my_variable.save_value({'value':int(ubidots_raw[2])})

 # ----------------------------------------------------------------------------#
 # MAIN APP
 # ----------------------------------------------------------------------------#
 if __name__=='__main__':

   # Create the serial object and connect to the mode
   # Do not check the serial port object as the function already does it
   s = connectMote(PORT)

   # Loop forever and wait for serial data
   while True:
     queue = s.inWaiting()
     if queue > 0:
       data = s.read(1000)
       process_data(data)
     sleep(0.2)

Step 5:Send data to Ubidots

Go to the folder /home/user/contiki-3.0/examples/sky
```
 cd /home/user/contiki-3.0/examples/sky
```
Upload the temperature.c code
```
 make TARGET=sky temperature.upload
```
Execute the script UbidotsPython.py
```
 python UbidotsPython.py
```
Wait approximately 15 seconds for the first data to appear in the dashboard of Ubidots.

Monday, June 13, 2016

Sending data from Sky motes to Mysql

Objective

The objective of this tutorial is to send temperature and light data from a sky mote connected to the USB port to a mysql database. This tutorial assumes that you already have mysql, python and contiki-3.0 running in your computer. This tutorial was tested in Ubuntu 14.04.

Step 1: Setting up python

In order to allow a communication between a sky mote and a mysql database a serial communication via the USB port is needed. Later in this tutorial, i will provide a python code that is in charge of getting the data from the sky mote and then save it into the mysql database. Python needs an extension called pySerial in order to be able to communicate with the serial port (USB). You can download the extension (pyserial-3.1.1.tar.gz) in the following link:

https://pypi.python.org/pypi/pyserial

Then, extract the archive, cd to the extracted folder and install the pySerial using the command.

python setup.py install

Now, you must install the MySQLdb library in order that python can talk with mysql. To that end, you must download the file MySQL-python-1.2.4b4.tar.gz from the following link:

http://sourceforge.net/projects/mysql-python/?source=dlp

Then, extract the archive, cd to the extracted folder and install the MySQLdb using the command.

python setup.py install

Step 2: Loading code to sky mote

The following code reads the temperature and light information from the sky mote, and then displays this information in the serial port (USB). This is a C code for Contiki-3.0

#include "contiki.h"
#include "/home/doctorado/contiki-3.0/dev/sht11/sht11-sensor.h"
#include "/home/doctorado/contiki-3.0/platform/sky/dev/light-sensor.h"
#include "/home/doctorado/contiki-3.0/core/dev/leds.h"
#include 

//Declare the process
PROCESS(send_sensor_info_process, "Print the Sensors Information");

//Make the process start when the module is loaded
AUTOSTART_PROCESSES(&send_sensor_info_process);

/*---------------------------------------------------------------------------*/
static int
get_light(void) //Get the light value from sensor
{
  return 10 * light_sensor.value(LIGHT_SENSOR_PHOTOSYNTHETIC) / 7;
}
/*---------------------------------------------------------------------------*/
static int
get_temp(void) //Get the temperature value from sensor
{
  return ((sht11_sensor.value(SHT11_SENSOR_TEMP) / 10) - 396) / 10;
}
/*---------------------------------------------------------------------------*/

int sequence = 0; // A sequence number that enumerates the data from 0 and increases in 1.

//Define the process code
PROCESS_THREAD(send_sensor_info_process, ev, data)
{


  static struct etimer et; // Struct used for the timer

  PROCESS_BEGIN(); 

 
  while(1){
 
  etimer_set(&et, CLOCK_SECOND * 5); // Configure timer to expire in 5 seconds

  SENSORS_ACTIVATE(light_sensor); // Activate light sensor
  SENSORS_ACTIVATE(sht11_sensor); // Activate temperature sensor

  PROCESS_WAIT_EVENT_UNTIL(etimer_expired(&et)); // Wait until timer expires 

  printf("Data\t", get_temp()); // Print the string "Data"
  printf("%d\t", sequence++);  // Print the sequence number
  printf("%d\t", get_temp()); // Print the temperature value
  printf("%d\n", get_light());  // Print the light value


  etimer_reset(&et); // Reset timer

  SENSORS_DEACTIVATE(light_sensor); // Deactivate light sensor
  SENSORS_DEACTIVATE(sht11_sensor);  // Deactivate temperature sensor

  }
  
  PROCESS_END();
}

Copy the code to an archive named TemperatureSky.c in the following folder of contiki-3.0

/home/YOUR_USER/contiki-3.0/examples/sky

In order to compile, to load the binary file to the sky mote and to visualize the temperature and light data you must write in terminal the following command

 
make TARGET=sky MOTES=/dev/ttyUSB0 TemperatureSky.upload && make login

Then, you must see in terminal the following

Data 0 28 182
Data 1 28 174

Where the first column is the sequence number which starts in 0 and increase always its value 1. I used this in order to enumerate the read data. The second column is the temperature value and the third column is the light value. Until now, you have configured the sky mote to send the temperature and light information via the serial port (USB). Then you must stop seeing the data pressing ctrl + c. This is important because you cann't see the data and run the python code at the same time.

Step 3: Creating a database in Mysql

To create a database and a table in Mysql which is appropiate for this application you must write the following commands in mysql. The first command creates a database called mote1, the second command uses this database, and the third command creates a table named weatherData. This table has 4 columns id, sequence, tempC and light.

CREATE DATABASE mote1;
USE mote1;
create table weatherData (id INT NOT NULL PRIMARY KEY AUTO_INCREMENT,sequence INT(11) NOT NULL,tempC INT(4) NOT NULL,light INT(4) NOT NULL);

Step 4: Creating a python code to read the temperature and light data and save it into mysql

The following python code reads the serial port (USB) and saves the temperature and light data into mysql, specifically python saves the information in the database named mote1 and in the table called weatherData.

#!/usr/bin/python
import serial 
import MySQLdb

#establish connection to MySQL. You'll have to change this for your database.
#dbConn = MySQLdb.connect("localhost","database_username","password","database_name") or die ("could not connect to database")
dbConn = MySQLdb.connect("localhost","root","1234","mote1") or die ("could not connect to database")
#open a cursor to the database
#cursor = dbConn.cursor()

#device = '/dev/tty.usbmodem1411' #this will have to be changed to the serial port you are using
device = '/dev/ttyUSB0' #this will have to be changed to the serial port you are using
try:
  print "Trying...",device 
  skyMote = serial.Serial(device, 115200) 
except: 
  print "Failed to connect on",device    

while True:
 #open a cursor to the database
 cursor = dbConn.cursor() 
 try:  
   data = skyMote.readline()  #read the data from the sky Mote
   pieces = data.split("\t")  #split the data by the tab
   print "pieces[0]=",pieces[0]
   print "pieces[1]=",pieces[1]
   print "pieces[2]=",pieces[2]
   print "pieces[2]=",pieces[3]
   #Here we are going to insert the data into the Database
   try:
     cursor.execute("INSERT INTO weatherData (sequence,tempC,light) VALUES (%s,%s,%s)", (pieces[1],pieces[2],pieces[3]))
     dbConn.commit() #commit the insert
     cursor.close()  #close the cursor
   except MySQLdb.IntegrityError:
     print "failed to insert data"
   finally:
     cursor.close()  #close just incase it failed
 except:
   print "Failed to get data from Sky mote!"

Copy the code in an archive named serial-mysql.py. Then run the code with the following command

python serial-mysql.py

Then, the you must see this in the terminal.

Dataes[0]= 
pieces[1]= 0
pieces[2]= 28
pieces[2]= 181

Dataes[0]= �
pieces[1]= 1
pieces[2]= 28
pieces[2]= 175

Finally, in order to see the table weatherData with the above values, type the following command in mysql

select * from weatherData;

You must see the following. That indicates that the values of temperature and light read by the sky mote are being saved in the mysql table.

+----+-----------+-------+-------+
| id | sequence  | tempC | light |
+----+-----------+-------+-------+
|  1 |         0 |    28 |   181 |
|  2 |         1 |    28 |   175 |