Skip to main content

IoT Hackathon Part IV : Using Web Services to send Sensordata

In the previous 3 posts, building towards the eProseed IoT Hackathon, I described how to setup your Raspberry Pi, and how to use the GrovePi sensors. The used example is a small weather-station that read temperature and humidity and shows the readings on a display. That is all very nice, however, the data remains local on the Raspberry Pi so there is nothing that we can do with this information from an 'enterprise' perspective. In this post I will explain how easy it is to send the data to whatever 'end point' by using a REST-JSON web-service.

The Database Tables
For this use case I decided I needed 2 tables. One to hold all my available sensors (yes, I know, I have only one) and one to store the measurements per sensor.


The Webservice

In order to store the data in the database tables I need to send it from the Raspberry Pi to the database. For that I use a simple REST/JSON webservice that has a POST method. I also implemented PUT, GET and DELETE methods, because they might come in handy later.

The Webserivce is described by the following WADL:
 <ns0:application xmlns:ns0="http://wadl.dev.java.net/2009/02">  
   <ns0:doc xmlns:ns1="http://jersey.java.net/" ns1:generatedBy="Jersey: 2.5.1 2014-01-02 13:43:00"/>  
   <ns0:doc xmlns:ns2="http://jersey.java.net/"  
        ns2:hint="This is simplified WADL with user and core resources only. To get full WADL with extended resources use the query parameter detail.  Link: http://yourhost.com:7101/IoTRestJsonService/resources/application.wadl?detail=true"/>  
   <ns0:grammars>  
     <ns0:include href="application.wadl/xsd0.xsd">  
       <ns0:doc title="Generated" xml:lang="en"/>  
     </ns0:include>  
   </ns0:grammars>  
   <ns0:resources base="http://yourhost.com:7101/IoTRestJsonService/resources/">  
     <ns0:resource path="iot">  
       <ns0:resource path="/sensordata">  
         <ns0:method id="createSensorData" name="POST">  
           <ns0:request>  
             <ns0:representation element="iotSensorData" mediaType="application/json"/>  
           </ns0:request>  
         </ns0:method>  
         <ns0:method id="updateSensorData" name="PUT">  
           <ns0:request>  
             <ns0:representation element="iotSensorData" mediaType="application/json"/>  
           </ns0:request>  
         </ns0:method>  
         <ns0:method id="findAllSensorData" name="GET">  
           <ns0:response>  
             <ns0:representation element="iotSensorData" mediaType="application/json"/>  
           </ns0:response>  
         </ns0:method>  
       </ns0:resource>  
       <ns0:resource path="/sensordata/{id}">  
         <ns0:param xmlns:xsd="http://www.w3.org/2001/XMLSchema" name="id" style="template" type="xsd:int"/>  
         <ns0:method id="deleteSensorData" name="DELETE"/>  
         <ns0:method id="getSensorDataById" name="GET">  
           <ns0:response>  
             <ns0:representation element="iotSensorData" mediaType="application/json"/>  
           </ns0:response>  
         </ns0:method>  
       </ns0:resource>  
     </ns0:resource>  
   </ns0:resources>  
 </ns0:application>  

The service can be tested from any REST client. You can use 'Test Webserice' from within JDeveloper, or use a tool such as postman to POST a test message.
Calling Webservices from Python

Database is in place, Webservice is up and running, so the only remaining thing is to call the service from my Python code. By now it starting to get pretty obvious that this also cannot be very difficult with Python. And indeed, it is very simple again. Python comes with two libraries that can be used for this purpose: The request library and the json library. When you use those two, it takes only a couple of minutes to implement the webservice call. The code sample below shows you how to add the import of the two libraries and then how to construct the request; The request of course needs a resource URL in our case the relevant part is'iot/sensordata'. We also need to tell the request that is a json type request. This is part of the header. Finally we need to construct our data string that contains the measurement data. Here we can construct the exact JSON string that is expected by the service. The data that is sent in the request is simply dumped as JSON string, by calling "json.dumps()". This is the exact same string as can be seen from the Postman screenshot, whit other data ofcourse.

Note that I use sensorid =1 because this is the unique identifier of this sensor. To be 100% dynamic we could also use the ip adres of the Raspberry Pi as sensorid. We could call out to http://httpbin.org/ip to get the ip adres. For now I keep it simple and stick with the hardcoded value of 1.

 # import for the ws  
 import requests  
 import json  
 # end import for the ws  

 # here we prepare and call the ws  
  url = "http://yourhost.com:7101/IoTRestJsonService/resources/iot/sensordata"  
  headers = {'Content-Type':'application/json'}  
  sensordata = {"dataDate":i.isoformat(),"dataType":"T", "dataValue":t,"dataValueNumber":t,"sensorid":"1" }  
  r=requests.post(url, data=json.dumps(sensordata),headers=headers)  
Finally I share the complete code of the Python script with you so you can see and learn how to do this.
 # grovepi_lcd_dht.py  
 #  
 # This is an project for using the Grove LCD Display and the Grove DHT Sensor from the GrovePi starter kit  
 #   
 # In this project, the Temperature and humidity from the DHT sensor is printed on the DHT sensor  
 from grovepi import *  
 from grove_rgb_lcd import *  
 import logging  
 import datetime  
 # for the ws  
 import requests  
 import json  
 # end for the ws  
 dht_sensor_port = 7          # Connect the DHt sensor to port 7  
 # lets log to file  
 logger = logging.getLogger('weather.logger')  
 logger.setLevel('DEBUG')  
 file_log_handler = logging.FileHandler('/home/pi/Desktop/Lucs_projects/weatherstation/weather.log')  
 logger.addHandler(file_log_handler)  
 stderr_log_handler = logging.StreamHandler()  
 logger.addHandler(stderr_log_handler)  
 formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')  
 file_log_handler.setFormatter(formatter)  
 stderr_log_handler.setFormatter(formatter)  
 while True:  
      try:  
           [ temp,hum ] = dht(dht_sensor_port,0)          #Get the temperature and Humidity from the DHT sensor  
           t = '{0:.2f}'.format(temp)  
           h = '{0:.2f}'.format(hum)  
           i = datetime.datetime.now()  
           logger.info("temp ="+ t + "C\thumidity ="+ h + "%")   
           setRGB(0,128,64)  
           setRGB(0,255,0)  
           setText("Temp:" + t + "C   " + "Humidity:" + h + "%")  
         # here we prepare and call the ws  
           url = "http://yourhost.com:7101/IoTRestJsonService/resources/iot/sensordata"  
           headers = {'Content-Type':'application/json'}  
         #prep the temp message  
           sensordata = {"dataDate":i.isoformat(), "dataType":"T", "dataValue":t,"dataValueNumber":t,"sensorid":"1"}  
           r=requests.post(url, data=json.dumps(sensordata),headers=headers)  
         #prep the humidity message  
           sensordata = {"dataDate":i.isoformat(), "dataType":"H", "dataValue":h,"dataValueNumber":h,"sensorid":"1"}  
           r=requests.post(url, data=json.dumps(sensordata),headers=headers)  
         # end ws call  
            time.sleep(60)                 
      except (IOError,TypeError) as e:  
           logger.error('Error')  
           logger.error(r)  
Note that, almost at the end of the script, I use a time.sleep(60). This actually puts the script to sleep for one minute. This means, that every minute the new data is measured and sent to the webservice and saved in the database. It is very simple to build an ADF Page to display the data in some graphs. That is beyond the scope of this blogpost, however, here is an image of the result.

Note
The core purpose of this post is to describe how to use webservices from python, to forward sensor data to a (cloud) server. In a later post I describe how to use the better suited MQTT protocol. An example of this is CloudMQTT. CloudMQTT are managed Mosquitto servers in the cloud.Mosquitto implements the MQ Telemetry Transport protocol, MQTT, which provides lightweight methods of carrying out messaging using a publish/subscribe message queueing model.

Resources
1) RESTful Web Service in JDeveloper 12c
2) Calling REST/JSON from Python

Comments

Popular posts from this blog

ADF 12.1.3 : Implementing Default Table Filter Values

In one of my projects I ran into a requirement where the end user needs to be presented with default values in the table filters. This sounds like it is a common requirement, which is easy to implement. However it proved to be not so common, as it is not in the documentation nor are there any Blogpost to be found that talk about this feature. In this blogpost I describe how to implement this.

The Use Case Explained
Users of the application would typically enter today's date in a table filter in order to get all data that is valid for today. They do this each and every time. In order to facilitate them I want to have the table filter pre-filled with today's date (at the moment of writing July 31st 2015).


So whenever the page is displayed, it should display 'today' in the table filter and execute the query accordingly. The problem is to get the value in the filter without the user typing it. Lets first take a look at how the ADF Search and Filters are implemented by the f…

ADF 12.2.x : Conditional Showing Message Instead of List of Values Popup

Today I had to fix an issue in one of our ADF applications. For those of you working with ADF (as I do too occasionally) this might be valuable, so I decided to share this. The application is based on an old Oracle Forms applications and I had to implement the following functionality:

FRM-41830 : List of Values contains no entries.

In other words (more ADF like) : If you are about to render a List of Values and that List of Values contains no rows, just show a message instead of showing the List of Values.

In this post I will describe how I was able to implement this although it turned out to be a challenge.
Below are the steps that I took. Note that the Application can be downloaded from GIT.

1) I use a UI pattern where we use readonly table and do edits in a popup. That means that any List of Values will add an additional popup to the UI.



2) The List of Values that I use in this sample only shows entries where the minimum salary is less then the employees current salary. Not that …

ADF 11g Quicky 3 : Adding Error, Info and Warning messages

How can we add a message programatically ? Last week I got this question for the second time in a months time. I decided to write a short blogpost on how this works.

Adding messages is very easy, you just need to know how it works.
You can add a message to your faces context by creating a new FacesMessage. Set the severity (ERROR, WARNING, INFO or FATAL ), set the message text, and if nessecary a message detail. The fragment below shows the code for an ERROR message.

1: public void setMessagesErr(ActionEvent actionEvent) {
2: String msg = "This is a message";
3: AdfFacesContext adfFacesContext = null;
4: adfFacesContext = AdfFacesContext.getCurrentInstance();
5: FacesContext ctx = FacesContext.getCurrentInstance();
6: FacesMessage fm =
7: new FacesMessage(FacesMessage.SEVERITY_ERROR, msg, "");
8: ctx.addMessage(null, fm);
9: }


I created a simple page with a couple of buttons to show the result of setting the message. When the butto…