Reading a Serial output from Arduino with Python V1

This is my first try at Python. It reads the Serial Output from Serial 3 on MotherHub. I had some problems with it randomly throwing exceptions, I seemed to have solved that with some If statements that count the amount of variables. The next 24 hours will be the big test as I just set the script up as a service for uninterrupted SQL insertions 🙂 I’m sure this script will expand over the coming months.

import serial
import MySQLdb
import datetime
from time import sleep
ser = serial.Serial('/dev/ttyAMA0', 9600, timeout=0) # This is using the serial GPIO pins on the Raspberry Pi, Pins 14 and 15
conn = MySQLdb.connect( host='Your MySQL server IP', db='SensorDB', user='SensorDBUser', passwd='SensorDBPass' ) #Change these to suit
cursor = conn.cursor()
count = 0
while True:
 data ="\r\n")
 parsed = data.split(",")
 if len(data) >= 54:
 print 'Serial Packet Length:', len(data)
 print 'Got:', data
 print 'parsed:', parsed
 templist = data.split(',')
 print 'Length:', len(templist)
# print templist
 if len(templist) < 12: #If there arent 12 items parsed from the Arduino output string then somethings not right, Go to Jail. Do not pass Go
 print 'templist[0]:', templist[0]
 print 'templist[1]:', templist[1]
 print 'templist[2]:', templist[2]
 print 'templist[3]:', templist[3]
 print 'templist[4]:', templist[4]
 print 'templist[5]:', templist[5]
 print 'templist[6]:', templist[6]
 print 'templist[7]:', templist[7]
 print 'templist[8]:', templist[8]
 print 'templist[9]:', templist[9]
 print 'templist[10]:', templist[10]
 print 'templist[11]:', templist[11]
 # Prepare SQL query to INSERT a record into the database.
 sql = "INSERT INTO tblSensorData(fldCommsMotion, fldCommsTemp, fldCommsBarometer, fldCommsHumidity, fldCommsLux, fldXbee1Temp, fldXbee1Volt, fldXbee1Battery, fldXbee1SolarV, fldTotalPowerUse, fl dHeatuse, fldLightPower) \
 VALUES ('%s', '%s', '%s', '%s', '%s', '%s', '%s', '%s', '%s', '%s', '%s', '%s' )" % \
 (templist[0], templist[1], templist[2], templist[3], templist[4], templist[5], templist[6], templist[7], templist[8], templist[9], templist[10], templist[11])
 print"sql ready for execution"
  # Execute the SQL command
 print"SQL Injected!"
 count = count + 1
 print 'SQL Update Count:', count #This is just so I can easily see how many updates have processed before an exception
# Commit your changes in the database
print 'not blocked'

MySQL Data is logging! This is massive motivation to get more sensors online. Even though I don’t have a frontend yet I know the data is there to play with.



In my travels with PHPMyAdmin I just noticed it has some basic graphing function! Amazing! When did they add that? Now I’ve noticed a problem with the power records


PHPMyADmin graphI have 2 sensors monitoring Power. One on the mains for the entire house and one on the Heating Circuit which does the hot water cylinder at the Heat pump. To calculate the lights and power circuit I simply minus the Heating circuit off the total power circuit. The graph however is showing that every time the heater comes on the power use for the house drops considerably. This is not correct. The difference between the Heating consumption and the total should always be consistent. I’m tempted to look further into this but at the same time I could put those efforts into completing my new energy monitor. I’ll think on it awhile and see if something comes to me


Posted in Arduino, Computing, Electronics, MySQL, phpMyAdmin, Python, Raspberry Pi, SensorNet | Tagged , , , , , | Leave a comment

Run Python code as a service

Testing of my python script that reads the serial output from MotherHub seems stable enough to run fulltime now. What I’ve found is though that everytime I turn my PC off my SSH session will time out and kill the python script from running. What I need is a way to run it all the time and auto start it after a reboot. Peiter Vanos has just the thing! I am reposting here for my own reference later. You can find the original code at

1. place your python script in the following folder: /usr/local/sbin/ (This can be anywhere you like)
2. create a new file in /etc/init.d/ in this case it will be example, so sudo nano /etc/init.d/example
3. Use the following code for the example file

#! /bin/sh
# /etc/init.d/example
case "$1" in
echo "Starting example"
# run application you want to start
python /usr/local/sbin/ &
echo "Stopping example"
# kill application you want to stop
killall python
echo "Usage: /etc/init.d/example{start|stop}"
exit 1
exit 0

4. give the file execute rights, sudo chmod 755 example
5. now execute: sudo update-rc.d example defaults

When you reboot raspbian it should automatically start the python script, else try service example start and service example stop


This worked like a charm for me. I could see SQL entries within a minute! Now to see if it can run 24/7 with no problems

Posted in Linux, Python, Raspberry Pi, SensorNet | Tagged , , | 1 Comment

Home SensorNet Overview

I have an ongoing project that involves my house and sensors. I’ve been working on and off on this as enthusiasm warrants but failing to actually post anything about it on dangertech. So this is it, I’m going to give a general overview interspersed with pictures as a starting point and see what happens. I plan on sensoring the shit out of my home. Sensors in every room that include at a bare minimum Temperature, motion and light levels. There are many more add-ons I want to achieve including Oven temperatures, Freezer temp, Water flow etc etc etc etc. I love stats and cant get enough.

SensorNet aka. Motherhub is an Arduino Mega2560 that runs the network of Xbees and arduinos around the house which so far consists of:

  • Arduino Mega2560 as MotherHub
  • Arduino UNO as data parser from power meter
  • Xbee Co-Ordinator as base unit for Xbee network
  • Outdoor Xbee Temp Sensor that also reports on battery level and solar Voltage
  • CurrentCost Energy Meter measuirng the mains power and outputting to above UNO

Ha. After listing that it doesn’t look like much for the hours of work I’ve poured in. There are also a few projects not completed enough that they are included in the above list:

Arduino UNO Weather unit

  • Wind Speed
  • Wind Direction
  • Rain Gauge
  • Temperature

This one is a little cheaty as it incorporates some Sparkfun parts. The wind and rain sensor are here and a convenient Ardy shield with some nice code on Github are here. To finish this I need to:

  • Xbee hooked up and coded
  • Power requirements calculated to run from solar and a battery
  • Housing designed and printed/built

Arduino House Foyer Sensor Node 

This is all Bread boarded up and tested. It includes:

  • 2 temperature sensors to monitor the hot water cylinder.
  • Room temp and humidity Sensors
  • Light Sensor
  • Motion Sensor

For completion it needs:

  • PCB to be designed and built
  • An enclosure

Arduino Power Monitor Node
This is to replace my CurrentCost device. It has served really well but since adding Solar to the house I need MOAR(tm) control. I currently monitor total power use and the HydroHeat circuits. The new unit will be capable of monitoring 4 circuits. Solar, power and lights, Hydroheat and Total Power consumption. As this sits by the front door it will also need to act as a doorbell, Light sensor, motion sensor and also temperature while communicating wirelessly to MotherHub through an xbee! A busy unit.

Things left to do:

  • Finish circuit board design
  • Enclosure
  • Arduino Code

Data Logging to MySQL and Web Frontend
Motherhub currently logs to Xively here. I want more control of the data and unlimited logging of that data over time which I haven’t seen offered by anyone else yet(for free). I’m sending data from MotherHub to a Python Script running on a Raspberry Pi at the moment. My NAS has an instance of MYSQL running so I log the data there. I could run python on the NAS or even my linux proxy server so save one device and its power consumption (not that the Pi uses much) but future wise I may move all of the sensor processing to the python script and more powerful Pi. Depends how MotherHub can handle 10-20 Xbees sending in packets and all the sensor processing 🙂 I’ve just made good progress on this by getting data into the database. No doubt I have a lot more time to be spent here.

  • Testing Script robustness when accepting data over serial from MotherHub
  • Running script as daemon on the Pi
  • PHP Front end to display graphs

Learning all this as I go but really starting to have fun with it. PCB design and Python being the latest skills I’ve started learning and have to admit enjoying immensely.

This is a great start to Documenting this whole thing! I have more of a plan after putting this to a post. I will do separate posts dealing individually with the different nodes that will include all the finer details of PCB designs and code. I know I said there would be pictures at the top of this post but i think its long enough already. I will add them in to the detailed posts of each arm of the project.If you have read all this then hopefully future posts can help you achieve your own SensorNet 2000 😛 I want to hear about it!

Posted in Arduino, Electronics, Raspberry Pi, SensorNet | Tagged , , , , , | Leave a comment

Looking for Wasted HD Space on the Raspberry Pi

Twice now I’ve ran out of space on my Raspberry Pi. First time I freed up some space and the second time that happened not to long after I went looking for the problem. This is how I found it…

danger@dangerpi ~ df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs          3.7G  2.9G  643M  82% /
/dev/root       3.7G  2.9G  643M  82% /
devtmpfs        180M     0  180M   0% /dev
tmpfs            38M  1.3M   37M   4% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            75M     0   75M   0% /run/shm
/dev/mmcblk0p1   56M   20M   37M  35% /boot
tmpfs            75M     0   75M   0% /tmp
/dev/sda1       925G  898G   27G  98% /mnt/DangerDrive

rootfs and /dev/root are the same thing in this case, the RPis SDCard. The above is after I found the culprit and cleaned it out. Now lets drill into it a little more

danger@dangerpi / $ sudo du -sh *
5.2M    bin
19M     boot
0       dev
4.8M    etc
3.3M    home
50M     lib
16K     lost+found
4.0K    media
898G    mnt
36M     opt
du: cannot access `proc/12707/task/12707/fd/4': No such file or directory
du: cannot access `proc/12707/task/12707/fdinfo/4': No such file or directory
0       proc
28K     root
1.3M    run
5.9M    sbin
4.0K    selinux
4.0K    srv
0       sys
0       tmp
1.3G    usr
661M    var

Aha! We can see which folders take up all the space!

danger@dangerpi /var/log $ sudo du -sh *
0       alternatives.log
4.0K    alternatives.log.1
4.0K    alternatives.log.2.gz
4.0K    alternatives.log.3.gz
4.0K    alternatives.log.4.gz
173M    syslog.1
0       syslog.1.gz
940K    syslog.2.gz
904K    syslog.3.gz
4.0K    user.log.4.gz
8.0K    wtmp
220K    wtmp.1
12K     Xorg.0.log

I’ve cut this down a a little but look at that syslog file! Perty big for a log file. What was happening once I had a look at it was transmission which runs on this particular Pi was spamming errors about not being able to find some files. I deleted the logs taking up all the space with

sudo rm -f syslog.*

and then removed the offending files from transmission. Hopefully this proves to be the root of the problems.

Posted in Computing, Linux, Raspberry Pi | Tagged , , | 1 Comment