Data listening & parsing
In this section we will work on the receiver software, that will talk to a receiver XBee and figure out what the sensor data means. I'll be writing the code in python which is a fairly-easy to use scripting language. It runs on all OS's and has tons of tutorials online. Also, Google AppEngine uses it so its a good time to learn!
This whole section assumes that you only have 1 transmitter and 1 receiver, mostly to make graphing easier to cope with. In the next section we'll tie in more sensors when we get to the datalogging part!
Raw analog input
We'll start by just getting raw data from the XBee and checking it out. The packet format for XBees is published but instead of rooting around in it, I'll just use the handy XBee library written for python. With it, I can focus on the data instead of counting bytes and calculating checksums.
To use the library, first use the pyserial module to open up a serial port (ie COM4 under windows, /dev/ttyUSB0 under mac/linux/etc) You can look at the XBee project page for information on how to figure out which COM port you're looking for. We connect at the standard default baudrate for XBees, which is 9600 and look for packets
from xbee import xbee
import serial
SERIALPORT = "COM4" # the com/serial port the XBee is connected to
BAUDRATE = 9600 # the baud rate we talk to the xbee
# open up the FTDI serial port to get data transmitted to xbee
ser = serial.Serial(SERIALPORT, BAUDRATE)
ser.open()
while True:
# grab one packet from the xbee, or timeout
packet = xbee.find_packet(ser)
if packet:
xb = xbee(packet)
print xb
Running this code, you'll get the following output:
<xbee {app_id: 0x83, address_16: 1, rssi: 85, address_broadcast: False, pan_broadcast: False, total_samples: 19, digital: [[-1, -1, -1, -1, -1, -1, -1, -1, -1], [-1, -1, -1, -1, -1, -1, -1, -1, -1], [-1, -1, -1, -1, -1, -1, -1, -1, -1], [-1 , -1, -1, -1, -1, -1, -1, -1, -1], [-1, -1, -1, -1, -1, -1, -1, -1, -1], [-1, -1 , -1, -1, -1, -1, -1, -1, -1], [-1, -1, -1, -1, -1, -1, -1, -1, -1], [-1, -1, -1 , -1, -1, -1, -1, -1, -1], [-1, -1, -1, -1, -1, -1, -1, -1, -1], [-1, -1, -1, -1 , -1, -1, -1, -1, -1], [-1, -1, -1, -1, -1, -1, -1, -1, -1], [-1, -1, -1, -1, -1 , -1, -1, -1, -1], [-1, -1, -1, -1, -1, -1, -1, -1, -1], [-1, -1, -1, -1, -1, -1 , -1, -1, -1], [-1, -1, -1, -1, -1, -1, -1, -1, -1], [-1, -1, -1, -1, -1, -1, -1 , -1, -1], [-1, -1, -1, -1, -1, -1, -1, -1, -1], [-1, -1, -1, -1, -1, -1, -1, -1 , -1], [-1, -1, -1, -1, -1, -1, -1, -1, -1]], analog: [[190, -1, -1, -1, 489, -1 ], [109, -1, -1, -1, 484, -1], [150, -1, -1, -1, 492, -1], [262, -1, -1, -1, 492 , -1], [423, -1, -1, -1, 492, -1], [589, -1, -1, -1, 492, -1], [740, -1, -1, -1, 492, -1], [843, -1, -1, -1, 492, -1], [870, -1, -1, -1, 496, -1], [805, -1, -1, -1, 491, -1], [680, -1, -1, -1, 492, -1], [518, -1, -1, -1, 492, -1], [349, -1, -1, -1, 491, -1], [199, -1, -1, -1, 491, -1], [116, -1, -1, -1, 468, -1], [108, -1, -1, -1, 492, -1], [198, -1, -1, -1, 492, -1], [335, -1, -1, -1, 492, -1], [ 523, -1, -1, -1, 492, -1]]}>
...which we will reformat to make a little more legible:
<xbee {
app_id: 0x83,
address_16: 1,
rssi: 85,
address_broadcast: False,
pan_broadcast: False,
total_samples: 19,
digital: [ [-1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1]],
analog: [ [190, -1, -1, -1, 489, -1],
[109, -1, -1, -1, 484, -1],
[150, -1, -1, -1, 492, -1],
[262, -1, -1, -1, 492, -1],
[423, -1, -1, -1, 492, -1],
[589, -1, -1, -1, 492, -1],
[740, -1, -1, -1, 492, -1],
[843, -1, -1, -1, 492, -1],
[870, -1, -1, -1, 496, -1],
[805, -1, -1, -1, 491, -1],
[680, -1, -1, -1, 492, -1],
[518, -1, -1, -1, 492, -1],
[349, -1, -1, -1, 491, -1],
[199, -1, -1, -1, 491, -1],
[116, -1, -1, -1, 468, -1],
[108, -1, -1, -1, 492, -1],
[198, -1, -1, -1, 492, -1],
[335, -1, -1, -1, 492, -1],
[523, -1, -1, -1, 492, -1]]
}>
OK now its clear whats going on here. First off, we get some data like the transmitter ID (address_16) and signal strength (RSSI). The packet also tells us how many sample are available (19). Now, the digital samples are all -1 because we didn't request any to be sent. The library still fills them in tho so thats why the non-data is there. The second chunk is 19 sets of analog data, ranging from 0 to 1023. As you can see, the first sample (#0) and fifth sample (#4) contain real data, the rest are -1. That corresponds to the hardware section where we setup AD0 and AD4 to be our voltage and current sensors.
We'll tweak our code so that we can extract this data only and ignore the rest of the packet.
This code creates two arrays, voltagedata and ampdata where we will stick the data. We throw out the first sample because usually ADCs are a bit wonky on the first sample and then are good to go after that. It may not be necessary, though.
#!/usr/bin/env python
import serial
from xbee import xbee
SERIALPORT = "COM4" # the com/serial port the XBee is connected to
BAUDRATE = 9600 # the baud rate we talk to the xbee
CURRENTSENSE = 4 # which XBee ADC has current draw data
VOLTSENSE = 0 # which XBee ADC has mains voltage data
# open up the FTDI serial port to get data transmitted to xbee
ser = serial.Serial(SERIALPORT, BAUDRATE)
ser.open()
while True:
# grab one packet from the xbee, or timeout
packet = xbee.find_packet(ser)
if packet:
xb = xbee(packet)
#print xb
# we'll only store n-1 samples since the first one is usually messed up
voltagedata = [-1] * (len(xb.analog_samples) - 1)
ampdata = [-1] * (len(xb.analog_samples ) -1)
# grab 1 thru n of the ADC readings, referencing the ADC constants
# and store them in nice little arrays
for i in range(len(voltagedata)):
voltagedata[i] = xb.analog_samples[i+1][VOLTSENSE]
ampdata[i] = xb.analog_samples[i+1][CURRENTSENSE]
print voltagedata
print ampdata
Now our data is easier to see:
Voltage: [672, 801, 864, 860, 755, 607, 419, 242, 143, 108, 143, 253, 433, 623, 760, 848, 871, 811] Current: [492, 492, 510, 491, 492, 491, 491, 491, 492, 480, 492, 492, 492, 492, 492, 492, 497, 492]
Note that the voltage swings from about 100 to 900, sinusoidally.
Normalizing the data
Next up we will 'normalize' the data. The voltage should go from -170 to +170 which is the actual voltage on the line, instead of 100 to 900 which is just what the ADC reads. To do that we will get the average value of the largest and smallest reading and subtract it from all the samples. After that, we'll normalize the Current measurements as well, to get the numbers to equal the current draw in Amperes.
#!/usr/bin/env python
import serial
from xbee import xbee
SERIALPORT = "COM4" # the com/serial port the XBee is connected to
BAUDRATE = 9600 # the baud rate we talk to the xbee
CURRENTSENSE = 4 # which XBee ADC has current draw data
VOLTSENSE = 0 # which XBee ADC has mains voltage data
# open up the FTDI serial port to get data transmitted to xbee
ser = serial.Serial(SERIALPORT, BAUDRATE)
ser.open()
while True:
# grab one packet from the xbee, or timeout
packet = xbee.find_packet(ser)
if packet:
xb = xbee(packet)
#print xb
# we'll only store n-1 samples since the first one is usually messed up
voltagedata = [-1] * (len(xb.analog_samples) - 1)
ampdata = [-1] * (len(xb.analog_samples ) -1)
# grab 1 thru n of the ADC readings, referencing the ADC constants
# and store them in nice little arrays
for i in range(len(voltagedata)):
voltagedata[i] = xb.analog_samples[i+1][VOLTSENSE]
ampdata[i] = xb.analog_samples[i+1][CURRENTSENSE]
# get max and min voltage and normalize the curve to '0'
# to make the graph 'AC coupled' / signed
min_v = 1024 # XBee ADC is 10 bits, so max value is 1023
max_v = 0
for i in range(len(voltagedata)):
if (min_v > voltagedata[i]):
min_v = voltagedata[i]
if (max_v < voltagedata[i]):
max_v = voltagedata[i]
# figure out the 'average' of the max and min readings
avgv = (max_v + min_v) / 2
# also calculate the peak to peak measurements
vpp = max_v-min_v
for i in range(len(voltagedata)):
#remove 'dc bias', which we call the average read
voltagedata[i] -= avgv
# We know that the mains voltage is 120Vrms = +-170Vpp
voltagedata[i] = (voltagedata[i] * MAINSVPP) / vpp
# normalize current readings to amperes
for i in range(len(ampdata)):
# VREF is the hardcoded 'DC bias' value, its
# about 492 but would be nice if we could somehow
# get this data once in a while maybe using xbeeAPI
ampdata[i] -= VREF
# the CURRENTNORM is our normalizing constant
# that converts the ADC reading to Amperes
ampdata[i] /= CURRENTNORM
print "Voltage, in volts: ", voltagedata
print "Current, in amps: ", ampdata
We'll run this now to get this data, which looks pretty good, theres the sinusoidal voltage we are expecting and the current is mostly at 0 and then peaks up and down once in a while. Note that the current is sometimes negative but thats OK because we multiply it by the voltage and if both are negative it still comes out as a positive power draw
Voltage, in volts: [-125, -164, -170, -128, -64, 11, 93, 148, 170, 161, 114, 46, -39, -115, -157, -170, -150, -99]
Current, in amps: [0.064516129032258063, -1.096774193548387, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.096774193548387,
0.0, 0.0, 0.0, -0.064516129032258063, 0.0, 0.0, -0.70967741935483875, 0.0, 0.0]
Basic data graphing
Finally, I'm going to add a whole bunch more code that will use the numpy graphing modules to make a nice graph of our data. Note that you'll need to install wxpython as well as numpy, and matplotlib!
At this point, the code is getting waaay to big to paste here so grab "wattcher.py Mains graph" from the download page!
Run it and you should see a graph window pop up with a nice sinusoidal voltage graph and various amperage data. For example this first graph is of a laptop plugged in. You'll see that its a switching supply, and only pulls power during the peak of the voltage curve.
A laptop plugged in, switching power supply (above)
Now lets try plugging in a 40W incandescent light bulb. You'll notice that unlike the switching supply, the current follows the voltage almost perfectly. Thats because a lightbulb is just a resistor!
40W lightbulb (above)
Finally, lets try sticking the meter on a dimmable switch. You'll see that the voltage is 'chopped' up, no longer sinusoidal. And although the current follows the voltage, its still matching pretty well.
Light bulb on dimmer switch (above)
Graphing wattage!
OK neat, its all fun to watch waveforms but what we -really want- is the Watts used. Remember, P = VI otherwise known as Watts = Voltage * Current. We can calculate total Watts used by multiplying the voltages and currents at each sample point, then summing them up over a cycle & averaging to get the power used per cycle. Once we have Watts, its easy to just multiply that by 'time' to get Watt-hours!
Download and run the wattcher.py - watt grapher script from the download page
Now you can watch the last hour's worth of watt history (3600 seconds divided by 2 seconds per sample = 1800 samples) In the image above you can see as I dim a 40-watt lightbulb. The data is very 'scattered' looking because we have not done any low-pass filtering. If we had a better analog sampling rate, this may not be as big a deal but with only 17 samples to work with, precision is a little difficult.
Done!
OK great! We have managed to read data, parse out the analog sensor payload and process it in a way that gives us meaningful graphs. Of course, this is great for instantaneous knowledge but it -would- be nice if we could have longer term storage, and also keep track of multiple sensors. In the next step we will do that by taking advantage of some free 'cloud computing' services!
Text editor powered by tinymce.
