Wednesday, January 2, 2008
The Energy Detective - TED Footprints (Computer Power Monitoring Software)

Update: The folks at Energy are working on an SDK! So great news for us wanting to write native applications to interface with the TED!
Back in August I blogged about the TED 1000 and how it includes a serial port which can be hooked up to a PC. Well, the folks over at The Energy Detective finally released the software. During my conversations with them, it was my understanding that they were going to release the serial protocol for the unit as well. Instead, they just released the software that logs the information the TED is monitoring (it does log to HTML or Excel files so one could parse those). Not exactly what I expected but acceptable. Since my units firmware is v6.xx, I will need to upgrade before I can buy the monitoring software for $44.95 — which was also unexpected… I thought my unit would work when the software was released, since I bought it for this purpose. Oh, well.
For those interested the TED Footprints software does the following:- Dashboard view of real-time usage, displaying all data stored within TED on your PC
- View data in kW, Dollar, or in CO2 format at the push of a button
- Logs Time of Use in kWh, Dollars and Voltage
- Graphs/charts all logged data in kWh, Dollars, or Voltage
- Graphs/charts are printable
- Calendar for viewing is flexible: view day, week, month, year.
- Data is logged in background
- Data is logged in Excel format, as well as .htm format for further user analysis
- Load-Profile major appliances, such as Water Heater, HVAC system, or other heavy loads. Records time-of-use, run-time and cost-to-run specified loads.
- The Energy Detective Footprints Software
SPONSORED ADVERTISEMENTS
COMMENTS
I too was led to believe since I ordered a new TED 1001, that once the software was available, I would receive the software at some cost, though wouldn’t have to send in my unit. Very upsetting I now would have to send the unit in to take advantage of this option.
I too was expecting the protocol to be released.
Disappointing to say the least. I have a couple of applications for the serial data but no Windows machines available.
I wish I would have found this article sooner, I purchased the 1001 last week and thought the product came with the software but to my error it doesn’t.
I have a brand new unit version 8 that they want sent back in so the can activate the port!
This is normally done by pressing a couple of keys or in specific order but they say no they must open it up, I call BS.
When I asked about their return policy they said 3 days again I call BS and will not recommend this to anyone as all they are doing is price gouging.
You can easily modify the TED to get the raw one second data from either the serial port or USB port without interfering with the opration of the unit.
I think we need a user forum so we can discuss stuff like Klaxon’s idea. The data gathering service, TEDService.exe, is only 7 kilobytes, so I suspect that it is doing a fairly easy job.
It’s obviously feeding the data into a database, based I think on Open Source product called called sqlite3.
The database it builds is called TED.db.
The front-end program that the user interacts with is TEDviewer.exe, which seems to be a big Flash player that also does the database export functions and sets up the amount of each record type, and warns you if the logger isn’t getting data from the serial port.
There’s lots I don’t know. Anyone want to fill in the gaps? I look forward to an SDK — but what I really want is the spec so I can write a simple logger for a unix box. Trying to keep a windows machine up and running all the time is a real mistake. :)
Regards,
tai (eastpole.ca)
I have been running the TED1001 and Footprint software for almost two months now. No complaints with the TED unit. It does everything as stated. The accuracy, in my case, is within 1kWHr over a 15 day period. Can’t ask for much better than that.
It has been very helpful in reducing the electrical usage by about 33% so far. It is a great tool to show kids where electrical usage is going and has had a positive effect on them in helping reduce electrical cost.
I do however have issues with the software and very disappointed with it’s performance. I have no issues with the interface or the information that is given.
My big problem has been the following:
1.) Persistent COM errors. Near Midnight EVERY night, there is a COM error. The software usually recovers when this error occurs at midnight. However, the error does occur during the day and usually action on my part has to be taken to recover, usually unplugging and re-inserting the USB cable. This causes you to loose hours or days of electrical use history.
2.) The so called TEDService grows in size to about 200M with the TedViewer following close behind in size. These two programs use more resources than all other processes combined. I restarted this morning at 12M and it is now at 28M for TedService. Ted Viewer is at 23M. Again, both will grow in size.
3.) So far my request for help has gone unanswered.
I’m still trying to determine if the COM issue is on my end or not. I will update with any findings, and would be interested if others have seen this.
Just found out about this device ans was very interested in the PC interface but the comments here are concerning. Seems perhaps the software is not what it should be and there is no API?
Has anyone found a better method of accessing the data directly from the device so we can make custom software instead of using the included software?
I have expect the serial port model should be easiest to figure out the protocol if you have a spare Linux box with a pair of serial ports. Just connect the TED to one serial port, and then a null-modem cable from the other port to a Windows box running the Footprints s/w. Write a program on the Linux box to copy and log the data passing between the ports. Then watch the footprints display and compare the numbers with the data.
USB is harder as I don’t think that you can do a pass-thru with USB so easily. Maybe you could run windows as a guest in KVM or XEN and get the Linux box to log the data? If someone with a TED 1001 could plug it into a USB port on a Linux box and report on what “udev” reports for the USB device detected, that would help.
With any luck, the protocol is simply 1-way transmission of voltage and kW every second in some not-too-hard-to-decode way.
I am having similar issue with the TED 1001 as gazn357 in the earlier post. The support staff sent me a new unit (original was V8.0) with V8.01 installed and the port already unlocked. A new version of the Footprints was included. I uninstalled and reinstalled everything. I was having a problem with the original unit locking up and not logging to the computer thru the USB port. New unit is at least logging to the computer every second. I have the same comm port issues in that the data stops flowing to the console at various times after a reboot. Sometimes 5-10 minutes and sometimes several hours. Support had me list TED programs in windows firewall and virus software as exclusions to make sure those programs were not scanning the executables. Still no better. I am getting packet discarded messages in the log file when the TED loses connection to the console. Usually as you said, unplugging the USB cable fixes it for awhile. Support seems to stumped but seems to be interested in fixing this and had turned over screen shots and log files I sent to their devlopment staff to review. Hope to hear back in a day or so. This is not much good to me without the computer interface and history capability. I have tried several different ports with no better success. -Cliff
The latest driver for USB device can be obtained from FTDI website: http://www.ftdichip.com/Drivers/VCP.htm
The recent linux kernels have the driver compiled in.
Here is the content of /proc/bus/usb/devices
T: Bus=02 Lev=01 Prnt=01 Port=03 Cnt=01 Dev#= 2 Spd=12 MxCh= 0
D: Ver= 2.00 Cls=00(>ifc ) Sub=00 Prot=00 MxPS= 8 #Cfgs= 1
P: Vendor=0403 ProdID=6001 Rev= 6.00
S: Manufacturer=FTDI
S: Product=FT232R USB UART
S: SerialNumber=XXXXXXXX
C:* #Ifs= 1 Cfg#= 1 Atr=a0 MxPwr= 90mA
I:* If#= 0 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=ff Prot=ff Driver=ftdi_sio
E: Ad=81(I) Atr=02(Bulk) MxPS= 64 Ivl=0ms
E: Ad=02(O) Atr=02(Bulk) MxPS= 64 Ivl=0ms
The unit sends 280 bytes every second at 19200 bps.
There is a 4 byte marker at the beginning - ‘TED ‘
The current power consumption is reported in 10W units in two bytes at the offset 136(0x88)
“TED”,
I see the ‘TED ’ bytes… I’m looking for the 10W units now. So all we get right now is the power consumption — is that all the unit reports? I guess we can determine the rest ourselves in code.
Great start!
Danny
I got the current power reading figured out… thanks for the info! Now, how do we get the current voltage reading? I’m logging the data from the serial to try to figure out the stream… it may take me a couple days to see any patterns.
Danny
Some more info on the packet data.
The size of the packets is not constant. It is 280 most of the time, but there are packets with sizes up to 283. There are some fields that increase by one every minute (?) The offset of the power field varies at most by 1, no matter what the packet size. You could figure out what offset to use by using another field which changes by one when the power field is shifted. Watching my python dumper program via strace it turns out ‘TED ’ is not the beginning of the frame, ( a 850ms delay when reading from the serial port is in a different place) but there is no reliable marker for the frame start. The actual frame start is 113 bytes before ‘TED ’ If you are waiting for a full 280 bytes to be read you will be spanning frames and be delayed one second. Better strategy will be to find ‘TED ’ and read 140 ( or up to 167 bytes ) bytes from there as these bytes are guaranteed to be within the same frame.
I did not notice any changes in the content of the first 138 bytes of the frame ( from ‘TED ’ - 113 ) This is in over two days of data.
voltage reading:
2 bytes at offset 140(0x8C) from ‘TED ’
in 0.1V units
we could really use a forum.. There’s a guy on flickr who is doing some interesting stuff, appears to know what he’s doing.
I think this thing is great, even with limitations. I’ve got a $500 a month electric bill and am tracking down the energy hog. Clearly we need an new AC, mine uses 6 kilowatts, it’s a 40 yo ground water to air unit. And my little beverage fridge in the garage is eating electricity, how bad do i need that thing.
hacking this would be nice…
James,
I’ve got a lot of material now to create a .NET application to poll the TED directly. I will be posting some stuff soon.
Thanks,
Danny
I think I finally figured it out. The reason for different packet sizes is the special meaning of byte 0x10. This byte is used in markers for packet start and end. When transmitted in data it is encoded as two bytes - 0x10 0x10.
Here are the markers:
packet start - 0x10 0x04
packet end - 0x10 0x03
Each packet after conversion 0x10 0x10 -> 0x10 should have size 280 bytes
Power is at offset 249, voltage offset 253.
I am attaching a simple python program that reports these two values every second.
Save to a file and run ‘base64 -d file|gunzip’
to decode.
H4sICLTfWUgCA3MucHkArVNdT9swFH33r7h0qhJvaT4KnUa1IsHopgk2KiggjfHgNk6JSG3LcQb8
+13bSVTQHvcS2+d+nntu3u0lTa2TVSkS9WIepCCk3CqpDZhyy4nRL1MC0EJNrRThz2uujEWVLoWB
wWCA9y+yqXIQ0uz6xrCoOKs55PJJVJLlUGi5taEPxqhpkpQrpqVg8UbKTcUV2/A6Xstt4goBXK8a
YRpMxXUNaybsDQKmzGjDsZCoDasq8H2PbExAfDf8uTRhRglZXFwuYQZBkvM/iTEv11cnaUDIj+PL
Mwv/Tsf4PD05bwF3fHAHWZwtr5bHLryF0T09CKxh/vP0NbyPSW3A919zNIw/pbb07fzSPg4Oyc3F
+fL4mzNN9gnJeQHIAKcXqkcTgSwKaqeiuWm0AKlzi98hfE+xwHjyEd7voohl99TnyZutUmz9yF0u
lwZPrITfWHNVsTUPW4aRa5laQQuouHARMJtB27oN7mS18se10YW9hMHwKwyXAYUIBsNJfgvDgzgr
bgYwhHCXiSNNsdssjV5RbCdAIUFTnGLzqAZ26dbkiuuSVQtcm9AKFsE4TTE+O8STklVTFFxbuXDI
Tw9lxWGpG26bxQXQlusos6KLXMnavQDw7V29y2dIPbkuwieNi1LkYac0dR44Gu901MUA9C34y51z
mN631i5n+iZ+tpNAc/ZI/pEKpcRBoFAsD+2lFLesxLFtQurb8TpUnKsQ5zax4LrRnmjas2y59zR7
ly17Dv0jekMZdzjq/HCfaM++xY4g67pHsCXe2ka4f7Dn/4DOaUeAPqvdMV+K9l7/dRQ72992OPVd
4I8DbgveSufNqN1foucM1foEAAA=
Anyone know simple math to integrate the power into KHWs?
I wrote a simple server which broadcasts the power/voltage out on my home network.
I want to write a client, to store power usage.
Right now I’m collecting this:
Power 390.00W Voltage 120.00V
Power 390.00W Voltage 120.00V
Power 390.00W Voltage 120.00V
Power 400.00W Voltage 120.00V
Power 410.00W Voltage 120.00V
Power 410.00W Voltage 120.20V
Power 410.00W Voltage 120.20V
From 390.00W, if the load was constant, every hour I would add .39 KW/h to the total. If it goes up, to 0.4 KW/h, what would I do then?
The ted unit seems to send out an update every second; I take it 60 minutes * 60 seconds = 3600, divide the power by 3600, and add it to a running total constantly?
WE need a forum to share code etc.
Anyone know is this data actually in the data packet from the Ted unit?
Sean
My USB is having trouble connecting—- add to that my laptop throws out EMI that knocks out the data stream from the sending unit. Not a very good start… I would like a data logger that I could connect and down load occasionally. Keeping a computer up and running to log energy savings is a bit counter-intuitive.
If someone made a small logger that would be great. As far as I can tell the Footprints does not pull out data from the TED— say last week’s hour by hour numbers. I think it stores a single 24 hour data set.
Thank you,
George
I’m not sure how new this is, but it appears that the makers of TED are now offering an API here: http://www.theenergydetective.com/developer-/-api-request.html
I have not purchased a unit yet, but I am considering it - more so now that I see this.
the entire protocol has been decoded by these nice poeple:
http://tinyurl.com/5dt6m7
I have a forum setup for this… nothing started yet as it’s part of my my:ro control forum. Feel free to use it if you’d like.
http://www.myrocontrol.com/forum/viewforum.php?f=13
Danny
James Jones> There’s a guy on flickr who is doing some interesting stuff…
No James, if you mean me, I don’t know what I’m doing. I’m going to grab that little python script now and see if it works for me! Thanks Ted!

Welcome to mavromatic! My goal is to bring to your attention the latest DIY projects, the greatest home automation and home theater products. As well as, the occasional offbeat entry, to spice things up a bit. 


