Raw data ======== These files contain the raw data that was logged by the gateway sensor mote. Originally, until 26th June 2013, the sensor mote was connected to a Crossbow Stargate single-board computer based around the Intel PXA255 processor. This originally ran a Postgres database, but this was replaced early in the project by a simple script that listened to the serial port and logged data to a logfile with a timestamp and a string of hexadecimal values. Files logged by the Stargate are named 'snifferNNNNNN.log'. The name 'sniffer' is a little misleading as the gateway sensor node was running a program named XMeshBase - which is not a packet sniffer. XMeshBase does not actually log every packet that it receives to the serial port - only data, health statistics and neighbourhood health packets are output to the serial port. (Crossbow did have another program named XSniffer that would have listened promiscuously and logged every packet, including various route-update and acknowledgement packets - but this would not have functioned as a root node for data collection.) The XMeshBase program running on the gateway node also logged 'heartbeat' packets to the serial port periodically. These do not correspond to any packets received by the gateway - the function of the heartbeat packets is as a diagnostic aid to distinguish between the case where there are no working sensor nodes in the network able to transmit to the gateway (in which case the gateway will only log heartbeat packets), and the case where the gateway mote itself has crashed or is otherwise non-functional (in which case it will log nothing at all). On 26th June 2013, the Crossbow Stargate was retired and replaced by a Raspberry Pi single-board computer connected to a Crossbow MIB520 gateway board. The original logging script was replaced by a functionally equivalent Python script. The logfile format did not change. However, the filename convention was changed to be data-YYYYMMDD-ptNN.log where YYYY, MM and DD are the year, month and day when the file was created and NN is an file 'part number' which increases by 1 when a new file is created. Logfile format -------------- Each line in the log file starts with a timestamp. There then follows a variable number of data bytes shown as two-digit hexadecimal numbers. The format of the data bytes depends on the packet type. These consist of a number of 1-byte or 2-byte fields. The 2-byte fields are transmitted and stored with the least significant byte first. Heartbeat packets: ~~~~~~~~~~~~~~~~~ Byte Offsets 0,1 TOS address: Always = 0x007e 2 Active Message Type: Always = 0xFD (AM_HEARTBEAT) 3 Group ID: Always = 0x7E = 127 for this deployment 4 Packet length: Always = 0x2 5,6 Heartbeat number: Monotonically increases 7,8 Checksum: e.g.: Byte Offset 0 1 2 3 4 5 6 7 8 1373705614.684 7E 00 FD 7E 02 94 0A 99 2A heartbeat packets are recorded every 60 seconds (approximately). MTS400 data packets: ~~~~~~~~~~~~~~~~~~~~ Byte Offsets 0,1 TOS address: Always = 0x007e 2 Active Message Type: Either 0x0B (AM_DATA2BASE) or 0x0D (AM_DATAACK2BASE) 3 Group ID: Always = 0x7E = 127 4 Packet length: Always = 0x25 7,8 Node ID: 11 Socket ID: Always = 0x33 12 Board ID: Always = 0x85 13 Packet ID: Always = 0x86 14,15 Parent Node ID 16,17 Battery Voltage 18,19 Humidity: Sensirion SHT11 raw humidity value 20,21 Temperature: Sensirion SHT11 raw temperature value 22,23 'calibW0' Intersema MS5534AM callibration coefficient word 0 24,25 'calibW1' Intersema MS5534AM callibration coefficient word 1 26,27 'calibW2' Intersema MS5534AM callibration coefficient word 2 28,29 'calibW3' Intersema MS5534AM callibration coefficient word 3 30,31 'prtemp': Intersema MS5534AM callibration raw temperature 32,33 'press': Intersema MS5534AM callibration raw pressure value 34,35 'taosch0': TAOS TSL2550D Light sensor Ch0 36,37 'taosch1': TAOS TSL2550D Light sensor Ch1 38,39 'accel_x': Analog ADXL202JE accelerometer X value 40,41 'accel_y': Analog ADXL202JE accelerometer Y value 42,43 Checksum e.g.: Byte Offset 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 1373705624.681 7E 00 0B 7E 25 00 00 06 00 00 00 33 85 86 0B 00 67 01 D6 03 20 15 C4 C1 D7 B4 1D B8 2A BF A6 62 32 46 80 FF 80 FF C6 01 C3 01 7C D8 Only the Sensirion SHT11 [1]/SHT71 [2] data were used in the project described in the accompanying paper. The TOAS Light [3] Sensor data and Analog Devices Accelerometer [4] data are stored in the database, but the Intersema Pressure sensor [5] data is only available from logfiles of the raw data. The conversion functions used by Crossbow in their XServe software were: Analog ADXL202E Accelerometer X Acceleration = 1000.0 * (1.0 - (500 - 'accel_x')/((500 - 400)/2)) Y Acceleration = 1000.0 * (1.0 - (500 - 'accel_y')/((500 - 400)/2)) TAOS TSL2550D Light sensor Light level = (16.5*((1 << ((taosch0 & 112) >> 4))-1) + (taosch0 & 15)*((1 << ((taosch0 & 112) >> 4))-1))*0.46 / exp(3.13*((16.5*((1 << ((taosch1 & 112) >> 4))-1) + (taosch1 & 15)*(( 1 << ((taosch1 & 112) >> 4))-1))/(16.5*((1 << ((taosch0 & 112) >> 4))-1) + (taosch0 & 15)*((1 << ((taosch0 & 112) >> 4))-1)))) Intersema MS5534AM Pressure sensor Pressure = ((((a << 1)+(d >> 6)*(y-(8*(((a & 1) << 10) | (b >> 6))+20224))/1024.0+24576)*(x-7168.0))/16384 - ((((c & 63) << 6) | (d &63))*4 + ((c >> 6)-512.0)*(y-(8*(((a & 1) << 10) | (b >> 6))+20224))/1024))/32.0 +250.0 Where: x = 'press' y = 'prtemp' a = 'calibW0' b = 'calibW1' c = 'calibW2' d = 'calibW3' The above functions were obtained by inspecting the mts400_mesh_config.xml file in the MoteView\xserve2\configxml folder on a machine with Crossbow XServe and MoteView[6] installed. This file unfortunately did not indicate which real-world units would result from these conversions. It is likely that the Accelerometer reading is in 'g' (or milli 'g'), the Light sensor is in 'Lux' and the Pressure is in millibar. Since these data were not used in the project, no effort was made to calibrate or check the results returned by these sensors. The descriptions are included here only for completeness. Additional information may be found in the Crossbow XMesh User's Manual[7] and MTS/MDA Sensor and Data Acquisition Board User's Manual [8] respectively. Inclinometer data packets: ~~~~~~~~~~~~~~~~~~~~~~~~~~ Byte Offsets 0,1 TOS address: Always = 0x007e 2 Active Message Type: Either 0x0B (AM_DATA2BASE) or 0x0D (AM_DATAACK2BASE) 3 Group ID: Always = 0x7E = 127 4 Packet length: Always = 0x1D 7,8 Node ID: 11 Socket ID: Always = 0x33 12 Board ID: Always = 0x81 13 Packet ID: Always = 0x86 14,15 Parent Node ID: 16,17 Battery Voltage: 18,19 Humidity: Sensirion SHT11 raw humidity value 20,21 Temperature: Sensirion SHT11 raw temperature value 22,23 Inclination: VTI SCA103T inclinometer [3] 24,25 Humidity2: Sensirion SHT71 [2] raw humidity value 26,27 Temperature2: Sensirion SHT71 raw humidity value 28,29 Validity flags: Bitfield indicating which data fields are valid 30,31 Reserved: 32,33 Reserved: 34,35 Checksum e.g.: Byte Offset 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 1231100731.840 7E 00 0B 7E 1D 00 00 0C 00 00 00 33 81 86 00 00 6E 01 D4 02 0F 13 12 0B BA 02 E4 12 1F 00 00 00 00 00 77 E7 The Validity flags field will be 0x001f if all sensors are 'valid' (no read errors): Valid Humidity = 0x0001 Valid Temperature = 0x0002 Valid Inclinometer = 0x0004 Valid Humidity2 = 0x0008 Valid Temperature2 = 0x0010 The inclinometer value is converted to a signed 16-bit integer before it is INSERTed into the 'inclo' field in the Anchorage-data table: 'inclo' = -32768 + (Inclination & 0x7fff) This can then be converted to engineering units using: Tilt (degrees) = (`inclo` * 18.5 ) / 32767.0 Health Statistics packets ~~~~~~~~~~~~~~~~~~~~~~~~~ Byte Offsets 0,1 TOS address: Always = 0x007e 2 Active Message Type: Always = 0x03 (AM_DEBUGPACKET) 3 Group ID: Always = 0x7E = 127 4 Packet length: Always = 0x1D 5,6 Source Address: 7,8 Node ID: 11 Socket ID: Always = 0x03 12 Old type: 0xF1 for a Health Statistics packet 13 Version: Always = 0x01 14 Type: Always = 0x01 for a Health Statistics packet 15,16 Health Packets: 17,18 Node Packets: 19,20 Forwarded: 21,22 Dropped: 23,24 Retries: 25 Battery: Battery voltage in tenths of 1V 26,27 Power Sum: 28 Board ID: 0x85 for a MTS400, 0x81 for an inclinometer (node 12). Also known as 'appid' 29,30 Parent Node ID: 31 Rx/Tx Link Quality: Top 4 bits - Rx link quality, Bottom 4 bits - Tx link quality 32 Path code: 33 Parent RSSI: Parent's Received Signal Strength Indicator 34,35 Checksum e.g.: Byte Offset 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 1373705638.776 7E 00 03 7E 1D 00 00 01 00 00 00 03 F1 01 01 19 00 D4 00 00 00 11 00 01 00 24 00 00 85 0B 00 FF 08 80 56 C9 Neighbour Health packets ~~~~~~~~~~~~~~~~~~~~~~~~ Byte Offsets 0,1 TOS address: Always = 0x007e 2 Active Messag Type: Always = 0x03 (AM_DEBUGPACKET) 3 Group ID: Always = 0x7E = 127 4 Packet length: Variable. Depends on number of neighbours 5,6 Source Address: 7,8 Node ID: 11 Socket ID: Always = 0x03 12 Old type: 13 Version: Always = 0x01 14 Type: Always = 0x02 for a Neighbour health packet 15,16 Neighbour1 ID: 17 Rx/Tx Link Quality: Top 4 bits - Tx link quality, Bottom 4 bits - Rx link quality 18 Path Cost: 19 RSSI Received Signal Strength Indicator 20,21 Neighbour2 ID: 22 Rx/Tx Link Quality: Top 4 bits - Tx link quality, Bottom 4 bits - Rx link quality 23 Path Cost: 24 RSSI Received Signal Strength Indicator 25,26 Neighbour2 ID: 27 Rx/Tx Link Quality: Top 4 bits - Tx link quality, Bottom 4 bits - Rx link quality 28 Path Cost: 29 RSSI Received Signal Strength Indicator 30,31 Checksum According to the documentation from Crossbow, there could be up to five neighbouring sensor nodes listed in each Neighbour Health packet. However, the maximum in practice appears to be 3 neighbours per packet. e.g.: Byte Offset 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1373705758.118 7E 00 03 7E 19 00 00 14 00 00 00 03 F3 01 02 15 00 FF 18 00 03 00 EC 09 00 0A 00 FF 04 00 F7 DA References ========== [1] Sensirion. Datasheet SHT1x (SHT10, SHT11, SHT15). Humidity and Temperature Sensor IC. Version 5 - December 2011. https://web.archive.org/web/20201216134918/https://www.sensirion.com/fileadmin/user_upload/customers/sensirion/Dokumente/2_Humidity_Sensors/Datasheets/Sensirion_Humidity_Sensors_SHT1x_Datasheet.pdf [2] Sensirion. Datasheet SHT7x (SHT71, SHT75). Humidity and Temperature Sensor IC. Version 5 December 2011. https://web.archive.org/web/20201216135053/https://www.sensirion.com/fileadmin/user_upload/customers/sensirion/Dokumente/2_Humidity_Sensors/Datasheets/Sensirion_Humidity_Sensors_SHT7x_Datasheet.pdf [3] VTI Technologies. SCA1003T Series Datasheet. The SCA103T Differential Inclinometer Series. Doc. Nr. 8261700 Rev.A https://web.archive.org/web/20170822104657/https://www.murata.com/~/media/webrenewal/products/sensor/gyro/selectionguide/sca103t_inclinometer.ashx [4] Analog Devices. ADXL202E*. Low-Cost +/- 2g Dual-Axis Accelerometer with Duty Cycle Output https://web.archive.org/web/20180127142556if_/http://www.analog.com:80/media/en/technical-documentation/data-sheets/ADXL202E.pdf [5] Intersema. MS5534A Barometer Module. DA5534_022.doc. July 17th, 2002. https://web.archive.org/web/20201218185331/https://datasheet.datasheetarchive.com/originals/distributors/Datasheets-9/DSA-179266.pdf [6] Crossbow MoteView Users Manual. Revision B, June 2006. PN: 7430-0008-04. https://web.archive.org/web/20060627163903/http://www.xbow.com/Support/Support_pdf_files/MoteView_Users_Manual.pdf [7] Crossbow XMesh User's Manual, Revision D, April 2007. PN: 7430-0108-01. https://web.archive.org/web/20071013051719/http://www.xbow.com/Support/Support_pdf_files/XMesh_Users_Manual.pdf [8] Crossbow MTS/MDA Sensor and Data Acquisition Board User's Manual. Rev. A, January 2006. Document 7430-0020-04. https://web.archive.org/web/20060909053632/http://www.xbow.com/Support/Support_pdf_files/MTS-MDA_Series_Users_Manual.pdf