Diagnostics Logs

Each Vector has a built in log of messages showing significant event and issues during its lifetime. The space available for logging error messages is very large and can store many tens of thousands of entries before old entries are deleted to make room for new ones.

in most cases the compete life of the device can be seen in the error log file - from first boot up and programming in the factory, right up to present day.

To view the log file, connect to the Vector and then go to the Diagnostic tab. Logs will be shown live, but can also be downloaded and viewed offline by clicking the Download button on the right. After a few seconds a file will download which can be opened in our preferred editor.

A lot of the log is written in human readable English, but unless you know what it means it is of little use. 

The below messages are explained in detail here as they are relevant to the diagnoses of many different issues.

 

Message Format

This is an example entry:

<Entry Code="27" Type="0x00000000" Level="Info" Timestamp="2022-09-15T11:38:49.0000000Z" Data="" FriendlyMessage="Connection: SSL accepted connection" />

The important parts are as follows...

  • The "Entry Code=" part shows the index number of each entry. Each line in the log will have its own unique index number so that you can see the correct order, starting at 0. This also means that setting of the date is not required to read the messages.
  • The "Level=" shows the type of entry. Valid Levels are 
    • 'Info' - An event happened. E.g. a setting was changed, the accelerometer was initialised, an MQTT connection succeeded 
    • 'Warning' - An event failed or went wrong, but counting performance in unaffected. E.g. an MQTT connection failed
    • 'Error' - Something went wrong which is affecting performance. E.g. there is a detected hardware fault, the Vector boot sequence failed.
  • The "Timestamp=" is the date/time the entry was made. Note that the time must have been set by someone as the date starts from 01/01/1970 otherwise. Use the Index number for correct time sequencing.
  • The "FriendlyMessage=" gives you the human readable entry of the actual entry.

 

Common Entries and Their Meaning

Boot message

<Entry Code="5007" Type="0x00000000" Level="Info" Timestamp="2022-09-15T09:21:53.0000000Z" Data="" FriendlyMessage="--- uImage: 2.0.145, uBoot: 1.0.2, webPackage: 9.3.0.9, HW: 3, BootMode: mmc ---" />

This is the first message written following a bootup. It contains a few very useful pieces of information.

Whilst a Vector cannot write a power off message in the event of power loss, by checking the timestamp of this message and the preceding one you can narrow the time window of the power outage and how long the device was actually off for.

The uImage shows the firmware version onboard. 

The BootMode should show mmc meaning it is booting correctly off the SD card. If it says nor then there is a fault with the SD card itself - either corrupt or missing/ejected. This would also be indicated with the below additional message:

<Entry Code="25" Type="0x00000000" Level="Warning" Timestamp="2022-09-15T11:38:14.0000000Z" Data="" FriendlyMessage="UImageBackup: failed - Boot mode is nor" />

Diagnosing further you may also see the following messages:

<Entry Code="23" Type="0x00000000" Level="Error" Timestamp="2022-09-15T11:38:14.0000000Z" Data="" FriendlyMessage="KCS: webfiles partition not found, or could not be mounted" /> <Entry Code="22" Type="0x00000000" Level="Error" Timestamp="2022-09-15T11:38:14.0000000Z" Data="" FriendlyMessage="KCS: system partition not found, or could not be mounted" /> <Entry Code="21" Type="0x00000000" Level="Error" Timestamp="2022-09-15T11:38:14.0000000Z" Data="" FriendlyMessage="KCS: data partition not found, or could not be mounted" />

Importantly, seeing all three partition missing messages indicates that the SD card has failed completely or more likely has been inadvertently removed. If you only see one or two of these messages then the card is probably not faulty, but has become corrupt, and requires re-imaging.

 

KCS Alive

<Entry Code="6676" Type="0x00000000" Level="Info" Timestamp="2020-09-01T00:00:00.0000000Z" Data="" FriendlyMessage="KCS is still alive" />
This message is written once a day as a kind of 'all is ok' record. Note the time of 00:00:00.
This should be expected every 24hrs. If any are missing then the device was powered off over the midnight period. Use this in conjunction with the above Boot Message to determine long periods of power outage.

 

Reboot Message

<Entry Code="5000" Type="0x00000000" Level="Info" Timestamp="2022-09-15T09:21:10.0000000Z" Data="" FriendlyMessage="1 seconds to reboot." /> <Entry Code="4999" Type="0x00000000" Level="Info" Timestamp="2022-09-15T09:21:09.0000000Z" Data="" FriendlyMessage="2 seconds to reboot." /> <Entry Code="4998" Type="0x00000000" Level="Info" Timestamp="2022-09-15T09:21:08.0000000Z" Data="" FriendlyMessage="3 seconds to reboot." /> <Entry Code="4997" Type="0x00000000" Level="Info" Timestamp="2022-09-15T09:21:08.0000000Z" Data="" FriendlyMessage="Reboot countdown activated: 3 seconds to reboot." />

Whilst a power outage results in no entry, these four entries indicate that someone chose the 'Reboot' option from with the RIFT setup software.

 

Network Issues

<Entry Code="4843" Type="0x00000000" Level="Warning" Timestamp="2022-09-15T09:15:27.0000000Z" Data="" FriendlyMessage="CommsParser: Could not fit TxBufferFull message on to TX buffer" /> <Entry Code="4842" Type="0x00000000" Level="Warning" Timestamp="2022-09-15T09:15:27.0000000Z" Data="" FriendlyMessage="CommsParser::SendPacket failed on V4D-22060090:1 - Tx buffer full" />

Either or both of these messages indicate a poor network that the Vector is having issues communicating on.

The below message may also been seen. If seen soon after a power on event this is normal, but continued logging of this message indicates a network issue.

<Entry Code="6946" Type="0x00000000" Level="Warning" Timestamp="2021-01-11T10:03:04.0000000Z" Data="" FriendlyMessage="UDP socket error during TX - Network is unreachable" />

 

Time Change

<Entry Code="4725" Type="0x00000000" Level="Info" Timestamp="2022-09-11T13:47:56.0000000Z" Data="" FriendlyMessage="SYSTEM CLOCK CHANGED from 2022-09-11T13:47:52+0000" />

A time synchronisation event occurred. Note the Timestamp of the entry and the new time in the main part of the entry - this sync removed four seconds of drift.

Time syncs can occur automatically when an Estate Manager or NTP connection is available, or if software sets the time programmatically using the REST API.

Having more than one time sync mechanism will cause the clock to change unnecessarily often, which could lead to a failure of the clock IC inside the Vector. Therefore if you see lots of time sync messages in the logs you should change the configuration so that there is only one time sync mechanism. I.e. via NTP or via Estate Manager, or via the REST API.

 

Tilt Warning

<Entry Code="4084" Type="0x00000000" Level="Warning" Timestamp="2022-08-26T08:31:09.0000000Z" Data="" FriendlyMessage="DepthTracker: detected tilt ({&quot;x&quot;:0.1265864074230194,&quot;y&quot;:0.0024239951744675636,&quot;z&quot;:-0.991952657699585}) significantly differs from configured ({&quot;x&quot;:0.026485448703169823,&quot;y&quot;:0.0034018007572740316,&quot;z&quot;:-0.9996434450149536})" />

This means that the stored angles now differ from the measured ones. This could indicate an issue relating to a change in the installation - someone has knocked the device, or moved the ceiling tile that it was installed on - but could be an indiction that the device has not been completely setup yet.

To fix this issue perform an Auto-Height to update the tilt angles. You will then see a corresponding 'Tilt Warning Cancelled' message:

<Entry Code="4085" Type="0x00000000" Level="Info" Timestamp="2022-08-26T08:31:10.0000000Z" Data="" FriendlyMessage="DepthTracker: tilt warning cancelled" />
A Tilt Warning entry is often seen in conjunction with a Height warning entry.

 

Height Warning

<Entry Code="4095" Type="0x00000000" Level="Warning" Timestamp="2022-08-26T11:00:36.0000000Z" Data="" FriendlyMessage="DepthTracker: detected height (3.546319) significantly differs from configured (3.700943)" />

Similar to the Tilt Warning above, the stored height is different to the most recently measured height. Again, this could indicate an issue relating to a change in the installation - someone has knocked the device, or moved the ceiling tile that it was installed on - but could be an indiction that the device has not been completely setup yet.

To fix this issue perform an Auto-Height to update the height setting to the measured one. You will then see a corresponding 'Height Warning Cancelled' message:

<Entry Code="4096" Type="0x00000000" Level="Info" Timestamp="2022-08-26T11:22:24.0000000Z" Data="" FriendlyMessage="DepthTracker: height warning cancelled" />

 

Auto Configuration Successful (Auto-Height successful)

<Entry Code="4061" Type="0x00000000" Level="Info" Timestamp="2022-08-26T08:25:25.0000000Z" Data="" FriendlyMessage="DepthTracker: auto-configuration successful @ 3.700943" />
If someone runs the 'Auto Height' routine you should expect to see this log entry.

 

Factory Reset Button

    <Entry Code="166" Type="0x00000000" Level="Info" Timestamp="1970-01-01T05:44:48.0000000Z" Data="" FriendlyMessage="FactoryResetButtonMonitor::factoryResetButtonLoop: Button pressed for &gt; 5 second(s), resetting to defaults." />    <Entry Code="165" Type="0x00000000" Level="Info" Timestamp="1970-01-01T05:44:44.0000000Z" Data="" FriendlyMessage="FactoryResetButtonMonitor::factoryResetButtonLoop: Button pressed for &gt; 1 second(s), setting default IP." />

The two above messages will be seen if someone presses the factory reset button for longer the 5 seconds, and only the first will be seen if pressed for between 1 and 5 seconds. See the Factory Defaults Button section for more details of what happens when the button is held down.

Note that the Factory Reset Button Monitor message just means that the button is now being monitored following a reboot and does not means someone has pressed the button:

<Entry Code="5011" Type="0x00000000" Level="Info" Timestamp="2022-09-15T09:21:53.0000000Z" Data="" FriendlyMessage="FactoryResetButtonMonitor::FactoryResetButtonMonitor: Initialized User Button." />

 

Configuration Changes

These entries can be very large. Example message shows just the start:

<Entry Code="5235" Type="0x00000000" Level="Info" Timestamp="2022-09-15T10:53:50.0000000Z" Data="" FriendlyMessage="ConfigManager updated: config.json:.............................

Each entry shows the full settings file, but with careful analyzing of the text following the config.json part of the entry what was actually changed can be determined. Essentially due to the complexity of the settings text, it is recommended to treat this as a message indicating a config change only, and use the date and time to help determine more information if required.

 

Outbound Connection Issues

A Vector could be configured with one or more outbound connections to an Estate Manager instance, MQTT broker, or HTTP Post server, but if that is not reachable, expect to see a similar message to any of the below:

    <Entry Code="16602" Type="0x00000000" Level="Warning" Timestamp="2022-09-14T17:24:02.0000000Z" Data="00" FriendlyMessage="ClientConnection: safecountdevices.irisyscloud.net:5000 - timed out" />    <Entry Code="16601" Type="0x00000000" Level="Warning" Timestamp="2022-09-14T17:22:57.0000000Z" Data="00" FriendlyMessage="ClientConnection: safecountdevices.irisyscloud.net:5000 - disconnected" />

The below message indicating an issue resolving the address may also be seen. It it is continually seen and no outbound connection is ever made to a hostname, double check your dns settings in Network Settings and that there is a connection to a valid dns server.

<Entry Code="16639" Type="0x00000000" Level="Warning" Timestamp="2022-09-15T06:02:56.0000000Z" Data="00" FriendlyMessage="ClientConnection: safecountdevices.irisyscloud.net:5000 - could not resolve address" />

Once a network issue is corrected, expect to see one or more of the following 'success' messages:

    <Entry Code="16644" Type="0x00000000" Level="Info" Timestamp="2022-09-15T06:03:42.0000000Z" Data="00" FriendlyMessage="ClientConnection warnings cleared" />    <Entry Code="16643" Type="0x00000000" Level="Info" Timestamp="2022-09-15T06:03:41.0000000Z" Data="00" FriendlyMessage="ClientConnection: resolved host safecountdevices.irisyscloud.net:5000" />