Introduction
This section is provided to assist partners and end users with performing unbiased, fair and precise, manual count audits on installed counting systems without affecting or influencing the count data directly, or unintentionally.
Pre-requisites
Before auditing it is important that the counting system has been installed and configured correctly by an approved and experienced installer, or - in the case of new installers - by someone who has read and understood the Installation and configuration instructions.
The doorway/entrance/environment has not changed significantly since the counting system was installed and configured. Changes to the layout of a doorway could lead to 'holes' in the setup where people might be missed.
The data logger used for retrieving the count data is correctly interfaced to the counting system. This applies to systems which can only report their counts on an external device, or 'box' - usually counting devices which output a relay pulse for each count increment. Clearly if the interfacing is not correct then valid counts may not be transmitted and/or received correctly resulting in an unfair 'missed count'.
The counting system is configured to count individuals and not groups of people or ‘shopping units’. Counting groups of people is different to counting individuals. A ‘shopping unit’ can be a very subjective entity which is very difficult to audit and is therefore outside the scope of this document.
Video Verification
Auditing of a complete counting system involves comparing visual observations from the scene with the count data that has been logged and/or reported, over the same period, to ensure that the performance is as good as expected/required. In other words, you are simply watching and counting people enter and leave, and then seeing if the counting system counted the same as you.
The Vector devices provide a video view of the area beneath the counter (from an integrated video camera) which is overlaid directly on to the Time of flight, ground plane view. This combined view, allows real time (remote) viewing of a counter for configuration and auditing purposes. When used in conjunction with the Irisys ‘Validation Tool’ functionality, recordings of the combined view can be scheduled for later playback. It’s dedicated playback controls allow accuracy audit reports to be produced quickly and efficiently.
When scheduling of a recording is not possible (or not allowed), on site manual validations must be performed instead.
Audit Instructions
If performing an on-site audit, take a position well away from the counting area so that you are not ‘visible’ to the counting system and therefore will not inadvertently generate additional counts. This will also ensure that you do not influence or affect people who are walking through the counting area. You should also inform any on site staff about the audit process so that they too can, temporarily, keep away from the counting area during the audit.
If video recording is not possible, or not allowed, then it is recommended that at least two observers should carry out the manual audit count and corroborate on the result later. If there is a disagreement in the result between the observers, then counting issues outlined in this document should be re-stated and the count repeated. It must never be assumed that a manual count carried out by a lone observer is accurate; it is very difficult for one person to maintain attention over a long time-period, especially if the store/counting area is busy and concentration is affected by other things going on.
Each door or entrance - which has its own count data report - must be audited individually, with IN and OUT traffic being counted separately. Equally, if a counting system is installed over a wide entrance consisting of several physical doors and each of these are reported separately from within a single system then these too must be audited individually, with IN and OUT traffic being counted separately. Any other directions/flows apart from simple IN and OUT counting must also be audited separately. By doing this, any problems which may be highlighted in the audit results, can more easily be traced to the relevant entrance, counter, or counting direction, in order that an installer can subsequently put it right. When multiple entrances and directions have been configured, they should be audited in groups of two count lines, possibly by using additional auditors as necessary. Remember that Irisys counters installed as a ‘wide opening network’ can be configured to count and report IN and OUT counts for multiple doors independently (using the units multiple count lines ability) or as a single system regardless of number of counters, reporting a total IN and a total OUT count for all doors. Consequently, the correct operation of a wide opening installation should be verified and then audited appropriately.
What to count
Count all adults and all children who are walking independently, and those people in wheel chairs.
Where a counter has been configured to not count people below a certain height, this gives only a very rough count of children as it cannot differentiate between a short adult and a tall child. Because of this, it is recommended that you count everyone rather than trying to subjectively count adults and children separately. The manual count total can then be compared with the system total afterwards.
Groups of people should be counted as individuals, so a family group of 5 people entering, for example, must be manually counted as 5 people.
If a child is carried through the counting area by a parent, then they will not realistically be expected to be counted separately by the counting system so you must decide if you count such an event in your manual count.
Similarly, if a small child walks through the counting area very close to the parent then, again, they might not be counted separately by the counting system. Because of this, it should be established, before starting, how far apart a parent and child must be in order to include them separately in the manual count.
If a buggy has a child in it then that child should be counted separately. If the buggy is empty, then it should not be counted manually. Do not count shopping trolleys.
If a person enters, but then turns around and leaves straight away, it is possible that the counting system will consider that person either as an IN count, or an OUT count, or as an IN and an OUT count, and in some cases will be ignored completely. This is dependent on the counting system and how it is configured. In some cases, a retailer will want the system to report how many people enter and leave straight away, as this can be considered a ‘lost sales opportunity’, but others will want to ignore this behaviour, and the counting system will be configured appropriately. Therefore, it should be established how far a person must travel through the counting area in order that they are deemed to have entered the shop or building and therefore should be included in the manual count. This could be just past the lease line, or door, or could be a certain distance from the door. Note also that this point will mostly be different for the IN and OUT directions.
Vectors include the ability to differentiate between staff and visitors, but only if configured to do so and only if staff are wearing the Irisys staff detection lanyards. Therefore the configuration of the device must be known so as to include or ignore staff. Note that in cases where staff exclusion is not configured, you must always include staff in your manual counts. This also applies to security guards, delivery drivers, postal workers – anyone who passes through the counting area as they enter or leave, must be included in the manual count. Similarly, no counting system can recognise the same person twice, so, if a person leaves and re-enters later, they will be counted again; therefore, when counting manually, you must not disallow a count because you recognise and remember having counted that person previously.
If a person enters but then stops in the counting area, the counting system should not count that person more than once. The manual audit count should also only count this person once if they have passed the previously established threshold, as above. But, if that person leaves the counting area completely and then returns to it, then they should be manually counted again, if applicable.
Starting a Manual Count:
To compare manual counts with logged counts, you must be sure that you are counting through the same period. Always ensure that the manual counting, and system counting, are ‘time synchronised’ so that the data can be directly compared afterwards. Similarly, if comparing counting devices from different manufacturers these too needs to be time synced, so that each starts and stops at the beginning and end of the audit period.
Note that Vectors audited with the Irisys Validation Tool software, will have the time synchronisation handled automatically for you.
If the data logger is the type which displays the current IN and OUT counts, which change in real-time as people enter and leave, then you should make a note of the numbers displayed before you start the audit, for later use. Always note any time delay between people leaving the counting area and the display updating and allow for this in the audit
process. Additionally, if the logging system only updates the ‘current’ counts at a specific time interval (such as every 15 minutes, or every 30 minutes, etc) then you should always start and finish the manual audit on one of these intervals: i.e. start and finish immediately after an update.
Irisys counters can be configured to give, either; a count increment once a person crosses a count line AND leaves the field of view; or a count on immediately on crossing the count line. This means that there may be a delay between a person passing through an entrance and the increment being given dependant on how the counter has been configured. A similar delay may be experienced with other counting systems - allow for this in the audit process.
You should not start a count audit session during a period when people are passing directly through the counting area. Always start the audit when the counting area is clear of people (if only for a few seconds) so that there is no ambiguity as to whether the counting system did or did not count a person who was still in the field of view, and whether they should be included in the manual count. If it is too busy and not possible to wait for a clear area, then the number of people in the area at the start, who may have been counted, should be noted and allowances made later when comparing manual audit data with the counting system data.
Finishing a Manual Audit:
When coming to the end of a manual audit make sure that you stop when the counting area is clear of people as with the start of an audit, so that all counts ‘in-the-loop’ are accounted for in the system and can be correctly correlated with
the manual counts.
If the data logger is the type which displays the current IN and OUT counts, which change in real-time as people enter and leave, then you should make a note of the numbers displayed when you stop and then take away the numbers noted at the start to get the actual numbers during the audit period. This can then be compared with the manual counts for IN and OUT.
Analysing the Results:
Always compare the observed and recorded manual IN count, with the IN count from the counting system for the same period that you audited, to assess the accuracy for that direction. Do the same for the OUT direction, and any other additional directions and flows that you may be auditing. To calculate the error as a percentage, use the formula:
For example, an audit based on a manual count of 1000 people through an entrance, resulted in a system count of 993. Therefore, the calculation is (1000-993)/1000 which gives an error rate of 0.7%, i.e. the system is under counting IN traffic very slightly. A negative value would indicate over counting.
Never assume that IN and OUT counts should be the same. There are too many factors that can affect this assumption, for example, people entering in one door but leaving via another, will skew the figures from the outset. Additionally, if an IN count did equal an OUT count this could mean that both are counting 100% but equally it could mean that both are under counting, or both are over counting, by the same amount. Any assessment which compares system IN to system OUT counts in this way should be discarded. See here for more details of the issues affecting IN/OUT "Occupancy" counting.