This decision concerns an application relating to a video-based method and system for detecting whether goods or items have been properly scanned at point-of-sale or cashier terminals. However, since the distinguishing feature was considered to refer to a subjective perception, the EPO refused grant. Here are the practical takeaways from the decision T 2156/17 (Detecting suspicious activity using video analysis / NCR Corporation) of June 10, 2022 of the Technical Board of Appeal 3.4.01.
The invention underlying the present decision may be summarized as follows:
Conventional cashier or point-of-sale (POS) systems that provide for purchase of items using a scanner or other automated identification of items via code suffer from a variety of deficiencies. In particular, operation of such systems can be compromised either knowingly or unknowingly by an operator in a manner that allows a customer to receive possession of one or more items without paying for them. In particular, such systems are susceptible to “pass-throughs”, also know as “sweethearting” in which an operator purposefully or accidentally fails to scan an item as that item moves through the transaction area. In such cases, the POS system never detects the unscanned item and the item is thus never totaled into the purchase price. In such cases, the customer effectively receives the item for free. Retail chains lose millions of dollars a year to operator error or fraudulent activity of this nature. In a non-fraudulent example, an operator may unknowingly pass an item through the scanning area during a transaction and place the item into the item output area such as a downstream conveyor belt, but no scan of the item took place. Perhaps the operator was not paying attention and did not notice (or did not care) that the scanner failed to beep during scanning of an item (cf. WO 2006/105376 A2, p. 2, l. 9-25).
The system according to the invention uses video data analysis techniques as will be explained to detect activity such as sweethearting or pass-throughs. In particular, the system detects incidents of theft or loss of inventory at the cash register, POS or other transaction terminal when an operator such as a customer or store employee passes one or more items around the scanner (or RFID reader) without being scanned, or when the operator scans or manually enters an incorrect code into the transaction terminal for an item. The system can also detect items which may be mislabeled with an incorrect bar code to be misread by the scanner or entered as the wrong item by the operator (cf. WO 2006/105376 A2, p. 3, l. 30-37).
Fig. 1 of WO 2006/105376 A2
Claim 1 of the first Auxiliary Request
A computer-implemented method for detecting a transaction outcome, the method comprising:
obtaining video data associated with a transaction area;
analyzing at least a portion of the video data to obtain at least one video parameter concerning at least a portion of a transaction associated with the transaction area, wherein the step of analyzing comprises automatically analyzing frames of video from at least one region of interest in the at least a portion of the video data to identify a respective event indicating the presence of an item associated with the transaction;
obtaining at least one transaction parameter originated from a transaction terminal associated with the transaction area, wherein the step of obtaining comprises analyzing at least a portion of transaction data from a transaction terminal to identify a sequence of items identified as being transacted in the transaction; and
automatically comparing the at least one video parameter to the at least one transaction parameter to identify a transaction outcome, wherein the step of automatically comparing comprises analyzing the sequence of items identified as being transacted in the transaction from the transaction data in comparison to the events produced from analysis of the video data to determine if the item represented by at least one event produced from the analysis of the video data is indicated as an item for transacting in the at least a portion of transaction data, and correlating video timestamps of events from the analysis of the video data to transaction timestamps of items reflected as having been transacted in the transaction data to identify events indicating an item in the video data that does not have a corresponding record in the transaction data,
obtaining video data further comprising:
obtaining video data originating from at least one elevated video camera that monitors a transaction area defining the region of interest; and
wherein analyzing at least a portion of the video data comprises:
analyzing the video data to track items involved in the transaction in the transaction area; and
wherein automatically comparing the at least one video parameter to the at least one transaction parameter to identify a transaction outcome comprises:
comparing the video analysis of the tracked items to transaction data produced from a transaction terminal to identify suspicious activity,
producing a set of detection events indicating detection of items by at least one detector within at least one region of interest of at least one portion of the video data;
wherein automatically comparing the at least one video parameter to the at least one transaction parameter comprises:
for each detector, comparing the set of detection events for that detector to at least a portion of transaction data taken at the time of the event to identify at least one apparent discrepancy in a number of items detected by that detector from a number of items indicated in the at least a portion of the transaction data; and
identifying an overall suspicion level for the transaction based on apparent discrepancies identified by the at least one detector.
Is it patentable?
First of all, the Board in charge rejected the Main Request because of a lack of inventive step (cf. Reasons for the Decision, items 1-30). Then the Board turned to the first Auxiliary Request and determined the distinguishing features over the closest prior art document D6 as follows:
33. The claimed method is further distinguished from the method of D6 in that the process is automatic, in that it incorporates a step of comparing the set of transaction events with the transaction data in order to identify a discrepancy in the respective number of items, and in the further step of identifying an overall “suspicion level” for the transaction.
However, according to the Board, specifically the comparison step to identify a discrepancy to generate a suspicion level would lack technical character:
34. Contrary to the appellant’s view, the step of comparing the set of detection events with the set of transaction data to identify a discrepancy is not technical. Independently of the nature of the events, it must be stressed that the comparison carried out simply compares data. All in all, the claimed step of comparing said data amounts to a mere comparison of lists by computer means.
35. The origin of the data are without bearing on the claimed step of comparing. The technicality of the transaction, as it manifests itself in the movement of items in a sequence of video frames, is lost when it comes to the step of comparing the data. The Board further rejects the view that the count that results from the comparison is technical in the context of the invention. The mere fact that the identification of a discrepancy is used to generate a “suspicion level” is also not sufficient to confer technical character to said method. The notion of “suspicion” is essentially subjective and as such not technical. The same applies to any parameters that may be derived therefrom.
Against this background, the Board concluded that the only feature of technical nature that distinguishes the claimed method from the teaching of D6 resides in the automation of the claimed process (cf. decision, item 36). However, since this would simply reflect a trend in technology, this feature would not be sufficient for an inventive step.
Since all other requests would either be inadmissible or would lack inventive step, the Board in charge dismissed the appeal.
You can read the full decision here: T 2156/17 (Detecting suspicious activity using video analysis / NCR Corporation) of June 10, 2022.
Patrick is a European patent attorney at BARDEHLE PAGENBERG. He specializes in software patents in Europe both from a prosecution and litigation point of view.