FIELD OF THE EMBODIMENTS

Embodiments are described regarding devices that transmit data, including those that may also sense input, produce output, receive, process and exchange data with end-nodes or networks of end-nodes. Some of the embodiments described herein relate to retail implementations. In such retail implementations, items may be sold to users that simply walk in, select an item or items, and then walk out without stopping at a cash register. Systems associated with a store can include a plurality of sensors of various types, which are used to identify various things, such as user identification, user interaction with items, confirmation that a user actually took an item off a shelf, confirmation that the item is not later dropped off in some other location if the user changes his or her mind, and adding of the item or items to a shopping cart managed by an online application, e.g., cloud, and then transacting payments for items confirmed to have been purchased.

BACKGROUND

Over the years, there has been much advancement in the area of processing devices and devices that communicate over networks. For example, electronic devices are typically designed for specific applications. Some devices are more versatile, such as the common day smartphone or general purpose computers. These devices, although versatile and sometimes powerful, require network connections to send and receive data. Network connections, for example, are those provided by internet service providers (ISPs). Such connections can be obtained for private use, e.g., homes and businesses, and some can be obtained in public places. In either form, users are required to setup connections or initiate connections, e.g., to the Internet, via their devices and interfaces, before access is enabled.

As a result, devices provided with network access to exchange data must contend with setup procedures for obtaining access, must provide interfaces for managing setups, and must also provide sufficient power to process data and perform the communication. These typical requirements are, unfortunately, impediments to the simplification of devices that could benefit from data exchanges over networks. It is in this context that a need is present for embodiments described in this disclosure.

SUMMARY

The embodiments described herein relate to communications devices, communication methods, and methods for utilizing digital detection, sensing and tracking systems to determine interactions with retail items, in order to enable processing of cashier-less purchase transactions. In one embodiment, method are provided to enable tracking of items in a store, and detect interactions with said items using one or more sensors. The interactions may be identified and classified, in order to determine if the user is intending to purchase the item and enabling charging of the user's account. In various embodiments described herein, a smart retail outlet is capable of classifying shopping activity for one or more shoppers into one or more categories, where upon a determination that a user fits in any particular category, or changes category, an action can be taken to dynamically respond to the state of the shopper. Described herein, a store tracks the taking of items into the possession of the shopper as well as the detection of returning items to the shelf, and is able to characterize the misplacement of returned items to the wrong shelves. In one embodiment, machine learning algorithms are used to predict a shopping list for a shopper. In one configuration, a retail outlet uses a mix of input data with machine learning model to classify shopping behavior and track events such as takes and returns and enables a family or group of shoppers to participate in the taking and returning of items to a single shopping account. Furthermore, the present invention includes capability to make a determination of the identity of each shopper taking or returning items from the shelf even when multiple shoppers are proximate to the shelf and interacting with the shelf at the same time.

In one embodiment, devices, systems, and method are provided for tracking items in a store for processing a cashier-less purchase transaction. In one example, a method includes tracking a shopper in a store using one or more sensors to process an account of shopping activity performed by the shopper. The method further includes monitoring the shopper in the store using said one or more sensors. Output of the one or more sensors is processed by a processing entity associated with the store to produce input feature data for aspects of a current scenario involving actions of the shopper and one or more items in the store. The processing entity associated with the store processes the input feature data to characterize aspects of the current scenario. The processing entity characterizes aspects of the current scenario in part by accessing a trained machine learning model that is maintained for a profile of the shopper. The input feature data is for actions of the shopper in relation to the scenario. The input feature data is processed by one or more classifiers of the trained machine learning model to produce the characterized aspects of the scenario relating to the account of said shopping activity of the shopper and said one or more items in the store.

In some embodiments, devices are integrated with a wireless communication chip and integrated power generating or delivering device. In one embodiment, these devices are referred to herein as wireless coded communication (WCC) device. Such devices are configured to harness power to cause or enable activation of a communication device to transmit data. The data can be pre-configured or coded to report occurrence of an event, log an event, log state, cause an action, send a message or request data from one or more end nodes. In some embodiments, the devices enable communication over a wireless network, which enables access to the Internet and further enables cloud processing on data received or processing for data returned or communicated.

Broadly speaking, a WCC device is one that has or is coupled with a wireless transmission capability (e.g., a transmitter, a transceiver, Wi-Fi chip, Bluetooth chip, radio communication chip, etc.), and a power pump or a power supply.

In one embodiment, a method for tracking items in a physical store for processing cashier-less purchase transactions is provided. The method includes detecting a portable wireless coded communication (WCC) device in the physical store. The WCC device is associated with an on-line account of a shopper. Then, receiving, by a server, sensor data regarding the WCC device, its location in the physical store and proximity to items in the physical store. The method includes receiving, by the server, interaction data of an item on a shelf of the physical store by the shopper using one or more sensors of the physical store and the WCC device to determine that the item is one targeted for purchase. The interaction data is configured to identify a type of said item and add it to an electronic shopping cart of the shopper having the on-line account for processing said cashier-less purchase transactions. The method further includes processing, by the server, an electronic charge to a payment service of the shopper for the item. The WCC device may be a portable device of the user, and an application for said cashier-less purchase transactions can be run by the WCC device or portable device of the user.

In one configuration, a power pump device is configured to receive a force or movement input from a user or object. The force can be a direct force that is intentionally input by the user, e.g., by pushing a button, moving a slider, lifting a tab, shifting a lever, etc. Another type of force is an indirect force, which is one that the user is not intending to provide. Instead, force is created when the user moves, opens, lifts, shifts, closes, or somehow changes the position or orientation of a physical object.

For example only, a physical object may be a door, and closing of the door is the intent of the user. In some embodiments, the physical object may be a store shelf, a store item on a shelf, e.g., retail store and retail items. The indirect force of closing the door is then, in one embodiment, transferred as force to a WCC device. As such, even though the user did not have intent to provide input or force to the WCC device, the WCC device received the force. In the case of a battery operated WCC, the WCC device could also receive the input indirectly. That is, when the door is closed, an input trigger or sensor can be set or impacted or triggered, and this input to the WCC device is incidental. The input is then used to activate a process of the WCC device and to automatically transmit data or request data wirelessly over a network.

In one embodiment, a method for tracking items in a store for processing cashier-less purchase transactions is provided. The method includes receiving, by a server, sensor data regarding items associated with shelves of a store, and receiving, by the server, interaction data of an item of a shelf of the store by a user. The interaction data is configured to identify a type of said item and add it to an electronic shopping cart of the user, if a take event is confirmed. The user account is used for processing said cashier-less purchase transactions. The embodiment includes receiving, by the server, data indicative of said user leaving an area of the store that is indicative that the user has purchased the item. The server is configured to process or instruct processing of an electronic charge to a payment service of the user for the item upon confirming that said user left the area of the store.

In some embodiments, a volume or area in which the item is located is tracked to obtain said interaction data. The interaction data is used to determine movement or removal of said item from the shelf and determine if a take event has occurred with respect to the item and the item is added to said electronic shopping cart.

In some embodiments, said movement is identified at least using image data from a camera. The camera is configured to produce image data that is processed as part of said sensor data that is received by said server.

In some embodiments, a confidence level of identifying said movement is increased by using input from one or more other sensors, each sensor is provided a weighting that gives a respective sensor or sensors more or less significance in said identifying of the movement.

In some embodiments, the method further includes receiving, by a server, data indicative of a user device being present in the store. The data from the user device is used to identify the user account of the user for enabling use of the electronic shopping cart for buying one or more items in the store via one of said cashier-less purchase transactions.

In some embodiments, the method further includes receiving, by a server, tracking data regarding movement of the user in the store, the movement of the user is indicative of browsing of said items on one or more of said shelves. The tracking includes tracking a hand of the user, the hand of the user caused by the user to touch, lift, move, and/or remove said item from the shelf, the hand tracking is further usable as said interaction data with the item.

In some embodiments, the method further includes sending, by the server, display information for rendering on a display screen that is proximate to the shelf having the item for which the user is interacting. The display information is customized for the user based on information obtained from a profile of the user associated with the user account.

In some embodiments, the method further includes sending, by the server, audio data for outputting from a region of the store that is proximate to the shelf having the item for which the user is interacting, the audio data being customized for the user based on information obtained from a profile of the user associated with the user account.

In some embodiments, the audio data is configured to be output in a directional tracking format, the directional tracking format implements audio steering to direct said audio data toward a head of the user, wherein said audio steering focuses sound for the user while excluding distribution of sound substantially beyond the head of the user.

In some embodiments, the method further includes receiving, by the server, eye or head gaze information of the user. The eye or head gaze information is indicative of actions taken by the user with the item for which the user is interacting. The eye or head gaze information is collected to determine product information that is of interest to the user. The server is configured to collect the product information that is of interest to the user and using said information to provide recommendations to the user regarding other items that may be of interest and/or to provide discount or promotional information for the item or other items.

In some embodiments, the method further includes receiving, by the server, first data indicative that a take event occurred for the item and second data indicative that the item having a tag communicates with a user device of the user. The user device of the user verifying that the item was subjected to the take even and was not returned prior to the user leaving the area of the store before processing the electronic charge. The user device is configured to communicate with at least one sensor in the store that monitors when the user device is leaving the store.

In some embodiments, the method further includes receiving, by the server, proximity data of the user and a second user, the user and the second user configured to be proximate to each other near the shelf. The proximity data of the user and the second user is usable to distinguish the user and the second user. Cameras are disposed in the store to produce part of the sensor data, and the cameras producing image data that is trackable to identify limbs of each of the user and the second user, and said limbs trackable in motion and used to verify whether the user or the second user is interacting with the item on the self.

In some embodiments, the interaction data includes one or more features that are descriptive of detected interaction. The features is processed as inputs to a model that classifies the interaction to be one of a take event or a returning event of the item from the shelf or back to the shelf.

In some embodiments, classifying is provided, which uses the model to assists in predicting an action to be taken by the user or other users. The model is one that refines its classification over time based on prior interactions and assists in determining whether to add the item or other items to the electronic shopping cart of the user or other users.

In some embodiments, the classifying uses the model to assist in generating recommendations to the user or other users based on learned likely actions to be taken by users when interacting with the item or items, the classifying being a process that continually learns to optimize the model, and said continual learning uses inputs of features from the store and/or other stores that are connected to a shopping service that enables said cashier-less purchase transactions.

In one embodiment, a method for tracking items in a store for processing a cashier-less purchase transaction is provided. The method includes receiving sensor data regarding items associated with shelves of a store. The method also includes identifying a user entering the store. The identifying uses at least a device having wireless communication. The device has an application that is executed in associated with a user account of the user. Also, tracking the user movements while in the store. The tracked movements include detecting proximity of the user to a shelf having an item. And, detecting interaction data for the item on the shelf of the store by the user. The interaction data is configured to identify a type of said item and enable add said item to an electronic shopping cart of the user having said user account for enabling processing of said cashier-less purchase transaction. The method includes receiving data indicative of said user leaving an area of the store while the item is in the electronic shopping cart. Leaving is indicative that the user intends to purchase the item. An exit of the area includes one or more sensors for confirming that the user has left the area. The method includes processing an electronic charge to a payment service associated with the user account of the user for the item based on said confirming.

In one embodiment, if the WCC device uses a power pump, the force or physical pressure or movement received can be imparted or communicated to a mechanically flexible device or element. In one embodiment, the flexible element may be a piezoelectric device. A piezoelectric device is one that can produce a voltage when the force or movement is imparted to it, and the result is that a voltage is produced. That voltage can be harvested and stored to a storage device. In one embodiment, the storage device can be capacitive storage device (e.g., a capacitor). In other embodiments, the storage device can be a battery or a rechargeable battery cell or cells. In one implementation, if the storage device is a capacitive storage device, the device will serve as the power source to an integrated circuit of the WCC device.

In one embodiment, the integrated circuit can be programmed to communicate data or a code or a message to an end node. The end node can store the received data or perform an action, e.g., based on pre-programming.

In one embodiment, the device is a wireless coded communication (WCC) device. The WCC device, in one configuration, is passive, in that the device is not connected to active power (e.g., a battery or power plug). However, once a force, e.g., mechanical force is applied to the power pump of the WCC device, the WCC device is activated for a period of time sufficient to process and transmit coded data to the end point or end-node. In some embodiments, a WCC can be programmed with more than one method or function to perform. For example, the selection of the method or function can be based on a pre-activation setting. In other cases, a method is automatically selected based on the power available. If more power is available, methods that require more power are performed. In certain embodiments, an input feature such as a selection control, switch, slider, touch pad, may be coupled to the WCC, which generate a payload. The payload may be transmitted remotely to an end node or processed by the WCC locally, or both in part or whole.

In some implementations, functions may be selected and triggered according to results of processing of the payload before, during or after transmission of the payload. In certain embodiments, the WCC includes the capability to detect or image the identity or attribute of a user, a biometric signature, a fingerprint, a voice, a sound, a scene, an object position, QR code, RFID code, barcode, status, temperature, pressure, absence or presence of conditions, environmental condition, vibration, and any signal or source coupled to, near or within sensing range of one or more sensors coupled to, or integrated with, a WCC device.

In other embodiments, devices, systems, and method are provided. One such device is a wireless coded communication (WCC) device, which may be configured for wireless communication with other devices, e.g., over a network. A WCC device is a type of internet of things (IOT) device that can sense data, process data, send data, respond to data requests and exchange data with other WCC device, a network device, a user device, and/or systems over the internet. In some configurations, a WCC device may include a power source that enables usage of low power, e.g., to send data that is sensed, request data and/or communicate data wirelessly. WCC devices maybe function as standalone devices or may be integrated into other devices. In some configurations, a WCC device may include power harvesting circuitry. A WCC device may be pre-configured or coded to report occurrence of an event, log an event, log state, cause an action, send a message or request data from one or more end nodes. In some configurations, the devices enable communication over a wireless network, which enables access to the Internet and further enables cloud processing on data received or processing for data returned or communicated.

