FIELD OF THE DISCLOSURE

The present disclosure is generally directed to linking screenshots to other resources. In particular, the present disclosure describes techniques for analyzing images to identify a screenshot and provide a link to another resource or content associated with the screenshot.

BACKGROUND OF THE DISCLOSURE

Companies can use the Internet to advertise or promote their brands and products. Users of client devices can view content provided by companies using a web browser or other application. Some users may capture screenshots relating to content that interests them on their client devices.

BRIEF SUMMARY OF THE DISCLOSURE

Systems and methods of the present solution are directed to linking screenshots to other resources. Consumer products companies often produce content that is published via the Internet. For example, a company may have its own website including web pages dedicated to its products or to brands associated with the company. In some instances, companies may promote their brands and products through third party web pages, such as social media sites. For example, many companies have one or more accounts on websites such as Facebook, Instagram, Pinterest, and Twitter. Users can view content published by a company by visiting the company's web pages or third party web pages on which the company publishes content. In some cases, users also may view content published by a company using a dedicated application on a client device, such as an application associated with a social media website on which the company publishes content.

Users can save to their devices images of content that interests them on the Internet. Users can accomplish this by capturing a screenshot of a display of the client device while the client device displays selected content. Typically, a screenshot refers to an image representing a portion or the entire graphical user interface of a display of a client device. Thus, a screenshot captured while the client device is being used to display content published by a company (for example, in a web browser or a third party application), will include the relevant content that the user wishes to save, and may also include other elements of the graphical user interface of the client device. A screenshot is often saved as an image file on the client device. In some client devices, the screenshots are saved in an image folder of the client device. The image folder is capable of storing the screenshots captured by the client device, regardless of the content or the application displayed on the client device at the time the screenshot was captured. In this way, the image folder can store screenshots of content displayed by multiple different applications of the client device.

Because screenshots typically include only image data, it can be difficult for a user to access content relevant to what is depicted in a screenshot when the user only has access to the screenshot itself. For example, while the screenshot may provide an image of a product that the user would like to purchase, the image file associated with the screenshot itself does not provide any interface with which the user can interact to purchase the product shown in the screenshot. The present disclosure provides techniques for linking a screenshot to other relevant resources and/or content, such as a shopping interface.

One aspect of the present disclosure can be implemented in a method or system for providing an online store associated with a screenshot. A screenshot linking system including one or more processors can identify a screenshot from images stored on a device, such as a photo roll on a mobile device storing both photos and screenshots. The screenshot linking system can identify an image included within the screenshot. The screenshot linking system can match the identified image with a reference image of a plurality of reference images maintained by the screenshot linking system. Each reference image can be associated with a link to content. In some implementations, the link can be a link to content on a mobile application or content on a webpage. In some implementations, the content to which the reference image is linked can include the reference image. Responsive to the screenshot linking system matching the image with a reference image, the screenshot linking system can then identify a link associated with the image. In some implementations, the screenshot linking system can generate and provide a notification to the device associated with the screenshot. The notification can include the link to content corresponding to the image identified in the screenshot. In some implementations, if the image in the screenshot is an image of a product, the link provided in the notification can be a link to a page of a mobile application or a web server through which the user of the device can purchase or otherwise receive additional information about the product. For example, if the image in the screenshot is of a shirt, the screenshot linking system can identify a link to a product page where the shirt is available for purchase, and can provide a notification including a link to the product page. In some implementations, responsive to the user visiting the page via the link, the screenshot linking system can receive, from the device, a request to purchase the product and execute the request to purchase the product. In some implementations, it can be determined that an image file corresponds to a screenshot, and the screenshot can be analyzed to identify at least one application associated with the screenshot.

As discussed above, users of computing devices may wish to save images of content that interests them, such as from web pages on the Internet or screens of mobile applications, in the form of screenshots. A screenshot can be processed to generate a link to other relevant content. In some implementations, it may be useful to determine an application associated with a screenshot for purposes of generating such a link. For example, a deep link may be a link to a particular page or portion of a particular application, such as the application that was used to generate the screenshot. Therefore, for the screenshot linking system to generate a deep link, the screenshot linking system may identify the identity of the application active on the display of the client device at the time the screenshot was captured. Furthermore, the screenshot linking system may also be useful to process only a subset of the screenshots saved on a client device, such as screenshots that are generated by only a subset of applications. For example, some applications may be added to a white list of applications whose screenshots are to be processed and linked to other content. Screenshots associated with applications not included in the white list may not be processed in order to save computing resources.

One aspect of this disclosure can be implemented in a method or system for determining an identity of an application that is associated with a particular screenshot. The screenshot linking system can identify an image file and determine that the image file corresponds to a screenshot. The screenshot linking system can identify one or more visual elements of the screenshot. The screenshot linking system can then determine an identity of an application corresponding to the screenshot based on the identified visual elements.

As discussed above, a screenshot may capture the graphical content displayed on a display screen of a client device. However, some of the visual elements of the screenshot may be irrelevant, such as elements of a user interface of the active application or other information that may be displayed on a client device, such as the time, date, and battery level of the client device. Therefore, it can be useful to separate the relevant content of a screenshot, such as an image provided by a content publisher, from the less relevant content that may be included within the screenshot. After an image has been identified and/or extracted, the image may be processed to determine a watermark or fingerprint associated with the image. Watermarks and fingerprints can be used to match the images with reference images provided by content publishers. In this way, if a user takes a screenshot of an image that matches a reference image of a content publisher, the screenshot linking system can provide the user, via the device of the user, a notification that includes a link to content corresponding to the reference image that matched the image of the screenshot. As a result, content publishers or advertisers can provide relevant content to users based on screenshots captured by the users.

One aspect of this disclosure can be implemented in a method or system for analyzing an image included within a screenshot. An image can be identified and/or extracted from a screenshot. A watermark of the image can be identified. The watermark can include additional information corresponding to the image. In some implementations, the watermark can include data representing a link to a page of a mobile application or web server that includes content associated with the image. The link can be a link to content of a product shown in the image. In some implementations, the link can be a link to content of an advertiser or entity that published or provided the image. In some implementations, responsive to identifying the link to content, the screenshot linking system can provide a notification to the client device including the link to content to allow the user of the client device to retrieve additional information relating to the image included in the screenshot.

Another aspect of this disclosure also can be implemented in a method for analyzing an image included within a screenshot. An image can be identified and/or extracted from a screenshot. The screenshot linking system or a mobile application corresponding to the screenshot linking system can generate a fingerprint of the image can be generated. The screenshot linking system can then identify a fingerprint of a reference image that matches the fingerprint of the image of the screenshot. The screenshot linking system, responsive to determining that the fingerprint of the image of the screenshot matches the identified fingerprint of the reference image, can identify a link to content associated with the image. The screenshot linking system can then generate and provide a notification including the link to the user of the device. In this way, the screenshot linking system can provide the user relevant content based on images included in screenshots captured by the user. As a result, advertisers or content publishers can send links relevant to the user to increase user engagement.

Users who capture screenshots depicting particular types of products may not be interested in other unrelated types of products. However, products similar to those that are depicted in the screenshots that a user captures may be of more interest to the user. As a result, the user may be more likely to purchase a product that is similar to the products depicted in the screenshots captured by the user. In some implementations, the screenshots and purchasing decisions of the user, as well as those of other users, can form the basis for a product recommendation for the user.

One aspect of this disclosure can be implemented in a method for providing a product recommendation to a user. To provide recommendation to one or more users, the screenshot linking system can build one or more recommendation engines. The recommendation engines can rely on models that process input data to generate output data in the form of recommendations. The input data can be referred to as training data that is provided to the recommendation engine to train the model. The models can be adaptive and continually learning.

In some implementations, the recommendation engine can be configured to determine recommendations for a user based on a user's screenshots. The recommendation engine can determine recommendations of products that are similar to products identified in the user's screenshots. In some implementations, the recommendation engine can rely on screenshots of other users that the recommendation engine determines are similar to screenshots captured by the user. In this way, the recommendation engine can generate product recommendations for the user based on screenshots and images included in the screenshots captured by the other users of the screenshot linking system.

In some implementations, the recommendation engine can be configured to identify users that are similar to one another based on the screenshots captured by each of the users. In this way, the recommendation engine can recommend products in which users similar to the user have expressed an interest. Similarly, advertisers or other content providers may choose to generate marketing or advertising campaigns directed towards users that are similar to one another and that are likely interested in the products of the advertisers. The recommendation engine can be configured to generate a mode to identify users similar to one another based on the screenshots of each of the users.

According to one aspect, a first plurality of user profiles can be generated for a plurality of users based on screenshots associated with the plurality of users. A plurality of categories to which the user profiles belong can be determined. Each user profile can be assigned to at least one of the plurality of categories. A recommendation model can be built for each category. A second user profile, not included in the first plurality of user profiles, can be identified. A category to which the second user profile should be assigned can be identified. A product recommendation can be provided to the second user based on the category to which the second user profile is assigned.

According to another aspect, the screenshot linking system can receive a plurality of screenshots captured by a device of a user. The screenshot linking system can identify images included in the screenshot. The screenshot linking system can, from the images, identify one or more categories to which the images correspond. In some implementations, the categories can be based on topics, such as clothing, furniture, or electronics, among others. In some implementations, the categories can be based on brands. In some implementations, the categories can be based on other attributes. Responsive to determining the categories to which the screenshots belong, the screenshot linking system can generate a user profile based on the categories of the screenshots. The screenshot linking system, via a recommendation engine that uses data from a large number of users, can provide recommendations of products (or content in general) based on a comparison of the user's profile with user profiles of other users and the screenshots corresponding to the other users.

At least one aspect of this disclosure can be implemented as method for analyzing screenshots to identify links to resources. The method can include accessing, by an application executing on a computing device from storage to which a screenshot captured using a screenshot capture function of the computing device is stored, the screenshot including an image and a predetermined marker. The method can include detecting, by the application, the predetermined marker included in the screenshot. The method can include identifying, by the application, using the predetermined marker, a link to a resource mapped to the image included in the screenshot. The method can include accessing the resource via the link.

In some implementations, the storage to which the screenshot is stored can be accessible by multiple applications executing on the computing device. In some implementations, the predetermined marker can include a visual marker added to the image included in the screenshot. In some implementations, the predetermined marker can include metadata uniquely identifying the image included in the screenshot or the image can be encoded with data corresponding to the predetermined marker. In some implementations, the predetermined marker can include a code comprising a string of characters.

In some implementations, the method can include comparing the predetermined marker to a plurality of reference markers. The method also can include determining a match between the predetermined marker and one of the plurality of reference markers. The method also can include selecting the link to the resource based on the one of the plurality of reference markers.

In some implementations, accessing the resource via the link can include accessing the resource responsive to selection of the link from a notification including the link. In some implementations, the application can be a first application and the link can include a deep link corresponding to a resource within a second application executable on the computing device.

Another aspect of this disclosure can be implemented as system for analyzing screenshots to identify links to resources. The system can include a computing device including a processor coupled to a memory and a display screen configured to display content. The system can include an application stored on the memory and executable by the processor. The application can include a screenshot receiver configured to access, from storage to which a screenshot of the content displayed on the display screen captured using a screenshot function of the computing device is stored, the screenshot including an image and a predetermined marker. The application can include a marker detector configured to detect the predetermined marker included in the screenshot. The application can include a link identifier configured to identify, using the predetermined marker, a link to a resource mapped to the image included in the screenshot, the resource accessible by the computing device via the link.

In some implementations, the storage to which the screenshot is stored can be accessible by multiple applications executing on the computing device. In some implementations, the predetermined marker can include a visual marker added to the image included in the screenshot. In some implementations, the predetermined marker can include metadata uniquely identifying the image included in the screenshot or the image can be encoded within data corresponding to the predetermined marker. In some implementations, the predetermined marker can include a code comprising a string of characters.

In some implementations, the system also can include a screenshot linking server. The screenshot linking server can be configured to receive the predetermined marker from the computing device. The screenshot linking server can be configured to compare the predetermined marker to a plurality of reference markers. The screenshot linking server can be configured to determine a match between the predetermined marker and one of the plurality of reference markers. The screenshot linking server can be configured to select the link based on the one of the plurality of reference markers. In some implementations, the application can be a first application and the link can include a deep link corresponding to a resource within a second application executable on the computing device.

Another aspect of this disclosure can be implemented as a method for analyzing screenshots to identify links to resources. The method can include accessing, by an application executing on a computing device from storage to which a screenshot has been captured by a screenshot function of the computing device, the screenshot including an image. The method can include extracting, by the application, data corresponding to the image included in the screenshot. The method can include generating, by the application, a fingerprint identifying the image, the fingerprint generated based on the data corresponding to the image. The method can include identifying, by the application, a link to a resource mapped to the fingerprint of the image. The method can include accessing the resource via the link.

In some implementations, the method can include identifying the image from the screenshot by performing image entropy analysis on the screenshot to determine a portion of the screenshot that corresponds to the image. In some implementations, the application can be a first application and the link can include a deep link corresponding to a resource within a second application that can execute on the computing device. In some implementations, identifying, by the application, the link to the resource mapped to the fingerprint of the image can further include identifying the link responsive to determining a match between the fingerprint and a plurality of reference fingerprints based on a comparison of the fingerprint of the image to a plurality of reference fingerprints.

Another aspect of this disclosure can be implemented as a system for analyzing screenshots to identify links to resources. The system can include a computing device including a processor coupled to a memory and a display screen configured to display content. The system can include an application stored on the memory and executable by the processor. The application can include a screenshot receiver configured to access, from storage to which a screenshot of the content displayed on the display screen captured using a screenshot function of the computing device is stored, the screenshot including an image. The application can include an image extractor configured to extract data corresponding to the image included in the screenshot. The application can include a fingerprint generator configured to generate a fingerprint identifying the image included in the screenshot based on the data corresponding to the image. The application can include a link identifier configured to identify, using the fingerprint, a link to a resource mapped to the image included in the screenshot. The application can include a resource display component configured to access the resource via the link.

In some implementations, the image extractor can be further configured to identify the image from the screenshot by performing image entropy analysis on the screenshot to determine a portion of the screenshot that corresponds to the image. In some implementations, the application can be a first application and the link can include a deep link corresponding to a resource within a second application that can execute on the computing device. In some implementations, the link identifier can be configured to identify the link to the resource responsive to determining a match between the fingerprint and a plurality of reference fingerprints based on a comparison of the fingerprint of the image to a plurality of reference fingerprints.

