BACKGROUND

Humans may engage in human-to-computer dialogs with interactive software applications referred to herein as “automated assistants” (also referred to as “chatbots,” “interactive personal assistants,” “intelligent personal assistants,” “conversational agents,” etc.). For example, humans (which when they interact with automated assistants may be referred to as “users”) may provide commands and/or requests using spoken natural language input (i.e. utterances) which may in some cases be converted into text and then processed, and/or by providing textual (e.g., typed) natural language input. Automated assistants are typically reactive, as opposed to proactive. For example, at the onset of a human-to-computer dialog session between a user and an automated assistant (e.g., when there is no currently-applicable conversational context), the automated assistant may provide at most a generic greeting, such as “Hi,” “Good morning,” etc. The automated assistant does not proactively obtain and provide specific information of potential interest to the user. Consequently, the user must provide initial natural language input (e.g., spoken or typed) before the automated assistant will respond with substantive information and/or initiate one or more tasks on behalf of the user.

SUMMARY

Techniques are described herein for configuring automated assistants to proactively incorporate, into existing or newly-initiated human-to-computer dialog sessions, unsolicited content of potential interest to a user. In some implementations, an automated assistant configured with selected aspects of the present disclosure—and/or one or more other components acting in cooperation with an automated assistant—may perform such incorporation when it determines that in an existing human-to-computer dialog session, the automated assistant has effectively fulfilled its obligations to the user (e.g., the automated assistant is awaiting further instructions). This can be as simple as the user uttering “Good morning” and the automated assistant providing a generic response, “Good morning to you.” In such a scenario, the user may likely still be engaged (at least briefly) with the human-to-computer dialog session (e.g., a chatbot screen showing an ongoing transcript of the human-to-computer dialog may still be open, the user may still be within earshot of an audio input/output device through which the human-to-computer dialog is implemented, etc.). Accordingly, any unsolicited content that is incorporated into the human-to-computer dialog session is likely to be consumed (e.g., heard, seen, perceived, understood, etc.) by the user.

Incorporating unsolicited content of potential interest to a user into a human-to-computer dialog session may have several technical advantages. The automated assistant may appear more “lifelike” or “human” to the user, which may incentivize increased interaction with the automated assistant. Also, the incorporated content is likely to be interesting to the user because it is selected based on one or more characteristics of the user, such as the user's current content (e.g., location, status, etc.), travel plans, known topics of interest, etc. Consequently, a user may be relieved of affirmatively soliciting such content, which may conserve computing resources that otherwise would be used to process the user's natural language input, and/or which may be helpful for users having diminished capabilities to provide inputs (e.g., driving, physical limitations, etc.). Additionally, the user may receive potentially helpful content that it may not otherwise have occurred to the user to solicit. As yet another example, incorporation of unsolicited content may provide a user with information that the user otherwise may have sought by submitting additional requests. Avoiding such additional requests may conserve computing resources (e.g., network bandwidth, processing cycles, battery power) required to parse and/or interpret these requests.

In some implementations, the automated assistant may initiate a human-to-computer dialog (for incorporation of unsolicited content) and/or incorporate unsolicited content into an existing human-to-computer dialog in response to various events. In some implementations, the event may comprise a determination that the user is within earshot of the automated assistant. For example, a standalone interactive speaker operating an automated assistant may detect, e.g., using various types of sensors (e.g., an IP webcam, or a motion sensor/camera incorporated into an appliance such as a smart thermostat, smoke detector, carbon monoxide detector, etc.) or by detecting co-presence of another computing device carried by the user, that the user is nearby. In response, the automated assistant may audibly provide unsolicited content to the user, such as “don't forget your umbrella today, rain is expected,” “Don't forgot today is your sister's birthday,” “Did you hear about last night's injury to <sports team's> power forward?”, or “<stock> has gone up 8% over the last several hours.”

In some implementations, an automated assistant operating on a first computing device of a coordinated ecosystem of computing devices associated with a user may receive one or more signals from another computing device of the ecosystem. Such signals may include computing interactions by the user (e.g., the user is performing a search, researching a topic, reading a particular article), states of application(s) operating on another computing device (e.g., consuming media, playing a game, etc.), and so forth. For example, suppose a user is listening to a particular musical artist on a standalone interactive speaker (which may or may not operate an instance of the automated assistant). An automated assistant on the user's smart phone may audibly detect the music and/or receive one or more signals from the standalone interactive speaker, and in response, incorporate, into a newly-initiated or preexisting human-to-computer dialog, unsolicited content such as additional information about the artist (or the song), recommendations regarding other similar artists/songs, notification(s) of upcoming tour dates or other artist-related events, etc.

Unsolicited content that is incorporated into a human-to-computer dialog may include information of potential interest to a user (e.g., weather, scores, traffic information, answers to questions, reminders, etc.) and/or actions of potential interest to a user (e.g., play music, create reminder, add item to shopping list, create etc.). Information and/or actions of potential interest be selected based on a variety of signals. In some implementations, the signals may include past human-to-computer dialogs between a user and an automated assistant. Suppose that during a first human-to-computer session, a user researches flights to a particular destination, but does not purchase any tickets. Suppose further that a subsequent human-to-computer dialog between the automated assistant and the user is triggered, and that the automated assistant determines that it has responded to all natural language inputs from the user. In such a scenario, the user has not yet provided any additional natural language input to the automated assistant. Accordingly, the automated assistant may take the initiative to incorporate unsolicited content including information relating to the user's previous flight search, such as “Did you ever buy a ticket to your destination,” or “I don't know if you're still looking for a flight, but I found a good deal on <website>.”

Other signals that may be used to select information and/or actions to be incorporated as unsolicited content into a human-to-computer dialog include, but are not limited to, user location (e.g., which might prompt the automated assistant to proactively suggest particular menu items, specials, etc.), calendar entries (e.g., “Don't forget your anniversary is next Monday”), appointments (e.g., an upcoming flight may prompt an automated assistant to proactively remind the user to perform online checkin and/or to start packing), reminders, search history, browsing history, topics of interest (e.g., interest in a particular sports team might cause an automated assistant to proactively ask the user, “Did you see the score last night?”), documents (e.g., an email including an invitation to an upcoming event might prompt the automated assistant to proactively remind the user of the upcoming event), application states (e.g., “I've performed updates of these three applications,” “I see that you still have several applications open, which may strain the resources of your device,” “I see you're currently streaming <movie> to your television; did you know that <trivia about movie>?”, etc.), newly-available features (e.g., “Welcome back. While you were away I learned to call a taxi. Just tell me whenever you need one”), weather (e.g., “I see that it is nice outside. Would you like me to search for restaurants with outdoor dining?”), and so forth.

In some implementations, a method performed by one or more processors is provided that includes: determining, by one or more processors, that in an existing human-to-computer dialog session between a user and an automated assistant, the automated assistant has responded to all natural language input received from the user during the human-to-computer dialog session; identifying, by one or more of the processors, based on one or more characteristics of the user, information of potential interest to the user or one or more actions of potential interest to the user; generating, by one or more of the processors, unsolicited content indicative of the information of potential interest to the user or the one or more actions of potential interest; and incorporating, by the automated assistant into the existing human-to-computer dialog session, the unsolicited content. In various implementations, at least the incorporating is performed in response to the determining that the automated assistant has responded to all natural language input received from the user during the human-to-computer dialog session.

These and other implementations of technology disclosed herein may optionally include one or more of the following features.

In various implementations, the unsolicited content may include unsolicited natural language content. In various implementations, the identifying may be based at least in part on one or more signals obtained from one or more computing devices operated by the user. In various implementations, the one or more computing devices operated by the user may include the given computing device currently operated by the user.

In various implementations, the one or more signals may be received from another computing device of the one or more computing devices operated by the user that is different than the given computing device currently operated by the user. In various implementations, the one or more signals may include an indication of a state of an application executing on the another computing device. In various implementations, the indication of the state of the application may include an indication that the application is providing media playback. In various implementations, the indication of the state of the application may include an indication that the application has received a search query from the user or provided search results to the user.

In various implementations, the method may further include determining, by one or more of the processors, a desirability measure that is indicative of the user's desire to receive unsolicited content, wherein the desirability measure is determined based on one or more signals, and wherein at least the incorporating is performed in response to a determination that the desirability measure satisfies one or more thresholds. In various implementations, the unsolicited content may include one or more user interface elements, each user interface element being selectable, by the user, to cause the automated assistant to provide the information of potential interest to the user or to trigger one or more of the actions of potential interest to the user.

In another aspect, a method may include: determining, based on one or more signals, that a user is within earshot of one or more audio output devices; identifying, based at least in part on one or more characteristics of the user, information of potential interest to the user or one or more actions of potential interest to the user; generating unsolicited content indicative of the information of potential interest or the one or more actions of potential interest; and incorporating, into an audible human-to-computer dialog session between the automated assistant and the user, the unsolicited content. In various implementations, the incorporation may be performed by the automated assistant in response to the determination that the user is within earshot of the one or more audio output devices.

In addition, some implementations include one or more processors of one or more computing devices, where the one or more processors are operable to execute instructions stored in associated memory, and where the instructions are configured to cause performance of any of the aforementioned methods. Some implementations also include one or more non-transitory computer readable storage media storing computer instructions executable by one or more processors to perform any of the aforementioned methods.

It should be appreciated that all combinations of the foregoing concepts and additional concepts described in greater detail herein are contemplated as being part of the subject matter disclosed herein. For example, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as being part of the subject matter disclosed herein.

