Good for the Many or Best for the Few? A Dilemma in the Design of Algorithmic Advice

Abstract: Applications in a range of domains, including route planning and well-being, offer advice based on the social information available in prior users' aggregated activity. When designing these applications, is it better to offer: a) advice that if strictly adhered to is more likely to result in an individual successfully achieving their goal, even if fewer users will choose to adopt it? or b) advice that is likely to be adopted by a larger number of users, but which is sub-optimal with regard to any particular individual achieving their goal? We identify this dilemma, characterized as Goal-Directed vs. Adoption-Directed advice, and investigate the design questions it raises through an online experiment undertaken in four advice domains (financial investment, making healthier lifestyle choices, route planning, training for a 5k run), with three user types, and across two levels of uncertainty. We report findings that suggest a preference for advice favoring individual goal attainment over higher user adoption rates, albeit with significant variation across advice domains; and discuss their design implications.

\section{Introduction}
Applications that offer advice to users have become commonplace across domains as diverse as route planning (e.g. Google Maps or Citymapper), financial investment (e.g. PlanMode), and lifestyle and well-being (e.g. Noom). Increasingly, this advice is generated algorithmically, based on machine learning analysis of large amounts of historic data relating to aggregated prior use. The advice on offer thereby combines social cues from other users' activity with up-to-date data from the domain in question. Two important questions that may be considered when formulating this advice are: 1) is the user likely to adopt the advice, if it is offered? and 2) if adopted, is the advice likely to lead to the user achieving their goal?

In many cases, offering the advice most likely to result in users achieving their goal will also result the highest rate of user adoption. However, this is not always the case. There are times when this advice may seem counter-intuitive; such as when traffic conditions result in a route much longer in distance being shorter in duration. In other situations, such as advising major changes in lifestyle associated with improving personal health, the `best' advice might appear exceptionally challenging.  For example, behavior in response to social distancing guidelines during the COVID-19 pandemic has highlighted the tension between the standalone value of public advice on one hand and its adoption by individuals who find it hard to follow on the other. Extant research shows that in cases like these people do not always act like models of rational decision making, and may not necessarily make decisions that maximize utility, or follow advice that is optimal to achieving their goal. Instead, their decisions are likely to be based on heuristics and judgments that follow predictable biases, e.g. \cite{tversky1974judgment, kahneman979prospect, kahneman2006anomalies, dellavigna2009psychology, thaler2018cashews, barberis2013thirty}. These circumstances pose a potentially serious design dilemma. Should the user be offered the advice which, if strictly adhered to, would be most likely to result in them achieving their objective, even if they are statistically less likely to adopt or adhere to it? Or alternatively, should the user be offered advice that they are more likely to choose to adopt and see through, and which may still be helpful, even though they are less likely to achieve their goal in full? We characterize this dilemma as being between: 1) \textit{Goal-Directed advice} that is more likely to lead a user who fully adopts it to achieve their goal, even if overall adoption rates are likely to be lower, and 2)  \textit{Adoption-Directed advice} which is likely to have a higher adoption rate, but which has a lower probability of resulting in a user fully achieving their individual goal.

This dilemma is brought even more clearly into focus when machine learning and data analysis techniques used to predict likely outcomes might also be used to predict user adoption rates. What might this mean for designers of applications aiming to algorithmically generate valuable advice in complex situations? Which design choice would be preferable from the perspective of a user? Which type of advice might be considered more ethical? This paper reports on a study in which participants in an online experiment were asked to indicate a preference between examples of these different advice types in one of four different scenarios, each presenting the potential dilemma in a different domain context with particular characteristics that influence how the dilemma might present. We varied the role participants were asked to adopt when indicating their preference, and controlled for several other potentially confounding factors. Our analysis found that participants consistently favored presenting Goal-Directed advice over Adoption-Directed advice, albeit the degree to which this preference was shown differed significantly across the scenarios that were presented. We discuss our findings with reference to prior research in decision-making under uncertainty, recommender systems, and behavioral economics, and outline implications for designing systems that offer algorithmically generated advice.

\subsection{The Advice Design Dilemma}

The research reported in this paper addresses a dilemma in presenting users with algorithmic advice that aims to support their decision making under uncertainty, particularly in challenging contexts. This dilemma can appear when advice, which may be optimal to an individual achieving their goal, appears counter-intuitive. Or when that advice seems challenging to such a degree that many users may either discount or ignore it. In these circumstances, is this advice still optimal? and should an application offer this advice? Or is it preferable for an application to offer advice that a higher percentage of users are more likely to adopt, but which is less likely to lead these users towards fully achieving their goal?

For the purposes of this study we define these two alternative advice types as follows:

\begin{itemize}
    \item \textit{Goal-Directed advice:} advice that is more likely to result in users who adhere to it achieving their goal, but which users are less likely to adopt and adhere to.
    \item \textit{Adoption-Directed advice} advice that is more likely to be adopted and adhered to by a greater number of users, but which is less likely to result in a user fully achieving their goal.
\end{itemize}

It should be noted that in the situations discussed in this paper, we make explicit the assumption that selecting either of the advice options is preferable to acting independently, and would therefore result in a better outcome for the user than were they to follow neither.

\subsection{Contributions}
This research contributes to the CSCW community's understanding of how to design applications that offer algorithmically generated advice based on social information aggregated from the prior behavior of a large number of users, and has both theoretical and practical implications. From a theoretical perspective, we 1) extend prior research at the intersection of CSCW and recommender systems that offer socially-informed advice based on aggregated data from prior use;  2) introduce and characterize the design dilemma posed when having to select between Goal-Directed and Adoption-Directed advice; and 3) quantify the trade-offs it represents. From a practical perspective we offer initial guidance on how to approach this dilemma when designing applications that offer algorithmically generated advice.

\section{Related Work}
This research was inspired by the observation that users may discount or ignore algorithmic advice intended to support the choices they make towards achieving particular goals, specifically when this advice may appear counter-intuitive or challenging to adhere to. In order to situate our work we provide an overview of related prior research in 1) decision making under uncertainty; and 2) online advice and recommendation.

\subsection{Decision-making under uncertainty}
A primary motivation for offering algorithmic advice is to support users' decision-making. Under uncertainty, decision-makers tend to be receptive to advice \cite{bonaccio2006advice,yaniv2000advice}, from experts \cite{cialdini1993influence,jowett2018propaganda,milgram1963behavioral,tyler1992relational} and from peers \cite{eysenbach2004health,bursztyn2014understanding,thaler2007behavioral}. This tendency has informed the design of interfaces for recommender systems. For example, in financial planning  \cite{adomavicius2005toward,wang2007recommendation} advice from software agents is increasingly replacing advice from humans \cite{adomavicius2011context,jannach2016recommender}, evidenced in the growth of financial `Robo-Advisors', such as Betterment \footnote{www.betterment.com}, which provide online advice with minimal human intervention.

Even where valuable advice may be available, decision-makers still tend to rely on simple heuristic principles to reduce the complexity of tasks such as assessing probabilities and predicting values \cite{kahneman979prospect,barberis2013thirty}. While often useful, heuristics can also lead to systematic errors or biases \cite{tversky1974judgment}. Their impact has been extensively studied in the context of personal finance \cite{benartzi1999risk,thaler2007behavioral}. Prior research shows the effectiveness of changing default participation to opt-in, but also the negative impact a reliance on simplistic heuristics has on asset allocation \cite{benartzi2007heuristics}. Decision-makers may excessively discount advice even when shown that the advice is good \cite{goodwin1999judgmental,lim1995judgemental}, because they have access to their own but not others' justifications \cite{yaniv2000advice}, or because of a bias toward egocentric or self-related information \cite{kruger1999lake,krueger2001role,clement2000primacy,dunning1996evidence}. As a result, decision-makers tend to exaggerate their own abilities \cite{kruger1999lake}, not take others' skills sufficiently into account, and display overconfidence and unrealistic self-assessments \cite{block1991overconfidence}. However, task complexity and quality of explanation may provide a counter-balance that reduces advice discounting \cite{onkal2009relative}, and decision-making can be affected by context driven challenges to self-control, which may represent inconsistent long-term and short-term preferences \cite{shefrin1988behavioral,thaler2018cashews}.

\subsection{Online Advice and Recommender Systems}
Insights such as these, from behavioral economics and social psychology, have influenced HCI, CSCW and information systems research (e.g. \cite{lee2011mining,gunaratne2015informing,caraban201923}) in areas such as recommender systems \cite{swearingen2001beyond, adomavicius2019reducing}. Recommender systems present advice to users in the form of suggestions \cite{nguyen2015perverse, wang2018tem} often based on collaborative filtering and a social context  \cite{herlocker2000explaining, tsai2017providing}. Recommender systems have been used to provide advice in a wide range of domains and use cases \cite{jannach2016recommender}, including advice on energy saving \cite{azaria2014advice}, nutrition \cite{zenun2017online}, finance   \cite{gunaratne2018persuasive}, and remedial behavior for computer programmers \cite{hartmann2010would}. Users' interactions with such systems are susceptible to similar confirmation biases as other instances of decision-making and advice selection \cite{jonas2001confirmation}, which has led researchers to investigate diverse recommendation and dissenting information strategies \cite{buder2012learning,nov2013personality}. In addition, prior research at the intersection of CSCW and recommendation systems found that negative social effects of activity transparency can result in users' increased adoption of mediocre advice \cite{nguyen2015perverse}. Another significant challenge to overcome, identified in this line of research, is the tendency for users to be put off by recommendations they consider demanding or challenging \cite{schwind2011will}. Moreover, the recommendations users find most useful may not always be the ones that by other measures are considered most accurate \cite{mcnee2006being}. An example is when optimal advice appears counter-intuitive, but following sub-optimal advice will still result in increased profit or satisfaction \cite{levy2016intelligent}. We extend this prior research by investigating participants' preferences between Goal-Directed and Adoption-Directed advice, under uncertainty and in counter-intuitive or challenging scenarios.

\section{Experimental Study}
To investigate the advice design dilemma, we probe participants' design preferences for which advice type to present in a future app. To study this, we conducted a between-subjects experiment online using Amazon Mechanical Turk (MTurk) as a recruitment platform. While acknowledging that the dilemma may appear in situations where there are more complex, ongoing interactions with applications that reoccur over time, and involve both short-term and evolving longer-term goals, e.g. when using diet and exercise apps,  we chose to echo a long tradition in economics, where simple, discrete-choice experiments are used to isolate a particular issue of concern and elicit preferences,  e.g. \cite{brynjolfsson2019using,bansback2012using,egesdal2015estimating,horne2005improving}. This degree of simplification allows us to gain purchase on the novel conceptual ground that the dilemma reflects, by allowing us to isolate the particular cases where advice might appear counter-intuitive or challenging, and to investigate the impact of factors, such as task domain, on these instances.

\subsection{Research Questions}
We ask the following research questions:
\begin{itemize}
    \item \textit{RQ1: Are participants' preferences between offering Goal-Directed advice and Adoption-Directed advice sensitive to different domain scenarios, or do they transcend specific settings?}
    \item\textit{RQ2: Are participants' preferences between offering Goal-Directed advice and Adoption-Directed advice sensitive to the different perspectives of `advice giver' and `advice receiver'?}
\end{itemize}

\subsection{Study Design}
We recruited a total of 1,589 US MTurk participants over the age of 18. Participation was limited to people with a record of at least 100 tasks at an approval rate of 95\% or higher.
Of these, 750 self-identified as women and 829 as men. Their self-reported ages ranged between 18 and 88, with a mean age of 36 (median = 33). Participants were paid a flat rate of \$0.65 for completing the survey based on an estimated time for completion of 5 minutes (median time spent on the study by participants was 5 minutes 30 seconds), and could take part in the study only once. Each participant was presented with a single selection task, asking for their preference between Goal-Directed and Adoption-Directed advice. Based on a 4X3 factorial design, participants were randomized into one of four scenarios, each set in a different domain, and three roles through which to consider their selection. We also controlled for other potentially confounding factors, such as the size of the gap between adoption and goal-effectiveness probabilities, and order of presentation. In addition to making their advice type selection, participants were required to answer an attention test question. Finally, participants were asked to briefly explain the reasons behind their selection. In the following sections we report the details of experimental procedure.

\subsection{Domain Scenarios}
To address RQ1, we selected four different domain scenarios representing different contexts where users might seek advice.  We designed each scenario to present a situation in which the advice most optimal for individual goal attainment is likely to appear counter-intuitive or seem challenging to adhere to. In these scenarios, we selected the details of both advice types based on realistic examples of advice offered in similar situations (sources referenced in the descriptions below). These scenarios were:

\begin{itemize}

    \item \textit{Route planning:} In this scenario (Figure ~\ref{fig:scenario1}), participants were asked to consider the case of a driver aiming to catch a flight and using an app to direct them to an airport in an unfamiliar location. It was based on examples of similar route planning (e.g. using Google maps) under different traffic conditions.

    \item \textit{Investment planning:} This scenario described a novice investor planning for retirement and aiming to recover from the previous year's poorly performing stock market. Two investment plans were shown. It was based on previous research into investment planning for retirement \cite{gunaratne2015informing,gunaratne2015influencing}.

    \item \textit{Training for a 5k run:} The third scenario presented a first time runner using a fitness-training app to guide their preparation for a 5K race in which they aim to finish in less than 26 minutes. It was based on training advice from \cite{gallwayrun,lacke2017run}.

    \item \textit{Making healthier lifestyle choices:} The fourth scenario described a user planning healthier lifestyle choices following a medical checkup. It was based on exercise and diet guidelines from \cite{usda2015diet}.

\end{itemize}

We selected four different domains in order to test whether participants' choices are sensitive to domain. The domains selected represent typical areas where algorithmic advice might be offered, but each has particular characteristics that make them interestingly different. For example, we selected `route planning' because the goal is immediate, whereas `training for a 5k run' offers a more long-term but well defined goal. In the `making healthier lifestyle choices' scenario the goal is both longer-term and less concrete, while we selected an `investment planning' scenario were the goal is in the longer-term future but decisions are made in response to historic activity. In each of these scenarios, participants were presented with a choice of two screen designs (e.g. Figure ~\ref{fig:image1} ). One screen shows Goal-Directed advice that is either counter-intuitive (e.g. in the route planning scenario the driver is required to double back and drive a significantly longer distance) or challenging to adhere to (e.g. in the making healthy lifestyle choices scenario the plan set strict rules for diet and exercise).  Participants were told that fewer people were likely to follow the advice presented on this screen, but that if they did manage to adhere to it, they would have a greater chance of achieving their goal. On the second screen design we presented Adoption-Directed advice that might appear more immediately intuitive (e.g. in the route planning scenario the route was shorter in distance and appeared more direct) or easier to adhere to (e.g. in the making healthier lifestyle choices scenario the plan set looser, less stringent, targets). Participants were told that more people were likely to follow this advice, but that they would have a reduced chance of completely achieving their goal.

\begin{figure}[t]
 \centering
  \begin{subfigure}{0.49\textwidth}
  \captionsetup{margin=0.1cm}
  \includegraphics[width=0.95\linewidth]{figures/route_planning_goal.jpg}
  \caption{`Goal-Directed' advice: Participants are shown a route that is longer in distance but shorter in duration. They are instructed that users will have a 90\% probability of catching their flight, but that only 15\% of users are likely to adopt it.}
  \label{fig:scenario1}
  \Description{Screenshot of the `Route planning' scenario, showing `Goal-directed' advice}
  \end{subfigure}
  \begin{subfigure}{0.49\textwidth}

  \captionsetup{margin=0.1cm}
  \includegraphics[width=0.95\linewidth]{figures/route_planning_adoption.jpg}
  \caption{`Adoption-Directed' advice: Participants are shown a route that is shorter in distance but longer in duration. They are instructed that users will have a 15\% probability of catching their flight, but that 90\% are likely to adopt it.}
  \label{fig:second_scenario}
  \Description{Screenshot of the `Route planning' scenario, showing `Adoption-directed' advice.}
\end{subfigure}

\caption{Screen designs from the `Route planning' scenario, where the goal-adoption gap is 90-15 and the `goal' term is given first in the description text. We show, (a) Goal-Directed advice and (b) Adoption-Directed advice. The text above each image explains the advice-type choices to participants.}
\label{fig:image1}
\end{figure}

\subsection{Participant Roles}
To address RQ2, and investigate whether preferences would manifest differently depending on whether participants were offering or receiving advice, we randomly assigned each participant to one of three roles, \textit{developer}, \textit{user}, and an \textit{ethical choice} role. By assigning participants the role of developer we placed them in a scenario where information about likely rates of goal success and adoption would be available, therefore highlighting the dilemma as it may manifest in realistic settings. These participants were asked, ``If you were the app developer, what would you show to the user in this situation?''. In assigning participants the role of user they were placed in a setting akin to a user study where they were presented with goal success and adoption information to elicit a hypothetical choice. These participants were asked, ``If you were the app user, what would you want to be shown in this situation?''. By assigning participants the ethical choice role we were presenting a scenario where they would need to know about the potential for individual goal success and likely adoption rates in order to weigh up the ethics of their advice type preference. These participants were asked, ``Which version of the screen below is more ethical advice to give to the user?''. We did not provide a definition of what `ethical' should be, rather preferring to leave this to participants' own conceptualization of the term. We did this in order to allow participants to focus on responding to the dilemma posed, rather than potentially taking issue with an externally imposed understanding of what is ethical. Our advice choice rationale question, discussed below, allowed us the opportunity to unpick possible issues that might arise from participants' different conceptualizations.

\subsection{Additional Study Design Considerations}
\subsubsection{Goal-adoption gap}
We also wanted to account for the possibility that participants' preferences would be influenced by the apparent size of the gap between the probability that users would achieve their goal and the probability that users would follow the advice. To control for this, we decided to create two variations and randomly assign participants to one of these. In the first, participants selected between a Goal-Directed advice option with a 90\% goal attainment probability but 15\% adoption rate, and an Adoption-Directed option with 15\% goal attainment probability but 90\% adoption rate. In the second variation, the Goal-Directed advice option presented 60\% goal attainment probability but 45\% adoption rate, and the Adoption-Directed option presented 45\% goal attainment probability but 60\% adoption rate. In all cases participants were told that choosing to follow \textit{neither} advice would result in the user having only a 1\% probability of achieving the goal. This was to highlight to participants that both advice options would be valuable to the user.

\subsubsection{Ordering effects of interface elements}
To control for possible ordering effects, we randomly varied the presentation of advice type options on the page, such that half the time Goal-Directed advice was on the left-hand side and Adoption-Directed advice was on the right, and the other half this was reversed. We also randomly varied the order of the terms referencing Goal-Directed and Adoption-directed advice in the description text above the screen designs. Half the time, the Goal-Directed probability was presented first, e.g. in the route planning scenario with a 90-15 goal-adoption gap the descriptions would read: ``Following the advice in this screen presentation, the user will have about 90\% probability of catching their flight; but about 15\% of users will follow this advice''. For an example see Figure ~\ref{fig:image1}. In the remainder, the Adoption-Directed probability was presented first, e.g. in the investment scenario with a 60-45 goal-adoption gap the descriptions would read: "About 60\% of users will follow this advice; but following the advice in this screen presentation, the user will have about 45\% probability of reaching their retirement goal".
\subsubsection{Attention test}
To filter out participants who simply rushed through the study without taking time to consider their responses, we also included a simple attention test question on the main study page based on the content of the particular scenario being shown. This type of test is commonly added to experimental tasks undertaken on MTurk.

\begin{figure}[t]
 \centering
  \includegraphics[width=0.9\linewidth]{figures/scenario2.jpg}
  \caption{Screenshot of the second scenario test of participants' understanding of independence between the goal attainment and adoption rate probabilities in the `Route planning' scenario.}
  \label{fig:second_scenario}
  \Description{Screenshot of scenario 2 the question testing understanding of the independence of goal attainment and adoption rate probabilities, in the `Route planning' scenario.}
\label{fig:image2}
\end{figure}

\subsubsection{Independence of goal attainment and adoption rate probabilities}
Because we were asking participants to make a design decision regarding which advice type to present to users, rather than which advice they would follow themselves, we wanted to be confident that participants understood the trade-off the dilemma poses. One important aspect of this was understanding that the predicted goal attainment and likely adoption rate probabilities were independent of each other, rather than on an interconnected `sliding-scale', such that an increase in one would automatically lead to an equivalent reduction in the other. To test for this, we added a multiple choice question set in a different domain scenario than the participant had been presented in the main study question. Here we presented participants with a single screen design, saying that it would offer users a 50\% probability of achieving their goal. We then asked them to select from 4 choices the proportion of users who would likely follow this advice (the correct response being: `The scenario does not provide enough information to answer', see Figure ~\ref{fig:second_scenario}).

\subsection{Advice choice rationale}
To better understand participants choice between advice types, and to probe the reasons behind these preferences, we also asked them to briefly explain their choice. First we reminded participants of the selection they had made in the main scenario, and then we asked them for a brief free-text response explaining why they selected their chosen advice type. We also probed participants about their responses to the second scenario in a similar fashion, using these responses to sanity check our decision to test for an understanding of goal attainment and adoption rate independence.

\section{Analysis}
In order to have more confidence in our findings, we adopted a conservative approach to data inclusion. First we discounted 3 submissions from participants who did not complete the task or who spent less than one minute on the task, as we considered these unreliable. We then discounted data from 434 participants who responded incorrectly to the attention check question, the answer to which could be readily found in the text describing the scenario. Such attention tests are commonly used to address potential concerns over data validity in studies where participants are recruited through MTurk   \cite{hauser2016attentive,landers2015inconvenient}, and this is not an atypical number, as up to 42\% of participants in MTurk studies have been found to be inattentive \cite{fleischer2015inattentive}. We then also removed data from 444 participants who responded incorrectly when tested about the independence of advice type probabilities. Data from these participants was discounted because we felt that a failure to understand the independence of the probabilities shown for Goal-Directed and Adoption-Directed advice indicated that participants may not clearly understand the choices available when the dilemma presented. This is a relatively large number of participants to exclude, and we speak further to this decision in the Discussion section. After the removal of data from 772 of 1,589 candidates, we were left with a final data set of 817 participants.

For our quantitative analysis, we first separated participants' response data according to their role assignment, i.e. user, developer, or ethical choice; and the scenario they were assigned, i.e. `route planning', `investment planning', `training for a 5k run', or `making healthier lifestyle choices'. The number of participants in each category is recorded in Table \ref{table:categories}.

\begin{table}
\centering
\begin{tabular}{|c | c c c c|}
 \hline
  & Scenario & & & \\
 \hline
 Role & Healthy Living & Investment Planning & Race Training & Route Planning\\
 \hline
 developer & 68 & 58 & 76 & 88\\
 ethical & 74 & 73 & 55 & 81 \\
 user & 54 & 55 & 53 & 82\\
 \hline
\end{tabular}
\caption{Number of participants in each experimental category.}
\label{table:categories}
\end{table}

We then compared the rate at which participants in each sub-group preferred the Goal-Directed advice to a 0.5 `indifference' threshold, using one-sample z-tests. A 0.5 reference threshold was used for comparison because we would expect participants to be \textit{indifferent} if 50\% of the sample selected the goal-directed advice and 50\% selected the adoption-directed advice. This initial analysis provides us with a high level view of the preferences of each of these sub-groups.

To extend our analysis, we used a logistic regression to investigate how these variables might influence an individual's probability of selecting the Goal-Directed advice. To do this we created three models. In Model 1 we included the participant's assigned role and scenario as main features in the regression. For Model 2 we also included terms for covariates: the goal-adoption gap, the side of the page on which the advice types were presented, the order that advice types were listed in the description text, participant gender, the log of participant age, and the log of time taken to complete the study. Finally, in Model 3, we included an interaction term between participant role and scenario. We use logistic regression because our dependent variable and the majority of our independent variables were categorical.

We also analyzed these participants' responses to the post-study question probing their advice choice rationale. We received responses from 814 of the 817 participants included in our quantitative analysis. These responses were brief, typically somewhere between a few words and a short paragraph in length, with a mean response length of 24 words. There was no need to sanitize these responses, most likely due to conservative approach we took to data inclusion (described above). We performed a simple thematic analysis \cite{braun2006using}, which involved an initial close reading and clustering led by the first author, followed by refinement and agreement seeking with one other researcher. For a first pass, the responses from all participants were considered together, resulting in a high-level set of main themes. Following this, we divided the responses into two groups so that those who selected a preference for Goal-Directed advice and those who selected Adoption-Directed advice were considered independently. This provided us with a more fine-grained understanding of the differences between the two groups and similarities within them. Having identified patterns across the data, and extracted key ideas, we sought agreement across interpretations, through discussion and where necessary testing alternative framings.

\begin{figure}[b]
 \centering
  \begin{subfigure}{0.49\textwidth}
  \captionsetup{margin=0.1cm}
  \includegraphics[width=0.9\linewidth]{figures/scenarioplot.png}
  \caption{Preference by Scenario}
  \label{fig:scenario_graph}
  \Description{Graph showing the proportion of people preferring Goal-Directed advice according to the scenario presented, with 95\% confidence intervals.}
  \end{subfigure}
  \begin{subfigure}{0.49\textwidth}
  \captionsetup{margin=0.1cm}
  \includegraphics[width=0.9\linewidth]{figures/roleplot.png}
  \caption{Preference by Role}
  \label{fig:role_graph}
  \Description{Graph showing the proportion of people preferring Goal-Directed advice according to the role assigned, with 95\% confidence intervals.}
 \end{subfigure}

\caption{Proportion of people preferring Goal-Directed advice according to: (a) the scenario presented; and (b) the role they were assigned, with 95\% confidence intervals. Red lines represent the 0.5 `indifference' threshold}
\label{fig:image3}
\end{figure}

\begin{table} [t]
  \caption{Results of logistic regression testing effect of role and domain on probability of selecting Goal-Directed advice. Model 1 is the model of main effects; Model 2 includes covariates; Model 3 includes an interaction effect between role and domain.}
  \label{tab:log_reg}
  \begin{tabular}{lllll}
    \toprule
     & &Model 1&Model 2&Model 3\\
    \midrule
    Intercept & & 2.07(0.26)***&3.26(1.44)* & 3.99(1.51)**\\
    Role & \textit{developer} & -0.08(0.22) &-0.07(0.22) & -0.54(0.54)\\
    &\textit{ethical} & 0.22(0.22)& 0.24(0.23)& -0.46(0.56)\\
    Domain & \textit{health} & -1.11(0.26)***& -1.11(0.27)***& -1.45(0.53)**\\
    &\textit{investment}& -0.75(0.28)**& -0.75(0.28)**&-1.45(0.53)** \\
    &\textit{race}& -1.08(0.27)***& -1.08(0.27)***& -1.51(0.53)**\\
    Gap size & \textit{90-15 split} & & 0.11 (0.18)& 0.10(0.18)\\
    Text order & \textit{goal first, adoption second} & & 0.09(0.18)& 0.07(0.18)\\
    Screen side & \textit{goal left, adoption right} & & -0.09(0.18)& -0.10(0.18)\\
    Gender & \textit{male} & & -0.21(0.18)& -0.19(0.19)\\
    &\textit{other} & & -0.04(1.15)& -0.05(1.15)\\
    Age & & & -0.41(0.31)& -0.45(0.31)\\
    Time on task& & & 0.07(0.19)& 0.03(0.19)\\
    Role*Domain&\textit{developer*health} & & & 0.42(0.68)\\
    &\textit{developer*invest.} & & & 1.16(0.72)\\
    &\textit{developer*race} & & & 0.31(0.67)\\
    &\textit{ethical*health} & & & 0.60(0.70)\\
    &\textit{ethical*invest.} & & & 0.94(0.71)\\
    &\textit{ethical*race} & & & 1.11(0.73)\\
    \midrule
    AIC & & 793.08&801.59 &808.00 \\
    Pseudo R$^2$ & &0.033 &0.037 &0.044\\
  \bottomrule
  \multicolumn{5}{l}{(Signif. codes:  $<0.001 "***"; 0.001 - 0.01 "**"; 0.01 - 0.005 "*"; 0.05 - 0.1 "^t" $)} \\
\end{tabular}
\end{table}

\section{Findings}
Of the 817 participants in our final data set, 658 (80.54\%) selected a preference for displaying Goal-Directed advice over Adoption-Directed advice (z=17.46, p<0.05, 95\% C.I.[0.78,0.83]). This preference was consistent across all scenarios and all roles, and in all cases this preference was statistically significant using a one-sample z-test of proportions relative to the 50\% indifference threshold. With regard to the different domain scenarios, we found that 74.49\% of participants shown the `making healthier lifestyle choices' scenario (z=8.31,p<0.05,95\% C.I.[0.70,0.79]), 80.64\% of participants shown the `investment planning' scenario (z=10.72, p<0.05, 95\% C.I. [0.77,0.84]), 74.45\% of participants shown the `training for a 5k run' scenario (z=8.24, p<0.05, 95\% C.I. [0.70,0.79]), and 89.64\% of participants shown the `route planning' scenario selected a preference for Goal-Directed advice (z=15.43, p<0.05, 95\% C.I. [0.87,0.92]). These rates are visualized in Figure~\ref{fig:scenario_graph}. Similarly, 78.28\% of participants assigned the role of developer (z=11.69, p<0.05, 95\% C.I. [0.75,0.82]), 83.04\% of those assigned the ethical choice role (z=13.97, p<0.05, 95\% C.I. [0.80,0.86]), and 80.32\% of those assigned the role of user (z=11.87, p<0.05, 95\% C.I. [0.77, 0.84]) selected a preference for Goal-Directed advice (Figure~\ref{fig:role_graph}).

Extending beyond these high level findings, the results of the logistic regression (Table~\ref{tab:log_reg}) indicates that the scenario participants were shown significantly influences the probability that they select a preference for Goal-Directed advice. In particular, Models 1 and 2 show that participants shown the `route planning' scenario were significantly more likely to select this preference  relative to the other three scenarios. By converting the coefficients on scenario from Model 2 to probabilities (holding age and time on task at their medians - 35 years and 312 seconds, respectively), we see that the predicted probability of an individual selecting Goal-Directed advice is 94.23\% when shown the `route planning' scenario. This is significantly higher relative to the other scenarios: 84.39\% in the `making healthier lifestyle choices' scenario ($\beta$ = -1.11, S.E. = 0.27, z=-4.14, p<0.000, 95\% C.I. [-1.63,-0.58]), 88.55\% in the `investment planning' scenario ($\beta$=-0.75, S.E. = 0.28, z = -2.67, p=0.008, 95\% C.I. [-1.30,-0.20]), and 84.79\% in the `training for a 5k run' scenario ($\beta$=-1.08, S.E. = 0.27, z = -3.99, p=0.0001, 95\% C.I. [-1.60,-0.55]). These comparisons are significant even when correcting for multiple comparisons using a Bonferroni correction with a threshold of p=0.017. On the other hand, Model 3 shows that participants' did not differ in their preference for Goal-Directed advice across roles.

Findings from our analysis of participants' responses to the post-study choice rationale question, explaining \textit{why} they selected a preference for presenting users with Goal-Directed advice or Adoption-Directed advice, offer further insight into these quantitative results. A number of examples from this analysis are included in the discussion that follows, here we briefly list the themes that emerged.

\begin{itemize}
    \item \textit{Relationships between individual users and the crowd:} Some participants selecting a preference for Goal-Directed advice explained their choice in terms of \textit{supporting an individual user in successfully achieving their goal}. Others stressed that it was \textit{the responsibility of individuals whether they choose to follow advice or not}, or \textit{wanted to reward those with the capacity to adhere to advice}, or adopted an \textit{egocentric standpoint} stressing their own qualities in comparison to others. For participants who selected Adoption-Directed advice, \textit{concern for the wider benefits of a greater number of people} or the \textit{pragmatic partial achievement of a goal by larger numbers of users} were important explanations.
    \item \textit{Personal insight and domain experience:} This was a strong theme among participants selecting Adoption-Directed advice, who made \textit{decisions based on heuristics} such as `diversify investment'. Those selecting Goal-Directed advice often made \textit{connections to their own lifestyles or practices}.
    \item \textit{Information detail and complexity:} For those selecting Goal-Directed advice, the \textit{discipline imposed by a more detailed plan} would make it easier to follow through to successful goal achievement. Participants selecting a preference for Adoption-Directed advice suggested the less detailed and demanding option would be \textit{a more realistic plan}.
    \item \textit{Success of the app or company:} Typically found among participants in the developer role, but across all scenarios and both advice types, this theme stressed success of the app, or the company behind it. For participants selecting Adoption-Directed advice the focus was on attracting a high number of downloads and \textit{large initial user base}, so that the app would be a viable business proposition. For participants selecting Goal-Directed advice the focus was more on \textit{user retention and satisfaction}, on minimizing bad reviews and increasing good ones.
\end{itemize}

\section{Discussion}
Our findings show participants indicating a clear preference for offering users Goal-Directed over Adoption-Directed advice, in the tasks we presented. This was consistent, albeit with varying rates of preference, across each of the four scenarios we tested in order to address RQ1, and was also invariant to the role participants were assigned in order to address RQ2.   This suggests that most participants focused on those users able to adopt and then adhere to the advice being offered, rather than the wider population. Such a finding is consistent with prior findings about  moral dilemmas, (e.g. \cite{bonnefon2016social,thomson1976killing}), which suggest that people generally avoid decisions that are sub-optimal at the level of individual utility, even if they increase aggregated social utility. Our  findings about participants' choices are echoed in our analysis of participants' responses to the advice choice rationale question, where one of the themes to emerge under the theme \textit{`Relationships between individual users and the crowd'} was \textit{`supporting an individual user in successfully achieving their goal'}, while another emphasized that it is \textit{`the responsibility of individuals whether they choose to follow advice or not'}, e.g.:

\begin{displayquote}
``The point of investment advice is to achieve the goal - it is up to the individual to decide if they will follow the advice'' (\textit{Investment planning, user, 90-15, Goal-Directed advice})
\end{displayquote}

\begin{displayquote}
``The app can't control who chooses to follow their advice, but they can control the quality of that advice, so I felt it was more ethical to provide investment advice that had a higher likelihood of helping a committed client reach his/her investment goal.'' (\textit{Investment planning, ethical, 90-15, Goal-Directed advice})
\end{displayquote}

Such views were not universal though. A significant number of participants selected Adoption-Directed advice, and the rationale question responses indicate that \textit{`concern for the wider benefits of a greater number of people'}, which might accrue through the \textit{`pragmatic partial achievement of a goal by a large number of people'}, were key motivators. This was fairly common among participants presented with the `investment planning' and `making healthy lifestyle choices' scenarios, but rarer among those shown the `training for a 5K run' scenario; and there were no examples at all amongst those in the `route planning' scenario. Example responses include:

\begin{displayquote}
``Getting more people to consider their investment goal is still very beneficial, even if they fall short of their goal.  This scenario meant that more people would be thinking about and taking action with their investments.'' (\textit{Investment planning, developer, 60-45, Adoption-Directed advice})
\end{displayquote}

\begin{displayquote}
``I think it would help more people make some steps toward a healthier lifestyle which is better to me than fewer people achieving completely healthy lifestyles.'' (\textit{Making healthy lifestyle choices, developer, 60-45, Adoption-Directed advice})
\end{displayquote}

These different preference explanations are one indicator of the influence scenario may play in the selection of advice types; suggesting greater nuance underneath the across the board preference we found in response to RQ2. Another is our quantitative finding that participants shown the `route planning' scenario were significantly more likely to select a preference for Goal-Directed over Adoption-Directed advice. Each of these points to the importance of task complexity and task familiarity in the decisions participants made.

\subsection{Task complexity}
The `route planning' scenario is arguably simplest, and most likely to manifest in the wild as a one-shot binary-selection. In this scenario, success only benefits the individual user, and there can be no partial achievement of the goal. The user either arrives in time to catch their flight, or they do not. Because of this, research showing the importance of loss aversion in decision-making e.g.\cite{kahneman979prospect,barberis2013thirty} might offer an explanation for our finding that participants shown this scenario were significantly more likely to prefer Goal-Directed over Adoption-Directed advice, when compared with participants shown any one of the other three scenarios. In contrast, the `making healthier lifestyle choices' scenario is likely the most complex when translated into real world experiences. Yet here too task complexity played a role. For participants preferring to offer Goal-Directed advice, the \textit{`discipline imposed by a more detailed plan'}, and the specific targets set would make it easier to follow through to successful goal achievement, e.g.:
\begin{displayquote}
``The instructions were much more rigid and difficult to achieve. For example, where one plan said to simply reduce alcohol consumption, the other listed an exact number to stay below. I felt that people that followed the more detailed plan received a better outcome than following a very basic and vague outline of a plan.'' (\textit{Making healthy lifestyle choices, user, 90-15, Goal-Directed advice})
\end{displayquote}

However, participants shown this scenario but preferring  Adoption-Directed advice suggested the less detailed and less demanding option would be simpler to understand and \textit{`a more realistic plan'}, that would also be easier to adhere to. This supports research suggesting users are put-off following advice from online recommender systems if it is considered demanding or challenging, e.g. \cite{schwind2011will}.
\begin{displayquote}
``Based on what I've seen, most people would take one look at the other plan and run away from it.  There's too much to it and people don't want to be bothered with programs like that.  I think more people would be willing to take the common sense approach, which is what the plan I chose was.'' (\textit{Making healthy lifestyle choices, developer, 60-45, Adoption-Directed advice})
\end{displayquote}

Further research is needed to better understand the impact of task complexity on participants' preferences. In this study, the choice between Goal-Directed and Adoption-Directed advice was presented as a binary selection between static screen designs. While this was intentional, and allowed us to isolate particular issues of concern, additional research is needed to fully investigate the nuances within more complex evolving interactions. Future studies should consider a longitudinal approach, that allows a deeper dive to study of how different options and trade-offs might be captured and expressed dynamically for users to explore and select between.

\subsection{Task familiarity}
In addition to being simplest, the `route planning' scenario may also be the most familiar to participants. The use of journey planners is widespread, and research indicates that in circumstances where travel time is the key factor users are likely to follow their advice \cite{samson2019exploring}. For the large number of participants who selected a preference for Goal-Directed advice when presented with this scenario, it may be that trust in this technology, and familiarity with the visualizations associated with it, reduces the apparent counter-intuitiveness of the advice offered. Further evidence of the impact that task or scenario familiarity might have on participants' advice type preferences comes via references to \textit{`personal insight and domain experience'} in their explanations of advice type selections. This was evident among participants that selected Adoption-Directed advice who often made \textit{`decisions based on heuristics'}, for example:
\begin{displayquote}
``Because historically, placing all your eggs in one basket is bad.  Better to spread the risk and be more diversified.  A fund that does well in the previous year will not necessarily do well in the future.'' (\textit{Investment planning, ethical, 90-15, Adoption-Directed advice})
\end{displayquote}

While those who selected Goal-Directed advice often  made \textit{`connections to their own lifestyles or practices'}, such as:

\begin{displayquote}
``I don't mind taking back roads or scenic detours as long as I get to my destination on time.  I know big city traffic can be horrible especially during rush hours and on popular roads.  Sometimes taking a round about route gets you to your destination faster.'' (\textit{Route planning, user, 90-15, Goal-Directed advice})
\end{displayquote}

We suspect that participants make these personal references for reasons closely related to those underlying research that suggests decision-makers privilege self-related information and adopt an egocentric standpoint, e.g. \cite{kruger1999lake,krueger2001role,clement2000primacy,dunning1996evidence}, and to research suggesting decision-makers may discount advice because they have access to their own justifications, but not to those of others \cite{yaniv2000advice}. Because of this, future studies might investigate how advice types can be personalized.

\subsection{Cultural differences}
Participants' average preference for Goal-Directed over Adoption-Directed advice may also reflect dominant attitudes towards individual achievement versus collective benefit in the society our sample is drawn from. We recruited MTurk workers with an account in the U.S, a society where individualism is highly valued \cite{markus1991culture,triandis1990multimethod}. It remains an open question whether our findings would translate to participants recruited from societies considered to place greater value on shared responsibility \cite{hacker2012shared} or collectivism \cite{markus1991culture,triandis1990multimethod}. In addition to potential biases associated with national cultures, there may also be a bias in the culture of participation in MTurk. Paid crowd workers may make different choices from participants recruited in other ways. Further research should investigate this, as potentially analogous behavior has been seen in comparisons between tasks undertaken by crowd workers and citizen scientists \cite{cartwright2019crowdsourcing}.

This potential bias could  explain the many examples in which participants who preferenced Goal-Directed advice adopted an \textit{`egocentric standpoint'} in their explanation that was explicitly made in opposition to others, or which stressed the individual qualities of the participant themselves, e.g.:

\begin{displayquote}
``I'm after the results. I don't care whether others are willing to put in the effort - I am.'' (\textit{Making healthy lifestyle choices, ethical, 60-45, Goal-Directed advice})
\end{displayquote}

\begin{displayquote}
``I don't need to feel validated by others' choices.  If I know I have a higher chance of reaching my goal I'm going to take it.'' (\textit{Route planning, user, 90-15})
\end{displayquote}

\begin{displayquote}
``I am a goal oriented person, once I start a goal I almost ALWAYS reach it. So for me, that scenario was more appealing.'' (\textit{Training for a 5k run, developer, 90-15, Goal-Directed advice})
\end{displayquote}

However, statements such as these may also reflect previous research which suggests decision-makers make overconfident self-assessments \cite{block1991overconfidence} and overestimate their own abilities \cite{kruger1999lake}. It could be that participants who are able to simply `make a change', e.g. when dieting, show a preference for Goal-Directed advice and simply assume that others are similar. Further research is needed to unpack the more nuanced impact of culture on peoples' advice type preferences, as design decisions based on social cue data gathered in a single country, or from within a single online community, may not translate into other contexts.

\subsection{Design values and ethical considerations}
In making decisions about what type of advice is best to offer users, designers are asked to make value judgments in complex and ethically ambiguous situations.  These judgements, and the values they operationalize, may be subject to different interpretations when employed in the service of an altruistic cause than when used to drive commerce \cite{chivukula2019analyzing}. Yet, despite the influence designers have on peoples' lives \cite{friedman2003human,redstrom2006persuasive,buchanan1985declaration}, prior discussion of the role of ethics in the design of technologies that offer advice or try to persuade users has tended to focus on the role they might play in the physical \cite{purpura2011fit4life}, material \cite{gunaratne2015influencing}, and psychological welfare \cite{kehr2012transformational} of individuals, considering societal good as reflecting value judgments on what might be considered `good' for the individual user, e.g. \cite{berdichevsky1999toward}, rather than possible wider impacts. Excepting `route planning', in each of the scenarios we presented there are potential benefits for users who partially achieve their goal. These benefits would be additional to the benefit gained by those who ultimately achieve their goal in full. The ambiguity surrounding such judgments is also reflected in themes from our analysis of the choice rationale responses. This highlights important ethical questions for designers who may have to choose between showing people what they apparently want as individuals, versus showing them what may have a larger positive impact on the greatest number of people. This contrast, between the individual and the group, adds an additional layer of complexity to decisions about `good for whom', in a similar way to how feminist HCI \cite{bardzell2011towards} and HCI for sustainability \cite{blevis2007sustainable,disalvo2010mapping} have called on designers to reconsider what it means to `do good'.

Related to the ethical considerations of `good for whom' are questions about `good for business'. Among participants assigned the developer role there was a repeating pattern of explanations that considered the \textit{`success of the app or company'}, to be the primary motivation for their selection of advice type to offer. This theme crossed all scenarios and both advice types, with subtle differences. For participants selecting Adoption-Directed advice the focus was on attracting a high number of downloads and \textit{`large initial user base'}, so that the app would quickly become a viable business proposition. For participants selecting Goal-Directed advice the focus was more on \textit{`user retention and satisfaction'}, on minimizing bad reviews and increasing good ones, so that the business was viable in a more ongoing sense. Examples include:

\begin{displayquote}
``If you are looking to generate money and a large user base with the app, then I think you would want to drive more traffic to your app.  In that case, you want the option where 90\% of people would participate.'' (\textit{Making healthy lifestyle choices, developer, 90-15, Adoption-Directed advice})
\end{displayquote}

 \begin{displayquote}
``Because I'd want the app to earn a good reputation by giving correct advice'' (\textit{Route planning, developer, 90-15, Goal-Directed advice})
\end{displayquote}

\subsection{Experimental design choices}
The choices made in designing any study are reflected in its findings. In this study, the choice between Goal-Directed and Adoption-Directed advice was intentionally presented as a one-shot binary choice, following a convention familiar in choice experiments undertaken by economists, e.g \cite{bansback2012using,egesdal2015estimating,horne2005improving}. We also asked participants to adopt one of three different roles when considering their selection of Goal-Directed or Adoption-Directed advice, and we selected four scenarios in which to probe participants' preferences. While similar techniques have been widely used in prior research, we should not exclude the possibility that differences observed are a function of the experiment itself. In order to gain more nuance to our understanding, and to more properly distinguish between possible experimental effects and deeper preferences, future research should consider longitudinal studies and qualitative methods. This would support investigation into if and how design choices can influence adoption rates for counter-intuitive or challenging advice, and therefore reduce the dilemma's impact. These might be considered in the context of health, where relationships with advice givers are likely to be ongoing, and where short and long-term goals interact. To better understand the dilemma, future research should also consider including additional conditions in which Goal-Directed and Adoption-Directed advice align. Adding conditions in which the probability of individual goal attainment and the probability that the advice will be adopted are both high, and conversely where they are both low, will help to further pick apart peoples' preferences.

\subsubsection{Testing for understanding the independence of advice type probabilities}
To be confident that we only included data from participants who understood the trade-offs that the dilemma poses, we included a test in a second domain. Failure of this test excluded the responses of 444 participants. While we consider this the correct approach for making an initial study into the advice dilemma we identify, we also acknowledge the impact of this choice. For this reason we compare the advice type preferences of this set of 444 participants with those of the 817 participants included in our full analysis. In this comparison, we first see an overall reduction in the preference for Goal-Directed advice over Adoption-Directed advice: 65.9\% (290/444) against 80.5\% (658/817). A chi-square test of independence between these two samples shows that these differences are significant ($\chi^{2}$=32.23, $p$<0.00). When broken down by domain scenario, participants in each group also indicated a reduced preference towards the Goal-Directed advice. In three of the four scenarios this difference was significant.  In the `investment planning' scenario the difference was 60.83 \% against 80.64\% ($\chi^{2}$ = 13.50, $p$ < 0.00); in the `training for a 5k run' scenario it was 54\% against 74.45\%  ($\chi^{2}$ = 11.40, $p$ < 0.00); and in the `route planning' scenario it was 76.56\% against 89.64\% ($\chi^{2}$=10.5, $p$=0.001).

Despite these differences, our initial results are robust and the findings remain largely the same with or without the exclusion. However, they do indicate that there is likely more nuance to the preferences people might have in actual use than is expressed in this initial experiment. We believe that this indicates the potential importance of numeracy skills on participants' selections, and in understanding the trade-offs the dilemma introduces. It also further indicates that research is needed into the way the dilemma manifests in the wild, where users may be exposed to repeated ongoing advice, making longitudinal decisions where short-term and long-term motivations and goals interact, and where issues of low numeracy and low engagement may be common.

\subsection{Algorithmic Advice Dilemmas: Future Research}
The experimental design of this study, i.e. a one-shot, discreet choice between advice types, was chosen in order to isolate the particular cases where advice might appear counter-intuitive or challenging, and echoes a tradition of similar experimental studies in economics. However, in practice this advice dilemma may also appear in cases where users have more complex, ongoing relationships with applications and recommender systems that involve use cases of repeated or adjusted advice \cite{wang2018tem}, often based on users' dynamic preferences and previous choices \cite{kapoor2015like, moore2013taste}, and other behavioral patterns such as in sequence-aware recommender systems \cite{quadrana2018sequence, quadrana2018sequence2}.

Our quantitative findings show that participants shown the `route planning' scenario, the scenario we consider most likely to be a one-shot decision, were also significantly more likely to select Goal-Directed advice. We also see from participants' response to the choice rationale question, that it was those shown the `making healthier lifestyle choices' scenario who most often mention task complexity, regardless of whether they selected a preference for Goal-Directed advice or Adoption-Directed advice. This is the scenario we consider most open ended and likely to present in a situation of regular ongoing interactions between a user and a system offering algorithmic advice. Taken in combination, these two findings strongly suggest that in the context of recommender systems, responses to the dilemma will need to be more subtle and nuanced. What a user prefers may vary across interactions, and may also vary through the temporal unfolding of interactions. Their preferences before advice is offered may vary from their preferences at the time of offering, they may vary again following success or failure in adhering to that advice, and they may vary yet again following information about the choices of other users. Future research, building on work at the intersection of CSCW and recommender systems \cite{chang2015using, koivunen2019understanding, xu2019think}, would therefore be needed to address the design dilemma outlined here in the context of repeated advice and collaborative filtering \cite{daskalova2018investigating}. For example, should algorithmic tools prioritize a sub-optimal `good' advice over optimal `best' advice if the sub-optimal advice is consistent with the choices of users in the network of the focal user, and therefore more likely to be followed? Such questions should be considered not only in terms of their effectiveness in driving users' choices, but importantly, in relation the ethical aspects of recommender systems and their design (e.g \cite{paraschakis2016recommender, milano2020recommender}).

\subsection{Design Considerations}
In this paper we probe a dilemma for designers of algorithmic advice systems. Here we discuss design considerations for these circumstances. These we consider complementary to existing guidelines, e.g. for  designing social recommender systems \cite{arazy2010theory, harman2014dynamics}.

Once identified, the dilemma between Goal-Directed and Adoption-Directed advice should be surfaced early on in the design process. Human-centered design methods advocate for iterative cycles of contextual inquiry, prototyping, and evaluation; and at each stage we would argue the need for comparative user studies, both qualitative and quantitative, to explore the dilemma as it plays out in-situ. This would allow not only comparison between different advice types, but also inquiry into specific ways they might be dynamically explored and compared by users; e.g. in situations such as route planning where the alternative recommendations may be presented simultaneously, but where designers must choose which advice to suggest as initially preferred. Our findings suggest not only that preferences may vary according to domain and scenario, but also that motivations underlying these preferences may vary too.

While our findings may indicate participants' clear preference for Goal-Directed advice, a substantial minority of participants selected a preference for Adoption-Directed advice. In practice, the advice design dilemma is often likely to present in complex situations where users have ongoing longitudinal interactions with applications offering advice. The variation we see in the strength of preference between the different domain scenarios, and the explanations participants gave for making these selections, across and within these scenarios, suggest that designers should investigate adaptive approaches to presenting algorithmic advice. At a simple level, this might involve making advice type a dynamic and/or configurable option, either when the user is completing initial setup or in use on a case by case situational basis. However, designers might also consider options that draw on prior research that inspired this work. For example, if we know that users have a tendency to discount good advice \cite{goodwin1999judgmental,lim1995judgemental} and display overconfidence in their own abilities \cite{block1991overconfidence} because of a bias towards self-related, egocentric information \cite{kruger1999lake,krueger2001role,clement2000primacy,dunning1996evidence}, can the choice of advice type be combined with challenges based on predicted goal success and adoption rates? and can the way this is presented to users guide these users towards more effective decision-making?  In such a case, we can imagine designing an application that uses machine learning to predict the likelihood of the advice dilemma, e.g. in advice relating to lifestyle changes, and which then adapts the advice offered dynamically, based first on data from aggregated prior use and then increasingly integrating this with the users' own responses. We can also imagine a potential user who receives advice, such as to quit smoking or to severely reduce salt or sugar consumption, but discounts the advice because they may be overconfident in their capacity to avoid the consequences, or because they are biased towards their current feelings of wellbeing. In this situation, the application might start - upon user consent - by varying the advice, at times using low adoption rates associated with Goal-Directed advice as an egocentric challenge to the user, and at other times using the less challenging but still beneficial goals associated with Adoption-Directed advice to provide a confidence supporting reward. Over time the application might blend its wider understanding of what represents Goal-Directed or Adoption-Directed advice, which is based on data aggregated from many users, with a more nuanced understanding of what these might mean for the particular user in question, and offer advice that responds to the dynamics of a particular situation in a more fine-grained way.

The example of applications that provide users with healthcare advice illustrates the considerations necessary for design and development in the medical domain. Digital therapeutics systems \cite{nebeker2019building}, for managing chronic diseases such as diabetes, are flourishing, with healthcare clinicians now prescribing apps to patients as supplements to their treatment regimens.  In this rapidly evolving situation, designers play an important role in helping shape the messaging that clinicians leverage through new digital therapeutic interactions.  Given a choice, does a physician prescribe the diabetes app that is most challenging to follow but has the best clinical outcomes when adhered to; or one that is more easily adopted by a larger number of patients, albeit with a lower effect size on those that follow its advice?  What factors should influence this decision?  And how might it be affected by the prescriber's inherent biases?  Is there a role for advanced analytics, based on social cues from historic use, in helping to personalize the advice these recommender systems offer, so that the clinician can promote adoption directed messages in patients who are likely to struggle with the more challenging path and goal directed advice for those who demonstrate a higher probability of completion? For the clinician, these trade-offs are not new. However, sharing the way these choices influence patients with designers or developers is.  This research indicates that new partnerships and collaborations between designers and healthcare clinicians are urgently needed to examine these trade-offs and develop best practices.

\section{Conclusion}
In this paper we have made an initial step towards better understanding the advice design dilemma that arises when advice most likely to maximize an individual's chances of achieving their goal is not the same as advice most likely to maximize the number of people who will adopt it. We studied participants' preferences when selecting between Goal-Directed and Adoption-Directed advice online in an experimental study spanning four domain scenarios, three user types, and two variations in the gap between goal attainment and advice adoption probabilities. We found an average preference for advice that favors individual goal attainment over higher user adoption rates, albeit with some variation across advice domains. The implications of these findings are both practical and theoretical. From a practical perspective, choosing between presenting advice that may be best in terms of individual goal attainment but which will reach few users, or advice that is merely good but will be adopted by many, is likely to be an increasingly common dilemma for designers to face in algorithmic advice design scenarios. From a theoretical perspective, the preferences shown for Goal-Directed advice (i.e. advice best for individual goal attainment that will be adopted by few users), and the explanations revealed in participants' survey comments, echo responses to moral dilemmas, e.g. \cite{bonnefon2016social,thomson1976killing}, where people eschew making decisions that are sub-optimal at the individual level, even if they are optimal at the population level. With the rapid growth in applications driven by algorithmic advice based on the social information derived from aggregated prior use, HCI and CSCW researchers and practitioners are likely to increasingly engage with these fundamental questions. Our findings will help inform this conversation.

\begin{acks}
This work was supported by the National Science Foundation award \#1928614.
\end{acks}

\bibliographystyle{ACM-Reference-Format}


The Age of the Galactic Disk

Abstract: I review different methods devised to derive the age of the Galactic Disk, namely the Radio-active Decay (RD), the Cool White Dwarf Luminosity Function (CWDLF), old opne clusters (OOC) and the Color Magnitude Diagram (CMD) of the stars in the solar vicinity. I argue that the disk is likely to be 8-10 Gyr old. Since the bulk of globulars has an age around 13 Gyr, the possibility emerges that the Galaxy experienced a minimum of Star Formation at the end of the halo/bulge formation. This minimum might reflect the time at which the Galaxy started to acquire material to form the disk inside-out.

\section{Introduction}
Disks are quite common structures in the Universe.
According to the classical theory of White \& Rees (1978)
spiral galaxies
are considered the seeds of galaxies assembly due to gas cooling
inside spinning dark matter halos.
Apart from spirals, disks are seen also
in the center of giant ellipticals; damped Lyman $\alpha$ clouds are
commonly believed to be gaseous disks. Finally, S0 galaxies
are expected to be disk galaxies depauperated of their gas.\\
However galaxian disks are fragile structures.
N-body simulations convincingly
show that the merging of two equal mass galaxies is fatal for disks.
They are simply destroyed.
The observational counterpart is represented -just to mention a source-
in the IRAS database, which shows many disk
on the way to be destroyed by major mergers.

\begin{figure*}
\centerline{\psfig{file=mw.ps,width=16cm}}
\caption{A sketch showing the vertical structure of the Galaxy. The solar
vicinity is shown as a cylinder with a radius of 1 kpc.}
\end{figure*}

Since our disk appears almost undamaged, this means that it did not suffer
from strong mergers since its formation, and that its past life was relatively
quiet. Of course we cannot forget that our disk has a warp, and that presently
a dwarf galaxy - Sagittarius - is going to merge with our Galaxy.\\
Just how the past life was quiet can be gauged by the ability of small
galaxies to heat or thicken the disk.
Quinn et al (1993) pointed out that even satellites with masses of order
$5\%-10\%$ of the disk mass can thicken the disk by a factor of two or more
if they dump all of their available orbital kinetic energy into the disk.
Hence the presence of disks with scale heights less than a few hundred
parsecs ({\it thin disks})
implies the absence of even minor encounters over the age of the
{\it thin disk}. Nonetheless
it may be possible for disks formed early in the lives of spirals
to have been heated by satellites forming a thick disk within which a new
{\it thin disk} was allowed to form. If this was the case, an estimate
of the {\it thin disk} age can fix the time at which the last encounter
occured.\\
The acute fragility of disks is a very important constraint on
the evolution of the environment of protogalaxies that develop into current
spirals. Either they were born in isolation (low density environment)
or they managed to remove all potentially disk-disturbing debris early
enough that a normal disk had time to form.
It is clearly very interesting to look from our favorite observational point,
the solar system, at the disk population to try to establish its age.
This is important to constrain the zero point of chemical evolution
models, the relationship between the disk and the other galaxy components,
and to fix a rough chronology for the disk development and origin.

\section{Age indicators}
In the following I shall discuss five different methods
devised to obtain an estimate of the age of the galactic disk,
discussing their feasibility, robustness and limitations.\\
I shall start making a crucial point.
Most methods to infer the age of the galactic disk pretend, or try,
to give an age estimate for the entire disk
(or even for the Galaxy as a whole), by using
age indicators located in the near solar vicinity (see Fig.~1).
Whether the solar neighborhood is really representative of all
the disk is an open question, which does not hold only
for this particular issue - the age of the disk -,
but more generally for determining
the global chemical evolution and the SF history of the galactic disk
looking at nearby indicators (Carraro et al 1998).
At present only old open clusters (although the sample is rather poor)
can be used to derive an estimate for the age of a
significant portion
of the galactic disk.
Moreover all the methods devised to address this topic make us of indicators
located well inside the {\it thin disk}. Therefore I am going to discuss the
age of the galactic {\it thin disk} (Gilmore et al 1989).

\begin{figure*}
\centerline{\psfig{file=hypp.ps,width=12cm}}
\caption{The CMD of nearby stars from Hypparcos. The red envelope of the subgiant region
is used to set a minimum age of the galactic disc.}
\end{figure*}

\subsection{Cool White Dwarfs Luminosity Function}
The idea that the coolest white dwarfs (WD) can be used to determine the
age of the local galactic disk dates back to Schmidt (1959).
His idea was the following: if we find the coolest white dwarf (just looking
at the downturn of their luminosity function)
and measure their cooling time,
we can find the age of the disk just adding to this time the lifetime
of the WD progenitor. Moreover WDs are easy to be modelled, much easier than
Main Sequence (MS) stars (Mestel 1952).
At that time it was not possible to detect the turn-down, and the idea failed.
Moreover some uncertainties in the models arose, in particular the crystallization.
In detail, at decreasing temperature ions start to crystallize, and the energy
necessary to maintain the ions lattice causes the WD to cool to invisibility
very rapidly.\\
More recently Winget et al (1987)
succeeded to find the turn-down, and since then many authors
built up CWDLF (Leggett et al 1998).
In addition theorists now agree that the onset of rapid cooling is
not reached for the dominant
stellar mass in the WD population ($0.6 M_{\odot}$) until well below the luminosity
of the observed shortfall.
The most recent work is from Knox et al (1999). They find 52 CWDs
using common proper motion binaries. Their sample is the only
complete one both in
luminosity and in proper motion. The resulting LF
(see their Figs.~19,20 and 22) shows a clear short-fall at
$log L/L_{\odot}~=~-4.2$,
and predicts a larger number of WDs per unit volume, as compared with
previous determination of the LF.
Knox et al (1999) compared their LF with two sets of stellar models,
by Wood  (1992) and Garcia-Berro et al (1997).
The best fit is obtained using Wood (1992) models, and provides a disk age
of $9.0\pm1.0$ Gyr. Garcia-Berro et al (1997) models do not fit the LF maximum,
and provide a somewhat greater age.

\subsection{The Hypparcos CMD}
The CMD provided by Hipparcos satellite
includes stars within about 150 parsecs from the Sun
(see Fig.~2).
It shows a well defined
MS, a prominent clump of He-burning stars and a subgiant region.
It represents of course a mix of stellar populations with spreads in age and
in metallicity.
The dating features used by Jimenez et al (1998) is the red envelope
of the sub-giant region, and the methods adopted is the isochrone fitting one.
Since the metallicity of the stars is not known - say from
observations - the authors assume that the spread in color of the clump
is a good indicator of a spread in metallicity. This is a rather crude
approximation, as discussed by Girardi (this meeting).
Nevertheless, it is possible to obtain a minimum age for this sample,
assuming that the star populating the lower red envelope
of the subgiant region has the maximum metallicity, since this metallicity
provides the minimum age.
So doing they obtain a minimum age for the disk of $8$ Gyr.

\subsection{Nearby $F$ \& $G$ stars}
Another method - although rather difficult -
is the direct age estimate of single stars, like the 187
stars sample of Edvardsson et al (1993).
To get an age estimate one needs star photometry, spectroscopy
and distance, to put them in the $M_{V}-logT_{e}$ plane. In the case of the
Edvardsson et al sample ages are inferred on the Vandenberg scale (1985).
Removing from the sample the presumed thick disk stars, or stars whose orbital
motions are not that of thin disk stars, a reasonable estimate for the age
of the local galactic disk is $9-11$ Gyr.
Recently the Edvardsson et al sample has been revised by Ng  \& Bertelli (1997),
who re-computed the ages of those stars taking into account new distances from
Hipparcos, correcting for the Lutz-Kelker effect, and putting them
in the Bertelli et al (1994) scale.
At older ages the new ages are slightly older, but
the conclusion on the disk age is roughly the same.

\subsection{The Radio-active Clock}
Butcher (1987)  proposed to derive the age of the Galaxy
by observing the radio-active nuclide $^{232}Th$ in stars of different ages,
and relating the nucleosynthesis timescale to the stellar and galactic evolution.
He considered the evolution of Nd, a stable nuclide
and Th (half life 14 Gyr).
The first point to stress is the extreme
weakness of the spectral features in the
measured stars spectra: in particular the Th line falls in a blend with Co.
The errors related to the derived abundance are around 0.1 dex.
The idea underlying this method is that after its formation,
a stars does not modify its envelope abundances of Th and Nd
but for radio-active decay.
By measuring the ratio of the abundances of these elements,
$[Th/Nd]$ in stars of different age, it is possible
to reconstruct the decay evolution of Th.
The basic assumption is that the growth rate of the two elements
is the same, although Th is a r-process and Nd is partly a r- and
partly a s-process.
So doing, they conclude that no reliable chemical evolution model
can account for this distribution without assuming an age less than
$9$ Gyr. The same result has been obtained by Morell et al (1992)
who made a new abundance analysis on the same sample.\\

Clayton (1988) criticized these results, stressing that although
the precise nature of r- and s- process is not clear enough,
in principle one has to take into account their different  evolution.
In particular assuming that the contribution to the
Nd abundance is about half from r-
and half from s-processes,
he showed that a simple model of chemical evolution
can account for the Butcher distribution
assuming ages greater than 12 Gyr,
and concluding: "{\it An unbiased look at all methods together favours
an age greater than 12 Gyr, although no single method is reliable. I point
out that each nuclear method is still amenable to further improvements,
but they alone will not be able to determine the Galaxy's age. Only a
detailed and specific and correct model for the growth and chemical evolution
of the solar neighbourhood can enable the galactic age to be inferred from
radioactivity}.

\begin{figure*}
\centerline{\psfig{file=be17.ps,width=12cm}}
\caption{The CMD of Berkeley~17 from the V,I
photometry of Phelps (1997). Overimposed is an isochrone
from Girardi et al 1999 for the parameters
listed in the plot.}
\end{figure*}

\subsection{Old Open Clusters}
Old open cluster are well suited to address
many issues concerning our disk (Friel 1995, Carraro et al 1998).
For the present topic, they are in principle
more suitable than the other indicators
to derive a lower limit for the age of the galactic disk.
In fact they are distributed in
a larger portion of the disk.
Good data have been obtained recently for clusters
older than M~67, so it is actually feasible to use this sample to determine
the age of the disk.
However there is still debate on the role of the oldest open cluster.
NGC 6791, often quoted as the oldest cluster (8-9 Gyr,
Carraro et al 1999c), is a rather special object.
Its nature is not completely clear, and somebody is suggesting that it could be
a bulge globular pushed away by the bar, or the core of a dwarf
spheroidal tidally stripped by our Galaxy (see Carraro et al 1999a
for a detailed analysis on this cluster).
Recently another cluster came out to be very old, Berkeley~17
(Fig.~3).
It is actually quite old, with an age around 9 Gyr (Carraro et al 1999b),
although optical photometry is not very good yet, and
should be improved, being this cluster so important.
If Berkeley~17 marks the age of the disk,
its minimum age is around 9 Gyr.\\
However the main drawback of open clusters is that their average lifetime
is of the order of some $10^8$ yr (Grenon 1990), so many old clusters might have
been destroyed.
Therefore the statistics of old open clusters is rather poor,
and in principle they can provide only a lower limit for the age
of the thin disk (Carraro et al 1999c). Anyhow their dating is rather
simple and robust.

\begin{figure*}
\centerline{\psfig{file=end.ps,width=12cm}}
\caption{Summary of disk ages from different methods.}
\end{figure*}

\section{Conclusions}
In the past several reviews were dedicated to the disk age issue. I would like
to remind the reviews from Sandage (1990), van den Bergh (1990)
and Grenon (1990).\\
Sandage at the 1989 Kingston Conference in Canada
concluded that the disk overlaps in age the halo,
whereas van den Bergh, at the same meeting, suggested that the disk
is somewhat younger than the halo. \\
Obviously the question of a possible star formation
delay at the end of the halo assembly depends on the age of the halo,
and on the age of the disk. The age of the halo comes from the mean
globular clusters age ($12-15$ Gyr at that time),
whereas the age of the disk comes from a variety
of methods, going from old open cluster to white dwarfs, from
radioactive decay to stars in the solar neighbourhood.
At that time the oldest open cluster was NGC~6791, with an age around 12 Gyr
according to Sandage, who quoted Janes (1988), and of 7 Gyr according to van den Bergh,
who quoted a preliminary work by Demarque et al (1992).\\
The conclusion of van den Bergh is supported also by Grenon (1990), who
in addition stressed that in the solar vicinity there is a group of
metal rich stars which seem to be older than open clusters, but whose
birthplaces might be inside the bulge.\\

\noindent
Ten years after the situation is not much different.\\
Gratton et al (1997) reported a mean age of the bulk of the halo globulars
around $13$ Gyr, which holds for all the halo clusters. There is indeed
the evidence of a population of very young globulars (Pal~12 for instance),
whose belonging to the halo is controversial.\\
Summarizing all the data I discussed above (see also Fig.~4) a plausible age for
the disk is in the range $8-10$ Gyr. Note that the age scale in the case
of F \& G stars and open clusters is the same as for globulars.\\
This seems to suggest the occurence of a hiatus, or minimun
in the star formation history
of the Galaxy, which might reflect the end of the halo/bulge formation.
Afterwards the Galaxy started to acquire material to form the disk in
an inside-out scenario.\\

\noindent
{\bf Acknowledgements}\\
I thank L. Girardi and M. Grenon for useful discussions, and Francesca Matteucci
for her invitation.

The Age of the Galactic Disk