Contents lists available at ScienceDirect Personality and Individual Differences journal homepage: www.elsevier.com/locate/paid Finding middle ground between intellectual arrogance and intellectual servility: Development and assessment of the limitations-owning intellectual humility scale Megan Haggarda,⁎, Wade C. Rowattb, Joseph C. Lemanb, Benjamin Meagherc, Courtney Mooreb, Thomas Fergusb, Dennis Whitcombd, Heather Battalye, Jason Baehrf, Dan Howard-Snyderd a Francis Marion University, Florence, SC, United States b Baylor University, Waco, TX, United States cHope College, Holland, MI, United States dWestern Washington University, Bellingham, WA, United States eUniversity of Connecticut, Storrs, CT, United States f Loyola Marymount University, Los Angeles, CA, United States A R T I C L E I N F O Keywords: Intellectual humility Scale development Limitations-owning Virtues Psychometrics A B S T R A C T Recent scholarship in intellectual humility (IH) has attempted to provide deeper understanding of the virtue as personality trait and its impact on an individual's thoughts, beliefs, and actions. A limitations-owning perspective of IH focuses on a proper recognition of the impact of intellectual limitations and a motivation to overcome them, placing it as the mean between intellectual arrogance and intellectual servility. We developed the Limitations-Owning Intellectual Humility Scale to assess this conception of IH with related personality constructs. In Studies 1 (n= 386) and 2 (n = 296), principal factor and confirmatory factor analyses revealed a three-factor model – owning one's intellectual limitations, appropriate discomfort with intellectual limitations, and love of learning. Study 3 (n = 322) demonstrated strong test-retest reliability of the measure over 5 months, while Study 4 (n = 612) revealed limitations-owning IH correlated negatively with dogmatism, closed-mindedness, and hubristic pride and positively with openness, assertiveness, authentic pride. It also predicted openness and closed-mindedness over and above education, social desirability, and other measures of IH. The limitations-owning understanding of IH and scale allow for a more nuanced, spectrum interpretation and measurement of the virtue, which directs future study inside and outside of psychology. 1. Introduction In 2014, Laszlo Block, a senior vice president at Google, detailed the personal qualities the company seeks in employees. "What we've seen is that the people who are the most successful here, who we want to hire, will have a fierce position. They'll argue like hell. They'll be zealots about their point of view. But then you say, 'here's a new fact,' and they'll go, 'Oh, well, that changes things; you're right'" (Friedman 2014). Instead of internship experience, high test scores, or even expertise, Block implicated intellectual humility as an important trait for success. Collins (2001) described that a paradoxical combination of strong professional will and humility are found in the best CEOs. Owens and colleagues (2016) also uncovered the important role humility plays among business teams and leaders. Intellectual humility (IH) has also proven to be a beneficial quality in other personal and interpersonal contexts as well. Krumrei-Mancuso (2017) found that IH positively predicted perspective-taking, empathetic concern, gratitude, altruism, and valuing benevolence and universalism. However, until recently, measurement of general humility and IH has lagged (Davis, Hook, Worthington, Van Tongeren, Gartner and Jennings, 2010), in part because of multiple philosophical and psychological perspectives on IH ranging from lacking intellectual arrogance to low concern for intellectual status to one of proper reliance on beliefs (Roberts & Wood 2003; Samuelson et al. 2014). A handful of different IH scales were developed within the past few years, but all focus on a binary interpretation of IH and intellectual arrogance (IA), where IH is primarily defined as a lack of IA (cf. Hill, Laney, & Edwards, 2014; Hoyle, Davisson, Diebels, & Leary 2016; Krumrei-Mancuso & Rouse 2015; Leary et al. 2017). Instead, we propose a conceptualization of IH that involves owning one's intellectual limitations, which lies on a https://doi.org/10.1016/j.paid.2017.12.014 Received 24 August 2017; Received in revised form 4 December 2017; Accepted 8 December 2017 ⁎ Corresponding author at: Department of Psychology, Francis Marion University, PO Box 100547, Florence, SC 29502, United States. E-mail address: mhaggard@fmarion.edu (M. Haggard). Personality and Individual Differences 124 (2018) 184–193 0191-8869/ © 2017 Published by Elsevier Ltd. T spectrum between IA and intellectual servility (IS). 1.1. Current conceptualization and measurement of intellectual humility Within positive psychology and virtue epistemology, there is strong consensus about what humility is not – arrogance (Davis, Worthington, & Hook 2010; Gregg & Mahadevan 2014), narcissism (Bollinger & Hill 2012), self-deprecation (Tangney 2009), low self-esteem (Chancellor & Lyubomirsky 2013; Tangney 2009), or modesty (Exline & Geyer 2004). Similarly, available theory and measures of IH tend to focus on this negative view applied to one's intellect, relying on factors such as lacking overconfidence in one's beliefs, separating ego and intellect, and low concern for intellectual status (Hill, Laney, & Edwards 2014; Krumrei-Mancuso & Rouse 2015; Leary et al. 2017). As research in general humility has emphasized, someone who is only self-deprecating is not necessarily humble. Davis and colleagues (2010) stress that a humble person is accurate in their self-appraisal, not over-confident or under-confident in their abilities. If one is a highly-respected scholar in an academic field, then one should not undersell his or her abilities in that area. However, if one is new to a particular realm of knowledge, one should be receptive to inviting multiple perspectives and altering views if new information arises. To account for the issue of accuracy, the General Intellectual Humility Scale (GIHS; Leary et al. 2017) and Comprehensive Intellectual Humility Scale (CIHS; Krumrei-Mancuso & Rouse 2015) include a willingness to change viewpoints in the face of appropriate evidence or respecting others' views. However, this creates unnecessary overlap between IH and open-mindedness, and centers IH as related only to what one knows or what others know, not an orientation toward knowledge and its attainment overall. Along these lines, there remains a need for a theory and measure of IH that includes both its deficiency, or IA, as well as its excess, which we posit is IS. Just as general humility is not simply thinking less of oneself, IH is not just endlessly questioning each piece of knowledge or thought, nor is it overthrowing long-held and tested beliefs when new or contradictory evidence arises. IH, then, emerges as the mean between the extremes of IA and IS. Within this framework, a person high in IH should be aware of their intellectual limitations and their potential impact and not overwhelmed by their existence, while a person lacking IH could be dismissive of criticism or completely overwhelmed by it. 1.2. The limitations-owning approach to intellectual humility To synthesize and clarify the existing conceptualizations of IH, Whitcomb, Battaly, Baehr, and Howard-Snyder (2015) provided a comprehensive philosophical analysis of the current state of the virtue. The innovative contribution of their work focuses on defining IH as owning one's intellectual limitations while being appropriately attentive to them. That is, not consumed by them, as with IS, but also not willfully ignorant of them, as in IA. The "owning" in question is made up of a suite of affective, motivational, behavioral, and cognitive dispositions toward knowledge. These dispositions, in turn, characteristically bring one to have accurate beliefs about one's intellectual limitations and the outcomes that are due to them, and to feel, act, and be motivated in certain ways in certain circumstances: for instance, to not feel hostile about them, to not lash out in anger about them, and to work to remedy them or perhaps come to peace with them. Excesses in these dispositions result in IS, in which an individual is so preoccupied with their limitations, he or she struggles to do anything about them, whereas deficiencies result in IA, in which an individual fails to recognize their intellectual shortcomings. When these dispositions come in the appropriate (i.e., non-deficient and non-excessive) degree, though, they make for IH. When they are motivated by a love of such goods as truth, knowledge, and understanding, they make for the virtue of IH, as opposed to the non-virtuous trait. This approach to IH is unique in that it captures the connection between IH and a desire/ openness to learn, allows it to be distinguished from intellectual pride, which is the proper owning of and attentiveness to one's intellectual strengths, and relies on a spectrum rather than binary understanding of IH. Whitcomb et al.'s (2015) conceptualization also allows for specific predictions about the kinds of behaviors, motivations, and feelings that an intellectually humble person would demonstrate. According to the limitations-owning conception, open-mindedness (considering alternative ideas) is distinct from IH, but is likely to be correlated with IH. For example, an intellectually humble person should be less likely to pretend to know something, more likely to consider alternative ideas, and less likely to treat intellectual inferiors with disrespect. This marks the first theory to provide testable hypotheses concerning how IH manifests within an individual. 1.3. Present research: developing and validating and limitations-owning IH scale To expand the current conceptualization of IH and empirically examine its connection to specific behaviors, motivations, and emotions, we sought to create a measure founded on the understanding of IH provided by Whitcomb et al. (2015). Based on discussions with a team of psychologists and philosophers, we identified three important factors needed to measure IH as a virtue – owning one's intellectual limitations, love of learning, and appropriate discomfort with one's intellectual limits, which taps into the servility component of limitations-owning. Combined, these three factors tap into the constellation of characteristics that define IH according to a limitations-owning understanding. First, the owning one's limitations factor focuses on the ability of an individual to admit to intellectual limitations, generally acknowledging and accepting that there are gaps in one's knowledge and that they may impact future feelings, thoughts, and behaviors. The appropriate discomfort component taps into the degree of attentiveness to intellectual limitations, especially the emotional experience associated with considering the gaps in one's intellect. This distinguishes between those who are attentive to their limitations but not preoccupied by limitations from those who are both attentive and preoccupied by them, miring themselves in IS. This inclusion is unique to our scale, as the others do not address outcomes related to an excess of IH. Lastly, the love of learning component captures the desire to gain more knowledge to bring about more understanding and helps to distinguish those who are virtuously intellectual humble, as reflected in Whitcomb et al.' (2015) argument that motivation to be intellectual humble must be for gaining epistemic goods, such as knowledge, truth, and understanding. It is similar in definition and function to Seligman and Peterson and Seligman (2004) love of learning virtue in their Values in Action (VIA) inventory, but the L-OIHS items are designed to capture a broader orientation to seek out new information compared to the VIA items. They overlap in their more abstract concerns, such as being a life-long learner, seeking truth, and finding out new information. However, the clearest differences are that the L-OIHS love of learning factor is not as concerned with the sources consulted or the ways in which knowledge is gained. 2. Study 1 To begin, 64 potential scale items were written by a team of philosophers and social-personality psychologists to match the criteria of IH as the three factors mentioned previously (available from first author). Thirty-seven of these items assessed owning one's intellectual limitations, 12 items measured love of learning, and 15 items measured appropriate discomfort with one's intellectual limits. M. Haggard et al. Personality and Individual Differences 124 (2018) 184–193 185 2.1. Method 2.1.1. Participants and procedure Three hundred eighty-six adults from the United States ranging in age from 18 to 75 (209 female;Mage = 36.59, SD = 12.18) completed a series of questionnaires on Qualtrics on Amazon's Mechanical Turk website. Amazon's MTurk has been shown to be a reliable source of quality data (Buhrmester, Kwang, & Gosling 2011). Each participant was paid $1.00 for completing a 20-minute survey, which is in line with best practices (Mason & Suri 2012). The sample was predominately Caucasian (80.9%), but also included African-Americans (7%), Asians/ Pacific Islanders (5.2%), Hispanics (5.2%), Native Americans (0.8%), and Other (0.5%). Most participants had at least a high school education (98.5%). Participants were asked to respond to each item using a 9-point Likert scale, ranging from 1 (Strongly Disagree) to 9 (Strongly Agree) as part of a larger study. 2.2. Results and discussion We conducted a principal factor analysis of the 64 candidate scale items, with oblique rotation (promax; Kappa = 4). Twenty-five of the items loaded on the first three factors, which accounted for 42% of the variance. To reduce redundancy, and avoid creating a conceptually bloated scale (with multiple items using the same words over and over), we retained the top four items with the highest factor loadings that did not cross load onto other factors (cut-off of 0.30). This reduced our original 64-item scale to a 12-item scale (see Appendix A), with four items measuring love of learning (0.74–0.86), four items measuring lacking discomfort with limitations (0.66–0.85), and four items measuring owning intellectual limitations (0.48–67). Next, we conducted principal axis factoring of the 12-item scale and found the same three factors, which accounted for 64% of the variance (see Table 1). The overall reliability of the 12-item scale was very good (Cronbach's α= 0.86), with moderate to high reliabilities for each of the factors – love of learning (α= 0.81), appropriate discomfort with limitations (α= 0.84), and owning intellectual limitations (α= 0.77). As expected, there were moderate, positive correlations between the factors (ranging from 0.31 to 0.54). Taken together, these 12 items create a consistent measure of the three theorized factors of the limitations-owning perspective of IH. 3. Study 2 Another issue inherent in the measurement of IH is understanding its model structure. While Krumrei-Mancuso and Rouse (2015) found better fit using a four-factor model, they opted to confirm a higherorder factor model instead. Similarly, we compare a theoretically supported three-factor model with a three-factor with higher order factor to understand the overall model fit and guide further research. 3.1. Method 3.1.1. Participants and methods Two hundred ninety-six adults between the ages of 19 and 74 (154 female; Mage = 38.90, SD = 12.48) completed a series of questionnaires on Qualtrics on Amazon's Mechanical Turk website in exchange for $2.00 Amazon.com credit. The sample was predominately Caucasian (77.1%), but also included African-Americans (8.5%), Asians/Pacific Islanders (6.1%), Hispanics (6.5%), Native Americans (1%), and Other (0.7%). Most participants had at least a high school education (97.6%). Participants completed the 12-item version of the LimitationsOwning Intellectual Humility Scale (L-OIHS) developed in Study 1 using the same rating scale. 3.2. Results and discussion We evaluated two models using structural equation modeling (maximum likelihood estimation). First, we tested the hypothesized three-factor model, followed by a model adding a higher-order factor to represent the common variance between them. As shown in Fig. 1, the three-factor model without the higher-order factor had acceptable fit (Byrne, 2010; Hu & Bentler, 2002): χ2 = 158.94 (df= 61; p < 0.0001), CFI = 0.946, SRMR = 0.05, and RMSEA = 0.074. The three-factor higher-order model had slightly worse fit: χ2 = 180.24 (df= 61; p < 0.0001), CFI = 0.935, SRMR= 0.058, and RMSEA = 0.080. The 12-item IH measure (α = 0.87) and each factor (0.80–0.87) were internally consistent. Taken together, studies 1 and 2 indicate the L-OIHS taps into three facets and does not involve a higher-order factor. This confirms the theoretical structure of the L-OIHS as the unique combination of owning one's intellectual limitations, appropriate discomfort with limitations, and love of learning. In addition, the reliability estimates from this new sample were comparable to those obtained in the first study. In order to assess internal reliability and confirm the structure of the LOIHS, we next tested the test-retest reliability of the measure. 4. Study 3 Study 3 examines the internal consistency of the L-OIHS over a fivemonth time period using an extension of Amazon's Mechanical Turk that allows for contact of participants who have completed previous studies (see Litman, Robinson, & Abberbock 2016, for overview). 4.1. Method 4.1.1. Participants and methods Three-hundred and twenty-two U.S. adults aged 21 to 70 years (156 female; Mage = 36.44, SD = 10.66) from the sample featured in Study 4 were contacted approximately five months after and consented to Table 1 Factor pattern of principal-axis factor analysis of items with promax rotation (N = 386). Items Factor 1 Factor 2 Factor 3 When I don't understand something, I try hard to figure it out. 0.860 0.054 −0.069 I love learning. 0.837 0.073 −0.100 If I don't understand something, I try to get clear about what exactly is confusing to me. 0.787 0.082 0.090 I care about truth. 0.743 −0.053 −0.098 When I think about the limitations of what I know, I feel uncomfortable.⁎ −0.065 0.850 −0.152 I focus on my intellectual weaknesses too much.⁎ 0.157 0.823 −0.298 I tend to get defensive about my intellectual limitations and weaknesses.⁎ −0.092 0.682 0.212 When I know that I have an intellectual weakness in one area, I tend to doubt my intellectual abilities in other areas as well.⁎ 0.004 0.663 0.013 When someone points out a mistake in my thinking, I am quick to admit that I was wrong. 0.238 0.032 0.671 I am quick to acknowledge my intellectual limitations. 0.101 −0.086 0.646 I have a hard time admitting when one of my beliefs is mistaken.⁎ 0.041 0.301 0.608 I feel comfortable admitting my intellectual limitations. 0.300 0.202 0.480 M. Haggard et al. Personality and Individual Differences 124 (2018) 184–193 186 completed a second survey on Qualtrics through Amazon's Mechanical Turk in exchange for $2.00, which included the L-OIHS. The sample was predominately Caucasian (77.3%), but also included participants who identified as African American (7.8%), Asian/Pacific Islander (9.2%), Hispanic (3.5%), Native American (1.4%), and Mixed Race (0.7%). Nearly all of participants had a high school education (98.2%). Participants responded using a 9-point Likert scale that featured only endpoint descriptors of Strongly Disagree and Strongly Agree. 4.2. Results and discussion The test-retest correlation for the L-OIHS was 0.75 after five months, demonstrating extensive reliability according to Robinson, Shaver, and Wrightsman (1991). No other current measure of IH has demonstrated reliability over such a long time period, as others have been tested between 4 and 8 weeks. In addition, the test-retest correlations were strong for all factors – love of learning (0.67), appropriate discomfort with limitations (0.66), and owns limitations (0.62). All coefficients were significant at the p < 0.001 level. Next, we sought to compare the limitations-owning IH scale with existing measures of IH and related constructs as outlined by Whitcomb et al. (2015). Study 4 is also important as it is the first study, to our knowledge, that compares multiple self-report measures of IH. 5. Study 4 Study 4 further tests the L-OIHS by examining its relationships with personality and cognition-related variables. Whitcomb et al. (2015) outline expected correlations between L-OIH, cognitions, and behaviors in their work, specifically that increased IH is connected to increased cognitive reflection and reduced likelihood of displaying intellectual dominance. In addition, the L-OIHS was examined in comparison with other recently developed IH self-report scales, including the GIHS (Leary et al. 2017), CIHS (Krumrei-Mancuso & Rouse 2015), and Biola IH Scale (Hill et al. 2014). 5.1. Method 5.1.1. Participants and methods Six-hundred and twelve U.S. adults between the ages of 18 to 74 (285 female; Mage = 34.73, SD = 10.18) completed a large survey on Qualtrics through Amazon's Mechanical Turk in exchange for $4.00. The sample was predominately Caucasian (77%), but also included participants who identified as African American (9.2%), Asian/Pacific Islander (7.5%), Hispanic (4.1%), Native American (0.8%), and Mixed Race (0.5%). Nearly all of participants had a high school education (99.2%). A post-hoc power analysis revealed that adequate power (β = 0.80) was achieved with this sample size for detecting a small correlation (r = 0.11). 5.1.2. Measures Limitations-Owning Intellectual Humility Scale. The 12-item LOIHS was used to assess IH and determine connections with personality and cognition variables. Items were rated on a 9-point scale that included only the endpoint descriptors of Strongly Disagree and Strongly Agree. General Intellectual Humility Scale. Participants completed the 6item GIHS (Leary et al. 2017), which has shown to be a reliable measure of a single-factor intellectual humility (αs = 0.73–0.82). Responses were recorded using a 5-point Likert scale, ranging from 1 (not at all like me) to 5 (very much like me). Comprehensive Intellectual Humility Scale. Participants Fig. 1. Model Fit for confirmatory factor analysis (N = 298). M. Haggard et al. Personality and Individual Differences 124 (2018) 184–193 187 completed the CIHS (Krumrei-Mancuso & Rouse 2015). The CIHS includes four factors – independence of intellect and ego (5 items), openness to revising one's viewpoint (5 items), respect for others' viewpoints (6 items), and lack of intellectual overconfidence (6 items) – all measured using a 5-point Likert scale ranging from strongly disagree to strongly agree. Biola Intellectual Humility Scale. The 17-item Biola IH Scale (Hill et al. 2014) consists of three subscales – perspective-taking (6 items), low concern for intellectual status (7 items), and low intellectual defensiveness (4 items). Participants responded using a 5-point Likert scale ranging from Strongly Disagree to Strongly Agree. Big Five Personality Inventory. In order to assess the connection between theoretically related personality constructs, participants completed the 44-item version of the Big Five Personality Inventory (John & Srivastava 1999). All facets of the BFI were measured using a 5-point Likert scale, ranging from 1 (Disagree Strongly) to 5 (Agree Strongly). Narcissism. Narcissists have an inflated, grandiose sense of self and often appear to be arrogant, conceited, and entitled. Narcissism is often considered to be the antithesis of any type of humility. Participants completed the 40-item Narcissistic Personality Inventory (NPI: Raskin & Hall 1981). The NPI is an ipsative questionnaire, forcing individuals to choose between items such as "I try not to be a show off" and "I will usually show off if I get the chance". The number of narcissism items endorsed is summed to create a total score and has demonstrated good reliability (Cronbach's α= 0.74–0.90: Raskin & Terry 1988). Assertiveness. While an intellectually humble individual may not show intellectual dominance, this does not necessarily mean they lack assertiveness, the ability to defend one's rights without imposing on the rights of others (Jenerette & Dixon 2010). Participants completed the Short Form of the Simple Rathus Assertiveness Scale (Jenerette & Dixon 2010), which is highly positively correlated with the original version (r = 0.98, p < 0.01) and retains reliability (α= 0.81) with 19 items. Responses were coded using a 6-point Likert scale ranging from 1 (very unlike me) to 6 (very much like me). Pride Scale. Participants completed the 14-item Pride Scale (Tracy & Robins 2007), which is divided into Authentic (or proper) and Hubristic (or distorted) types. Authentic pride (α= 0.88) has been shown to be positively associated with agreeableness, conscientiousness, and self-esteem, whereas Hubristic pride (α= 0.90) has negative associations with these variables. Using a 5-point Likert scale ranging from 1 (not at all) to 5 (extremely), participants rated their agreement to statements such as, "I generally feel fulfilled" or "I generally feel smug". This will be the first time the relationship between pride and IH will be assessed empirically. Desirable Responding. Due to the fact that self-report measures, especially those concerning humility, are prone to self-enhancement (Davis, Hook, & Worthington, 2010), participants completed the SelfDeceptive Enhancement subscale of the Balanced Inventory of Desirable Responding (BIDR; Paulhus 1991). Typical reliabilities range from 0.67–0.77. Sample items include, "I am a completely rational person" and "I never take things that don't belong to me" and was measured using a 7-point Likert scale ranging from 1 (not true) to 7 (very true). Dogmatism. Dogmatism measures the extent to which individuals hold on to beliefs that may be unjustified. This is similar to several conceptions of IA, and thus should demonstrate a negative relationship with IH. Using the 20-item version of the measure developed by Altemeyer (1996, 2002), participants rated questions such as "My opinions are right and will stand the test of time" and "The people who disagree with me may well turn out to be right" (reverse-scored) using a 9-point Likert scale. The measure has demonstrated excellent reliability (Cronbach's α = 0.90–0.93). Closed-Mindedness. In order to assess connections with closedminded, limited thinking, participants completed the closed-mindedness subscale of the Need for Cognitive Closure Scale (Kruglanski, Webster, & Klem 1993). The subscale consists of eight items with responses ranging from strongly disagree to strongly agree on a 6-point Likert scale. Cognitive Reflection Task. In order to assess the relationship between IH and slower, more reflective thinking, the Cognitive Reflection Task (CRT: Toplak, West, & Stanovich 2011) was used. The CRT serves as a unique measure of miserly cognition performance over and above measure of cognitive ability, such as general intelligence, executive functioning, and thinking dispositions, including open-mindedness. The CRT consists of 3 items, questions that all have a quick, intuitive, and incorrect answer that must be overridden in favor of slower thinking to arrive at the correct answer. For example, one question reads, "A bat and ball cost $1.10 in total. The bat costs a dollar more than the ball. How much does the ball cost?". In this case, participants must override the initial inclination to answer 10 cents, which would be incorrect, and instead consider the problem long enough to arrive at the correct answer of 5 cents. While all of the questions are simple and do not require advanced mathematics, it is not unusual for more than half of participants to not correctly answer any (Toplak et al. 2011). Scores ranged from 0 (all incorrect) to 3 (all correct). Social Vigilantism Scale. In order to assess the relationship between IH and intellectual dominance, participants completed the Social Vigilantism Scale (SVS: Saucier & Webster 2010). The SVS measures the tendency to see one's own beliefs as completely correct and the need to correct others' inferior beliefs. The SVS contains 14 items (α = 0.81), including "I need to win any argument about how people should live their lives" and "There are a lot of ignorant people in society", measured on a 9-point Likert scale, ranging from 1 (disagree very strongly) to 9 (agree very strongly). 5.2. Results and discussion Comparison with other IH Measures. As expected, the L-OIHS demonstrated moderate positive correlations with the CIHS (r = 0.52, p < 0.001) and GIHS (r = 0.43, p < 0.001). When examining the relationships between measures and the three subscales of the L-OIHS, differences emerged, particularly with the GIHS. While each of the subscales were positively correlated with the total CIHS (Love of Learning r = 0.38, p < 0.001; Limitations Owning r = 0.28, p < 0.001; Appropriate Discomfort r = 0.49, p < 0.001) and the Biola IH Scale (Love of Learning r = 0.30, p < 0.001; Limitations Owning r = 0.27, p < 0.001; Appropriate Discomfort r = 0.47, p < 0.001), the GIHS correlated positively with Love of Learning (r = 0.50, p < 0.001) and Limitations Owning (r = 0.46, p < 0.001) subscales, but not the Appropriate Discomfort subscale (r = 0.08, p = 0.069). In addition, a second-order factor analysis was conducted to determine if all of the newly created IH measures, including the L-OIHS, CIHS, GIHS, and Biola IH (Hill et al. 2014) loaded on to a single factor. Using principal axis factoring with an oblique rotation, as we expected at least some inter-correlation between the variables (promax, Kappa = 4), all measures loaded on to one factor that explained 67% of the variance (see Table 2 for loadings). Correlations with Personality Variables. The L-OIHS scale demonstrated small correlations with Extraversion (r = 0.21, p < 0.001) and Openness (r= 0.36, p < 0.001) and moderate correlations with Agreeableness (r = 0.40, p < 0.001), Conscientiousness (r = 0.49, Table 2 Factor pattern of principal-axis factor analysis of IH measures with promax rotation (N = 612). Measures Factor 1 Comprehensive intellectual humility scale (CIHS; Krumrei-Mancuso & Rouse 2015) 0.953 Biola intellectual humility scale (Hill et al. 2014) 0.760 General intellectual humility scale (GIHS; Leary et al. 2017) 0.697 Limitations-owning intellectual humility scale (L-OIHS) 0.587 M. Haggard et al. Personality and Individual Differences 124 (2018) 184–193 188 p < 0.001), and Neuroticism (r = −0.49, p < 0.001). There was no correlation with NPI (r= 0.05, p = 0.26) and a moderate correlation with desirable responding – self-deceptive enhancement (r = 0.49, p < 0.001). Correlations with Theoretically Related Constructs. The L-OIHS scale was also negatively correlated with measures of restrictive cognitions or beliefs, including Dogmatism (r = −0.23, p < 0.001) and Closed-Mindedness (r =−0.48, p < 0.001). There was no relationship between L-OIHS and Cognitive Reflection, (r = 0.07, non-significant), although there were small positive correlations with the Love of Learning and Appropriate Discomfort with Limitations subscales (see Table 3). Assertiveness was moderately positively correlated with LOIHS, (r = 0.43, p < 0.001), but no other IH scale was correlated. Measures of Authentic and Hubristic pride also correlated with L-OIHS; a moderate positive correlation with Authentic pride (r = 0.41, p < 0.001) and a small negative correlation with Hubristic pride (r = −0.26, p < 0.001). Interestingly, while CIHS (r= −0.42, p < 0.001), GIHS (r =−0.17, p < 0.001), and Biola IH (r= −0.32, p < 0.001) all negatively correlated with Hubristic Pride, none demonstrated relationships with Authentic Pride (rs= −0.04–0.05, nonsignificant). See Table 4 for all correlations. Correlations with Theoretically Unrelated Constructs. The LOIHS demonstrated no relationships with general religiousness (r = −0.03), social vigilantism (r = −0.01), participant age (r = −0.02), or participant sex (r = 0.09). See Table 5 for all correlations. Hierarchical Regressions. Incremental validity of the L-OIHS was assessed using three hierarchical regressions, shown in Tables 6 and 7 in three panels. Panel A demonstrates that L-OIHS (Step 3) predicted variance in Openness to Experience over and above years of education and social desirability – self-deceptive enhancement (Step 1, R2 = 0.07, F(2, 580) = 22.89, p = 0.001) as well as the CIHS, GIHS, and Biola IH Scales (Step 2, R2 = 0.105, F(3, 577) = 24.46, p = 0.001). The L-OIHS accounted for 2% additional variance above education, social desirability, and the other IH scales in Openness to Experience (Step 3, R2 = 0.020, F(1, 576) = 13.88, p= 0.001). Panel B shows L-OIHS predicting 0.6% additional variance in Closed-Mindedness, R2 = 0.006, F(1, 576) = 7.80, p= 0.005, beyond education and social desirability (Step 1, R2 = 0.048, F(2, 580) = 14.70, p = 0.001) and the IH scales (Step 2, R2 = 0.467, F(3, 577) = 186.42, p= 0.001). However, Panel C shows that the L-OIHS did not predict additional variance in Dogmatism, R2 = 0.002, F(1, 576) = 2.15, p = 0.143, beyond education and social desirability (Step 1, R2 = 0.059, F(2, 580) = 18.17, p = 0.001), and other IH scales (Step 2, R2 = 0.492, F(3, 577) = 211.59, p = 0.001). Overall, the recently developed IH measures tested all strongly loaded on to a single factor, which indicates that all are tapping in to the same construct of IH (see Table 2). Additionally, the L-OIHS demonstrated unique correlations with theoretically related constructs compared to other IH measures, notably positive associations with assertiveness and authentic pride. Though it uniquely predicted variance in Openness to Experience and Closed-Mindedness over and above education, social desirability, and three other IH measures, it did not predict additional variance in Dogmatism. Table 3 Bivariate correlations and reliabilities, Study 4 (N = 612). 1 2 3 4 5 6 7 α 1. Love of learning – 0.81 2. Appropriate discomfort 0.24⁎⁎ – 0.83 3. Owns limits 0.39⁎⁎ 0.24⁎⁎ – 0.78 4. L-OIHS 0.68⁎⁎ 0.76⁎⁎ 0.73⁎⁎ – 0.85 5. GIHS 0.50⁎⁎ 0.07 0.46⁎⁎ 0.35⁎⁎ – 0.97 6. CIHS 0.38⁎⁎ 0.29⁎⁎ 0.49⁎⁎ 0.52⁎⁎ 0.67⁎⁎ – 0.90 7. IH – biola 0.30⁎⁎ 0.27⁎⁎ 0.47⁎⁎ 0.48⁎⁎ 0.48⁎⁎ 0.74⁎⁎ – 0.86 8. Openness 0.43⁎⁎ 0.08 0.27⁎⁎ 0.36⁎⁎ 0.21⁎⁎ 0.06 −0.07 0.87 9. Conscientiousness 0.38⁎⁎ 0.44⁎⁎ 0.23⁎⁎ 0.49⁎⁎ 0.13⁎⁎ 0.20⁎⁎ 0.21⁎⁎ 0.88 10. Extraversion 0.19 0.29⁎⁎ 0.09 0.21⁎⁎ 0.04 −0.02 −0.14⁎⁎ 0.91 11. Agreeableness 0.33⁎⁎ 0.31⁎⁎ 0.31⁎⁎ 0.40⁎⁎ 0.25⁎⁎ 0.34⁎⁎ 0.36⁎⁎ 0.84 12. Neuroticism −0.24⁎⁎ −0.54⁎⁎ −0.19⁎⁎ −0.49⁎⁎ −0.08 −0.19⁎⁎ −0.15⁎⁎ 0.91 13. NPI 0.05 0.14⁎⁎ −0.11⁎⁎ 0.05 −0.17⁎⁎ −0.31⁎⁎ −0.45⁎⁎ 0.93 14. Cognitive reflection 0.09⁎ 0.09⁎ −0.002 0.07 0.11⁎⁎ 0.15⁎⁎ 0.10⁎ 0.76 15. BIDR self-deception 0.27⁎⁎ 0.50⁎⁎ 0.24⁎⁎ 0.49⁎⁎ −0.03 0.08⁎⁎ 0.19⁎⁎ 0.79 ⁎Note. GIHS = General Intellectual Humility Scale; CIHS = Comprehensive Intellectual Humility Scale; L-OIHS Limitations-Owning Intellectual Humility Scale. NPI = Narcissistic Personality Inventory ⁎p < 0.05, ⁎⁎p < 0.01. Table 4 Correlations between measures of intellectual humility and theoretically related constructs, study 4 (n = 612). 1 2 3 4 5 6 7 8 9 10 11 α 1. Love of learning – 2. Appropriate discomfort 0.24⁎⁎ – 3. Owns limitations 0.39⁎⁎ 0.24⁎⁎ – 4. L-OIHS 0.68⁎⁎ 0.76⁎⁎ 0.73⁎⁎ – 5. GIHS 0.50⁎⁎ 0.07 0.46⁎⁎ 0.35⁎⁎ – 6. CIHS 0.38⁎⁎ 0.29⁎⁎ 0.49⁎⁎ 0.52⁎⁎ 0.67⁎⁎ – 7. Biola IH 0.30⁎⁎ 0.27⁎⁎ 0.47⁎⁎ 0.48⁎⁎ 0.48⁎⁎ 0.74⁎⁎ – 8. Dogmatism −0.24⁎⁎ −0.06 −0.24⁎⁎ −0.23⁎⁎ −0.63⁎⁎ −0.46⁎⁎ −0.65⁎⁎ – 0.93 9. Closed-mindedness −0.45⁎⁎ −0.22⁎⁎ −0.42⁎⁎ −0.48⁎ −0.06 −0.43⁎⁎ −0.61⁎⁎ 0.47⁎⁎ – 0.78 10. Assertiveness 0.27⁎⁎ 0.49⁎⁎ 0.12⁎⁎ 0.43⁎⁎ 0.06 −0.05 0.06 0.09⁎ −0.21⁎⁎ – 0.89 11. Authentic pride 0.26⁎⁎ 0.41⁎⁎ 0.18⁎⁎ 0.41⁎⁎ 0.05 −0.04 0.05 0.11⁎ −0.22⁎ 0.56⁎⁎ – 0.93 12. Hubristic pride −0.17⁎⁎ −0.18⁎⁎ −0.22⁎⁎ −0.26⁎⁎ −0.17⁎ −0.42⁎⁎ −0.32⁎⁎ 0.15⁎ −0.10 0.10⁎ 0.10⁎ 0.92 Note. GIHS = General Intellectual Humility Scale; CIHS = Comprehensive Intellectual Humility Scale; L-OIHS Limitations-Owning Intellectual Humility Scale. ⁎ p < 0.05. ⁎⁎ p < 0.01. M. Haggard et al. Personality and Individual Differences 124 (2018) 184–193 189 These results confirm the subtle differences between different measures of IH, which is understandable given their theoretical origins. Of primary interest to the limitations-owning understanding, the positive associations with openness to experience, agreeableness, conscientiousness, assertiveness, and authentic pride indicate that higher levels of L-OIHS are connected to a person who may be open and agreeable, but will also stand up for their beliefs and take pride in them, avoiding IS. The negative correlations between L-OIHS and reliance on restricted/shallow thinking styles (i.e., dogmatism, closed-mindedness, and hubristic pride) also show that these individuals are also more open-minded cognitively and about their own accomplishments. However, the connections (or lack thereof) between L-OIHS and Table 5 Correlations between intellectual humility and theoretically unrelated constructs, study 4 (n = 612). 1 2 3 4 5 6 7 8 9 10 α 1. Love of learning – 2. Appropriate discomfort 0.24⁎⁎ – 3. Owns limitations 0.39⁎⁎ 0.24⁎⁎ – 4. L-OIHS 0.68⁎⁎ 0.76⁎⁎ 0.73⁎⁎ – 5. GIHS 0.50⁎⁎ 0.07 0.46⁎⁎ 0.35⁎⁎ – 6. Biola IH 0.38⁎⁎ 0.29⁎⁎ 0.49⁎⁎ 0.52⁎⁎ 0.67⁎⁎ – 7. CIHS 0.30⁎⁎ 0.27⁎⁎ 0.47⁎⁎ 0.48⁎⁎ 0.48⁎⁎ 0.74⁎⁎ – 8. General religiousness −0.01 0.01 −0.05 −0.03 −0.20⁎⁎ −0.09⁎ −0.15⁎⁎ – 0.89 9. Social vigilantism 0.18⁎⁎ −0.07 −0.09⁎ −0.01 −0.06 −0.45⁎⁎ −0.35⁎⁎ 0.09⁎ – 0.88 10. Age 0.08⁎ 0.19⁎⁎ 0.05 −0.02 −0.07 −0.03 −0.05 0.03 0.00 – – 11. Sex (1 = male; 2 = female) 0.10⁎ −0.01 0.12⁎⁎ 0.09 0.01 0.10⁎ 0.06 0.17⁎ −0.20⁎ −0.01 – Note. GIHS = General Intellectual Humility Scale; CIHS = Comprehensive Intellectual Humility Scale; L-OIHS Limitations-Owning Intellectual Humility Scale. ⁎ p < 0.05. ⁎⁎ p < 0.01. Table 6 Hierarchical regression models demonstrating incremental validity.⁎ Panel A: L-OIHS predicting openness to experience beyond education, self-deceptive enhancement, and intellectual humility scales (n = 583) Openness to experience B (SE) Β ΔR2 Tolerance VIF Step 1 0.07 Education 0.01 (0.06) 0.009 0.99 1.01 Social Desirability 0.12 (0.02) 0.27⁎⁎⁎ 0.99 1.01 Step 2 0.105⁎⁎⁎ Education −0.01 (0.01) −0.05 0.97 1.04 Social Desirability 0.19 (0.17) 0.30⁎⁎⁎ 0.98 1.02 GIHS 0.14 (0.02) 0.39⁎⁎⁎ 0.52 1.92 CIHS −0.03 (0.04) −0.04 0.31 3.16 Biola IH −0.15 (0.04) −0.24⁎⁎⁎ 0.45 2.24 Step 3 0.020⁎⁎⁎ Education −0.01 (0.01) −0.05 0.96 1.04 Social desirability 0.09 (0.02) 0.21⁎⁎⁎ 0.68 1.47 GIHS 0.12 (0.02) 0.34⁎⁎⁎ 0.50 2.01 CIHS −0.05 (0.04) −0.08 0.31 3.25 Biola IH −0.17 (0.04) −0.27⁎⁎⁎ 0.43 2.30 L-OIHS 0.06 (0.02) 0.20⁎⁎⁎ 0.48 2.06 Total R2 0.197 Panel B: L-OIHS predicting closed-mindedness beyond education, self-deceptive enhancement, and intellectual humility scales (n = 583) Closed-Mindedness B (SE) Β ΔR2 Tolerance VIF Step 1 0.048 Education −0.04 (0.01) −0.11⁎⁎ 0.99 1.01 Social Desirability −0.19 (0.04) −0.19⁎⁎⁎ 0.99 1.01 Step 2 0.467⁎⁎⁎ Education −0.01 (0.01) −0.05 0.97 1.04 Social Desirability −0.18 (0.03) −0.19⁎⁎⁎ 0.98 1.02 GIHS −0.38 (0.03) −0.48⁎⁎⁎ 0.52 1.92 CIHS −0.42 (0.07) −0.30⁎⁎⁎ 0.32 3.15 Biola IH 0.06 (0.06) 0.04 0.45 2.24 Step 3 0.006⁎⁎⁎ Education −0.01 (0.01) −0.04 0.96 1.04 Social desirability −0.13 (0.03) −0.13⁎⁎⁎ 0.68 1.47 GIHS −0.36 (0.03) −0.45⁎⁎⁎ 0.50 2.01 CIHS −0.39 (0.07) −0.27⁎⁎⁎ 0.31 3.25 Biola IH 0.08 (0.06) 0.06 0.43 2.30 L-OIHS −0.07 (0.03) −0.12⁎⁎⁎ 0.49 2.06 Total R2 0.521 Note. GIHS = General Intellectual Humility Scale; CIHS = Comprehensive Intellectual Humility Scale; L-OIHS Limitations-Owning Intellectual Humility Scale. ⁎ p < 0.05. ⁎⁎ p < 0.01. ⁎⁎⁎ p < 0.001. M. Haggard et al. Personality and Individual Differences 124 (2018) 184–193 190 extraversion, narcissism, and cognitive reflection were unexpected and require further exploration. 6. General discussion Recent developments in the theoretical understanding of IH have prompted new research into its potential benefits, yet have lagged due to a lack of consistent measurement and theoretical grounding. The present work is the first to attempt to develop a concise self-report measure of IH using a limitations-owning perspective (Whitcomb et al. 2015), which conceptualizes true, virtuous IH as owning one's intellectual limitations in combination with a motivation to continue learning and appropriate discomfort with those limits, the mean between IA and IS. Across two diverse community samples, exploratory and confirmatory factor analyses revealed a three factor, 12-item scale of LOIHS. The scale was also found to be psychometrically robust in terms of convergent, discriminant, and incremental validity, as well as extensive test-retest reliability over a five-month period. It is important to note that the hierarchical regressions conducted were extremely conservative, given that the L-OIHS was competing with three other measures of IH. Even with these restrictions, L-OIHS predicted additional variance in Openness to Experience and Closed-Mindedness. Each of the factors – owning intellectual limitations, appropriate discomfort with intellectual limitations, and love of learning – uniquely contribute to the overall understanding, where lacking in any of these aspects may signal false or non-virtuous IH. Self-report measures of all types of humility can be prone to self-inflation, but often are not equipped to distinguish between under-inflation of weaknesses by narcissists or over-inflation of weaknesses by servile individuals (Davis et al., 2010). L-OIHS is more protected against these issues, as the limitations-owning perspective places IH as the mean between two extremes, acknowledging that a truly intellectually humble individual will not only own their intellectual limitations, but also not be preoccupied with them and pursue further understanding and knowledge (Whitcomb et al. 2015). Indeed, our spectrum interpretation, as opposed to others' binary interpretation, may account for why the L-OIHS had the lowest factor loading among the developed IH measures. By including IS as an endpoint, IH can be interpreted as the mean between it and IA, which adds needed complexity to its measurement. Individuals who are too subservient to their intellectual limitations may mirror those who are helplessly humble about their abilities (Bollinger & Hill 2012). Instead of pursuing continued learning and truth, they simply bemoan that they will never be as smart as Einstein and call it quits. Clearly, as Tangney (2009) pointed out, this is not true, virtuous IH. Therefore, it is important to recognize that IS may mar IH just as much as IA. The CIHS (Krumrei-Mancuso & Rouse 2015), GIHS (Leary et al. 2017), and Biola IH Scale (Hill et al. 2014) all tap into key aspects related to the binary explanation of IH, including openness to revising one's viewpoints, recognition that one's beliefs may be fallible, and lack of intellectual over-confidence. However, they neglect the impact that IS may have, tenuously overlap with the virtue of open-mindedness, and do not account for proper motivation in overcoming one's intellectual limitations. Our inclusion of the love of learning component may also help to confirm a strong unity amongst intellectual virtues (Wilson 2017), which is suggested by Krumrei-Macuso's work detailing the interconnectedness between IH and other virtues such as gratitude, altruism, and empathetic concern (2017). However, it is also important to note that while our measure may provide a distinct and needed perspective on IH, there were also some unexpected relationships with personality constructs. First, L-OIHS correlated highly with self-deceptive enhancement factor of social desirability. This denotes that individuals may be prone to positively inflate their responses, which could be altered using different endpoints (i.e., very much like me to not at all like me) or different phrasing (i.e., asking how often do respondents feel this way). Meagher, Leman, Bias, Latendresse, and Rowatt (2015) also found positive associations between IH and self-enhancement. Even with this limitation, the L-OIHS still predicted Openness to Experience and Closed-Mindedness over and above social desirability, and this is the first study to examine measures of IH with the BIDR, a more rigorous scale of desirable responding. In addition, while the correlation may be strong, 86.6% of participants had scores at or below the traditional scoring schema's mean (10 out of 20 items endorsed at 6 or higher), which may also be over-inflated (see Vispoel & Kim 2014, for review). Second, there was a strong negative correlation between L-OIHS and neuroticism, which appears to be due primarily to the appropriate discomfort with limitations factor, which is not a factor in the other measures. While unexpected, this suggests that individuals who are characteristically emotionally unstable may be less Table 7 Hierarchical regression models demonstrating incremental validity. Panel C: L-OIHS predicting dogmatism beyond education, self-deceptive enhancement, and intellectual humility scales (n= 583) Dogmatism B (SE) β ΔR2 Tolerance VIF Step 1 0.242 Education −0.03 (0.01) −0.10⁎⁎⁎ 0.99 1.04 Social desirability 0.19 (0.04) 0.21⁎⁎⁎ 0.99 1.47 Step 2 0.492⁎⁎⁎ Education −0.01 (0.01) −0.04 0.97 1.04 Social desirability 0.21 (0.03) 0.24⁎⁎⁎ 0.98 1.01 GIHS −0.23 (0.03) −0.32⁎⁎⁎ 0.52 1.92 CIHS −0.61 (0.06) −0.47⁎⁎⁎ 0.32 3.15 Biola IH 0.03 (0.05) 0.03 0.45 2.24 Step 3 0.002 Education −0.03 (0.01) −0.04 0.96 1.04 Social desirability 0.19 (0.04) 0.21⁎⁎⁎ 0.68 1.47 GIHS −0.38 (0.03) −0.33⁎⁎⁎ 0.50 2.01 CIHS −0.62 (0.07) −0.48⁎⁎⁎ 0.31 3.25 Biola IH 0.02 (0.06) 0.02 0.43 2.30 L-OIHS 0.04 (0.02) 0.06 0.49 2.06 Total R2 0.743 Note. GIHS = General Intellectual Humility Scale; CIHS = Comprehensive Intellectual Humility Scale; L-OIHS = Limitations-Owning Intellectual Humility Scale. ⁎ p < 0.05. ⁎⁎ p < 0.01. ⁎⁎⁎ p < 0.001. M. Haggard et al. Personality and Individual Differences 124 (2018) 184–193 191 likely to be intellectually humble, which provides evidence that IH involves emotions, not only cognitions or behaviors. In comparison to prior examinations of IH measures, our hierarchical regressions were especially restrictive and conservative, as we included three other measures of IH to compete with L-OIHS. First, this demonstrated that while the measures may overlap and share variance, they are not so similar that they tread into issues of multicollinearity when used together. Second, it showed impressive incremental validity that the L-OIHS accounts for additional variability over and above the additional IH measures in both closed-mindedness and openness to experience. It was by far the most stringent and rigorous assessment of an IH scale to date. In addition, it is important to consider that the current research is focused squarely on self-reporting. A fuller examination of the utility of the L-OIHS should include consensus with other-reports, behavioral studies, and observational studies. Prior research has shown that reaching consensus regarding the IH of others requires a great deal of interaction or time (Meagher et al. 2015). Self-other agreement about a target's IH was not reached after 45 min of group interaction on tasks that strained humility; rather consensus about whether a person is intellectually humble was reached after working in a group for several months (Meagher et al. 2015). However, this work did not employ the scales developed or tested here, which may facilitate more accurate interpersonal judgments. Still, this work also demonstrates the unique contribution of not only the L-OIHS, but also the other IH measures to the overall understanding of the virtue. Though the CIHS, GIHS, and Biola IH Scale were all constructed from differing theoretical viewpoints, each measure provides a consistent yet distinctive understanding of IH. Further work incorporating other-reports, behavioral measures, and extensive statistical methods, as well as investigation into intellectual pride, will not only improve the quality of research surrounding IH, but will also help to bolster the usefulness of each individual measure. Acknowledgement This work was generously supported by the John Templeton Foundation under Grant 29630 and through the Institute for the Study of Human Flourishing at the University of Oklahoma, Grant 57481. The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of the John Templeton Foundation or the Institute for the Study of Human Flourishing. Appendix A 12-item Limitations-Owning Intellectual Humility Scale I. Love of Learning 1. If I don't understand something, I try to get clear about what exactly is confusing to me. 2. When I don't understand something, I try hard to figure it out. 3. I love learning. 4. I care about truth. II. Appropriate Discomfort with Limitations 1. I focus on my intellectual weaknesses too much.⁎ 2. When I know that I have an intellectual weakness in one area, I tend to doubt my intellectual abilities in other areas as well.⁎ 3. When I think about the limitations of what I know, I feel uncomfortable.⁎ 4. I tend to get defensive about my intellectual limitations and weaknesses.⁎ III. Owning Intellectual Limitations 1. I have a hard time admitting when one of my beliefs is mistaken.⁎ 2. When someone points out a mistake in my thinking, I am quick to admit that I was wrong. 3. I am quick to acknowledge my intellectual limitations 4. I feel comfortable admitting my intellectual limitations. ⁎denotes item is reverse-scored References Altemeyer, B. (1996). The authoritarian specter. Cambridge, MA, US. Harvard University Press. Altemeyer, B. (2002). Dogmatic behavior among students: Testing a new measure of dogmatism. The Journal of Social Psychology, 142(6), 713–721. http://dx.doi.org/10. 1080/00224540209603931. Bollinger, R. A., & Hill, P. C. (2012). Humility. In T. G. Plante (Ed.). Religion, spirituality, and positive psychology: Understanding the psychological fruits of faith (pp. 31–47). Santa Barbara, CA, US: Praeger/ABC-CLIO. Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon's mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6(1), 3–5. http://dx.doi.org/10.1177/1745691610393980. Chancellor, J., & Lyubomirsky, S. (2013). Humble beginnings: Current trends, state perspectives, and hallmarks of humility. Social and Personality Psychology Compass, 7(11), 819–833. http://dx.doi.org/10.1111/spc3.12069. Collins, J. (January 2001). Level 5 leadership: The triumph of humility and fierce resolve. Harvard Business Review, 67–76. Davis, D. E., Hook, J. N., Worthington, E. L. J., Van Tongeren, D. R., Gartner, A. L., & Jennings, D. J. I. (2010). Relational spirituality and forgiveness: Development of the spiritual humility scale (SHS). Journal of Psychology and Theology, 38(2), 91–100. Davis, D. E., Worthington, E. L. J., & Hook, J. N. (2010). Humility: Review of measurement strategies and conceptualization as personality judgment. The Journal of Positive Psychology, 5(4), 243–252. http://dx.doi.org/10.1080/17439761003791672. Exline, J. J., & Geyer, A. L. (2004). Perceptions of humility: A preliminary study. Self and Identity, 3(2), 95–114. http://dx.doi.org/10.1080/13576500342000077. Friedman, T. L. (2014, February 22). How to get a job at Google. The New York Times. Retrieved from http://www.nytimes.com/2014/02/23/opinion/sunday/friedmanhow-to-get-a-job-at-google.html. Gregg, A. P., & Mahadevan, N. (2014). Intellectual arrogance and intellectual humility: An evolutionary-epistemological account. Journal of Psychology and Theology, 42(1), 7–18. Hill, P. C., Laney, E. K., & Edwards, K. E. (2014). Unpublished intellectual humility scale. La Mirada, CA: Rosemead School of Psychology, Biola University. Hoyle, R. H., Davisson, E. K., Diebels, K. J., & Leary, M. R. (2016). Holding specific views with humility: Conceptualization and measurement of specific intellectual humility. Personality and Individual Differences, 97, 165–172. http://dx.doi.org/10.1016/j.paid. 2016.03.043. Jenerette, C., & Dixon, J. (2010). Developing a short form of the simple Rathus assertiveness schedule using a sample of adults with sickle cell disease. Journal of Transcultural Nursing, 21(4), 314–324. http://dx.doi.org/10.1177/ 1043659609360712. John, O. P., & Srivastava, S. (1999). The big five trait taxonomy: History, measurement, and theoretical perspectives. In L. A. Pervin, & O. P. John (Eds.). Handbook of personality: Theory and research (2nd ed.) (pp. 102–138). New York, NY, US: Guilford Press. Kruglanski, A. W., Webster, D. M., & Klem, A. (1993). Motivated resistance and openness to persuasion in the presence or absence of prior information. Journal of Personality and Social Psychology, 65(5), 861–876. http://dx.doi.org/10.1037/0022-3514.65.5. 861. Krumrei-Mancuso, E. J. (2017). Intellectual humility and prosocial values: Direct and mediated effects. The Journal of Positive Psychology, 12(1), 13–28. http://dx.doi.org/ 10.1080/17439760.2016.1167938. Krumrei-Mancuso, E. J., & Rouse, S. V. (2015). The development and validation of the comprehensive intellectual humility scale. Journal of Personality Assessment, 98(2), 209–221. http://dx.doi.org/10.1080/00223891.2015.1068174. Leary, M. R., Diebels, K. J., Davisson, E. K., Isherwood, J. C., Jongman-Sereno, K. P., Raimi, K. T., ... Hoyle, R. H. (2017). Cognitive and interpersonal features of intellectual humility. Personality and Social Psychology Bulletin, 43(6), 793–813. http:// dx.doi.org/10.1177/014616721769769. Litman, L., Robinson, J., & Abberbock, T. (2016). TurkPrime.com: A versatile crowdsourcing data acquisition platform for the behavioral sciences. Behavior Research Methods, 1–10. http://dx.doi.org/10.3758/s13428-016-0727-z. Mason, W., & Suri, S. (2012). Conducting behavioral research on Amazon's mechanical Turk. Behavior Research Methods, 44(1), 1–23. http://dx.doi.org/10.3758/s13428011-0124-6. Meagher, B. R., Leman, J. C., Bias, J. P., Latendresse, S. J., & Rowatt, W. C. (2015). Contrasting self-report and consensus ratings of intellectual humility and arrogance. Journal of Research in Personality, 8, 35–45. http://dx.doi.org/10.1016/j.jrp.2015.07. 002. Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.). Measures of personality and social psychological attitudes (pp. 17–59). San Diego, CA: Academic Press. Peterson, C., & Seligman, M. E. P. (2004). Character strengths and virtues: A handbook and classification. Washington, DC, US. New York, NY, US: American Psychological Association. Raskin, R., & Hall, C. S. (1981). The narcissistic personality inventory: Alternate form reliability and further evidence of construct validity. Journal of Personality Assessment, 45(2), 159–162. http://dx.doi.org/10.1207/s15327752jpa4502_10. Raskin, R., & Terry, H. (1988). A principal-components analysis of the narcissistic M. Haggard et al. Personality and Individual Differences 124 (2018) 184–193 192 personality inventory and further evidence of its construct validity. Journal of Personality and Social Psychology, 54(5), 890–902. http://dx.doi.org/10.1037/00223514.54.5.890. Roberts, R. C., & Wood, W. J. (2003). Humility and epistemic goods. In L. Zagzebski, & M. DePaul (Eds.). Intellectual virtue: Perspectives from ethics and epistemology (pp. 257– 279). New York, NY, US: Oxford University Press. Samuelson, P. L., Jarvinen, M. J., Paulus, T. B., Church, I. M., Hardy, S. A., & Barrett, J. L. (2014). Implicit theories of intellectual virtues and vices: A focus on intellectual humility. The Journal of Positive Psychology, 0(0), 1–18. http://dx.doi.org/10.1080/ 17439760.2014.967802. Saucier, D. A., & Webster, R. J. (2010). Social vigilantism: Measuring individual differences in belief superiority and resistance to persuasion. Personality and Social Psychology Bulletin, 36(1), 19–32. http://dx.doi.org/10.1177/0146167209346170. Tangney, J. P. (2009). Humility. In S. J. Lopez, C. R. Snyder, S. J. Lopez, & C. R. Snyder (Eds.). Oxford handbook of positive psychology (2nd ed.) (pp. 483–490). New York, NY, US: Oxford University Press. Toplak, M. E., West, R. F., & Stanovich, K. E. (2011). The cognitive reflection test as a predictor of performance on heuristics-and-biases tasks. Memory & Cognition, 39(7), 1275–1289. http://dx.doi.org/10.3758/s13421-011-0104-1. Tracy, J. L., & Robins, R. W. (2007). The psychological structure of pride: A tale of two facets. Journal of Personality and Social Psychology, 92(3), 506–525. http://dx.doi.org/ 10.1037/0022-3514.92.3.506. Vispoel, W. P., & Kim, H. Y. (2014). Psychometric properties for the balanced inventory of desirable responding: Dichotomous versus polytomous conventional and IRT scoring. Psychological Assessment, 26(3), 878–891. http://dx.doi.org/10.1037/a0036430. Whitcomb, D., Battaly, H., Baehr, J., & Howard-Snyder, D. (2017). Intellectual humility: Owning our limitations. Philosophy and Phenomenological Research, 94(3), 509–539. http://dx.doi.org/10.1111/phpr.12228. Wilson, A. (2017). Unity of the intellectual virtues. Oral presentation at the virtue epistemology conference, Norman, OK. M. Haggard et al. Personality and Individual Differences 124 (2018) 184–193