Ethical issues of cost effectiveness analysis and guideline setting in mental health care
Symposium on evidence based medicine
R Berghmans1,
M Berg2,
M van den Burg1,
R ter Meulen1
1Institute for Bioethics, Health Ethics and Philosophy, Department of Caring Sciences, University of Maastricht, Maastricht, the Netherlands
2Institute of Health Policy and Management, Erasmus University, Rotterdam, the Netherlands
Correspondence to:
R Berghmans
Institute for Bioethics, Health Ethics and Philosophy, Department of Caring Sciences, University of Maastricht, PO Box 616, 6200 Maastricht, The Netherlands; r.berghmansige.unimaas.nl
This article discusses ethical issues which are raised as a result of the introduction of economic evidence in mental health care in order to rationalise clinical practice. Cost effectiveness studies and guidelines based on such studies are often seen as impartial, neutral instruments which try to reduce the influence of non-scientific factors. However, such rationalising instruments often hide normative assumptions about the goals of treatment, the selection of treatments, the role of the patient, and the just distribution of scarce resources. These issues are dealt with in the context of increased control over clinical practice by third parties. In particular, health insurers have a great interest in economic evidence in clinical care settings in order to control access to and quality of (mental) health care. The authors conclude that guideline setting and cost effectiveness analysis may be seen as important instruments for making choices in health care, including mental health care, but that such an approach should always go hand in hand with a social and political debate about the goals of medicine and (mental) health care. This article is partly based on the results of a research project on the normative aspects of guideline setting in psychiatry and cardiology which was conducted under the guidance of the Royal Dutch Medical Association.
cost effectiveness analysis
http://dx.doi.org/10.1136/jme.2003.007047
Mental health care is one of the areas in which one can notice a growing interest in evidence based approaches regarding the effectiveness and cost effectiveness analysis (CEA) of psychiatric and psychotherapeutic treatments. One of the reasons for this interest is the increasing scarcity of resources in the health care system, caused by a growing demand for care on the one hand and diminishing resources on the other. The scarcity of resources is particularly felt in mental health care: in the Netherlands—for example—there are long waiting lists for mental health care services, particularly for outpatient care. One way to cope with scarcity of resources in mental health care is to have more specific diagnoses, and to limit the duration of treatment by introducing guidelines based on scientific studies on the effects and costs of treatments.
Clinicians have mixed feelings about the increasing emphasis on evidence based guidelines, particularly those intended to ration scarce or expensive medical treatments. Many of them resist the idea that clinical decisions should be guided by economic considerations instead of the needs of the patient. This is particularly true for mental health care, which is generally very patient oriented: more than in other areas in health care, the values and preferences of the patient are taken into account in treatment decisions. None the less, professionals in mental health care are very well aware of the problem of scarcity of resources and the need to make fair and just decisions about the kind and length of treatment as well as the access to it. Guidelines, then, could play a useful role when such difficult decisions need to be made. An important question therefore is to what extent the rational, economic world of guidelines can be reconciled with the moral values which guide the practice of mental health care.
In this article we discuss a number of moral values to which practitioners in mental health care adhere, and compare these values with the normative structure of cost effectiveness studies and guidelines based on these studies. We will make use of the results of a research project on the normative aspects of guideline setting in psychiatry and cardiology which was conducted under the guidance of the Royal Dutch Medical Association. This project aimed at investigating how guidelines for "appropriate medical care" should be constructed. The project took as a starting point that explicit attention should be given to ethical and political considerations, in addition to data about costs and effectiveness. In this project the values of cardiologists and psychiatrists regarding the treatment of angina pectoris and depression were compared with the normative structure of economic evaluations which are the basis for guideline development.1 For a report of this project in English see: M Berg, M van den Burg, R ter Meulen.2
DEFINITION OF THE GOALS OF TREATMENT
The history of mental health care and psychiatry is characterised by the emergence of different schools of thought, each of which advocates the "single best way" to explain and treat mental disorders. Today, this divergence in the goals and methods of mental health care still exists. One can point to the struggle between the biological and the psychological schools in the treatment of depression. In biologically oriented psychiatry, therapy of depression focuses on the control and reduction of specific symptoms. According to biological psychiatry, this objective can be achieved best by psychopharmacological means. In contrast, with a psychodynamic approach, the objective of therapy lies beyond mere symptom control. The goal of psychodynamic therapy is generally for the patient to gain insight into the structure of his or her personality. This is considered to be a way to gain reduction of, and control over, symptoms as well as to lessen the vulnerability of patients in the long run.
In our research project, it appeared that practitioners were less concerned about the "school" or line of thought in order to define the goal(s) of treatment. In fact, all of them took the view that the therapist should ask himself what is the most effective or cost effective treatment for that particular patient. Is a reduction of symptoms the best we can do for a patient, or is the patient better served by more insight in to his or her personality structure, in order to reach longer term improvement of the psychological condition, to prevent relapse, or to provide the means to manage difficult situations? As soon as the treatment goal has been defined, one can determine the most effective way to reach it.
If I were sure long term psychotherapy for patients suffering from depression,...if it were really very clear that it would make a big difference for them in the years to come, then I would introduce it into the health care system. I would strongly promote it. Because it really would make a big difference.... Whereas, if I think there is no point at all, that it would do no good, then I would not refer patients (Van den Burg, et al,1 pp 31–2).
Biologically oriented psychiatrists will generally use Hamilton scores as a criterion for treatment effect and success, whereas for psychoanalytically oriented psychiatrists experienced insight of the patient into unconscious conflicts will be a more relevant criterion for effective treatment. A socially oriented psychiatrist by comparison will put more emphasis on the change of social circumstances as a goal of treatment.
As a result, effectiveness is less easy to define or to make operational in a psychodynamic or socially oriented therapy than in the case of symptom control by antidepressant drugs, which can be made operational and measured by standardised instruments. As cost effectiveness studies have a strong bias towards the use of such "objective measures"—that is, Hamilton scores–therapeutic strategies with goals other than mere symptom control are considered to be less cost effective. If the psychiatrist and/or the patient consider the gain of insight or the change of social circumstances to be an important goal of treatment above symptom control, then they may be discriminated against in case only Hamilton scores are used as effect measures in CEAs.
This example illustrates how studies of the cost effectiveness of treatment may include normative assumptions about the definition of effectiveness. Psychiatric treatments are considered effective to the extent to which they succeed in measurable control and reduction of specific symptoms. Insight into the structure of a personality or changes in social circumstances are less easy to measure and disappear as acknowledged goals of psychiatric treatment. To avoid narrowing down the effectiveness of psychiatric treatment to mere symptom reduction, one should ask in every individual case what is the best goal for the patient, and what is the best method to reach that goal.
Another issue to consider is that effect measures in guidelines are based on ideal populations of specific patient groups included in randomised controlled trials (RCTs) and do not necessarily parallel individual measures of effect in clinical practice, in which not ideal but real patients with often complex and comorbid complaints present themselves. Also, when the RCT is considered as the gold standard for evidence, researchers and their sponsors will focus more on drug trials, because drugs fit very well into the RCT model. If policy decisions, particularly reimbursement decisions, are based on the "best available evidence", drug based treatment regimens could derive an unfair advantage over psychotherapy.3
THE ROLE OF PATIENT SPECIFIC FACTORS
In psychiatric practice, patient specific factors play an important role in the treatment of individual patients. Patient specific factors determine to a large degree the effectiveness of treatment. Examples of such factors include: a patient's values, expectations and interactions with others, social economic background, and trust in the psychotherapist.
In our project, one of the psychiatrists told us that the wish of the patient was an important consideration regarding the choice of a therapy and the determination of the goal(s) of the treatment.
This differs from patient to patient; it is very important what people want. You try to formulate treatment goals together with the patient. Treatment goals, this is what you want. For most people this comes down to decreasing or completely eliminating the symptoms of depression, and any problems that occurred in relation to those symptoms. If someone has great conflict at work and as a result ends up on sick leave with depression, then it is possible to try and treat him or her for that depression, but this means that you close your eyes to the reality of the context in which that person lives. You cannot treat the disease in isolation from the patient's life. With each and every patient you must look to see what people want. And most people want to get rid of all symptoms (Van den Burg, et al,1 p 34).
Not only the wishes and preferences of the patient, but also their social and cultural circumstances play a role in determining the best treatment and what this treatment should achieve. Patients from lower income groups—for example—often stop psychotherapeutic treatment prematurely. The reason for this lack of compliance can be sought in their specific social and cultural lifestyles, incorporating a particular value system and use of language which are difficult to reconcile with psychotherapeutic attitudes, styles, and methods. Also, the level of knowledge a patient has about himself or herself and the causes of the problems will influence the decisions about which therapy will be applied and which goals should be achieved.
Look, of course I can say I think these and those factors are very important in a patient, but if I then think that the person involved cannot do anything about those factors.... When you have ideas about treatment options, you must immediately ask yourself whether the person involved is capable of undergoing or cooperating with such a treatment. A simple example is extensive explorative psychotherapy in a mentally retarded patient. Not only does the treatment need to be indicated, but the patient must be capable of taking part in the treatment. This is an important point.... The treatment must also be feasible (Van den Burg, et al,1 p 34).
Sometimes, decisions about the choice of the therapy and the goals of treatment are the result of negotiations between the patient and the psychiatrist.
For example, if a patient has been given antidepressants, such as Prozac, and responds to some extent, but—to my way of thinking—is not recovering fully, yet the patient says, "Well, I think this is quite acceptable and I prefer not to take another drug", well, yes, then that is it.I may think a different drug or a different approach would be better but... I think that what I should do is inform people more, let them know clearly what my preference is, and why I have that preference, and that I base my preference on my experience with other patients. Because patients' experience is often unique, you know, they only know about their own experience. I think that is the way it should go.(Van den Burg, et al,1 p 35).
In some cases the psychiatrist proposes a certain treatment, but in other cases the patient might come up with a proposal, too. It depends on the psychiatrist how much room is given to the patient in the decisions about the treatment.
Sometimes, the preferences of the patient are quite influential:
But if the patient wants a different therapy, then I would not prevent him from getting it. But of course you do weigh up the pros and cons. If someone says, "Well, I don't want this, but I do want to jump from a bridge", well, yes, then I wouldn't immediately say, "Go ahead". That would not be therapy. You must yourself have some notion, a vague idea that something may not be the best treatment, but you must think it is at least a reasonably useful alternative.... If someone says, "I am going to that quack because he helped my neighbour's wife so much", then I don't accept that just like that. But okay, I will try to sort of go along with it. I'd say, "Well fine, go there, and then I will hear from you in two weeks' time whether it worked. Is that okay?" I usually try to approach it in that way. We have an idea what might work, it is not that we can enforce it and then say this is what should happen, and if you don't agree then you can leave. If someone says no, then you can give him a number of alternatives. And you have an outpatient service and you have community services, so there is a chance that you will actually find what you are looking for (Van den Burg, et al,1 p 35).
Because of the relevance of patient specific factors, it is important to build up a relationship based on trust, in which a patient can identify with the method and the goals of the treatment. Without such identification, treatment may have a small chance of success.
The degree to which a patient has faith in a treatment, of course also makes a difference. You can imagine that if someone has no faith in it at all—well, yes, perhaps it doesn't make much difference in electric shock treatment—but in psychotherapy when someone has absolutely no faith, then there is nothing you can do. If someone is unwilling to engage in conversations, then you have to stop. Then the treatment is impossible.... But also in the case of medication it is an important element. If possible, you would like to make use of a placebo effect. And the effect of a treatment is of course determined by the actual pharmacological effect plus the placebo effect. For me as a therapist, it doesn't make much of a difference whether you use the placebo effect or the real effect. You are happy with every result that you achieve (Van den Burg, et al,1 p 34).
The trust of the patient will be increased when the treatment has meaning for the patient.
One of the main problems regarding the application of scientific evidence in mental health care, however, is that it says little about the meaningfulness of a treatment for the individual. In the process of standardisation of psychiatric treatment, effectiveness and meaningfulness are constantly entangled. Patients may even have a totally different view of the effectiveness of a treatment as compared with the definition of effectiveness in guidelines. Guidelines are generally based on measurable criteria of effectiveness, such as the Hamilton score, which do not necessarily correspond with the values of patients. This is particularly true for patients for whom, in the light of treatment goals, the effect of treatment is difficult to measure, for example, because the treatment aims at affecting their deeper subjective experiences (such as mental pain and suffering, loss of meaning) and less the more overt symptoms of their mental illness, that is, vital signs.
As guidelines tend to neglect patient specific factors, they reduce patient autonomy in mental health care decision making. Furthermore, the emergence of trust in the patient/therapist relationship can be undermined.
PROFESSIONAL AUTONOMY OF THE THERAPIST
The introduction of guidelines and evidence based medicine into mental health care practice will not only affect patient autonomy and trust, but also the autonomy of the professional. Professionals will be under increased pressure to work according to the guidelines developed for the treatment of a specific illness, particularly when guidelines and scientific evidence constitute the basis for reimbursement policies. As a result, there will be less room for independent, professional judgment, based on training, experience, and affinity with a particular treatment approach.
On the one hand, evidence based guidelines may prevent inappropriate treatment decisions, in which case the limitation on the autonomy of the professional is justifiable. On the other hand, when guidelines leave little or no room for therapist discretion (as will be the case if guidelines play a role in reimbursement decisions), appropriate deviations from guidelines in individual cases will be ruled out, thus frustrating the therapist and patient in their attempts to do what is best for the patient.
What I find to be a disadvantage of guidelines, is that people treat them as gospel truth, as if they are recipe books they are using. I think guidelines are guidelines; you base your actions on them, but they also allow you a certain degree of freedom, because you can never write a guideline that applies to every single individual. So you will always have to individualise a guideline. So I would almost say, about the expert systems that prescribe exactly what you should do, that, well, I don't believe in them either. A guideline should have a certain degree of flexibility, otherwise it ignores the fact that people are individuals (Van den Burg, et al,1 p 41).
In the same way as subjective patient factors are excluded from the development and application of guidelines, subjective professional factors such as intuition and experience are becoming less important. Such factors lack scientific value and are therefore considered not to be useful for mental health care practice. According to Redman,4 clinical practice guidelines may eventually limit patients' and providers' autonomy by accepting without question the values contained within scientific research as the only basis on which judgments can be made, and then codifying these values in reimbursement decisions.
Most of the interviewed psychiatrists, even those who advocate a tough, evidence based approach, take the view that psychiatrists should have the freedom to deviate from the guideline.
The most important thing about guidelines is that they help you to come to the most effective treatment as quickly as possible. As long as you don't apply them like a straitjacket, but keep realising, as a therapist, that something is a guideline, and not an obligation, then I see no disadvantage. Then I can only see the advantage that I am doing something which is testable. I see too many examples of people who are not being treated according to the guidelines, and who are not getting well. You could say they are being mistreated, but in fact it is maltreatment (Van den Burg, et al,1 pp 40–1).
The psychiatrists in our project stressed the importance of individual variation and tailor made care. This freedom is very important, but could be compromised when decisions about the application of guidelines fall into the hands of reimbursing parties. In that scenario, the freedom of the practitioners might be seriously restricted, and that process could have detrimental effects on the wellbeing of the patients.
THE ROLE OF GUIDELINES
As stated at the beginning of this article, evidence based guidelines are increasingly seen as important instruments for rationalising mental health treatment and, at the same time, for controlling costs. In our project, however, it appeared that psychiatrists had different views about the role these guidelines should play in their decision making. Some psychiatrists strongly advocated the use of hard evidence and uniform guidelines as the best road to success in the treatment of depression:
In principle guidelines should be, just like a protocol, a translation of scientific knowledge. And scientific knowledge, that is what we build up to make sure that what we do is grounded and solid, means it is better to do something on the basis of studies according to method B than according to method A. So in that sense guidelines and protocols are quite strict propositions for your clinical actions. I think it is a great idea to set up content specific uniform guidelines.... I don't think we should discover time and again that water boils at 100 degrees centigrade, and that sort of thing. I think everybody is entitled to a treatment method that has been proved, in the most reliable way, to be of benefit. This is quite essential. In my view, that is how you should practise medicine. I think you must be able to show that what you are doing makes sense. And if the treatment makes sense, that must be made known to the specialists, so they can incorporate it into their practice on the basis of rational considerations. That is essential (Van den Burg, et al,1 p 39).
Other psychiatrists have less confidence in the evidence which is collected in scientific research.
I think the power of evidence based psychiatry is over-rated. The main problem with many of those types of treatment is that they are carried out on patient populations who are not representative of our daily practice.... And that is a problem. What we need is research, yes, simply much more naturalistic research of effectiveness, in which you simply, in real institutions, such as the outpatient service here, where people come in randomly, investigate treatment, and look at how many people want those treatments, whether they are effective, and whether they also mean things will improve in patients' lives much more quickly. You must look at the whole thing from a much wider perspective (Van den Burg, et al,1 p 40).
These psychiatrists are, consequently, less positive towards the role of guidelines and have, to say the least, ambivalent feelings about the introduction of guidelines.
A score on a scale—I have always had a resistance against such a thing, I have never liked to do things in a standard way. I think it is actually so boring. But on the other hand it enables you to express something in a measurable way. If you do something on the basis of a number of structured questions and you then note down the answers to those questions—you cannot capture it in a figure—but I think you can know more accurately by measuring.... If in the future protocols—for example—mean that I will always see people while having a questionnaire in front of me, then I personally don't think that would be nice. At the same time I realise that it is true, so to speak, that you do your work on a sounder basis.(Van den Burg, et al,1 p 40).
MORE ROOM FOR THE PATIENT
A possible way to balance the onesidedness of evidence based approaches in psychiatric care is to include patients in the process of development and application of guidelines. One option might be to collect patient preferences or utilities at a general level and to use these as an outcome measure for the success of a treatment. However, such a measure, no matter how patient based it might be, could easily be seen as an abstract measure which, like the traditional outcome measures of the evidence based approach, is too far removed from the concrete experiences of the individual patient. If one argues for more patient involvement in guidelines, one would do better to think of procedural steps which create room for the patient, such as: shared decision making with the patient; listening to the patient; evaluating the treatment together with the patient; discussing compliance with the therapy; giving the patient a role in the determination of the success of the treatment; and so on. In this way not only the autonomy of the patient could be reinforced, but his or her responsibility as well. When the patient has been involved in the decision making regarding the treatment and the goals of treatment, he or she may feel more responsible for the course of the treatment, whether that be the use of medication or participation in therapeutic sessions. In fact, increasing and supporting the responsibility of the patient might be a more effective and cost effective measure than the uncritical and unbalanced application of hard economic evidence.
Some authors claim that the position of patients may be strengthened by the application of evidence based guidelines.5 They welcome evidence based medicine as a critical response to the inadequacies of traditional medicine in general, and the paternalistic authority of doctors and other health professionals in particular. With the outcomes of systematic reviews in their hands, patients and their representatives could claim a more central role in medical decision making. It is questionable, however, whether evidence based approaches will indeed result in a medical democracy. There is a concern among doctors and other health care practitioners that the increasing dominance of evidence based medicine may lead to a shift of power from doctors (and their patients) to managers and purchasers. Policy makers and insurance companies are often tempted to see guidelines and cost effectiveness analyses as an ideal way to control clinical practice and health care delivery and thus to control the costs of health care. In theory, evidence based guidelines could improve the position of patients in clinical decision making. In practice, however, third parties are gaining more power to control the delivery of care services and will in fact determine which kinds of treatment and how many will be reimbursed. In continental health care systems—for example—the introduction of market approaches into the health care system will result in a shift of power towards insurance companies. Because those companies bear greater financial risks, they could force providers, including hospitals and institutions for mental health care, to make use of the most cost effective treatments, such as drug based therapies instead of psychotherapy or social care services for which there is less hard evidence available.
When third parties are using guidelines for their own purposes, there is also a danger that they make use of guidelines that give directions about the best treatment, which in fact is only relevant for a specific context. The Agency for Health Care Policy and Research (AHCPR) guidelines for depression treatment offer an example of this.6 Because of the characteristics of the settings for which these guidelines are intended—general practitioners' (GPs) offices—these guidelines are aimed at short term treatments, and at symptom reduction. The assumption is not that short term treatments are always better, but that they are optimal within this specific setting. Due to the way in which the AHCPR perceives the organisation of GPs' practices, patient populations and expertise, this guideline limits itself to this mode of treatment and is not applicable to long term treatment which has no place in a GP's practice. It is crucial to be attentive to such assumptions, because once a guideline or cost effectiveness analysis has been developed for a particular context, it becomes part of a set of tools. In the case of the AHCPR guidelines, these assumptions are particularly relevant because external parties (such as insurance companies) may neglect this specific background and attempt to push for a general application of the guidelines. By simplifying the treatment of depression in order to address the needs of primary care physicians working under time and cost containment pressures, broader, more patient centred criteria for the assessment of the benefits of treatment will be rejected.
ISSUES OF JUSTICE
Guidelines and CEAs are usually associated with the aim to rationalise medical work, to render medical action more objective, and to introduce scientific reasoning into the core of the work of doctors and other health care professionals. They are intended to enhance the efficacy and efficiency of medical care, and they should help to reduce unwanted variations in physicians' practices.7–9 Cost effective analyses and guidelines based upon them are seen as the most important way to solve the problem of priority setting, because they are considered to be neutral and impartial instruments.
From an ethical point view, however, the neutrality and impartiality of cost effectiveness analyses and guideline setting are debatable. Ethical and political considerations are embedded in any guideline and CEA. Whether implicitly or explicitly, every such tool passes judgment on the acceptability of risk, the meaningfulness of treatment for a specific group of patients, the quality of life in a limited health condition, and so forth. These normative issues can take the form of—implicit or explicit—assumptions that are embedded in a tool. An example is the guideline that pharmacological therapy for depression is to be preferred to psychodynamic therapy. Alternatively, normative issues can be at stake when a concrete medical situation has to be represented by a guideline or another formal model, such as a decision tree or a critical pathway. Such translations inevitably imply abstractions, simplifications, and selections, and require constant, pragmatic choices. What is the role of the patient? What is the role of intuition and experience? How many therapy sessions may be held? Where should the cut off point be? It is in these often pragmatic choices that much of the normativeness creeps in.10
Using CEAs to make choices between treatments and/or diagnostic strategies is another example of a specific translation of such a normative choice into economic considerations, and a utilitarian notion of a just distribution of scarce resources. Such a strategy pushes alternative considerations or concepts of justice into the background.2 The limited budget for health care expenses is defined as the starting point for rationalising individual treatment decisions, which is in itself a normative decision. Moreover, the problem of treatment or diagnostic strategy selection is reformulated as a collective problem. Founding treatment decisions on CEAs in health care implies an approach in which individual initiatives are made subordinate to collective considerations. This does not only imply that the actions of individual physicians are collectively regulated: this also happens in ordinary (non-CEA based) guidelines. Rather, the utilisation of CEAs implies a fundamental shift from a liberal, individual initiative approach to the operation of health care, to a collective, top down approach motivated by the utilitarian question: "What is best for society as a whole?"11,12
Guideline setting and CEAs may indeed be seen as important instruments for making choices in health care, including mental health care. The development and application of these instruments should always take place, however, with the clear involvement of practitioners and patients, in order that the professional values of practitioners, the needs and values of the patients, and the need of society for a just distribution of scarce resources will be balanced and, hopefully, to some extent, reconciled with each other.
Van den Burg M, Berg M, Berghmans R, et al.Normatieve aspecten van richtlijnontwikkeing en kosteneffectiviteitsstudies in de cardiologie en psychiatrie. Utrecht: KNMG-project Passende Medisch Zorg, 1999.
Berg M, van den Burg M, ter Meulen R. Guidelines for appropriate care: the importance of empirical normative analysis. Health Care Anal2001;9:77–99.
Ter Meulen R, Dickenson D. Into the hidden world behind evidence based medicine. Health Care Anal2002;10:231–41.
Redman BK. Clinical practice guidelines as tools for public policy: conflicts of purpose, issues of autonomy and justice. J Clin Ethics1994;5:303–9.
Hope T. Evidence based patient choice. Report to the King's Fund. London: King's Fund, 1996.
Depression Guideline Panel. Clinical practice guideline number 5: depression in primary care, 2: treatment of major depression. Rockville, MD: US Department of Health and Human Services, Agency for Health Care Policy and Research, 1993. AHCPR publication 93–0551.
Eddy DM. The challenge. JAMA1990;263:287–90.
Wennberg JE. Dealing with medical practice variations: a proposal for action. Health Aff1984;3:6–32.
Sackett DL, Rosenberg WMC. The need for evidence based medicine. J R Soc Med1995;88:620–24.
Feinstein AR. Clinical judgment revisited: the distraction of quantitative models. Ann Intern Med1994;120:799–805.
Wynia MK. Economic analyses, the medical commons, and patients' dilemmas: what is the physician's role? J Investig Med1997;45:35–43.
Granata AV, Hillman AL. Competing practice guidelines: using cost effectiveness analysis to make optimal decisions. Ann Intern Med1998;128:56–63.
Is the methodological quality of guidelines declining in the US? Comparison of the quality of US Agency for Health Care Policy and Research (AHCPR) guidelines with those published subsequently
R Hasenfeld et al., Quality and Safety in Health Care, 2003
Quality of web based information on treatment of depression: cross sectional survey
Kathleen M Griffiths et al., BMJ, 2000
Depression in primary care: part 2—management
Parashar Ramanuj et al., BMJ, 2019
Smoking Cessation Guidelines for Health Professionals—A guide to effective smoking cessation interventions for the health care system
MARTIN RAW et al., Thorax, 1998
Efficacy and moderators of psychological interventions in treating subclinical symptoms of depression and preventing major depressive disorder onsets: protocol for an individual patient data meta-analysis of randomised controlled trials
David D Ebert et al., BMJ Open, 2018
Integrated primary care for patients with mental and physical multimorbidity: cluster randomised controlled trial of collaborative care for patients with depression comorbid with diabetes or cardiovascular disease
Peter Coventry et al., BMJ: British Medical Journal, 2015
Bipolar depression: a review of treatment options
Kate Levenberg et al., General Psychiatry, 2022
Evidence synthesis, practice guidelines and real-world prescriptions of new generation antidepressants in the treatment of depression: a protocol for cumulative network meta-analyses and meta-epidemiological study
Yan Luo et al., BMJ Open, 2018
Association of intranasal esketamine, a novel 'standard of care' treatment and outcomes in the management of patients with treatment-resistant depression: protocol of a prospective cohort observational study of naturalistic clinical practice
Gustavo Vazquez et al., BMJ Open, 2022
International Committee on Mental Health in Cystic Fibrosis: Cystic Fibrosis Foundation and European Cystic Fibrosis Society consensus statements for screening and treating depression and anxiety
Alexandra L Quittner et al., Thorax, 2015