Beyond Diagnosis and Comorbidities—A Scoping Review of the Best Tools to Measure Complexity for Populations with Mental Illness. (2024)

Author(s): Grace Kapustianyk [1]; Anna Durbin [2]; Ali Shukor [3]; Samuel Law (corresponding author) [4,*]

1. Introduction

Starting with and extending beyond diagnostics, the proper understanding of patient complexity is an evolving science that is at the core of improving healthcare quality, affecting patient understanding and engagement, referral and service matching and coordination, outcome evaluation, and system-level care planning [1]. Contemporary conceptualizations of patient complexity can be traced back to Rudolf Virchow’s development of the field of social medicine, which arose in response to the detrimental social, physical, and mental health effects of industrialization in the 19th and early 20th centuries [2]. The science of complexity was furthered by advancements in general systemology [3], scientific movement away from reductionism [4], development of integrative medicine [5], as well as George Engel’s model of biopsychosocial medicine in the 1970s [6]. Furthermore, advances in complexity theory offer sophisticated insights and frameworks that contribute to the conceptualization and understanding of patient complexity [7,8,9]. The properties and features of complex systems, such as adaptation, nonlinearity, interactive causal structures, emergence, self-organization/spontaneous order, and feedback loops, provide insights into mental health and illness, particularly in relation to the dynamic context of how individuals and systems respond to stresses and changes [10,11]. With concepts like the “edge of chaos”, which explores complex adaptive coping with chaotic environments through flexibility, agility, and innovation, fostering adaptability and resilience, chaos theory also contributed to the understanding and development of complexity measurements [12].

Researchers and clinicians have built on some of these concepts and developments to arrive at a variety of tools that enable a more rigorous measurement and assessment of patient complexity, becoming increasingly multidimensional and multidisciplinary in perspective. Early concepts of the complex patient were disease-oriented, based largely on the strength of the diagnosis and attendant condition of the illness, plus the presence and number of multi-morbidities; however, this has been progressively recognized as being far too narrow and inadequate in reflecting the realities of complexities [13,14]. In the mid-1990s, the concept and research on complexity moved more fully to include various determinants of health, integrating social, economic, political, and environmental aspects [15].

As a steadily growing field, the definition of patient complexity remains fragmented and, often, quite subjective. For example, a review of the literature for definitions and descriptions of complexity found three broad themes: multimorbidity (including the key diagnosis), resource utilization, and psychosocial complexity, with multimorbidity being the formative and most investigated term [1]. In this category, most measures on complexity still emphasize chief diagnosis, physical health issues, and comorbidity (e.g., the Charlson Comorbidity Index [16], the Elixhauser Comorbidity Index [17], and the Johns Hopkins’ Adjusted Clinical Group system [18]), and largely exclude social determinants of health, despite overwhelming evidence of their relationship with health outcomes [19]. The resource utilization category takes up complexity as an outcome of resource utilization, impact, and efficiency (e.g., the Kaiser Permanente Chronic Conditions Management pyramid [20]), while the psychosocial category emphasizes some combination of psychosocial and environmental factors that include social isolation, psychiatric illness, socio-demographic vulnerability, addictions, and access to care, among others. Given these various definitions of complexity, researchers have proposed models that are based on the chief diagnosis, but extend beyond this to include various aspects of a patient’s health status as vectors in a multidimensional space, with each of the important dimensions represented. Analyzing the relationships and interactions within this vector space helps to capture the overall complexity. Some vector models focus on clinical complexity [21,22], while others highlight the balance between patient workload demands and patient capacity [23] or offer a systematic and holistic perspective to understand care needs and patient challenges to ultimately match services and inform guidelines and policies [1].

Within the field of complexity measurement, populations with mental illness present unique challenges as their diagnoses are typically without any hard anchors of biological markers and rely on reported and observed psychological and behavioral information, using an epidemiologically and statistically driven approach that heavily relies on subject experts [24,25]. The current chief diagnostic references and guidelines are the Diagnostic Statistical Manual (DSM-V) and International Classification of Diseases (ICD-11) [26]. Psychiatric diagnoses and complexity arise from biological, psychological, and social components that are fundamentally more intertwined and interactive; in addition, acute and chronic physical health conditions, addictions issues, and neurocognitive disorders often coexist [11]. Their complexity is multifactorial and is also mediated by patient-level factors, clinician practice factors, systemic-level issues and barriers, and health policy factors [27]. Moreover, social and cultural forces, such as stigma, insight and acceptance of the illness, help in seeking patterns, adherence to treatment, and availability and access to treatment and care, all affect the understanding, approach, and treatment of mental illness. These intersecting challenges often require special understanding and consideration [23]. Furthermore, within those with mental illness, a unique subgroup of people with serious mental illness (SMI), or formerly termed severe and persistent mental illness (SPMI)—mainly consisting of people with challenging diagnoses, such as schizophrenia or psychotic spectrum disorders, major mood disorders such as bipolar disorder and severe depression, often with comorbid addiction disorders—is known to have very high levels of resource utilization as well as poor health outcomes (including non-psychiatric illness, such as cardiovascular disease). According to care guidelines [28,29], people with SMI require intensive multidisciplinary team-based care, with special attention to individualization of services (i.e., person orientation), taking into account the individual’s biopsychosocial context. Improving both resource allocation and health outcomes require sophisticated case-mix adjustment tools, while functions such as team-based care, integrated care, and individualization of care require person-oriented complexity assessment tools [30]. Therefore, proper complexity measurement for people with SMI presents additional challenges.

Given the many recent developments in tools or algorithms to measure complexity, and a lack of attention paid specifically to mental health in the field, we sought to review the scope of complexity tools available and consequently determine which could best serve patients within mental health settings, including for those with SMI. Properly conducted, measurements of complexity could aid in mental healthcare through the proper assignment of services based on diagnostic groups, improved admission and discharge decisions, allocation of resources, evaluation of performance and quality of care, and designs of care teams and governance structures—all of which could aid in improving care from a deficit-oriented focus on mental illness to a strength-focused approach in promoting mental health [15]. Adjustments for complexity, or case-mix, are also key to understanding baseline differences in groups when interpreting outcome data and making attribution claims and policy decisions [31]. This review aims to:

1. Undertake a review of the tools that define, operationalize, and measure patient diagnoses and complexity, while being considerate of the fact that there is not one agreed upon definition by complexity practitioners or researchers.

2. Identify tools that could be applied specifically to patients with mental health diagnoses, including those with SMI.

2. Methods

A scoping review methodology informed by the PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews) was used to identify tools and measures of patient complexity [32] to determine how patient complexity is defined and operationalized through such tools and to select tools that could be applied specifically to populations with mental illness. Relevant databases (i.e., PubMed, Google Scholar, PsycINFO, and APA PsychInfo, Embase) were searched between March 2020 and November 2023. There were no restrictions on publication date or language. For Google Scholar, the authors limited the search to 1000 results. Key terms searched in combination included: complexity, patient complexity, tools, serious mental illness, severe mental illness, severe and persistent mental illness, and mental health. There were also no restrictions related to symptoms or diagnoses to keep the search as broad as possible.

The authors also searched for reviews with each combination (e.g., mental health AND complexity AND review). In addition, the authors consulted a key review in the literature by Schaink and colleagues [1], and hand-searched the bibliography and future references to this review, particularly for articles and reviews that focused on patient complexity in our target population with psychiatric diagnoses. Inclusion criteria: Articles were included if they substantially engaged in operationalizing a tool for patient complexity that included biopsychosocial domains and were, at minimum, field-tested in a setting or population with mental health diagnoses. This allowed us to extend beyond the diagnoses, conceptual narratives (e.g., diagrams or profiles), and frameworks (e.g., vectors) to find tools that could be used and implemented in healthcare settings. Articles were excluded if the tool related to only one particular dimension but did not address a fuller scope of patient complexity (e.g., multimorbidity or experience with hospitalizations only but lacking any psychosocial determinants data). The objective was to avoid tools that would have only captured the “traditional” understanding of complexity, like multimorbidity or resource utilization, while acknowledging that there is no single agreed upon definition of “patient complexity”. Information on the respective tool’s definition of complexity, aim and utilization, setting/sector intended, geography and reach, study sample and whether the tool was mental health population field-tested or validated, level of validation, primary age group and socioeconomic status (SES) of the sample, tool funding source, and historical complexity tools referenced was extracted and outlined in summary tables.

While not an exclusion criterion established a priori, we prioritized tools that included discrete mental health or psychiatric diagnostic AND psychosocial domains. Additional information on these domains and their relevant sources, outcomes assessed in relation to complexity measured, scoring, and recommendations provided were extracted. We also included additional comments based on the team’s expertise and experience using the tool, such as ease and mode of use, time taken to complete the tool, adaptability (e.g., can one use a portion of the tool to shorten it but still have useful outcomes?), government or national association recommendations, any patient specificity or recommended settings, and whether they are available to use for free and in what languages. This was information often missing in general reviews, and it was extracted with the intention of providing practical information required by users when deciding if, and when, to use any given tool or measure.

Given the nature of a scoping review, there was no required oversight by an institutional review board. The scoping review protocol was registered with Open Science Framework (OSF) at https://doi.org/10.17605/OSF.IO/4QSDG (accessed on 5 November 2023).

3. Results

3.1. Search Results

The search returned 5345 articles (4727 articles after deduplication) from PsychINFO (741 articles), EMBASE (81 articles), PUBMED (349 articles), Google Scholar (4142 articles), and hand searching (32 articles published until mid-November 2023). After the initial title and abstract screening, there were 165 articles included for full-text review, of which 14 were considered relevant for the final selection as determined by the inclusion and exclusion criteria. For full-text review, the main reasons for exclusion were not a tool/operationalized (71 articles), not patient complexity (63 articles), not field-tested or validated in a mental health population or setting (15 articles), other language duplicate (1 article), and protocol with no associated discussion paper (1 article). (Please see Supplemental Information for PRISMA-ScR flow diagram and Checklist.) Some excluded papers that proposed frameworks for patient complexity or contributed in discussions regarding aspects of complexity—such as multimorbidity, level of care, continuity of care, quality of life, and self-sufficiency (e.g., the Dutch Self-Sufficiency Matrix) [33]—were reviewed for their content and are referenced in this review to capture the breadth of complexity and enrich the discussion.

3.2. Description of the Final Complexity Tools and Papers Reviewed

The 14 final complexity tools and related papers, along with all the extracted details for the respective tools, are presented in Table 1a,b. There was considerable variety in complexity definition, measurement indicators, setting, and use. While all included tools captured mental health diagnoses and complexity, only eight included a definition of complexity that was used to guide their tool development, ranging from vague (e.g., COMPRI [34]: “Complexity of care”; and OCCAM [35]: “The number of different factors that affect the illness and its management”) to more comprehensive definitions (e.g., VCAT-CM [30]) that used a multidimensional person-oriented profile, AMPS [36], and MCAM [37] that recognized medical and non-medical factors that interfere with care and improved health; and PCAM [38] that included “Social determinants of health that characterize socioeconomic disadvantage…, which lead to a complex interplay of biological, psychological, and social factors”.

The geographic context in which each of these tools was developed was also wide-ranging. For examples, VCAT [30], HexCom [39], Escalation tool [40], and OCCAM [35] were developed for specific geographic regions in Vancouver (Canada), Catalonia (Spain), Sydney (Australia), and Oxford (England), respectively, while other tools were developed for national (e.g., PCAT [41]—United Kingdom) or multinational use (e.g., COMPRI [34]—Spain, Italy, Hungary, Netherlands, Portugal, Germany, Denmark). Importantly, these tools were also developed for a myriad of healthcare settings and uses. The Care Process Model (CPM) [42] was developed for the general patients of the Utah’s Intermountain Healthcare Hospital system in the US to determine which level of care best suits the patient; the Minnesota Complexity Assessment Model (MCAM) [37] was developed for clinicians to quickly articulate and take account of “unnamed medical factors” that interfere with care delivery in that US state; the Complexity Checklist [43] was developed to discriminate between patients of Primary Care Providers (PCPs) that did and did not require complex care in the Colorado Division of Mental Health. Finally, the INTERMED [44], the formative complexity tool developed in the Netherlands, which many tools in this review are adapted from or informed by, has been further adapted and validated for use in multiple settings, including general healthcare, mental healthcare, and those with multiple diagnoses (substance use disorders and physical and mental disorders) (see Table 1a,b for a complete overview of all the above characteristics of each tool).

Table 1: (a) Characteristics of the final tools selected. (b) Characteristics of the final tools selected (continued).

3.3. Complexity Tools Useful for People with Mental Health Diagnoses and Challenges

Six of the fourteen tools met the specific focus of this review as they were field-tested or validated in a mental health setting—AMPS [36], CPM [42], INTERMED [44], VCAT [30], Escalation tool [40], and Complexity checklist [43]—of which four were directly developed for and tested in a mental health setting (AMPS, INTERMED, VCAT, and Escalation Tool) and two (Complexity checklist and CPM) had strong representation or identified individuals with mental illness and psychiatric diagnoses in a general healthcare setting. Each of the six tools has unique attributes that define them, and these attributes help to guide end-users in their decision on which tool(s) to use based on their unique needs. These attributes—namely the domain types and domain-related data sources, domain and overall tool scoring, and recommendations provided on scoring—all focus on different key aspects of patient complexity and are described and analyzed in depth below. A summary is presented in Table 2a,b.

Domain types: While each tool contains at least one specific psychological domain (e.g., “Psychological”, “Suicide assessment”, and “Medical, addictive, and psychiatric comorbidity” (i.e., multiple diagnoses permitted)), only the INTERMED, Escalation Tool, VCAT, and AMPS had separate, comprehensive psychosocial domains. These additional domains ranged from “family and living situation”, “previous complexity or escalation”, and “activities of daily living” to “risk of harm”. Of note, no tool had a specific race, ethnicity, or inquiry on experiences of racism domains.

Domain sources: Most tools (INTERMED, Escalation Tool, AMPS, and Complexity Checklist) are all primarily scored from the clinician’s perspective, while the VCAT and CPM use multiple domain-specific sources that include a combination of hospital admission records, patient/clinician perspectives, and validated questionnaires, like the Patient Health Questionnaire-9 (PHQ-9) [48] or the Health of the Nations Outcome Scale (HONOS) [49].

Domain and tool scoring: The VCAT, INTERMED, CPM, and AMPS all have discrete scoring with summary scores and sub-scores available, whereas the Escalation Tool and Complexity Checklist are checklists that PCPs used to determine the levels of care needed (e.g., if a PCP checked off three or more domains that they believe contributed to the patient’s complexity, the patient is escalated to a higher level of care).

Score-based recommendations from the tool: Dependent on the overall tool scores or thresholds surpassed, the tools may provide recommendations on the next steps and action items for the patient and/or practitioner. For example, the CPM recommends a care manager and mental health specialist in addition to routine, PCP-based care for those with a higher complexity, while the INTERMED simply flags a higher complexity for consideration of higher need for care. Of note, the VCAT’s summary score was initially used to gauge whether the existing patients were meeting the mandate of the services provided, but later leveraged the scores to decide on access to team-based care within community-health centers.

Additional considerations: All but the CPM and VCAT are pencil and paper tools, available within the published papers; the others are computer-based. There is little information about licensing or potential costs associated the tools. Most tools have received positive feedback on their feasibility and use. The INTERMED is available in nine languages for the self-assessment version, while the other five currently appear to be in English only. The AMPS, INTERMED, and Complexity Checklist have versions that are available online. Only the Complexity Checklist and INTERMED papers report the time taken to complete the tool, which are an average of 1.4 min and 20 min, respectively (see Table 3 for details of these additional considerations).

3.4. Complexity Tools for People with SMI

People with SMI diagnoses have a diverse set of complexities and unique clinical and psychosocial challenges. With this in mind, the INTERMED [44] and VCAT [30] were identified as the best suited tools as they include both psychological and social domains, represent both clinician-scored and multiple-sourced tools, are field-tested or validated in a mental health setting, and have varying levels of adaptability required. More specifically, the INTERMED is a four domains-by-three time perspectives grid approach directly rated by clinicians. The four domains are biological, psychological, social, and healthcare, and the three time perspectives are history, current state, and prognosis. The 4 × 3 grid approach generates at least 12 specific complexity data categories (e.g., biological x history may include intensity of prior treatment, etc.), and each of the 12 categories can be subdivided into finer points of information (e.g., social x current State can include residential instability and impairment in social integration, etc.).

On the other hand, the VCAT is a computer-based instrument that utilizes information from healthcare databases and clinician input to generate a comprehensive picture of complexity. The VCAT has nine domains, conceived as vectors arrayed in parallel, and they form a profile of complexity rather than a singular index. The nine domains include quality of attachment to care; service density in terms of multitudes of care and their access and coordination; social and environmental factors related to social barriers, such as housing instability, poverty, etc.; psychosocial factors, such as cognitive, behavioral, and/or functional impairment; relationships issues, such as inability to maintain lasting personal or professional relationships; the functional level of activities of daily living (ADLs); medical complexity, such as chronic disease, concurrent disorders, or communicable diseases; acute (hospital) utilization, such as emergency department use or hospitalization; and Risk of harm to self or others [30]. VCAT leverages routinely collected administrative and clinical data to support more consistency and reliability in data collection as well as having potential for larger level scalability across the system and for program evaluations.

Although a small suite of biopsychosocial complexity tools derived from the Dutch INTERMED tool address biopsychosocial complexity (MCAM, PCAM, etc.), item subjectivity and reliance on additional data collection are key barriers to their widespread use.

4. Discussion

This current review aims to contribute to the understanding of patient complexity tools available, with a special emphasis on those with psychiatric diagnoses being served in mental health settings, in view of the limited definitional clarity surrounding this topic and the wide-ranging developments and available choices. Our findings present an overview of the patient complexity tools that specifically include psychological and psychosocial domains and are operationalized for use in mental health settings, with additional attention to those that are suitable to people with SMI who have unique challenges. Despite the progress in concept, it is evident from this review that the number of currently available complexity measuring tools that consider a broader set of biopsychosocial aspects of complexity is still limited, and fewer still are best suited for populations with mental health challenges. Encouragingly, the review did identify a few readily available, useful tools that could serve those with a wide spectrum of psychiatric diagnoses and illnesses.

Previous major reviews of complexity measurement have focused on how complexity is defined [16] or thematically classified [1], what conceptual frameworks of medical complexity are available [50], provider narratives on complexity [14], or how comorbidity does not fully reflect complexity [51]. While each of these reviews had merits and examined some key components for measuring complexity, they did not offer a comprehensive framework that informed the state-of-the-art measurement of complexity for mental health settings. Bearing in mind that no single universal definition for complexity exists, the current review addressed these limitations by purposefully identifying a broad range of perspectives, tools, and their respective strengths and carried out literature-informed data extraction and comparisons. The current review potentially contributes to making complexity measurement for people with mental illness more understandable, more accessible, and more widely utilized, which could in turn improve mental health outcomes across various contexts.

One of the findings from the review is the high level of heterogeneity, even among the tools that met the inclusion criteria. Given the diversity both of the definitions of complexity and the contexts under which each of the tools was developed, it is not surprising that the tools differ a great deal in their orientations and foci. Only one (i.e., INTERRMED) is specifically adapted for people with mental illness [44]. The other few that were found to be suitable for people with mental illness have more explicit psychological and/or psychosocial domains, but they may not be specifically developed for this population. At the least, the current review is helpful in understanding how such heterogeneity came about and realizing where the limits are and could inform further specific developments in this field for people with mental illness.

Of the six complexity tools for patients with mental illness identified—the AMPS [36], CPM [42], INTERMED [44], VCAT [30], Escalation tool [40], and Complexity checklist [43]—the current review summarized and compared highly specific details on the characteristics and indicators of each tool to assist with decision-making processes when selecting a tool, depending on one’s needs, population, and resources available. The review found that, while each of the six tools included a psychological domain, only four (i.e., the AMPS, INTERMED, VCAT, and Escalation Tool) also included a psychosocial domain. For end-users interested in a more holistic approach, the current review would endorse these four, as they have more mental health-inclusive considerations through additional domains, such as family and supportive relationships, living situation, financial capacity, and history of substance abuse, among others. The current review is explicit in advocating the inclusion of such important psychosocial issues in understanding a mental health patient’s complexity, as the field has come a long way from the days of the more unidimensional consideration of patient complexity [16].

The current review also shows that the more recently developed, comparatively more sophisticated, biopsychosocially informed tools—such as the VCAT and INTERMED—cover the various perspectives and address the shortcomings of the previous tools by having good “cross-mapping” with many of the specific dimensions of the other more limited complexity tools (e.g., the MCAM [37], AMPS [36], etc.). However, to further improve complexity measurement, some experts have advocated capturing better the interrelationship among the domains [52] and to not only focus on the deficits involved in complexity, but consider wellbeing and well functional status too [53]. In addition, it is very important that concepts from complexity theory—such as nonlinearity, interactive causal structures, and the “edge of chaos” phenomenon from chaos theory—be further explored and incorporated into the conceptualization, development, and refinement of complexity tools [8,12].

A more holistic understanding of complexity has substantial implications for patients, physicians, administrators, policymakers, and insurers alike. Some researchers pointed out that, for example, if the demanding care of complex patients is not appropriately considered and compensated, and healthcare provision are not harmonized with complexity, service providers have a powerful incentive to select healthier patients and neglect those who are more complex [54]. As a field, primary care has demonstrated that proper complexity measurements have contributed to improvements in provider–patient communication, service empanelment, team-based care, service matching, coordination and continuity of care, and outcome research [55,56,57]. Mental health as a specialty, being less exposed to such care improvements through incorporating complexity measurements, may find it attractive to adapt such measurements [42,58].

In terms of access and ease of implementation, four of the six mental health compatible tools identified (i.e., the INTERMED, AMPS, Complexity checklist, and the Escalation tool) rely on the clinician’s perspective. This approach does carry compromises on the aforementioned need for comprehensiveness as well as the usual limitations of inefficiencies and subjectivity of questionnaire-based methodology. On the other hand, as they do not require administrative or other sources of data, they have a low barrier of use. Also, these tools also allow the clinicians to make real-time specific recommendations for further action, making them more attractive with immediate utility, and the ease of use could mean a higher level of utilization and potential impact [59]. Of the two that need administrative and other clinical data, such as hospital admission records or electronic health records. (i.e., the INTERMED and VCAT), the INTERMED does offer a self-assessment version as well, making it more accessible. The VCAT, while harder to use, shows promise as a very powerful tool with mounting evidence of adaptability [30,60]. If fully realized, the VCAT developers envision that it can achieve—from clinical to organizational governance levels—improvements related to assessing whether certain patients meet the service organization’s mandate, optimizing and balancing the client panels of healthcare providers, optimizing the composition and organization of multidisciplinary teams, assessing workload content, enabling the recognition of client needs and the tailoring of individualized care plans, and the monitoring and assessment of changes in individual and population complexity profiles and outcomes over time [30]. It is hypothesized that complexity profiles or scores are quite sensitive to change and possibly able to identify or predict clinical presentations (e.g., the future onset of frailty in aging populations) based on multitudes of relevant information (e.g., relationships, ADLs, disability status, substance use, etc.) [30]. In practical ways, such utility in mental health settings could mean a better matching and prioritizing of patients who require intensive community outreach services, such as Assertive Community Treatment teams, which are always on short supply and the waiting lists are long; so, the proper targeting and management of care provision becomes more paramount [61,62]. Typical barriers to its implementation may include a lack of awareness of such potential, a lack of budget and commitment, and the absence of expertise and support, among others. The current review aims to promote and begins such recognition.

Questions on the cross-cultural adaptability of the reviewed tools are also of note [63,64]. All the measurements were developed and/or field-tested in North America, the United Kingdom, or Europe. Some tools, like the CPM, include specific domains designed to be generalizable in Western medical settings (e.g., PHQ-9 scores [48] the HoNOS [49]). Other tools have domains that leave room for broader interpretation; for example, the INTERMED has been adapted in many different cultural contexts and settings. Further cross-cultural validity research of the instruments is needed, particularly for mental health settings where the cultural interpretation and idioms of stress and illness explanatory models vary considerably from culture to culture [65]. The concept of complexity is known to be deeply and historically embedded in cultural epistemologies of health, wellbeing, and illness. A notable example is the Medicine Wheel, which Indigenous Peoples in North America use to conceptualize the complex interactions between the physical, mental, emotional, and spiritual domains [66]. In an modern example, the “Aaniish Naa Gegii” tool (Ojibwe for “How are you?”, also known as the Aboriginal Children’s Health and Wellbeing Measure (ACHWM)) comprises sixty-two questions across the four aspects of health represented by the Medicine Wheel [67]. The tool is used to assess the health and wellbeing needs of youths and to evaluate program and service delivery. It is important to recognize and study such non-Western approaches to complexity, as they provide valuable and complementary insights into advancing complexity science.

Lastly, from an end-user choice perspective, the current review notes the heterogeneity of the tools, which highlight all the key health system functions—across macro- (system and policy), meso- (organizational), and micro- (clinical) levels. No single tool fits neatly in any particular level, but for those interested in a macro-level tool, the VCAT is the most comprehensive and digitally advanced, developed to serve from an individual level up to population health management (PHM) and quality-of-care improvement at a system level. Furthermore, the VCAT was specifically designed for vulnerable patient populations with complex psychosocial and severe mental health and addiction needs [30]. Its drawback is the reliance on a multi-sourced database. It is most ready for the increasingly computer and large data-driven approach in formalizing a patient’s case conceptualization [68]. The INTERMED, on the other hand, with its ease of use and demonstrated success in being adapted and implemented in many settings and contexts, is useful at a meso- and micro-levels [44]. The INTERMED tool is suited for improving functions relating to organizing integrated healthcare for highly complex populations, including those with chronic psychiatric disorders, substance abuse, pain, and unexplained medical complaints [69]. Of note, both the VCAT and INTERMED were found to be well-suited for use in individuals with SMI at any level given their comprehensive and multi-domain approaches. The MCAM [37] and PCAM tools [38], both derived from the INTERMED tool, can be recommended for micro- and some meso-level use, as they are both best researched for improving primary care interdisciplinary teamwork and referrals to social services. Furthermore, the AMPS [36] tool could be considered for micro-level use, as it was specifically designed for complex populations, focusing on functions related to individualized care planning and outcome assessment.

This current review has several limitations. One is related to the classical problem within psychiatry in terms of a lack of biological markers to determine the diagnostic accuracy of the chief diagnosis, introducing potential complexity in and of itself. There was also a limited number of results returned from our academic search strategy. Trade and gray literature, or modified versions of the current tools, or media reports of successful implementations were not searched for or reviewed, and some suitable tools may have been missed. This was remediated to a degree by our hand search strategy, which reviewed all tools featured in the major review by Schaink and colleagues [1] and another major report from Shukor and colleagues [30]. Another limitation is related to the fact that addictions were not specifically included in the current review search terms, and addiction was not examined as a separate entity. The review does capture substance abuse issues; since addictions are often prevalent as comorbidities in people with mental illness, some tools have addiction-relevant domains or items. Despite efforts to locate and test-run the different tools, information on utility metrics, such as ease of use, time taken to complete, ideal databases needed, adaptability, and availability and accessibility for users, are not robust. Part of this reflects that the developers of the tools did not report such useful information in academic journals.

Based on the current review, some reflections on the future directions of the field are of interest. One relates to the general neglect of the patient perspective. Most of the tools are limited by the lack of input from the patients themselves, which is a major component of any reliable and valid patient complexity tool. This lack also reflects the general state of healthcare, where subjective information is often missing in health data or considerations [70,71]. Some tools integrate the data sources that currently exist (e.g., the VCAT), which means that it might be easier to include existing or future patient self-reported data. Another future improvement may be working towards more of an evidence-based consensus on the definition of complexity. The current divergent definitions may reduce user confidence in adapting such tools. There has been more convergence in the field on agreeing that complexity extends beyond the medical model. Focusing on higher data quality is another desirable direction as the quality of the complexity tool is only as good as the quality, rigor, and validity of the data informing it. Starting to use complexity measures where one can, using the data that are available, paying attention to missing and gaps in the data, and striving for better data may drive a better understanding and appreciation of the different domains of complexity, and this may in turn improve patient care overall.

5. Implications for Behavioral Health

This scoping review provides practical information on the concept and current state of patient complexity measurement for, potentially, a wide range of stakeholders in mental health, from clinicians to health system managers and administrators. The review provides key insights on strengths, uniqueness, and limitations through the analysis and comparisons of a concrete number of readily available tools for those interested or curious about using them. Complexity measurements—starting with diagnostic group identification—could help mental health services to leverage what data they have; be guided purposively to think and collect the key data needed to measure complexity and generate a comprehensive understanding of a patient’s current clinical picture, special considerations, challenges, and needs; and be better guided to meet the services needs and improve care. Faced with long waiting times, inappropriate matching of services, and often fraught doctor–patient relationships, complexity measurements could greatly aid in proper admission to matched services and gauge the duration, level, and intensity of services to provide for and assess the readiness to transition to other services or graduate from more intensive services, creating better transparency, clarity on guidelines, objectivity in decision making, and overall enhancing service use and outcomes in mental healthcare settings. Where traditional disease-focused management fails in terms of paying attention to complexity, the current review promotes the utilization of tools that are biopsychosocially oriented and more accurately understand and serve those in need.

Author Contributions

G.K. and S.L. conceptualized the review. G.K. performed the literature search and analysis and wrote the first draft. All authors contributed to the draft manuscript and provided critical revisions of the work. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

No ethical approval was involved in this scoping review study.

Conflicts of Interest

The authors have no relevant financial or non-financial interests to disclose. While A. Shukor supported the development of one of the tools included in the results, he was not involved in the review process or initial write-up of the review and recommendations.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Supplementary Materials

The following supporting information (PRISMA flow diagram for scoping and systematic reviews) can be downloaded at: https://www.mdpi.com/article/10.3390/diagnostics14121300/s1 [72].

References

1. A.K. Schaink; K. Kuluski; R.F. Lyons; M. Fortin; A.R. Jadad; R. Upshur; W.P. Wodchis A Scoping Review and Thematic Classification of Patient Complexity: Offering a Unifying Framework., 2012, 2,pp. 1-9. DOI: https://doi.org/10.15256/joc.2012.2.15.

2. T.M. Brown; E. Fee Rudolf Carl Virchow: Medical scientist, social reformer, role model., 2006, 96,pp. 2104-2105. DOI: https://doi.org/10.2105/AJPH.2005.078436.

3. M. Drack; W. Apfalter; D. Pouvreau On the making of a system theory of life: Paul A Weiss and Ludwig von Bertalanffy’s conceptual connection., 2007, 82,pp. 349-373. DOI: https://doi.org/10.1086/522810.

4. Beyond reductionism: New perspectives in the life sciences. Arthur Koestler & John Raymond Smythies (eds.)-1969-London, UK: Hutchinson. Boenink, A.D.; Huyse, F.J. Arie Querido (1901–1983): A Dutch psychiatrist: His views on integrated health care., 1997, 43,pp. 551-557.

5. A. Querido The Shaping of Community Mental Health Care., 1968, 114,pp. 293-302. DOI: https://doi.org/10.1192/bjp.114.508.293.

6. G.L. Engel The need for a new medical model: A challenge for biomedicine., 1977, 196,pp. 129-136. DOI: https://doi.org/10.1126/science.847460.

7. C.G. Hudson Theory on systems, complexity and chaos., Routledge: London, UK, 2019,

8. M. Olthof; F. Hasselman; F. Oude Maatman; A.M. Bosman; A. Lichtwarck-Aschoff Complexity theory of psychopathology., 2023, 132,p. 314. DOI: https://doi.org/10.1037/abn0000740.

9. L. Crielaard; M. Nicolaou; A. Sawyer; R. Quax; K. Stronks Understanding the impact of exposure to adverse socioeconomic conditions on chronic stress from a complexity science perspective., 2021, 19, 242. DOI: https://doi.org/10.1186/s12916-021-02106-1.

10. E.I. Fried; D.J. Robinaugh Systems all the way down: Embracing complexity in mental health research., 2020, 18, 205. DOI: https://doi.org/10.1186/s12916-020-01668-w.

11. R.M. Bilder; K.S. Knudsen Creative cognition and systems biology on the edge of chaos., 2014, 5,p. 112333. DOI: https://doi.org/10.3389/fpsyg.2014.01104.

12. A.D. Innes; P.D. Campion; F.E. Griffiths Complex consultations and the ‘edge of chaos’., 2005, 55,pp. 47-52.

13. F. Webster; K. Rice; O. Bhattacharyya; J. Katz; E. Oosenbrug; R. Upshur The mismeasurement of complexity: Provider narratives of patients with complex needs in primary care settings., 2019, 18,p. 107. DOI: https://doi.org/10.1186/s12939-019-1010-6.

14. E.I. Fried Studying Mental Health Problems as Systems, Not Syndromes., 2022, 31,pp. 500-508. DOI: https://doi.org/10.1177/09637214221114089.

15. E. Manning; M. Gagnon The complex patient: A concept clarification., 2017, 19,pp. 13-21. DOI: https://doi.org/10.1111/nhs.12320.

16. M.E. Charlson; P. Pompei; K.L. Ales; C.R. MacKenzie A new method of classifying prognostic comorbidity in longitudinal studies: Development and validation., 1987, 40,pp. 373-383. DOI: https://doi.org/10.1016/0021-9681(87)90171-8.

17. A. Elixhauser; C. Steiner; D.R. Harris; R.M. Coffey Comorbidity measures for use with administrative data., 1998, 36,pp. 8-27. DOI: https://doi.org/10.1097/00005650-199801000-00004.

18. R. Hosar; A. Steinsbekk Identifying individuals with complex and long-term health-care needs using the Johns Hopkins adjusted clinical groups system: A comparison of data from primary and specialist health care., 2023,p. 14034948231166974. DOI: https://doi.org/10.1177/14034948231166974.

19. R.N. Kuo; M.S. Lai Comparison of Rx-defined morbidity groups and diagnosis-based risk adjusters for predicting healthcare costs in Taiwan., 2010, 10, 126. DOI: https://doi.org/10.1186/1472-6963-10-126.

20. M. Porter; M. Kellogg Kaiser Permanente: An Integrated Health Care Experience., 2008, 1,pp. 1-8.

21. G.R. Corazza; C. Klersy; P. Formagnana; M.V. Lenti; D. Padula A consensus for the development of a vector model to assess clinical complexity., 2017, 12,pp. 1313-1318. DOI: https://doi.org/10.1007/s11739-017-1709-6.

22. M.M. Safford; J.J. Allison; C.I. Kiefe Patient Complexity: More Than Comorbidity. The Vector Model of Complexity., 2007, 22,pp. 382-390. DOI: https://doi.org/10.1007/s11606-007-0307-0.

23. N.D. Shippee; N.D. Shah; C.R. May; F.S. Mair; V.M. Montori Cumulative complexity: A functional, patient-centered model of patient complexity can improve research and practice., 2012, 65,pp. 1041-1051. DOI: https://doi.org/10.1016/j.jclinepi.2012.05.005.

24. R.C. Kessler Psychiatric epidemiology: Challenges and opportunities., 2007, 19,pp. 509-521. DOI: https://doi.org/10.1080/09540260701564914.

25. A. Frances The past, present and future of psychiatric diagnosis., 2013, 12,pp. 111-112. DOI: https://doi.org/10.1002/wps.20027.

26. G. Andrews; T. Slade; L. Peters Classification in psychiatry: ICD–10 versus DSM–IV., 1999, 174,pp. 3-5. DOI: https://doi.org/10.1192/bjp.174.1.3.

27. N. Sunderji; K. Powles; M. Tau; J. Hunter; A. Ghavam-Rassoul; D. Kljenak; G. Jansz, University of Toronto: Toronto, ON, Canada, 2018,p. 16.

28. J. Kreyenbuhl; R.W. Buchanan; F.B. Dickerson; L.B. Dixon The Schizophrenia Patient Outcomes Research Team (PORT): Updated Treatment Recommendations 2009., 2010, 36,pp. 94-103. DOI: https://doi.org/10.1093/schbul/sbp130.

29. R. Norman; T. Lecomte; D. Addington; E. Anderson Canadian Treatment Guidelines on Psychosocial Treatment of Schizophrenia in Adults., 2017, 62,pp. 617-623. DOI: https://doi.org/10.1177/0706743717719894.

30. A.R. Shukor; R. Joe; G. Sincraian; N. Klazinga; D.S. Kringos A Multi-sourced Data Analytics Approach to Measuring and Assessing Biopsychosocial Complexity: The Vancouver Community Analytics Tool Complexity Module (VCAT-CM)., 2019, 55,pp. 1326-1343. DOI: https://doi.org/10.1007/s10597-019-00417-5.

31. D.M. Needham; D.C. Scales; A. Laupacis; P.J. Pronovost A systematic review of the Charlson comorbidity index using Canadian administrative databases: A perspective on risk adjustment in critical care research., 2005, 20,pp. 12-19. DOI: https://doi.org/10.1016/j.jcrc.2004.09.007.

32. A.C. Tricco; E. Lillie; W. Zarin; K.K. O’Brien; H. Colquhoun; D. Levac; D. Moher; M.D. Peters; T. Horsley; L. Weeks et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation., 2018, 169,pp. 467-473. DOI: https://doi.org/10.7326/M18-0850.

33. S. Lauriks; M.A.S. de Wit; M.C.A. Buster; T.J.L. Fassaert; R. van Wifferen; N.S. Klazinga The use of the Dutch Self-Sufficiency Matrix (SSM-D) to inform allocation decisions to public mental health care for homeless people., 2014, 50,pp. 870-878. DOI: https://doi.org/10.1007/s10597-014-9707-x.

34. F.J. Huyse; P. de Jonge; J.P.J. Slaets; T. Herzog; A. Lobo; J.S. Lyons; B.C. Opmeer; B. Stein; V. Arolt; N. Balogh et al. COMPRI—An Instrument to Detect Patients with Complex Care Needs: Results from a European Study., 2001, 42,pp. 222-228. DOI: https://doi.org/10.1176/appi.psy.42.3.222.

35. O. Troigros; Y. Béjot; P.M. Rodriguez; F. Shoaib; H. Ellis; D. Wade Measuring complexity in neurological rehabilitation: The Oxford Case Complexity Assessment Measure (OCCAM)., 2014, 28,pp. 499-507. DOI: https://doi.org/10.1177/0269215513505300.

36. Health Connection HealthConnection Clinic Complexity Assessment Tool (AMPS). 2014,. Available online: https://paperzz.com/doc/6925931/healthconnection-clinic-complexity-assessment-tool--amps--an <date-in-citation content-type="access-date" iso-8601-date="2022-02-17">(accessed on 17 February 2022)</date-in-citation>.

37. C.J. Peek; M.A. Baird; E. Coleman Primary care for patient complexity, not only disease., 2009, 27,pp. 287-302. DOI: https://doi.org/10.1037/a0018048.

38. R. Pratt; C. Hibberd; I.M. Cameron; M. Maxwell The Patient Centered Assessment Method (PCAM): Integrating the Social Dimensions of Health into Primary Care., 2015, 5,pp. 110-119. DOI: https://doi.org/10.15256/joc.2015.5.35.

39. X. Busquet-Duran; E.M. Jiménez-Zafra; J.M. Manresa-Domínguez; M. Tura-Poma; O. Bosch-delaRosa; A. Moragas-Roca; M.C.G. Padilla; S.M. Moreno; E. Martínez-Losada; S. Crespo-Ramírez et al. Describing complexity in palliative home care through HexCom: A cross-sectional, multicenter study., 2020, 13,pp. 297-308. DOI: https://doi.org/10.2147/JMDH.S240835.

40. L.J. Carpenter; A. Bautovich; S. Sharma; M. Gatsi; A.P. Wand The development of pathways for responding to patient complexity in a liaison psychiatry setting., 2021, 29,pp. 256-260. DOI: https://doi.org/10.1177/1039856220937648.

41. L. Turner-Stokes; C.U. Krägeloh; R.J. Siegert The patient categorisation tool: Psychometric evaluation of a tool to measure complexity of needs for rehabilitation in a large multicentre dataset from the United Kingdom., 2019, 41,pp. 1101-1109. DOI: https://doi.org/10.1080/09638288.2017.1422033.

42. T.A. Oniki; D. Rodrigues; N. Rahman; S. Patur; P. Briot; D.P. Taylor; A.B. Wilcox; B. Reiss-Brennan; W.H. Cannon Computerization of Mental Health Integration Complexity Scores at Intermountain Healthcare., 2014, 2014,pp. 934-943.

43. R. Ellis; J. Wackwitz; M. Foster Uses of an empirically derived client typology based on level of functioning: Twelve years of the CCAR., 1991, 18,pp. 88-100. DOI: https://doi.org/10.1007/BF02518603.

44. P. de Jonge; F.J. Huyse; J.P.J. Slaets; W. Söllner; F.C. Stiefel Operationalization of biopsychosocial case complexity in general health care: The INTERMED project., 2005, 39,pp. 795-799. DOI: https://doi.org/10.1080/j.1440-1614.2005.01684.x.

45. C. Hudon; M. Bisson; M.F. Dubois; Y. Chiu; M.C. Chouinard; N. Dubuc; N. Elazhary; V. Sabourin; A. Vanasse CONECT-6: A case-finding tool to identify patients with complex health needs., 2021, 21, 157.

46. M.L. Martin-Rosello; M.R. Sanz-Amores; M.R. Salvador-Comino Instruments to evaluate complexity in end-of-life care., 2018, 12,pp. 480-488. DOI: https://doi.org/10.1097/SPC.0000000000000403.

47. J.K. Mount; R.M. Massanari; J. Teachman Patient care complexity as perceived by primary care physicians., 2015, 33,p. 137. DOI: https://doi.org/10.1037/fsh0000122.

48. K. Kroenke; R.L. Spitzer; J.B.W. Williams The PHQ-9., 2001, 16,pp. 606-613. DOI: https://doi.org/10.1046/j.1525-1497.2001.016009606.x.

49. J.K. Wing; A.S. Beevor; R.H. Curtis; S.G.B. Park; J. Hadden; A. Burns Health of the Nation Outcome Scales (HoNOS): Research and development., 1998, 172,pp. 11-18. DOI: https://doi.org/10.1192/bjp.172.1.11.

50. L.L. Zullig; H.E. Whitson; S.N. Hastings; C. Beadles; J. Kravchenko; I. Akushevich; M.L. Maciejewski A Systematic Review of Conceptual Frameworks of Medical Complexity and New Model Development., 2016, 31,pp. 329-337. DOI: https://doi.org/10.1007/s11606-015-3512-2.

51. R. Nardi; G. Scanelli; S. Corrao; I. Iori; G. Mathieu; R. Cataldi Amatrian Co-morbidity does not reflect complexity in internal medicine patients., 2007, 18,pp. 359-368. DOI: https://doi.org/10.1016/j.ejim.2007.05.002.

52. S. Nicolaus; B. Crelier; J.D. Donzé; C.E. Aubert Definition of Patient Complexity in Adults: A Narrative Review., 2022, 12,p. 26335565221081288. DOI: https://doi.org/10.1177/26335565221081288.

53. R.E.G. Upshur; L. Wang; R. Moineddin; J. Nie; C.S. Tracy The complexity score: Towards a clinically-relevant, clinician-friendly measure of patient multi-morbidity., 2012, 2,pp. 799-804.

54. B.J. Turner; L. Cuttler The Complexity of Measuring Clinical Complexity., 2011, 155,pp. 851-852. DOI: https://doi.org/10.7326/0003-4819-155-12-201112200-00009.

55. T. Bodenheimer; A. Ghorob; R. Willard-Grace; K. Grumbach The 10 Building Blocks of High-Performing Primary Care., 2014, 12,pp. 166-171. DOI: https://doi.org/10.1370/afm.1616.

56. P. de Jonge; F.J. Huyse; F.C. Stiefel; J.P.J. Slaets; R.O.B. Gans INTERMED—A Clinical Instrument for Biopsychosocial Assessment., 2001, 42,pp. 106-109. DOI: https://doi.org/10.1176/appi.psy.42.2.106.

57. B. Starfield Measurement of Outcome: A Proposed Scheme., 2005, 83, DOI: https://doi.org/10.1111/j.1468-0009.2005.00429.x.

58. H. Kaneko; A. Hanamoto; S. Yamamoto-Kataoka; Y. Kataoka; T. Aoki; K. Shirai; H. Iso Evaluation of Complexity Measurement Tools for Correlations with Health-Related Outcomes, Health Care Costs and Impacts on Healthcare Providers: A Scoping Review., 2022, 19, 16113. DOI: https://doi.org/10.3390/ijerph192316113.

59. M. Gao; P. Kortum; F. Oswald Psychometric Evaluation of the USE (Usefulness, Satisfaction, and Ease of use) Questionnaire for Reliability and Validity., 2018, 62,pp. 1414-1418. DOI: https://doi.org/10.1177/1541931218621322.

60. R. Joe; G. Sincraian; A.R. Shukor The Vancouver Community Analytics Tool (VCAT): Software Enabling Operationalization of the Building Blocks of High-Performing Primary Care at Community Health Centers in British Columbia, Canada., 2020, 25,p. 20.050. DOI: https://doi.org/10.7812/TPP/20.050.

61. A. Nakhost; F. Sirotich; K.M.F. Pridham; V. Stergiopoulos; A.I. Simpson Coercion in outpatients under community treatment orders: A matched comparison study., 2018, 63,pp. 757-765. DOI: https://doi.org/10.1177/0706743718766053.

62. A. Kassam; M. Beder; J. Maher; S. Sediqzadah; N. Kirwan; M. Ritts; M. Levy; S. Law The Impact of the Covid-19 Pandemic on Assertive Community Treatment Team Functions, Clinical Services, and Observable Outcomes—A Provincial Survey in Ontario, Canada., 2023, 42,pp. 53-65. DOI: https://doi.org/10.7870/cjcmh-2023-016.

63. S.M. Hunt; J. Alonso; D. Bucquet; M. Niero; I. Wiklund; S. McKenna Cross-cultural adaptation of health measures., 1991, 19,pp. 33-44. DOI: https://doi.org/10.1016/0168-8510(91)90072-6.

64. L. Gjersing; J.R. Caplehorn; T. Clausen Cross-cultural adaptation of research instruments: Language, setting, time and statistical considerations., 2010, 10, 13. DOI: https://doi.org/10.1186/1471-2288-10-13.

65. L.J. Kirmayer; D. Bhugra Culture and Mental Illness: Social Context and Explanatory Models., John Wiley & Sons, Ltd.: Hoboken, NJ, USA, 2009,pp. 29-40.

66. Z. Yu; A. Steenbeek; M. Biderman; M. Macdonald; L. Carrier; C. MacDonald Characteristics of Indigenous healing strategies in Canada: A scoping review., 2020, 18,pp. 2512-2555. DOI: https://doi.org/10.11124/JBISRIR-D-19-00385.

67. W. Glauser Indigenous youth co-develop a new way to measure their health., 2020, 192,pp. E327-E328. DOI: https://doi.org/10.1503/cmaj.1095854.

68. J. Burger; D.C. van der Veen; D.J. Robinaugh; R. Quax; H. Riese; R.A. Schoevers; S. Epskamp Bridging the gap between complexity science and clinical practice by formalizing idiographic theories: A computational model of functional analysis., 2020, 18, 99. DOI: https://doi.org/10.1186/s12916-020-01558-1.

69. E. Hoogervorst; P. de Jonge; B. Jelles; F. Huyse; I. Heeres; H.M. van der Ploeg; B.M.J. Uitdehaag; C.H. Polman The INTERMED: A screening instrument to identify multiple sclerosis patients in need of multidisciplinary treatment., 2003, 74,pp. 20-24. DOI: https://doi.org/10.1136/jnnp.74.1.20.

70. F.K. Del Boca; J.A. Noll Truth or consequences: The validity of self-report data in health services research on addictions., 2000, 95,pp. 347-360.

71. J. Shingler; M.E. Van Loon; T.R. Alter; J.C. Bridger The importance of subjective data for public agency performance evaluation., 2008, 68,pp. 1101-1111. DOI: https://doi.org/10.1111/j.1540-6210.2008.00958.x.

72. M.J. Page; J.E. McKenzie; P.M. Bossuyt; I. Boutron; T.C. Hoffmann; C.D. Mulrow; L. Shamseer; J.M. Tetzlaff; E.A. Akl; S.E. Brennan et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews., 2021, 372,p. n71. DOI: https://doi.org/10.1136/bmj.n71.

Tables

Table 2: (a) Tool domains and sources—scoring, outcomes, and recommendations. (b) Tool domains and sources—domains and sources.

Table 3: Utility of tools for practitioners.

Author Affiliation(s):

[1] St. Michael’s Hospital, 17th Floor, 30 Bond Street, Toronto, ON M5B 1W8, Canada

[2] MAP Centre for Urban Health Solutions, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health Toronto, 209 Victoria Street, Toronto, ON M5B 1T8, Canada

[3] Department of Public and Occupational Health, Amsterdam University Medical Center (UMC), Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands

[4] Department of Psychiatry, University of Toronto, St. Michael’s Hospital, 17th Floor, 30 Bond Street, Toronto, ON M5B 1W8, Canada

Author Note(s):

[*] Correspondence: [emailprotected]; Tel.: +1-416-864-6060 (ext. 2415); Fax: +1-416-864-5007

DOI: 10.3390/diagnostics14121300