Problems in defining cutoff points of continuous prognostic factors: Example of tumor thickness in primary cutaneous melanoma

doi:10.1016/S0895-4356(97)00155-8

Journal of Clinical Epidemiology

Volume 50, Issue 11, November 1997, Pages 1201-1210

https://doi.org/10.1016/S0895-4356(97)00155-8 Get rights and content

Abstract

Continuous prognostic factors are often categorized by defining optimized cutoff points. One component of criticism of this approach is the problem of multiple testing that leads to an overestimation of the true prognostic impact of the variable. The present study focuses on another crucial point by investigating the dependence of optimized cutoff points on the observed distribution of the continuous variable. The continuous variable investigated was the vertical tumor thickness according to Breslow, which is known to be the most important prognostic factor in primary melanoma. Based on the data of 5093 patients, stratified random samples were drawn out of six artificially created distributions of tumor thickness. For each of these samples, Cox models were calculated to explore optimized cutoff points for tumor thickness together with other prognostic variables. The optimized cutoff points for tumour thickness varied considerably with the underlying distribution. Even in samples from the same distribution, the range of cutoff points was amazingly broad and, for some of the distributions, covered the whole region of possible values. The results of the present study demonstrate that optimized cutoff points are extremely data dependent and vary notably even if prerequisites are constant. Therefore, if the classification of a continuous prognostic factor is necessary, it should not be based on the results of one single study, but on consensus discussions including the findings of several investigations.

References (45)

C Schmoeckel et al.
Low- and high-risk malignant melanoma. I. Evaluation of clinical and histological prognosticators in 585 cases
Eur J Cancer Clin Oncol
(1983)
JE Woods et al.
Is the BANS concept for malignant melanoma valid?
Am J Surg
(1985)
AR Feinstein et al.
A comparison of multivariable mathematical methods for predicting survival. I. Introduction, rationale and general strategy
J Clin Epidemiol
(1990)
SD Walter et al.
A comparison of multivariable mathematical methods for predicting survival. II. Statistical selection of prognostic variables
J Clin Epidemiol
(1990)
CK Wells et al.
A comparison of multivariable mathematical methods for predicting survival. III. Accuracy of predictions in generating and challenge sets
J Clin Epidemiol
(1990)
J Concato et al.
Importance of events per independent variable in proportional hazards analysis. I. Background, goals and general strategy
J Clin Epidemiol
(1995)
P Peduzzi et al.
Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates
J Clin Epidemiol
(1995)
A Breslow
Thickness, cross-sectional areas and depth of invasion in the prognosis of cutaneous melanoma
Ann Surg
(1970)
CM Balch et al.
A multifactorial analysis of melanoma: Prognostic histopathological features comparing Clark's and Breslow's staging methods
Ann Surg
(1978)
FL Meyskens et al.
Cutaneous malignant melanoma (Arizona Cancer Center Experience). I. The natural history and prognostic factors influencing survival in patients with stage I disease
Cancer
(1988)

WH Clark et al.

Model predicting survival in stage I melanoma based on tumor progression

J Natl Cancer Inst

(1989)

CP Karakousis et al.

Tumor thickness and prognosis in clinical stage I malignant melanoma

Cancer

(1989)

CM Balch et al.

An analysis of prognostic factors in 8500 patients with cutaneous melanoma

DL Morton et al.

Multivariate analysis of the relationship between survival and the microstage of primary melanoma by Clark level and Breslow thickness

Cancer

(1993)

C Garbe et al.

Primary cutaneous melanoma: Identification of prognostic groups and estimation of individual prognosis in 5093 patients

Cancer

(1995)

CL Day et al.

The natural break points for primary-tumor thickness in clinical stage I melanoma

N Engl J Med

(1981)

EP van der Esch et al.

Stage I melanoma of the skin: Evaluation of prognosis according to histologic characteristics

Cancer

(1981)

E Marubini et al.

Prognostic factors and risk groups: Some results given by using an algorithm suitable for censored survival data

Stat Med

(1983)

GS Rogers et al.

Effect of anatomical location on prognosis in patients with clinical stage I melanoma

Arch Dermatol

(1983)

K Sondergaard et al.

Survival with primary cutaneous malignant melanoma, evaluated from 2012 cases

Virchows Arch

(1985)

Cited by (40)

Association between preoperative hematologic markers and aggressive behavior in meningiomas
2023, Clinical Neurology and Neurosurgery
Meningiomas have varying degrees of aggressive behavior. Some systemic hematologic makers are associated with malignancy, but their value in predicting aggressive meningioma behavior is not fully understood.
To evaluate the association between preoperative markers such as neutrophil-lymphocyte ratio (NLR), neutrophil-monocyte ratio (NMR), monocyte-lymphocyte ratio (MLR), platelet-lymphocyte ratio (PLR), and prognostic nutritional index (PNI), and diagnostic and prognostic factors including WHO grade, proliferation index, presence of edema on preoperative MRI, and tumor recurrence.
A retrospective review of patients treated between 2000 and 2019 with a preoperative complete blood count (CBC) differential lab draw before intracranial meningioma resection was conducted. All preoperative steroid dosages were converted to dexamethasone equivalents. Primary outcomes included presence/absence of perilesional edema, WHO grade, Ki-67/MIB-index, and recurrence. Univariate and multivariable regression analyses were conducted.
A total of 209 meningioma patients were included. Of these, 143 (68 %) were WHO grade I, 61 (29 %) grade II and 5 (2 %) were grade III. Recurrence was reported in 19 (9.1 %) tumors. No hematologic markers were associated with recurrence. In separate multivariable logistic analyses, no biomarkers were associated with perilesional edema or WHO grade. MLR was associated with higher MIB-index (p = 0.018, OR 6.57, 95 % CI 1.37–30.91).
Most hematologic markers were not associated with meningioma invasiveness, grade, proliferative index, or aggressiveness. Preoperative MLR was associated with high proliferation index in patients undergoing surgery for intracranial meningioma. Higher MLR could be a surrogate for meningioma proliferation and has potential to be used as an adjunct for risk-stratifying meningiomas.
Subgroup analyses in randomized controlled trials frequently categorized continuous subgroup information
2022, Journal of Clinical Epidemiology
Citation Excerpt :
Instead, cutpoints could be chosen in a data-dependent way e.g., by using the median of the observed distribution of the subgroup-defining variable. However, this complicates the comparison of results between different studies [27]. Approaches that avoid using cutpoints may be more powerful [28].
To investigate how subgroup analyses of published Randomized Controlled Trials (RCTs) are performed when subgroups are created from continuous variables.
We carried out a review of RCTs published in 2016–2021 that included subgroup analyses. Information was extracted on whether any of the subgroups were based on continuous variables and, if so, how they were analyzed.
Out of 428 reviewed papers, 258 (60.4%) reported RCTs with a subgroup analysis. Of these, 178/258 (69%) had at least one subgroup formed from a continuous variable and 14/258 (5.4%) were unclear. The vast majority (169/178, 94.9%) dichotomized the continuous variable and treated the subgroup as categorical. The most common way of dichotomizing was using a pre-specified cutpoint (129/169, 76.3%), followed by a data-driven cutpoint (26/169, 15.4%), such as the median.
It is common for subgroup analyses to use continuous variables to define subgroups. The vast majority dichotomize the continuous variable and, consequently, may lose substantial amounts of statistical information (equivalent to reducing the sample size by at least a third). More advanced methods that can improve efficiency, through optimally choosing cutpoints or directly using the continuous information, are rarely used.
Serum levels of soluble programmed death-ligand 1 (sPD-L1): A possible biomarker in predicting post-treatment outcomes in patients with early hepatocellular carcinoma
2021, International Immunopharmacology
Citation Excerpt :
The cut-off identification is of secondary importance (due to its associated problems), but it will yield its usefulness in the future. This is due to the fact that the classification of a continuous variable (such as sPD-L1), should not be based on the results of a single study but on a consensus of different studies [25]. Our current research brings forward multiple novelties in the field.
There have been great advances in hepatocellular carcinoma management over the last years. However, there are still no prognostic biomarkers that can identify patients who will benefit the most from curative treatments. We aimed to investigate whether sPD-L1 levels measured before curative treatment is a prognostic biomarker of survival in patients with HCC.
HCC patients from a prospectively collected database were selected and soluble programmed death-ligand 1 (sPD-L1) levels were determined. The association of sPD-L1 levels and overall survival (OS) and disease-free survival (DFS) was assessed.
One hundred twenty-one patients with HCC were included. The best cut-off value of sPD-L1 for both DFS and OS was 96 pg/mL. Patients with a high sPD-L1 value (>96 pg/mL) had a shorter disease free survival and OS (hazard ratio 5.42, 95% confidence interval 2.28–12.91, p < 0.001, and hazard ratio 9.67, 95% confidence interval 4.33–21.59, p < 0.001). High sPD-L1 levels were associated with mortality independently from other known survival predictors. We found a positive correlation between sPD-L1 and PD-L1 expression in cancer cells (p = 0.01). In 16 out of 38 patients, sPD-L1 levels decreased from baseline value on week 6 after treatment and in 22 out of 38 patients, sPD-L1 levels increased from the baseline value. However, fluctuations of sPD-L1 in time had no influence on survival (p = 0.148).
We conclude that a high sPD-L1 level is a biomarker for a poor outcome in HCC. The predictive value of sPD-L1 levels for a successful anti-PD1/PD-L1 therapy should be investigated in the future.
Understanding the effects of dichotomization of continuous outcomes on geostatistical inference
2021, Spatial Statistics
Citation Excerpt :
Furthermore, in cases where the relationship is non-linear or non-monotonic, dichotomization or categorization into few classes may make that undetectable (Royston et al., 2006; Bennette and Vickers, 2012). All these issues are further exacerbated when the choice of specific cut-offs is inconsistent or, in some cases, even arbitrary (Harrell, 2004; Buettner et al., 1997; MacCallum et al., 2002). For example, cut-offs may vary within and across studies due to differences in the sample populations from which they are derived or due to changes in how they are defined according to clinical practice and operational policy.
Diagnosis is often based on the exceedance or not of continuous health indicators of a predefined cut-off value, so as to classify patients into positives and negatives for the disease under investigation. In this paper, we investigate the effects of dichotomization of spatially-referenced continuous outcome variables on geostatistical inference. Although this issue has been extensively studied in other fields, dichotomization is still a common practice in epidemiological studies. Furthermore, the effects of this practice in the context of prevalence mapping have not been fully understood. Here, we demonstrate how spatial correlation affects the loss of information due to dichotomization, how linear geostatistical models can be used to map disease prevalence and thus avoid dichotomization, and finally, how dichotomization affects our predictive inference on prevalence. To pursue these objectives, we develop a metric, based on the composite likelihood, which can be used to quantify the potential loss of information after dichotomization without requiring the fitting of Binomial geostatistical models. Through a simulation study and two applications on disease mapping in Africa, we show that, as thresholds used for dichotomization move further away from the mean of the underlying process, the performance of binomial geostatistical models deteriorates substantially. We also find that dichotomization can lead to the loss of fine scale features of disease prevalence and increased uncertainty in the parameter estimates, especially in the presence of a large noise to signal ratio. These findings strongly support the conclusions from previous studies that dichotomization should be always avoided whenever feasible.
Validation, clinical utility and limitations of the Amsterdam-Oxford model for primary sclerosing cholangitis
2019, Journal of Hepatology
Citation Excerpt :
The use of thresholds in clinical practice is widespread but has several limitations. Categorization results in loss of predictive information and thresholds are difficult to generalize to other populations.34,35 This study shows that with dichotomous application of the AOM, patients may switch in risk category during follow-up.
Recently the Amsterdam-Oxford model (AOM) was introduced as a prognostic model to assess the risk of death and/or liver transplantation (LT) in primary sclerosing cholangitis (PSC). We aimed to validate and assess the utility of the AOM.
Clinical and laboratory data were collected from the time of PSC diagnosis until the last visit or time of LT or death. The AOM was calculated at yearly intervals following PSC diagnosis. Discriminatory performance was assessed by calculation of the C-statistic and prediction accuracy by comparing the predicted survival with the observed survival in Kaplan-Meier estimates. A grid search was performed to identify the most discriminatory AOM threshold.
A total of 534 patients with PSC and a mean (SD) age of 39.2 (13.1) years were included. The diagnosis was large duct PSC in 466 (87%), PSC with features of autoimmune hepatitis in 52 (10%) and small-duct PSC in 16 (3%). During the median (IQR) follow-up of 7.8 (4.0–12.6) years, 167 patients underwent LT and 65 died. The median LT-free survival was 13.2 (11.8–14.7) years. The C-statistic of the AOM ranged from 0.67 at baseline to 0.75 at 5 years of follow-up. The difference between the predicted and observed survival ranged from −1.6% at 1 year to + 3.9% at 5 years of follow-up. Patients that developed AOM scores >2.0 were at significant risk of LT or death (time-dependent hazard ratio 4.09; 95% CI 2.99–5.61).
In this large cohort of patients with PSC, the AOM showed an adequate discriminative performance and good prediction accuracy at PSC diagnosis and during follow-up. This study further validates the AOM as a valuable risk stratification tool in PSC and extends its utility.
In our study we assessed whether the Amsterdam-Oxford model (AOM) is able to correctly estimate the risk of liver transplantation or death in patients with primary sclerosing cholangitis (PSC). This model uses 7 objective and readily available variables to estimate prognosis for individual patients at the time of PSC diagnosis. The AOM may aid in patient counselling and timing of diagnostic procedures or therapeutic interventions for complications of liver disease. We confirm that the model works well at PSC diagnosis, but also when the AOM is recalculated at different timepoints during follow-up, greatly improving the applicability of the model in clinical practice and for individual patients.
Melanoma epidemiology and trends
2009, Clinics in Dermatology
Citation Excerpt :
Melanoma can then arise in body regions with intermittent sun exposure. Interestingly, the body distribution of melanoma closely matches with the distribution of melanocytic nevi.44,77 This hypothesis likewise explains why melanoma can develop in young adults.
Rising incidence rates of cutaneous melanoma have been observed during the last four decades in white populations worldwide. The cancer statistics in the United States have revealed 6 cases per 100,000 and year at the beginning of the 1970s and 18 cases per 100,000 inhabitants and year at the beginning of 2000, demonstrating a threefold increase in incidence rates. Incidence rates in central Europe increased in the same time period from 3 to 4 cases to 10 to 15 cases per 100,000 inhabitants and year, which is very similar to the increase in the United States. Cohort studies from several countries indicate that the trend of increasing incidence rates will continue in the future for at least the next 2 decades; thus, an additional doubling of incidence rates is expected. The highest incidence rates have been reported from Australia and New Zealand, from 40 to 60 cases per 100,000 inhabitants and year. Mortality rates likewise slightly increased in the United States and in Europe during the 1970s and 1980s. In the 1990s, however, a leveling off of mortality rates was observed in many countries. Simultaneously, a clear decrease of Breslow tumor thickness was reported in the United States and European countries. This development indicates improved early recognition of cutaneous melanoma, which is presently the main factor for a more favorable prognosis.

View all citing articles on Scopus

^☆: This study was supported by the German Dermatological Society and by the Verband der Lebensversicherungsunternehmen e.V.

View full text

Original articleProblems in defining cutoff points of continuous prognostic factors: Example of tumor thickness in primary cutaneous melanoma☆

Abstract

Eur J Cancer Clin Oncol

Am J Surg

J Clin Epidemiol

J Clin Epidemiol

J Clin Epidemiol

J Clin Epidemiol

J Clin Epidemiol

Thickness, cross-sectional areas and depth of invasion in the prognosis of cutaneous melanoma

Ann Surg

A multifactorial analysis of melanoma: Prognostic histopathological features comparing Clark's and Breslow's staging methods

Ann Surg

Cutaneous malignant melanoma (Arizona Cancer Center Experience). I. The natural history and prognostic factors influencing survival in patients with stage I disease

Cancer

Model predicting survival in stage I melanoma based on tumor progression

J Natl Cancer Inst

Tumor thickness and prognosis in clinical stage I malignant melanoma

Cancer

An analysis of prognostic factors in 8500 patients with cutaneous melanoma

Multivariate analysis of the relationship between survival and the microstage of primary melanoma by Clark level and Breslow thickness

Cancer

Primary cutaneous melanoma: Identification of prognostic groups and estimation of individual prognosis in 5093 patients

Cancer

The natural break points for primary-tumor thickness in clinical stage I melanoma

N Engl J Med

Stage I melanoma of the skin: Evaluation of prognosis according to histologic characteristics

Cancer

Prognostic factors and risk groups: Some results given by using an algorithm suitable for censored survival data

Stat Med

Effect of anatomical location on prognosis in patients with clinical stage I melanoma

Arch Dermatol

Survival with primary cutaneous malignant melanoma, evaluated from 2012 cases

Virchows Arch

Original article
Problems in defining cutoff points of continuous prognostic factors: Example of tumor thickness in primary cutaneous melanoma☆