Article Text
PDF
Abstract
Aim The Papilloplex high-risk human papillomavirus (hrHPV) test (Genefirst, Oxford, UK) is a single tube real-time HPV test which provides multiplex detection and separate identification of 14 hrHPV types. Here, we present the clinical validation of the test in SurePath samples in comparison to a clinically validated reference test, the GP5+/6+Enzyme ImmunoAssay (GP5+/6+EIA) using the VALGENT (VALidation of HPV GENotyping Tests) framework.
Methods Clinical performance was assessed using 998 unselected, cervical screening samples enriched with 297 cytologically abnormal specimens (100 atypical squamous cells of unspecified significance, 100 low-grade squamous intraepithelial lesions, 97 high-grade squamous intraepithelial lesions). Cases were defined as women diagnosed with histologically confirmed cervical intraepithelial neoplasia two or more (≥CIN2, N=119) and controls defined as women with two subsequent negative cytology results (N=834).
Results The Papilloplex HR-HPV test has non-inferior sensitivity for detection of cervical precancer (p=0.0001 for ≥CIN2 and p=0.0005 for ≥CIN3) and non-inferior specificity, compared with GP5+/6+EIA (pni=0.0167)). The assay also showed excellent or good agreement for overall hrHPV and nearly all individual HPV types as compared with GP5+/6+EIA/Luminex.
Conclusion The Papilloplex HR-HPV applied on cervical specimens stored in SurePath medium fulfils the international clinical accuracy criteria for use in cervical cancer screening.
- HPV
- papillomavirus Infections
- polymerase chain reaction
- diagnostic screening programs
Data availability statement
Data are available on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Introduction
Detection of high-risk human papillomavirus (hrHPV) in cervical specimens is becoming the mainstay test used in cervical screening because it is more consistent and effective than cervical cytology1 2 and visual inspection.3 Currently, there are over 200 different assays that detect nucleic acids of HPV, but relatively few are clinically validated.4–6 Only clinically validated hrHPV, that assure an optimal balance in sensitivity and specificity for detecting cervical precancer should be used in screening.7 8
The Papilloplex HR-HPV test (Genefirst, Oxford, UK) is a single tube real-time HPV test which provides multiplex detection and separate identification of 14 hrHPV types 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66 and 68 together with a human control target, all reported individually. The assay is based on multiplex probe amplification (MPA) technology and uses differing melting-curve profiles to allow the differentiation of up to six targets per fluorescence channel of a real-time PCR reaction9 allowing for extended genotyping information.
The assay has previously shown good analytical sensitivity and specificity and reproducibility for the 14 hrHPV types.10 Preliminary clinical performance studies, using archived cervical Thinprep samples, demonstrated similar performance of Papilloplex for detection of cervical intraepithelial neoplasia grade 2 (≥CIN2) to that of two clinically validated HPV tests: RealTime hrHPV test (RealTime) and Hybrid Capture 2 HPV test (HC2), along with high agreement for HR-HPV and individual genotypes when compared with Linear Array (Roche) and Optiplex HPV test (Diamex).11 However, while encouraging, the data described above derived using a convenience sample rather than on a well-defined set of specimens that would enable clinical validation according to internationally recognised criteria.
To address this, the clinical performance of the Papilloplex HR-HPV test was assessed according to the international guidelines of Meijer et al 12 using the VALidation of HPV GENotyping Tests (VALGENT) framework.13 14 Among other things, VALGENT provides a ‘fast track’ to validation by the international guidelines by using disease-enriched sample panels, previously characterised for HPV status and cytopathological information. A total of four iterations of VALGENT have been performed to date. Valgent 2 and 3 (V2–V3) supported assay evaluations in ThinPrep media (Hologic, Bedford, Massachusetts, USA)15–23 while the first and the fourth iteration supported assay evaluations in SurePath (Becton Dickinson (BD), USA).24
Papilloplex is part of the fourth iteration of VALGENT,14 comprising eleven different HPV genotyping assays from eight different manufacturers, using GP5+/6 + PCR enzyme immunoassay (EIA) as comparator14 and SurePath collected cervical screening samples from an organised screening programme. Assays interrogated include those with extended genotyping capability such as Onclarity HPV assay (Becton Dickinson, New Jersey, USA),15 limited genotyping such as Cobas 4800 HPV assay (Roche, Basel, USA),25 Harmonia HPV (Liferiver, Shanghai, China),26 HPV-risk assay (Self-Screen BV, Amsterdam, The Netherlands)18 27 and those with full genotyping abilities—CLART HPV4 assay (Genomica, Madrid, Spain), HPV MassArray assay (Agena Bioscience, Hamburg, Germany), INNO-LiPA Genotyping Extra II test (FujiriBio Europe,Ghent, Belgium), Venus HPV (Liferiver, Shanghai, China) and Papilloplex HPV assay (Genefirst, Oxford, UK).10 21 25 The Papilloplex HPV assay is validated analytically on different extraction systems (including NucliSENS easyMAG and QIAamp DNA Mini Kit) and on a commonly used PCR machine (Applied Biosystems 7500) and provides individual genotyping results with no post-PCR steps.
In the current paper, we further explore clinical performance by determining whether its accuracy to detect cervical precancer is non inferior to a standard, clinically validated comparator HPV test (HPV GP5+/6+PCR EIA).
Materials and methods
Study population and processing
The VALGENT4 panel consists of 998 unselected, consecutive routine samples collected from women aged 30–59 from the Danish cervical cancer screening programme enriched with 297 cytologically abnormal specimens (100 atypical squamous cells of unspecified significance, 100 low-grade squamous intraepithelial lesions, 97 high-grade squamous intraepithelial lesions). A detailed description of the study population, sample collection and processing can be found in Bonde et al.14 Samples were collected at the Molecular Pathology Laboratory, Department of Pathology, Copenhagen University Hospital, Hvidovre and DNA was extracted using MagNA Pure96 system (Roche Diagnostics, Pleasanton, California, USA).14 In brief, four extractions were carried out using 1 mL sample each and the DNA eluates were pooled to get a 400 µL eluate. DNA was extracted in 2017 and an aliquot of DNA was sent to the HPV Research Group, University of Edinburgh, UK, to perform Papilloplex testing and to Delft Diagnostic Laboratories (DDL), Rijswijk, The Netherlands, for GP5+/6+EIA test. Papilloplex testing was carried out in 2018 and GP5+/6+EIA test was subsequently performed and completed on the GP5+/6+amplicons in 2018.
Screening and outcome history (average follow-up of 33 months (min 32, max 35 months)) was retrieved from the Danish Pathology Data Bank (PatoBank). All clinical follow-up was managed according to Danish guidelines, and the result of the VALGENT HPV test evaluations did not affect clinical follow-up recommendations. Samples associated with ≥CIN2 or worse were considered ‘cases’ and used for the evaluation of sensitivity.
In total, 122 cases with precancer were detected in VALGENT-4 which included 83 with CIN3 or worse (≥CIN3). Controls were defined based on two consecutive negative cytology results (at enrolment and at 12–24 months before enrolment) and used for specificity assessment (2x negative for intraepithelial lesion or malignancy (NILM)). In the final analysable cohort (after removing invalids in Papilloplex), there were 834 controls and 119 cases (including 82 ≥CIN3) (figure 1).
Flow chart representing the study population. A total of 119 ≥CIN2 and 82 ≥CIN3 samples were analysed. ASC-US, atypical squamous cells of unspecified significance; CIN2, cervical intraepithelial neoplasia grade 2; HSIL, high-grade squamous intraepithelial lesions; LSIL, low-grade squamous intraepithelial lesions; NILM, negative for intraepithelial lesion or malignancy.
Papilloplex HR-HPV test
Papilloplex HR-HPV test (Genefirst, Oxford, UK) is a CE marked multiplex real time PCR test targeting the L1 region for the detection of 14 hrHPV types (16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66 and 68) individually. The test includes an internal control (IC) targeting ARHGEF11 gene. The targets covered by different fluorophore channels are as follows: FAM- HPV16, 18, 31, 52, 59; HEX-39, 58, 68; ROX- 33, 35, 45, 51, 56, 66; Cy5-IC.
Testing was performed according to the instructions for use (IFU-P0111v3.0, May 2017). In brief, 4 µL of sample DNA was added to a total of 20 µL reaction mix. PCR was performed using the ABI7500 Real Time PCR System and amplification analysis using Applied Biosystems Sequence Detection Software V.1.4–7500 Fast System SDS software. Baseline settings for the assay were modified from the recommended IFU settings on discussion with the manufacturer to relative fluorescence values for different channels set as follows—Cy5-10 000, FAM-40 000, Hex-10 000 and ROX-40 000 (IFU recommendation was Cy5-50, 000, FAM-100 000, Hex-25 000 and ROX-100 000). Melting profile analysis and report creation was carried out using GeneFirst MPA Analysis Software, version 0.5. The PCR protocol takes approximately 2.5 hours. Testing was performed in batches along with kit controls. Baseline and threshold for the real time amplification curves were standardised between runs. Samples were considered valid if a CT value of ≤38 was obtained for IC and positive if the CT value was ≤36 for the HPV channels. Samples were deemed invalid if IC CT value was >38 and no HPV type was detected in the sample. Any samples that were invalid were repeated once.
Standard HPV comparator test GP5+/6+ EIA
GP5+/6+PCR EIA which detects the 14 hrHPV types (16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66 and 68) was used as the standard HPV comparator test to assess and compare the clinical accuracy of the Papilloplex HR-HPV test.12 Testing was performed according to manufacturer’s instructions.
Assessment of clinical performance and non-inferiority to comparator test
HrHPV positivity was based on presence of at least one of the defined 14 hrHPV types. Sensitivity and specificity of Papilloplex were compared with GP5+/6+EIA. The McNemar χ2 test was applied to assess differences between matched proportions (PMcN). A matched non-inferiority (pni) test was performed with a 0.90 relative sensitivity threshold and 0.98 relative specificity threshold. A p value level of 0.05 was set for significance for both pMcN and pni.28 All the analyses were carried out using STATA V.14. Data analysis was performed at the Unit of Cancer Epidemiology (Sciensano, Belgian Cancer Centre) in Brussels, Belgium.
Results
Hpv prevalence
All samples (n=1295) tested were valid for GP5+/6+EIA but 77 (5.9%) samples were invalid for Papilloplex as defined by a cut-off of ≥38 for IC with no HPV type detected (figure 1). HrHPV prevalence with the Papilloplex test in the screening cohort (n=927) was 15.1% (95% CI: 12.9% to 17.6%), while prevalence of the comparator assay, GP5+/6+EIA was 14.3%. Prevalence of individual HPV types detected by Papilloplex test ranged from 0.2% (HPV35) to 2.59% (HPV45) (table 1).
Prevalence of HPV types detected by Papilloplex HR-HPV test in the VALGENT-4 screening cohort
Overall hrHPV and type specific agreement between Papilloplex HR-HPV and GP5+/6+ EIA and GP5+/6+ Luminex, respectively
Table 2 shows the overall concordance for hrHPV (14 hrHPV types) between Papilloplex, HPV-HPV GP5+/6+EIA and the type specific concordance between Papilloplex HPV-HPV GP5+/6+-Luminex). Overall concordance of the two assays for 14 hrHPV types was 92.4% (κ=0.727), when compared with GP5+/6+EIA and 92.1% (0.823) when compared with GP5+/6+ (Luminex) LMNX, indicating excellent agreement. The level of agreement was also excellent for detection of HPV16 (k=0.867), HPV31 (k=0.903), HPV33 (k=0.809), HPV39 (k=1.000), HPV45 (k=0.802) and HPV66 (k=0.811). For all other HPV types the level of agreement was good except HPV59 which had moderate agreement between the two assays.
Type specific concordance between Papilloplex and GP5+/6+PCR-EIA/Luminex
Absolute sensitivity and specificity of Papilloplex HR-HPV test
Papilloplex detected 114 of 119≥CIN2 cases (95.8%, 95% CI: 0.91% to 0.99%) and 80 of the 82≥CIN3 cases (97.6%, 95% CI: 0.92% to 1.00%). Absolute specificity for the 2x NILM cohort for ≤CIN1 was 89.1% (95% CI: 0.87% to 0.91%).
Sensitivity and specificity of Papilloplex HR-HPV test when compared with GP5+/6+-EIA
Relative sensitivity and specificity can be seen in table 3. Papilloplex HR-HPV test has non-inferior sensitivity and specificity to GP5+/6+EIA for detection of ≥CIN2 and ≥CIN3 (p=0.0001 for ≥CIN2 and p=0.0005 for ≥CIN3). Of the 119≥CIN2 cases, Papilloplex detected 114 and GP5+/6+EIA detected 112 cases. Of the 82 ≥CIN3 cases, Papilloplex identified 80 cases while GP5+/6+EIA detected 78 cases. The specificity of Papilloplex was also non-inferior to GP5+/6+EIA (relative specificity: 1.003 (95% CI: 0.98 to 1.02, pni=0.0167)).
Clinical accuracy of the Papilloplex HPV test for detection of CIN2+ and CIN3+ compared with to GP5+/6+EIA
Discussion
Papilloplex HR-HPV assay is a PCR based HPV test that detects 14 hrHPVs individually. Based on a simple and rapid (<3 hours) real-time PCR system, the assay is validated on different extraction systems (including NucliSENS easyMAG and QIAamp DNA Mini Kit) and on a commonly used PCR machine (Applied Biosystems 7500) offering flexibility to laboratories using the assay. The ability to provide individual genotyping within a single closed tube assay offers advantages over other genotyping assays in reducing reaction times, number of steps required for assay setup and in reducing contamination related to multiple reactions. The assay was set up manually in this study and analysis was performed using two softwares. However, work around automation of the setup of PCR reactions and streamlining data analysis which will improve the throughput and ease of use of the assay for large scale screening is currently being undertaken by the manufacturer.
In this study, we had an invalidity rate, where samples were IC negative (CT value >38) and negative for HR-HPV types (CT value >38), of 5.8%. This is substantially above what has recently been reported for HPV-based cervical screening.29 Increasing the IC CT value cut-off to ≥39, ≥40 or ≥41 reduced the invalidity rate to 4.9%, 4.5% and 4.2%. However, some technical reasons might have contributed to the high invalidity rate, including issues with the PCR mix reagents reacting to certain plastics (PCR tubes vs PCR plates and type of plate holders) on the ABI7500, requiring a change of plastics prior to retesting, and altering the optimal baseline settings for the assay (Cy5 10 000, FAM 100 000, Hex 30 000 and ROX 100 000). Use of different baseline settings might have reduced the invalidity rate, but it was standardised in the study to ensure that manufacturer’s instructions were followed. Limitation of the starting material restricted further analysis into the matter.
Here, we report that the Papilloplex HR-HPV in SurePath samples shows near perfect agreement with the comparator test GP5+/6+EIA for all hrHPV types individually. Moreover, our data indicate that Papilloplex test is non-inferior for clinical sensitivity for both ≥CIN2 and≥CIN3 compared with GP5+/6+EIA. Additionally, clinical specificity was measured by analysis of 834 ≤CIN1 (2x NILM) which shows that Papilloplex is non-inferior to GP5+/6+EIA. Absolute specificity based on extended outcome which included women with two consecutive negative cytology results and women with normal colposcopy or ≤CIN1 (N=935) was 81.7% (95% CI 0.79% to 0.84%).
There are limited data on the clinical performance of Papilloplex. A previous study demonstrated similar sensitivity and specificity of Papillopex compared with two other clinically validated assays (RealTime HPV and HC2) on 500 Thinprep samples as based on the analysis of 87≥CIN2 samples within the cohort, the assay had similar sensitivity and specificity to the comparator assays.11
One limitation of our study was that intralaboratory and interlaboratory reproducibility was not assessed within this panel. Another potential limitation is that use of archived specimens may include a disadvantage to tests evaluated years after sample collection. However, we do not observe such time effect in this study nor in previous VALGENT iterations. For example, Oštrbenk et al found non-inferiority between three different assays tested up to 8 years apart.30
In conclusion, the Papilloplex HR-HPV test is a simple, rapid HPV test with high clinical sensitivity and specificity for detection of ≥CIN2 and ≥CIN3. The test fulfils the international clinical accuracy criteria for use in cervical cancer screening on SurePath samples. As a cross-platform PCR based test with full genotyping, Papilloplex HR-HPV has promise for both cervical screening and epidemiological workstreams. Further data on the use of the test in clinical settings and in the widely used ThinPrep PreservCyt medium along with intralaboratory and interlaboratory reproducibility of Papilloplex HR-HPV of the assay is warranted.
Take home messages
Novel single-tube PCR assay for human papillomavirus (HPV) genotyping is clinically validated according to international guidelines.
Multiplex probe amplification technology can be used for HPV genotyping.
There is need for robust clinically validated assays for cervical screening.
Data availability statement
Data are available on reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
The study was approved by the Danish Data Inspection Agency J. No. AHH-2017–024, I-Suite: 05356.
References
Footnotes
Handling editor Runjan Chetty.
Contributors MA coordinates the VALGENT framework. JHB and MA designed the VALGENT 4 protocol. KC and WGVQ co-ordinated testing within the laboratories. RB, EAB and DME performed the laboratory work, KC, RB, LX and MA analyzed the data. KC and RB drafted the manuscript. All authors were involved in writing the paper and had final approval of the submitted version. RB is the guarantor for the work.
Funding This study is investigator led but funded by Genefirst.
Competing interests KC, RB and EAB’s institution has received research funding or gratis consumables to support research from the following commercial entities in the last 3 years: Cepheid, Euroimmun, GeneFirst, SelfScreen, Hiantis, Seegene, Roche, Abbott and Hologic. MA and LX are supported by the RISCC Network (Grant No. 847845), coordinated by the Free University of Amsterdam (The Netherlands), and the SME Instrument Grant GA 806551 (HPV OncoPredict), funded by the Horizon 2020 Programme of DG Research and Innovation, European Commission (Brussels, Belgium). Sciensano, the institute where MA and LX are employed, received support from Becton Dickinson, Genomica SAU, Agena Biotech Gmbh, Zhanghai Biotech (LifeRiver), GeneFirst and FujiRebio for methodological and statistical work as described in the VALGENT protocol (Arbyn et al, J Clin Virol 2016). VALGENT is a researcher induced study protocol where manufacturers can participate under the condition of providing test kits and covering research costs. Researchers do not receive any financial advantage by collaborating in VALGENT.
Provenance and peer review Not commissioned; externally peer reviewed.