Immunohistochemistry And Gene Expression

Published Date: 02 Nov 2017

Professor and Chair, Department of Pathology and Laboratory Medicine

University of Tennessee College of Medicine

930 Madison, 5th Floor, Memphis, TN Tel: 901 448-7020 email: [email protected]

Anand Kulkarni Department of Pathology and Laboratory Medicine, University of Tennessee College of Medicine, Memphis, TN

James P. Grenert Department of Pathology, San Francisco General Hospital and Molecular Diagnostics Laboratory, University of California, San Francisco, CA

Lawrence M. Weiss Clarient, Inc. Aliso Viejo, CA

William M. Rogers Department of Pathology, El Camino Hospital, Mountain View, CA

Oliver S. Kim Pathology, Advocate Good Shepherd Hospital, Barrington, IL

Allen M. Gown PhenoPath Laboratories, Seattle, WA

Federico A. Monzon Baylor College of Medicine, Houston, TX

Meredith Halks-Miller Pathwork Diagnostics, Redwood City, CA

Glenda G. Anderson, 5Degrees Biosciences, San Jose, CA

Michael G. Walker WalkerBioscience, Carlsbad, CA

Raji Pillai Pathwork Diagnostics, Redwood City, CA

W. David Henner Pathwork Diagnostics, Redwood City, CA

Comparative Effectiveness of Immunohistochemistry and Gene Expression Profiling for the Accurate Identification of Metastatic Cancers

Charles R. Handorf, M.D., Ph.D., Anand Kulkarni, M.D., James P. Grenert, M.D., Ph.D., Lawrence M. Weiss, M.D., William M. Rogers, M.D., Oliver S. Kim, M.D., Allen M. Gown, M.D., Federico A. Monzon, M.D., Meredith Halks-Miller, M.D., Glenda G. Anderson, Michael G. Walker, Ph.D, Raji Pillai, Ph.D., and W. David Henner, M.D., Ph.D.

Abstract

Background

Metastatic tumors with an uncertain primary site can be a difficult clinical problem. In tens of thousands of patients every year, no confident diagnosis is ever issued making standard-of-care treatment impossible. Gene expression profiling (GEP) tests currently available to analyze these difficult-to-diagnose tumors have never been compared head-to-head with immunochemistry (IHC).

Methods

This prospectively conducted, blinded, multicenter study compares the diagnostic accuracy of GEP and IHC in identifying the primary site in 157 formalin-fixed paraffin-embedded specimens from metastatic tumors with known primaries, representing the 15 tissues on the GEP test panel. Four pathologists selected from 84 stains in two rounds to render diagnoses. Gene expression profiling was performed using the Pathwork Tissue of Origin test.

Results

Overall, GEP accurately identified 89% of samples, compared to 83% accuracy using IHC (P=0.013). In the subset of 33 poorly differentiated and undifferentiated carcinomas, GEP accuracy exceeded IHC (91% to 71%, P=0.023). In samples for which pathologists rendered their final diagnosis with a single round of stains, both IHC and GEP exceeded 90% accuracy. However, when the diagnosis required a 2nd round, IHC significantly underperformed GEP (67% to 83%, P<0.001).

Conclusions

Gene expression profiling has been validated as accurate in diagnosing the primary site in metastatic tumors. The Pathwork Tissue of Origin test used in this study was significantly more accurate than IHC when used to identify primary site with the most pronounced superiority observed in specimens that required a 2nd round of stains, and in poorly differentiated and undifferentiated metastatic carcinomas.

Over the past decade, multiplexed genomic tests have been developed to improve the diagnosis of cancer and decisions about its treatment. Studies that demonstrate clinical validity and utility are critical to the appropriate integration of these tests into the standard practice of medicine [1] . To date, this level-of-evidence has been met by very few gene expression-based tests1. The blinded, multicenter validation study presented here was designed to objectively compare the diagnostic accuracy of immunohistochemistry and gene expression-based tests for identification of metastatic tumors.

Primary site determination in metastatic cancer can be problematic. While oncologists and pathologists can usually collaborate to determine the tissue of origin based on clinical information, radiography, and histopathology, [2] in some cases tumor classification remains undefined even after blood tests, imaging, endoscopy, and advanced microscopic and molecular evaluation. [3] These cases are known as cancers of unknown primary (CUP), and it is estimated that there are 31,000 per year in the U.S. [4] The actual number, including borderline CUPs classified as known primaries for pragmatic reasons, may be closer to 100,000. [5] - [6] 

Accurate determination of a tumor’s origin is clinically important because it directs evidence-based treatment.2 Most chemotherapeutics and targeted biologic agents are tested, approved, and reimbursed within the context of tissue type. [7] , [8] While many CUP clinico-pathologic subsets resist treatment,2, [9] certain poorly differentiated metastatic tumors are known to respond to specific therapies once their lineage is identified.4 , [10] 

Immunohistochemistry (IHC) is the current standard in pathological evaluation of tumors of unknown origin.2 The availability of new IHC stains has progressively improved the pathologist’s diagnostic acuity and recent efforts to systematize IHC staining have reduced subjectivity.2, [11] , [12] , [13] , [14] , [15] Still, IHC is acknowledged to lack uniform specificity or sensitivity in this setting2,13, [16] and a small fraction of metastatic cancers with occult primaries remain unidentified despite exhaustive IHC investigations. [17] 

Gene expression profiling (GEP) tests have been developed to assist in the evaluation of difficult-to-diagnose tumors. [18] Most GEP tests use microarrays or reverse transcriptase polymerase chain reaction to quantify mRNA or microRNA. Reported accuracy of these new tests, established mainly in known primary and metastatic tumors, has been in the range of 75% to 90%. [19] , [20] , [21] , [22] , [23] Retrospective studies have suggested the ability of GEPs to predict the correct primary site in CUPs. [24] , [25] , [26] In prospective studies of true CUPs, however, the accuracy of any tissue-of-origin test is by definition impossible to gauge. This diagnostic lacuna plus the lack of large direct comparisons between IHC and GEP in metastatic tumors have left clinicians unclear about the relative diagnostic accuracies of these two methodologies and their potential areas of practical interface in the pathology laboratory.

In this blinded, multicenter study, we compared the accuracy of IHC and GEP in identifying the tissue of origin in formalin-fixed paraffin embedded specimens from metastatic tumors with known primaries.

Methods

Study design

The goal of the study was to compare the diagnostic accuracy of GEP to that of current, IHC-guided methods in a setting that resembles current clinical practice with specimens that are representative of the tumors seen in cancers of uncertain primary. The study protocol was developed with practicing pathologist input and tested in a pilot study, published separately. [27] Study design and workflow are illustrated in Figure 1 and summarized here.

Patient Specimens

Retained, paraffin-embedded tumor specimens from 153 patients, obtained under an Institutional Review Board-approved protocol, were sourced from four tissue archives. The principal investigator (C.H.), a board-certified pathologist, screened all specimens for inclusion in the study, based on five criteria: (1) metastatic tumor with a known primary site determined by clinical and imaging information but not IHC; (2) known primary site was one of 15 available on the GEP panel; (3) sufficient tissue for at least 25 5-μm-thick sections; (4) ≥60% non-necrotic tumor tissue; and (5) consistency between reported histology and that found on review by board-certified pathologists. Within these constraints, the specimens were selected to represent the range of uncertain primaries seen in most clinical practices, based upon the experience of the PI. Table 1 summarizes patient and specimen characteristics.

Slide preparation and staining

Each of the 160 specimens admitted to the study was assigned a masked identifier at the University of Tennessee, Memphis, where all H&E and IHC staining was performed in a CLIA-certified Central Pathology Laboratory. Stains could be ordered from a panel of 84, agreed upon in advance by all investigators (Table S1, Supplementary Appendix).

Digital imaging and review of stained slides

Each stained slide was digitized using a Whole Slide Imaging system (ScanScope®XT, Aperio® Technologies, Inc., San Diego, CA) at 0.25µm/pixel resolution and the quality verified by the principal investigator before being posted for the ordering Evaluating Pathologist (EP). Through a web-based interface, each EP was able to order stains, view requested slides as digital images, and record both diagnosis and confidence level after the initial H&E as well as after up to two rounds of stains. When more than one EP ordered a given stain for a given specimen, each was provided the same digitized image.

IHC Study arm

The Evaluating Pathologists (EPs) were blinded to the clinical history and primary site of the samples. The EPs operated independently, with the freedom to use their own professional judgment and experience to select stains to diagnose the primary site. The four EPs were chosen to represent a mix of experience and practice settings. All were board-certified with 3 to 30+ years of experience in academic centers, community practice and/or pathology reference laboratories. Each specimen was independently evaluated by 3 pathologists.

Gene Expression Profiling study arm

Gene expression profiling was performed at Pathwork Diagnostics Laboratory (PWDL) using the Pathwork Tissue of Origin Test (Pathwork Diagnostics Inc., Redwood City, CA). The processing steps described previously [28] included microdissection, RNA extraction, cDNA preparation, and microarray processing. The test reports similarity to the 15 tissues on the test panel as a Similarity Score, ranging from 0 to 100 and summing to 100 across all 15 tissues. As previously established28, samples with a maximum similarity score less than 20 are considered "non-evaluable". Two samples failed by this criterion. A third sample yielded inadequate cDNA. For the remaining 157 samples, the tissue corresponding to the highest Similarity Score was reported as the GEP result.

Statistical analysis

Statistical analyses were performed using SAS 9.3 and R 2.14. For all analyses, two-sided P values less than 0.05 were considered statistically significant.

The primary study endpoint was agreement of IHC with Reference Diagnosis compared with agreement of GEP with Reference Diagnosis. This endpoint was evaluated using conditional logistic regression and a one-to-many comparison (GEP:pathologists). The odds ratio (OR) and 95% confidence interval (CI) for the association between specimen classification accuracy of the two classifiers (GEP, EP/IHC; correct or incorrect) were calculated. The null hypothesis (OR=1) represents equivalent accuracy.

Power calculations indicated that a sample size of 160 specimens would provide 90% power (2-tailed, p-value <0.05) in determining an accuracy difference of at least 15% between the IHC and GEP, assuming (i) an accuracy of 70% for IHC and 85% for GEP in tissue of origin, (ii) a

1:1 matching of GEP to EP and (iii) non-agreements between the call and the truth were not consistent between GEP and EP. The higher observed accuracy, the use of 1:3 matching (GEP compared to 3 pathologists) and the consistency between GEP and EP with respect to correct calls (e.g., in well-differentiated specimens) resulted in greater power.

RESULTS

The study enrolled 160 metastatic tumor samples from 153 patients with a known primary cancer. The 157 specimens evaluable by both IHC and GEP are reported.

Overall Accuracy

Overall, Gene Expression Profiling was in agreement with the Reference Diagnosis in 89.2% of the samples tested, compared to 83.3% agreement with IHC-based methods. This difference is statistically significant (P=0.013) with an odds ratio of 2.9, and 95% confidence interval of 1.2 to 6.7.

Accuracy varied by known primary site (Table 2). Both methods identified the non-Hodgkin’s lymphoma, melanoma and thyroid samples with 100% accuracy, while gastric metastases proved most challenging, achieving less than 30% accuracy.

From the treating physician’s perspective, Positive Predictive Value (PPV) can often be a useful measure of test performance. PPV answers the question, "Given that this test reports X, how certain can we be that the tumor is truly X?" IHC achieved PPV greater than 98% for 6 of 15 tests, compared to 8 of 15 for GEP (Figure 2).

IHC Accuracy

Overall accuracies of the individual EPs were similar (84.1%, 82.2%, 83.2%, and 85.2%). Although the distribution of incorrect calls varied somewhat from EP to EP by primary site, pairwise kappa ranged from 0.76 to 0.83, indicating good-to-excellent concordance between participants [29] .

Slide and stain usage

Across all 157 specimens, EP evaluations required an average of 8.4 stains (range 7 to 10). Of these, more slides were used in first round stain assessment (median of 6) than in second round (median of 4). GEP microdissection and processing consumed fewer than half the number of unstained sections, averaging 3.8 (range 1 to 8). The difference of 4.6 slides per specimen is statistically significant (P<0.001).

Subgroup Analysis

Upon completion of the study, a systematic subgroup analysis was performed to identify characteristics that distinguish "challenging specimens" which might be readily assessed a priori by a pathologist performing the diagnostic work-up. Five candidate subgroups were identified for subgroup analysis: (1) poorly vs. well differentiated tumors; (2) adenocarcinoma vs. non-adenocarcinoma; (3) biopsy site; (4) number of 1st round stains to be ordered; (5) those for which a 2nd round of stains is required.

The results are presented in a forest plot of comparative accuracy (Figure 3). In all 57 poorly differentiated and undifferentiated metastatic tumors, GEP accuracy was 94.1%, significantly greater than IHC accuracy of 79.1% (P=0.016). In the subset of 33 poorly differentiated and undifferentiated carcinomas, GEP accuracy exceeded IHC (91% to 71%, P=0.023). By contrast, the IHC and GEP performed similarly in the well and moderately differentiated tumors. (85.3% and 86.8%, respectively; P=0.52). Within the adenocarcinoma subgroup, GEP accuracy is 86.4% compared to IHC of 79.8%; this difference is statistically significant (P=0.022).

Subgroup analysis by biopsy site proved more problematic due to sample size limitations. In the metastatic tumors biopsied from bone, brain, and lymph node, GEP accuracy appears to be 7-8% higher than IHC, but the difference is not statistically significant due to the small sample counts within each subgroup.

One interesting metric to identify the "challenging specimens" is the number of stains an individual pathologist intends to order for first round assessment--a median of 6 in this study. In 88 specimen-reviews, fewer than 6 stains were ordered and both IHC and GEP performed similarly (93% and 91% respectively, P=0.40). When 6 or more 1st round stains were used, in 142 specimen-reviews, GEP was more accurate than IHC (87.5% and 79.4% respectively, P=0.012).

A final subgroup analysis examined those specimen-reviews requiring a second round of stains. The EPs evaluated 157 samples, for a total of 472 specimen-reviews. (EPC and EPD divided the specimens between them, but inadvertently evaluated one specimen in duplicate) In 283 specimen-reviews the EP made a final diagnosis within the first round of stains ordered with accuracy of 94.3% compared to GEP accuracy of 93.3% on the same specimens (P=0.47). The remaining 189 specimen-reviews required a second round of stains, with IHC accuracy of 67.0% compared to GEP accuracy of 83.1% for these specimens (P<0.001).

DISCUSSION

Few assays have been rigorously validated for use in oncology; fewer still have been rigorously validated for use in diagnosing the primary site of metastatic tumors. This is the first prospectively conducted study to directly compare the diagnostic accuracy of gene expression profiling to IHC methods in a controlled, multicenter, clinical trial setting.

In the metastatic specimens included in the study, GEP was more accurate than the current gold standard of IHC. The comparative accuracy advantage of GEP was greatest in the poorly differentiated and undifferentiated carcinomas. These tumors have lost some or all of the morphologic features that pathologists typically use to identify likely primary tissue type and therefore it is not surprising that these cases would be more challenging to diagnose using morphology-based methods.

These findings confirm the ability of IHC to diagnose the majority of metastatic tumors. The results also suggest that GEP may be a reasonable alternative to additional IHC in the workup of occult primaries when the initial round of staining proves inconclusive. As shown in this study, there is typically no gain in diagnostic accuracy with a second batch of staining.

The goal of the pathologic workup in these situations is to improve overall diagnostic accuracy and allow more patients to receive tissue-specific treatment, which is presumed to improve survival in patients who present with cancers of uncertain origin.2 While this presumption has yet to be fully verified and is actively doubted by some who see CUPs and borderline CUPs as distinct biologic entities unbound from susceptibility to tissue-specific therapy, [30] recent studies do in fact indicate that successful profiling of highly ambiguous metastatic tumors can lead to more specific treatment and improved survival.9, [31] Thus, site-directed therapy remains the standard of care for these patients.

As increasing numbers of targeted anti-cancer agents are introduced, more accurate and efficient classification of the cancer tissue type will become even more important. Identification of tumor lineage and primary site allow the pathologist to then focus the search for mutational markers that dictate targeted therapy in that particular tumor type. 4,5,7,31, [32] , [33] In this emerging diagnostic algorithm, use of GEP instead of IHC on the most challenging cases will likely consume less tumor tissue in the hunt for the primary (in this study, less than half as much). Although not an issue in this trial which required abundant biopsy tissue, conserving biopsy material for biomarker evaluation may ease assessment of poorly differentiated tumors from needle biopsies or other situations in which limited diagnostic material is available.

In this study, we minimized elements of variability in both test methods. For IHC analysis, we centralized specimen handling and slide staining, utilized a set catalogue of antibodies, removed fiscal constraints on stain selection, used a common digitized slide for duplicate stain orders, and blinded pathologists to all clinical information other than gender and gross sample description. The general concordance of IHC results among EPs indicates that we were successful in probing mainly the core cognitive domain of the IHC workup (hypothesis generation, stain selection, slide interpretation) rather than the more variable communicative and site-specific aspects of the diagnostic process. [34] To maximize objectivity in GEP, we employed automatic prediction based on the highest expression score and made no allowance for pathologist score interpretation.

The study has several limitations. We used a broad range of metastatic specimens with known primaries instead of a collection of CUPs because we required a gold standard diagnosis of primary site in order to compare the accuracy of the two methods. As shown in the analyses the relative accuracy of the two methods may depend on the mix of well differentiated and poorly differentiated CUP samples seen in clinical practice. The study sought to mimic the diversity of specimens processed in the typical hospital pathology laboratory, but was restricted to the 15 tissues represented on the GEP test panel. This panel represents approximately 90% of the human solid tumor malignancies seen each year in the United States [35] ,36,37, but notably does not include neuroendocrine tumors which represent 3-5% of all CUPs38. Some specimens are not evaluable by GEP for reasons including RNA quality; 3 specimens were excluded for non-evaluability. However, a sensitivity analysis indicates that the GEP accuracy was still superior to IHC even if all three of these GEP results were imputed to be incorrect and included in the analysis (p=0.047). Only one commercially available GEP test was evaluated in this study. Other mRNA or miRNA-based expression tests which use alternative genes, algorithms and platforms may perform differently.

In summary, the controlled, multicenter study reported here finds that expression profiling with the Pathwork Tissue of Origin Test is more accurate than IHC-based diagnosis in determining the primary site of metastatic tumors. These results can be used to guide the appropriate integration of GEP into clinical practice to increase diagnostic accuracy, conserve biopsy tissue, and improve therapy.

Financial Disclosure:

Supported by a sponsored research grant to the University of Tennessee (CRH and AK) by Pathwork Diagnostics.

Drs. Halks-Miller, Henner and Pillai report holding equity ownership or stock options in Pathwork Diagnostics and being employed by Pathwork, the commercial entity that sponsored the study. Ms. Anderson reports holding equity ownership in Pathwork. Dr. Walker reports receiving consulting fees from Pathwork; Evaluating Pathologists who participated in the study were compensated for their time. Drs. Weiss and Monzon report receiving research funding from competitive GEP providers. 

Acknowledgments:

We thank Paul Courter for editorial assistance, Ashley Ezekiel for technical support, Jing Shi, MD, PhD and Rebecca Panos for statistical support; Catherine Dumur, PhD (Virginia Commonwealth University), Michael Idowa, MD (Virginia Commonwealth University), and George Sandusky, MD (Indiana University) for supplying some of the samples; Eloisa Fuentes, MD and Andrea Pingitore, MD for secondary review of tissue samples; personnel from the University Pathology Group for their assistance in tissue processing and staining; staff at Pathwork Diagnostics Laboratory for processing specimens through the Tissue of Origin test and Shawn Becker, MD for helpful discussions.

Author Contributions:

Study conceived and designed by: CRH, AK, RP, WDH, FAM, LMW, JPG and AG. Study conducted by: CRH, AK, JPG, LMW, WMR, OSK, MHM and RP. Statistical analysis performed by: MW. Paper written by: RP, WDH, CRH and GGA.

Figure 1. Study Design

Metastatic tumor specimens meeting criteria were obtained as archived formalin-fixed, paraffin embedded specimens from four institutions: the University of Tennessee (UT); Virginia Commonwealth University; Indiana University-Purdue University, Indianapolis; and Folio Biosciences. The Central Pathology Study Site (CPSS) at the University of Tennessee (UT) sectioned each block and sent digitized hematoxylin & eosin (H&E) images to four different Evaluating Pathologists (EP) and to the pathology team at Pathwork Diagnostic Laboratory (PWDL). The EPs and PWDL were blinded to the primary site. The EPs ordered stains from a panel of 84 immunohistochemistry and histochemical stains. All staining was performed at CPSS and digitized images of the requested stained slides were provided to only the ordering EP using the Web Interface. The EPs, who were trained on the Whole Slide Imaging system and the web interface before the trial and during a pilot study,26 could issue a Final Diagnosis at any stage in the workup. Coded slides sent to PWDL were analyzed using the FDA-approved Pathwork Tissue of Origin Test—a gene expression test (GEP) that relies on mRNA expression of 2000 genes to predict the primary site from a panel of 15 tumor tissue types.17,18 The accuracy of IHC and GEP were compared using statistical methods described.

Figure 2. Confusion Matrix, GEP and IHC. Known primary site vs. Test prediction / Final Diagnosis. Test Predictions and Final Diagnoses with Positive Predictive Value (PPV) of 98% or higher are highlighted in green.

Abbreviations: bladder (BL), breast (BR), colorectal (CO), gastric (GA), testicular germ cell (GC), kidney (KI), hepatocellular (LI), non-small cell lung (LU), non-Hodgkin’s lymphoma (LY), melanoma (ME), ovarian (OV), pancreas (PA), prostate (PR), thyroid (TH), and sarcoma (SC).

Test Prediction / Final Diagnosis

Known Primary Site

BL

BR

CO

GA

GC

KI

LI

LU

LY

ME

OV

PA

PR

SC

TH

TOTAL

BL

GEP Confusion Matrix

6

 

1

1

 

1

 

 

 

 

 

 

1

 

 

10

BR

 

25

 

 

 

 

 

 

 

 

 

 

 

 

 

25

CO

 

 

25

 

 

 

 

 

 

 

 

 

 

 

 

25

GA

 

 

3

2

 

1

 

 

 

 

 

1

 

 

 

7

GC

 

 

 

 

2

 

 

 

 

 

1

 

 

1

 

4

KI

 

 

 

1

 

12

 

 

 

 

 

 

 

1

 

14

LI

 

 

 

 

 

 

6

 

 

 

 

 

 

 

 

6

LU

 

 

 

 

 

 

 

5

 

 

1

 

 

 

 

6

LY

 

 

 

 

 

 

 

 

3

 

 

 

 

 

 

3

ME

 

 

 

 

 

 

 

 

 

12

 

 

 

 

 

12

OV

 

 

 

 

 

 

 

 

 

 

7

1

 

 

 

8

PA

 

 

 

2

 

 

 

 

 

 

 

3

 

 

 

5

PR

 

 

 

 

 

 

 

 

 

 

 

 

3

 

 

3

SC

 

 

 

 

 

 

 

 

 

 

 

 

 

17

 

17

TH

 

 

 

 

 

 

 

 

 

 

 

 

 

 

12

12

Total

6

25

29

6

2

14

6

5

3

12

9

5

4

19

12

157

PPV%

100

100

86

33

100

86

100

100

100

100

78

60

75

89

100

 

BL

IHC Confusion Matrix

(weighted average of 3 Final Diagnoses per case)

4.3

0.3

0.7

0.7

 

1

1.3

 

 

 

 

1.7

 

 

 

10

BR

 

21

 

0.7

 

 

 

2

 

 

0.3

1

 

 

 

25

CO

 

 

23

1.7

 

 

 

 

 

 

 

0.3

 

 

 

25

GA

 

 

1

2

 

 

 

 

 

 

 

4

 

 

 

7

GC

 

 

0.3

0.7

1.3

 

0.3

0.7

 

0.3

 

0.3

 

 

 

4

KI

 

 

 

 

 

14

 

 

 

 

 

 

 

 

 

14

LI

 

0.7

 

 

 

 

4.7

 

 

 

0.3

 

 

0.3

 

6

LU

 

 

 

 

 

 

 

6

 

 

 

 

 

 

 

6

LY

 

 

 

 

 

 

 

 

3

 

 

 

 

 

 

3

ME

 

 

 

 

 

 

 

 

 

12

 

 

 

 

 

12

OV

 

0.3

 

 

 

0.3

 

 

 

 

6

1

 

 

0.3

8

PA

 

 

0.3

1

 

 

 

 

 

 

 

3.7

 

 

 

5

PR

 

 

 

0.3

 

 

0.3

0.7

 

 

 

 

1.7

 

 

3

SC

 

 

 

 

 

0.3

 

0.3

 

 

 

 

0.3

16

 

17

TH

 

 

 

 

 

 

 

 

 

 

 

 

 

 

12

12

Total

4.3

22

25

7

1.3

16

5.3

11

3

12

6.7

12

2

16

12

157

PPV%

100

94

90

29

100

89

89

55

100

98

90

31

85

98

98

 

Figure 3. Forest plot of Comparative Accuracy, IHC compared to GEP, overall and subgroup analysis. Odds ratios less than one favor IHC, greater than one favor GEP. Statistically significant subgroups are shown in blue; lines indicate the 95% confidence interval.

Table 1: Patient and Tumor Characteristics (n = 160) enrolled in the study

Characteristics

N

Percent (%)

Gender (patients = 153)

Male

82

54%

Female

71

46%

Age

<49 years

36

24%

50-59 years

37

24%

60-69 years

57

37%

>70 years

23

15%

Ethnicity

African-American

41

27%

Caucasian

108

71%

Other

4

2%

Biopsy Site (samples = 160)

Bone

8

5%

Brain

34

21%

Liver

22

14%

Lung

18

11%

Lymph Node

43

27%

Peritoneum/Omentum

11

7%

Soft/Skin Tissue

24

15%

Tumor & Differentiation

Metastatic Specimens

160

100%

Poor/Undifferentiated

51

32%

Well/Moderate

109

68%

Primary Specimens

0

0%

Sample Source

University of Tennessee

81

51%

Indiana University-Purdue University

43

27%

Virginia Commonwealth University

32

20%

Folio Biosciences

4

3%

Table 2. Accuracy of immunohistochemistry (IHC) and gene expression profiling (GEP) stratified by Known Primary Site.

Known Number IHC GEP

Primary Site of specimens Accuracy† Accuracy†

Bladder 10 43% 60%

Breast 25 84% 100%

Colorectal 25 92% 100%

Gastric 7 29% 29%

Testicular Germ Cell 4 33% 50%

Kidney 14 100% 86%

Liver 6 78% 100%

Non-Small Cell Lung 6 100% 83%

Non-Hodgkin’s Lymphoma 3 100% 100%

Melanoma 12 100% 100%

Ovarian 8 75% 88%

Pancreatic 5 73% 60%

Prostate 3 56% 100%

Soft tissue sarcoma 17 94% 100%

Thyroid 12 100% 100%

Overall 157 83.3% 89.2%*

† IHC Accuracy calculated as weighted average agreement with Known Primary Site, includes three IHC Final Diagnoses for each case. GEP Accuracy is calculated as GEP agreement with Known Primary Site, using highest Similarity Score for 157 specimens.

* Difference in overall accuracy is statistically significant, P=0.013; odds ratio (95% confidence interval) 2.9 (1.3 to 6.7).

Supplementary Figures and Tables

Supplementary Table S1. Stains Available to Evaluating Pathologists in Study

1a. List of immunohistochemical stains

No.

IHC Stain

Clone

No.

IHC Stain

Clone

1

Actin, muscle specific

1A4

38

HMB 45

HMB45

2

AE1/AE3

Mouse Monoclonal

39

HMW Keratin

AE3

3

AFP

Rabbit Polyclonal

40

HNF-1

SAB2105022-5OUG

4

hCG

Rabbit Polyclonal

41

Inhibin

Alpha (R1)

5

CA-125

Mouse Monoclonal

42

CK, Oscar

OSCAR

6

Calcitonin

Rabbit Polyclonal

43

Leucocyte Common Antigen

RP2/18

7

Caldesmon

E89

44

Mammaglobin

Rabbit Monoclonal

8

Calretinin

Rabbit Polyclonal

45

Melan-A

MC-7C10

9

CAM 5.2

Cam 5.2

46

MOC-31

MOC-31

10

CD10

270

47

MUC 1

MRQ-17

11

CD117

Rabbit monoclonal

48

MUC 2

MRQ-18

12

CD138

VS38c

49

MUC5AC

MRQ-19

13

CD20

L-26

50

Myeloperoxidase

Rabbit Polyclonal

14

CD3

Rabbit Polyclonal

51

Myogenin

F5D

15

CD30

BerH2

52

Napsin A

Rabbit Polyclonal

16

CD34

QBEnd-10

53

NSE

E27

17

CD43

L-60

54

OCT-4

MRQ-10

18

CD45RO

A6

55

P504S

13H4

19

CD56

1B6

56

p53

BP 53-11

20

CD99

Mouse monoclonal

57

p63

Tap63a

21

CDX2

CDX2-88

58

PAX-2

2E4-11063

22

CEA-polyclonal

Mouse Monoclonal, II-7

59

PAX5

Rabbit Monoclonal SP34

23

Chromogranin

LK2H10(2)

60

PAX-8

Rabbit Polyclonal

24

CK 20

Ks 20.8

61

PLAP

NB10

25

CK 7

K72

62

Progesterone Receptor

Y85

26

CK17

Mouse Monoclonal

63

Prostate Specific Antigen

Rabbit Polyclonal

27

CK19

Mouse Monoclonal

64

RCC

Mouse Monoclonal PN-15

28

CK5/6

D5 & 16B4

65

S-100

4C4.9

29

Pancytokeratin

AE1, AE3, PCK26

66

Synaptophysin

Rabbit, SP11

30

Desmin

NCL-DE-R-11

67

Thrombomodulin

1009

31

E-cadherin

ECH-6

68

Thyroglobulin

1D4

32

EMA

Mouse Monoclonal

69

TTF1

8G7G3/1

33

Estrogen Receptor

6F11

70

Uroplakin

AU-1

34

GCDFP-15

Mouse Monoclonal

71

Villin

CWWB1 (Mouse Monoclonal)

35

GFAP

Polyclonal

72

Vimentin

V9

36

Glypican-3

1G12

73

WT-1

6F-H2 (Mouse Monoclonal)

37

Hep-Par1

OCHIES

1b. List of histochemical stains

No.

Histochemical stain

No.

Histochemical stain

1

Alcian blue- PAS

7

PAS

2

Argentaffin

8

PASD

3

Argyrophil

9

PTAH

4

Colloidal iron stain

10

Reticulum, Gomori’s

5

Elastic, Verhoeff’s

11

Trichrome, Masson’s

6

Mucicarmine

Supplementary Table S2. Association between known primary and biopsy site for all study specimens

Known Primary

Total

Count

Biopsy Site

Bone

Brain

Liver

Lung

Lymph node

Peritoneum/ Omentum

Soft/Skin Tissue

BL

10

1

1

1

6

1

BR

25

2

8

3

2

7

3

CO

25

4

9

2

4

6

GA

7

1

5

1

GC

4

1

1

2

KI

14

3

2

3

3

1

2

LI

6

2

1

2

1

LU

6

4

2

LY

3

2

1

ME

12

1

3

2

2

4

OV

8

2

1

3

2

PA

5

1

4

PR

3

1

1

1

SC

17

1

5

5

1

3

2

TH

12

2

3

1

5

1

Total

157

8

33

22

18

41

11

24

Abbreviations:

bladder (BL), breast (BR), colorectal (CO), gastric (GA), testicular germ cell (GC), kidney (KI), hepatocellular (LI), non-small cell lung (LU), non-Hodgkin’s lymphoma (LY), melanoma (ME), ovarian (OV), pancreas (PA), prostate (PR), thyroid (TH), and sarcoma (SC).

Supplementary Table S3. Immunohistochemistry Stain/Slide Usage Average number of slides requested by each evaluating pathologist. For comparison, GEP consumed an average of 3.4 slides per specimen (range 1-8).

Evaluating Pathologist (EP)

(number of specimens evaluated)

EP-A EP-B EP-C EP-D Overall Average (157) (157) (131) (27) per specimen

Overall 9.6 7.9 7.8 6.7 8.4

Tumor

Differentiation

Poor (n=51) 12.0 9.4 8.3 8.5 9.9

Well (n=106) 8.5 7.2 7.5 5.9 7.6

Workup

IHC Stage

First Round 7.6 6.2 5.6 6.2 6.5

Second Round 2.0 1.7 2.1 0.5 1.9

Final Diagnosis

Accuracy

Incorrect 14.7 10.3 9.9 10.3 11.6

Correct 8.6 7.4 7.3 6.1 7.7

Supplementary Table S4. IHC Accuracy by Workup Stage

Based on percent of initial, intermediate, or final calls in agreement with Reference Diagnosis at each stage for samples evaluated by each pathologist. Accuracy of gene expression test (single result) shown for comparison. To compare EP accuracy after each round of stain assessment, McNemar’s test for one-to-one matched pairs was used.

Evaluating Number After H&E After First After Second

Pathologist of Accuracy IHC Round IHC Round

(EP) Specimens (Confidence) Accuracy Accuracy

Evaluated (Confidence) (Confidence)

EP-A 157 64% (53%) 82% (85%) 84% (90%)

EP-B 157 64% (58%) 85% (82%) 82% (84%)

EP-C 131 66% (36%) 78% (69%) 83% (71%)

EP-D 27 63% (59%) 85% (92%) 85% (92%)

Weighted

Average 64% 82%* 83%**

GEP 157 N/A N/A 89%

* P<0.0001, significant change in 1st round vs H&E accuracy. (McNemar’s test)

** P=0.21, no significant difference in 2nd round versus 1st round IHC accuracy (McNemar’s test)

Supplementary Figure S1. Confusion Matrix, 157 specimens. Results for all tissue calls in all cases are represented in a confusion matrix. Reference Diagnosis shown in rows, Final Diagnosis shown in columns. Each cell shows how many times each EP or GEP made a specific prediction for the given Reference Diagnosis. Within each cell, GEP calls (n=157) are shown in red in the lower right corner and immunohistopathologic calls of EPs are shown in other corners (see Legend) [n=471 specimen-reviews, i.e.157 X 3 full-set reviews by EPs. All correct calls are represented on the diagonal, off-diagonal calls are incorrect.

Legend:

EP-A EP-B

EP-C/D GEPAbbreviations: bladder (BL), breast (BR), colorectal (CO), gastric (GA), testicular germ cell (GC), kidney (KI), hepatocellular (LI), non-small cell lung (LU), non-Hodgkin’s lymphoma (LY), melanoma (ME), ovarian (OV), pancreas (PA), prostate (PR), thyroid (TH), and sarcoma (SC).

Prediction

----------

Ref Dx 

BL

BR

CO

GA

GC

KI

LI

LU

LY

ME

OV

PA

PR

SC

TH

Grand

Total

 

BL

4

3

 

 

 

1

2

 

 

 

1

1

 

 

1

2

 

 

 

 

 

 

2

3

 

 

 

 

 

10

10

6

6 

1

 

1

1 

 

  1

 

 

1

1 

 

 

1

 

 

 

 

 

 

 

 

 

 

1 

 

 

 

 

10

 10

BR

 

 

19

21

 

 

2

 

 

 

 

 

 

 

2

2

 

 

 

 

1

 

1

2

 

 

 

 

 

 

25

25

 

 

23

 25

 

 

 

 

 

 

 

 

 

 

2

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

25

 25

CO

 

 

 

 

23

24

2

1

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

25

25

 

 

 

 

22

 25

2

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1

 

 

 

 

 

 

 

25

 25

GA

 

 

 

 

1

2

2

2

 

 

 

 

 

 

 

 

 

 

 

 

 

 

4

3

 

 

 

 

7

7

 

 

 

 

 

  3

2

 2

 

 

 

 1

 

 

 

 

 

 

 

 

 

 

5

 1

 

 

 

 

7

 7

GC

 

 

 

 

 

 

1

1

1

2

 

 

1

 

 

 

 

 

1

 

 

 

 

1

 

 

 

 

 

 

4

4

 

 

 

 

1

 

 

 

1

2 

 

 

 

 

2

 

 

 

 

 

 

 1

 

 

 

 

 

1 

 

 

4

 4

KI

 

 

 

 

 

 

 

 

 

 

14

14

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

14

14

 

 

 

 

 

 

 

 1

 

 

14

12 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1

 

 

14

 14

LI

 

 

 

 

 

 

 

 

 

 

 

 

5

6

 

 

 

 

 

 

 

 

 

 

 

 

1

 

 

 

6

6

 

 

2

 

 

 

 

 

 

 

 

 

3

 6

 

 

 

 

 

 

1

 

 

 

 

 

 

 

 

 

6

 6

LU

 

 

 

 

 

 

 

 

 

 

 

 

 

 

6

6

 

 

 

 

 

 

 

 

 

 

 

 

 

 

6

6

 

 

 

 

 

 

 

 

 

 

 

 

 

 

6

5 

 

 

 

 

 

 1

 

 

 

 

 

 

 

 

6

 6

LY

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

3

3

 

 

 

 

 

 

 

 

 

 

 

 

3

3

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

3

3 

 

 

 

 

 

 

 

 

 

 

 

 

3

 3

ME

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

12

12

 

 

 

 

 

 

 

 

 

 

12

12

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

12

 12

 

 

 

 

 

 

 

 

 

 

12

 12

OV

 

 

 

1

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

7

5

1

1

 

 

 

 

 

1

8

8

 

 

 

 

 

 

 

 

 

 

1

 

 

 

 

 

 

 

 

 

6

7 

1

 1

 

 

 

 

 

 

8

 8

PA

 

 

 

 

 

1

1

1

 

 

 

 

 

 

 

 

 

 

 

 

 

 

4

3

 

 

 

 

 

 

5

5

 

 

 

 

 

 

1

 2

 

 

 

 

 

 

 

 

 

 

 

 

 

 

4

3 

 

 

 

 

 

 

5

 5

PR

 

 

 

 

 

 

 

1

 

 

 

 

 

1

 

 

 

 

 

 

 

 

 

 

3

1

 

 

 

 

3

3

 

 

 

 

 

 

 

 

 

 

 

 

 

 

2

 

 

 

 

 

 

 

 

 

1

 3

 

 

 

 

3

 3

SC

 

 

 

 

 

 

 

 

 

 

 

1

 

 

 

1

 

 

 

 

 

 

 

 

 

 

17

15

 

 

17

17

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1

 

16

17 

 

 

17

 17

TH

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

12

12

12

12

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

12

 12

12

 12

Grand Total

4

3

19

22

24

28

10

6

1

2

15

16

6

7

9

11

3

3

13

12

8

5

12

13

3

1

18

15

12

13

157

157

6

6 

26

25 

24

 29 

5

 6 

1

 2

16

14 

3

 6

13

 5

3

 3

12

12 

7

 9

11

5 

2

 4

16

 19

12

 12

157

 157