A Comparison of the Intercept Oral Specimen Collection Device (IOSCD)® to

 

Laboratory Urinalysis among Baltimore City Treatment Clients

 

 

 

 

 

 

 

 

 

 

 

 

George S. Yacoubian, Jr., Ph.D.[1]

Eric D. Wish, Ph.D.

 

A Comparison of the Intercept Oral Specimen Collection Device (IOSCD)® to

 

Laboratory Urinalysis among Baltimore City Treatment Clients

International Journal of Drug Testing       Volume 3
http://www.criminology.fsu.edu/journal/Yacoubian-Wish.htm

 

 

Abstract

To date, few field studies have compared oral fluid (OF) analysis to urinalysis.  In the current pilot study, urine and OF specimens were collected from 163 adult intensive outpatient and methadone maintenance treatment clients in Baltimore City.  With laboratory urinalysis as the reference standard, the Intercept Oral Specimen Collection Device (IOSCD)® was 100% sensitive and 100% specific for benzodiazepines, 82% sensitive and 96% specific for cocaine, 100% sensitive and 92% specific for methadone, and 83% sensitive and 99% specific for opiates.  For marijuana, the sensitivity was 39% and the specificity was 93%.  After considering issues of personal invasiveness, ease of collection, cost, accuracy, and any consequences associated with false-positive or false-negative results, drug treatment personnel and local policymakers should determine if these rates of agreement are high enough to consider replacing or augmenting urinalysis with OF analysis.

 

Keywords:       Drug surveillance, Kappa, OF analysis, treatment clients, urinalysis

Introduction

Research subjects under criminal justice supervision underreport their recent use of illicit drugs (Yacoubian 2000; Wish et al. 1997; Harrison 1995; Mieczkowski et al. 1991).  To address this underreporting, biological specimens are often collected as objective measures of recent drug use.  While urinalysis is generally recognized as the industry standard, a variety of drug testing technologies have been developed within the past few years.  One of these innovations – oral fluid (OF) testing – may offer researchers an acceptable alternative to urinalysis because the detection period for both measures is approximately 48 hours (Cone 1993).  To date, however, few field tests have compared OF analysis to urinalysis (Wish & Yacoubian 2002; Yacoubian et al. 2001; Niedbala et al. 2001b, a; Moore et al. 2001; Speckl et al. 1999).    

To assess the accuracy of opiate detection, Speckl et al. (1999) collected 130 urine and OF specimens from patients participating in drug withdrawal therapy.  The concordance of OF analysis to urinalysis for opiate detection was 98% (Speckl et al. 1999).  Yacoubian et al. (2001) collected urine and OF specimens from 114 adult male arrestees in Anne Arundel, Charles, and Prince George’s Counties, Maryland, between April and July 2000.  With urinalysis as the reference standard, the Intercept Oral Specimen Collection Device (IOSCD) was 100% sensitive and 99% specific for cocaine metabolite and 88% sensitive and 100% specific for opiates (Yacoubian et al. 2001).  Niedbala et al. (2001b) collected urine and OF specimens from 149 cocaine-experienced subjects participating in research at three facilities across the United States.  The IOSCD was 95% sensitive and 88% specific (Niedbala et al., 2001b).  Most recently, Wish and Yacoubian (2002) collected urine and OF specimens from 284 adult arrestees in Baltimore City.  With laboratory urinalysis as the reference standard, the IOSCD was 95% sensitive and 98% specific for cocaine metabolite and 90% sensitive and 99% specific for opiates.  For marijuana, the sensitivity was 56%, and the specificity was 99% (Wish & Yacoubian 2002).

Urinalysis is used routinely with treatment clients to monitor illicit drug use.  Depending on the treatment program, a positive urine test may mean an increased treatment regimen or potential sanctioning by the criminal justice system.  Given these consequences, the need to explore objective measures of illicit drug use in treatment settings is critical.  The current pilot study collected urine and OF specimens from a sample of 163 adult treatment clients in Baltimore City.  Given the serious drug use problem within this population (Wish et al. 1997), Baltimore City offered a unique opportunity to compare OF testing to urinalysis in an urban treatment client population.  Two research questions are addressed. 

 

1)      Would treatments clients who provided a urine specimen also provide an OF specimen? 

2)      How did results from the IOSCD compare to those from urinalysis?   

 

Methods

The Baltimore Substance Abuse System (BSAS) funds and manages all drug treatment programs in Baltimore City.  To monitor illicit drug use, a mandatory urine specimen is collected from treatment clients twice a month.  Two intensive outpatient facilities and one methadone maintenance facility were selected for inclusion in the study.  These facilities were selected because of the large volume of clients served and their willingness to participate. 

Clients were approached for inclusion in the study as they reported to the facilities for treatment.  After the mandatory urine specimen was collected, facility staff members requested an OF specimen.  Clients were told that the purpose of the study was to compare OF analysis to urinalysis and that their participation was completely voluntary and confidential.  If the clients agreed to participate, they initialed a consent form.  Facility staff members then signed the consent form to indicate that they had read the consent form to the treatment clients and had informed them of all of their rights.    Demographic information (age, sex, and race) was collected from official records and recorded on a code sheet.  The Institutional Review Board (IRB) for Human Subjects at the University of Maryland approved the consent form and research protocol.

Urine specimens were shipped directly by the facility to their standard laboratory[2] and screened by the Enzyme Multiplied Immunoassay Test (EMIT) for amphetamines, benzodiazepines, marijuana, metabolite (crack and powder) cocaine, methadone, and opiates.  Cutoff levels were 300 ng/ml for amphetamines, 100 ng/ml for marijuana, and 300 ng/ml for benzodiazepines, cocaine, methadone, and opiates.  The OF specimens were shipped to a different laboratory[3] and screened by the Enzyme Linked Immunosorbent Assay (ELISA) for the same six drugs at comparable cutoff levels.  Because the standard BSAS procedures do not involve confirmation of urinalysis results, no confirmation tests were performed in the current study with either urinalysis or OF analysis.  While we recognize the lack of confirmation procedures precludes any general conclusions about the overall accuracy of either the EMIT or ELISA, and thereby a comparison between the two, our primary objective in the current study was assessing the technologies as they are used with Baltimore City treatment clients.  The EMIT and ELISA yielded positive or negative results for each of the six drugs.  A unique identification number linked the two specimens. 

To confirm the comparability of cutoff levels between the IOSCD and the laboratory urinalysis, clinical studies have been conducted for accuracy (OraSure Technologies 2003).  For each of the six drugs under scrutiny, the manufacturer made comparisons between urinalysis and the IOSCD at the respective cutoff levels identified above.  Percent agreements between the two measures were greater than 97% for each of the drugs (OraSure Technologies 2003).  These data suggest that the urinalysis and IOSCD cutoff levels were comparable for amphetamines, benzodiazepines, cocaine, methadone, opiates, and marijuana.

 

Results

Data analysis was accomplished in three phases.[4]  First, participation rates were calculated.  Second, descriptive statistics were computed.  Third, results from the IOSCD were compared to those from urinalysis.  Kappa statistics were computed and tests for specificity and sensitivity were conducted using urinalysis as the reference standard.  Because only one amphetamine-positive was detected by urinalysis, analyses were conducted only with benzodiazepines, cocaine, marijuana, methadone, and opiates. 

Participation rates

Of the 192 treatment clients who provided the mandatory urine specimen, 88% (n=169) provided an OF specimen.  Unfortunately, three OF specimens were of insufficient quantity to analyze and three urinalysis results were misplaced by one of the treatment facilities.  Statistical analyses for the current study were thus conducted on the 163 clients for whom both OF analysis and urinalysis results were available.

Description of sample

As shown in Table 1, a majority of the sample was male (62%), black (76%), and over the age of 39 (59%).  The mean age of the sample was approximately 43 years old.  A majority (60%) of the sample were outpatient residential clients.

Table 1.  Demographic Characteristics (n=163)

 

Comparison of OF analysis to urinalysis

The established method for evaluating the accuracy of a measurement tool is to determine its sensitivity and specificity compared to a reference standard (Lilienfield et al. 1994).  In the current study, urinalysis was the reference standard and the IOSCD was the tool being evaluated.  Sensitivity is the percentage of all persons with the condition (a positive OF test) who were correctly identified as having the condition based on the reference standard (urinalysis).  Specificity is the percentage of all persons who did not have the condition (according to the reference standard) who were correctly identified by the test as being free of the condition.  The higher the sensitivity and specificity, the greater the accuracy of the tool.  

As shown in Table 2, the results from urinalysis and OF analysis were within four percentage points for all of the drugs.  With laboratory urinalysis as the reference standard, the IOSCD was 100% sensitive and 92% specific for methadone, 82% sensitive and 96% specific for cocaine, 83% sensitive and 99% specific for opiates, and 100% sensitive and 100% specific for benzodiazepines.  For marijuana, the IOSCD was 39% sensitive and 93% specific.  Kappa, which measures the agreement between the evaluations of two raters (urinalysis and OF analysis) when both are rating the same object (recent drug use), is considered an appropriate measure of agreement when the time periods covered by both are similar (Magura & Kang 1996). The kappa statistics ranged from a low of .30 for marijuana to a high of 1.00 for benzodiazepines.  With the exception of marijuana, the kappa statistics suggest moderate to strong agreement between the urinalysis and OF analysis results. 

 

Table 2. Estimates of Drug Use from Biological Specimens (n=163)

 

 

As shown in Table 3, the overall agreement between urinalysis and OF analysis ranged from a low of 89% (marijuana) to a high of 100% (benzodiazepines).  Most of the errors occurred with marijuana (18) and cocaine (12).  For example, there were seven clients who were cocaine-positive by urinalysis, but cocaine-negative by OF analysis.  These seven clients would thus have avoided detection had OF analysis alone been used.  There were also five clients who were cocaine-negative by urinalysis, but cocaine-positive by OF analysis.  These five clients would have avoided detection for cocaine had urinalysis alone been used. 

Table 3. Summary of Agreement Between Urinalysis (UA) and IOSCD Analysis, by

Drug (n=163)

 

 

Discussion

Urine and OF specimens were collected from 163 adult treatment clients in Baltimore City between February and March 2002.  A large majority (88%) of the clients who provided a mandatory urine specimen were willing to provide the voluntary OF specimen.  While urinalysis and OF analysis detected almost identical rates of recent benzodiazepine, cocaine, methadone, and opiate use, there were a number of clients with false-positive and false-negative results. 

Three limitations should be noted.  First, the current data come solely from adult intensive outpatient and methadone maintenance treatment clients in a high cocaine- and heroin-using jurisdiction.  Because of the chronic and serious drug use that characterizes this population (Wish et al. 1997), our findings may not apply to other populations (e.g., juvenile treatment clients).  We recommend replicating the current study with additional drug-using populations to assess generalizability. 

Second, the relatively small sample size, and the small number of subjects in the marijuana- and benzodiazepine-positive cells, suggests that our results should be viewed cautiously.  While researchers typically do not report findings with cells less than 25,  providing the kappa statistics and the sensitivity and specificity coefficients was warranted because of the pilot nature of the study.  That said, future research should involve larger samples sizes to increase our confidence in the results.

Third, the current study was only a comparison between the urinalysis EMIT and the IOSCD ELISA.  Because no confirmation techniques [e.g., gas chromatography/mass spectrometry (GC/MS)] were used, the current study is not a comparison of urinalysis to OF testing per se.  It is important to stress, however, that the purpose of the current study was to compare the IOSCD to urinalysis as it is currently used by Baltimore City (and presumably other) treatment facilities.  Because the programs studied do not confirm their urinalysis results and because we had a limited budget for this pilot study, we were unable to confirm either of the two specimens by GC/MS.  Future research should confirm both urine and OF specimens in order to assess the accuracy of the two tests.  This caveat aside, the strength of our study is its ability to show how drug test results would have changed had the programs used the IOSCD in lieu of urinalysis to screen their clients.

In addition to these empirical findings, anecdotal reports from both interview staff and treatment clients indicated that the IOSCD was preferable to urine collection because of its ease of use and storage and its minimal personal invasiveness.  Moreover, the expenditures of urinalysis and the IOSCD are comparable – both cost approximately $10 per specimen, depending on volume.

The number of clients with false-positive and false-negative results may be a concern for treatment providers and policymakers.  There are serious ramifications for false-positive and false-negative results.  False-positive results may result in a criminal justice system violation or treatment consequences, while false-negative results mean that continued drug use or relapse would be missed by treatment personnel.  After considering issues of personal invasiveness, ease of collection, cost, accuracy, and any consequences associated with false-positive or false-negative results, drug treatment personnel and local policymakers will have to determine if these rates of agreement are high enough to consider replacing or augmenting urinalysis with OF analysis.

 

References

 

Cone, E.J. 1993. Saliva testing for drugs of abuse.  In: D. Malamud & L. Tabak

(Eds.) Saliva As a Diagnostic Fluid (pp. 86-127).  New York: The New York Academy of Sciences.

 

Harrison, L. 1995. The validity of self-reported data on drug use.  Journal of Drug

Issues 25: 91-111.

 

Lilienfield, D.E.; Stolley, P.D.; & Lilienfield, A.M. 1994. Foundations of Epidemiology.

New York: Oxford University Press.

 

Magura, S. & Kang, S. 1996. Validity of self-reported drug use in high-risk populations:

A meta-analytical review. Substance Abuse and Misuse 31: 1131-1153.

 

Mieczkowski, T.; Barzelay, D.; Gropper, B.; & Wish, E.D. 1991. Concordance of

three measures of cocaine use in an arrestee population: Hair, urine, and self-report. Journal of Psychoactive Drugs 23: 241-246.

 

Moore, L.; Wicks, J.; Spiehler, V.; & Holgate, R. 2001. Gas chromatography-mass

spectrometry confirmation of Cozart RapiScan saliva methadone and opiate tests.  Journal of Analytical Toxicology 25: 520-524.

Niedbala, R.S.; Kardos, K.; Fries, T.; Cannon, A.; & Davis, A. 2001b. Immunoassay

for detection of cocaine/metabolites in oral fluids. Journal of Analytical Toxicology 25: 62-68.

 

Niedbala, R.S.; Kardos, K.; Waga, J.; Fritch, D.; Yeager, L.; Doddamane, S.; & Schoener,

E. 2001a. Laboratory analysis of remotely collected oral fluid specimens by

immunoassay. Journal of Analytical Toxicology 25: 310-315.

 

OraSure Technologies. 2003. Intercept Oral Specimen Collection Device®, Package

Insert. Beaverton, OR: OraSure Technologies.

 

Speckl, I.M.; Hallbach, J.; Guder, W.G.; Meyer, L.; & Zilker, T. 1999. Opiate detection

in saliva and urine – A prospective comparison by gas chromatography-mass spectrometry. Clinical Toxicology 37(4): 441-445.

 

Wish, E.D.; Hoffman, J.A.; & Nemes, S. 1997. The validity of self-reports of drug use at

treatment admission and at follow-up: Comparisons with urinalysis and hair assays.  In: L. Harrison and A. Hughes (Eds.) The Validity of Self-Reported Drug Use: Improving the Accuracy of Survey Estimates.  Rockville, MD: National Institute of Drug Abuse.

Wish, E.D. & Yacoubian, G. 2002. A comparison of the Intercept Oral Specimen

Collection Device® to laboratory urinalysis among Baltimore City arrestees.  Federal Probation 66(1): 27-29.

 

Yacoubian, G. 2000. Reassessing the need for urinalysis as a validation technique. 

Journal of Drug Issues 30(2): 323-334.

 

Yacoubian, G.; Wish, E.D.; & Pérez, D.M. 2001. A comparison of saliva testing to

urinalysis in an arrestee population. Journal of Psychoactive Drugs 33(3):289-294.