






































Microsoft Word - Volume 14.doc


Commonwealth:  A Journal of Political Science       Volume 14, Number 1      December 2008 
Special Policy Issue on Public Health             2008 PPSA/LORL, PA House of Reps.  ISSN 0890-2410 

 

 

 
Public Health Shortage Areas in Pennsylvania: 

A Barrier to Health Information 
 
 

Alberto J.F. Cardelle, PhD, MPH, East Stroudsburg University 
Deidre Holland, MPH, School of Public Health, 

University of Medicine and Dentistry of New Jersey 
 

Abstract 

One of the essential functions of public health services is to “inform, 
educate, and empower people about health issues” (CDC 1994). 
Throughout the United States, the tendency is to have decentralized 
public health systems that leave the decision-making and a significant 
part of the financing to local county and municipal governments. This 
strong reliance on local government financing and control translates into 
extreme variations in per capita expenditures and access. This paper 
examines whether or not individuals residing in areas without a 
centralized public health infrastructure have more difficulty accessing 
health information to help them make informed decisions about healthy 
living and lifestyle choices.  The paper compares the ease and accuracy 
of accessing basic public health information in counties and 
municipalities without a Local Health Department1 (LHD) as compared 
to counties and municipalities served by a LHD. The study examines the 
case of Pennsylvania, because the state has the lowest ratio of public 
health workers per person in the country (Gebbie 2000), and it has only 
10 LHDs covering six counties and four municipalities.  

The study found that in areas without a LHD, residents had to make 
20% more calls and received useful information in only 64% of the 
inquiries. This is compared with locales served by LHDs which required 

                                                 
1An agency of local government, a local health department (LHD) develops and 
administers programs and services that are aimed at maintaining a healthy community. 
To ensure that these efforts address a community's most important health problems and 
concerns, the LHD encourages residents to participate in assessing public health needs 
and in formulating a community health plan. It also works with other community 
organizations to assure that needed services and programs are available. 



8 PUBLIC HEALTH SHORTAGE AREAS IN PENNSYLVANIA:  A BARRIER TO  
 HEALTH INFORMATION 
 

 

 

fewer calls in order to receive useful information, and in which useful 
information was attained 100% of the time. This assessment shows that 
the potential for callers to receive useful public health information in 
areas without a LHD was impacted by both the higher number of calls 
that were required and in the diversity of places to which callers were 
referred. In locales without LHDs, the caller was much more likely to be 
referred to a non-public health entity, and was statistically less likely to 
get to speak to a public health professional early on in their inquiry.  

 

Introduction 

Today most states in the United States organize their public health 
systems around county health departments (Mays et al. 2004).  Of the 
currently identified 2,865 local public health agencies in the United 
States, 73% cover a county, or a county and city, and 18% cover smaller 
geographic areas such as towns and townships (NACCHO 2005) 
(Beitsch et al. 2006). Sixty-two percent of LHDs in the United States 
serve populations of less than 50,000 persons while 40% of LHDs serve 
even less populated rural areas (NACCHO 2005). On the average, the 
majority of the funding, 65%, for these LHDs, comes from local 
government sources, the state, or are pass-through funds from the federal 
government.  

This decentralized approach to funding and control has meant that no 
one entity has comprehensive authority and responsibility for creation, 
maintenance, and oversight of the nation’s public health infrastructure 
which in turn has allowed the distribution of services in public health to 
be “fragmented and uneven” (Baker et al. 2005). 

Of the 67 counties in the Commonwealth of Pennsylvania, only six 
have a county health department and an additional four municipalities 
have local health bureaus.2 None of the 42 rural counties in Pennsylvania 
has a LHD. Counties without LHDs have services provided by various 
different governmental agencies and non-governmental organizations. 
The state Department of Environmental Protection provides 
environmental services (water supply testing), the Department of 

                                                 
2Counties with health departments are Montgomery, Bucks, Chester, Erie, Philadelphia 
and Allegheny; municipal bureaus include York, Allentown, Bethlehem and Wilkes-
Barre. 



Alberto J.F. Cardelle and Deidre Holland        9 
 
 
 

 

State Health Clinic
22%

Private Non-Profit 
16%

Other
25%

Hospital
16%SHIP Partner

9%

County Government
6%

County Human 
Service Org

6%

Agriculture provides restaurant inspections, and the state Department of 
Health provides the remaining public health services. The counties, 
which lack LHDs, have a state health center with a staff of one to four 
nurses that provide communicable disease clinical services including 
sexually transmitted disease and tuberculosis diagnosis and treatment, 
immunization, and HIV testing, counseling and education (PADOH 
2007). The health centers operate under the direction of district offices, 
whose staff provide coordination, consultative and administrative support 
to the health centers in communicable disease reporting and 
investigation, epidemiology, informational and referral, chronic disease 
prevention and intervention programs, and environmental health services 
(PADOH 2007). In addition, the state Department of Health contracts 
with local non-profit agencies for the provision of other public health 
services throughout counties without LHDs. 

 
Figure 1:  Organizations Providing Public Health Services in 

Areas without LHDs 
 

 

 

 

 

 

 

 

In 2004, an assessment of the existing public health infrastructure in 
areas of Pennsylvania without a LHD was completed using a survey of 
agencies and organizations (identified through reputational sampling) 
carrying out public health functions in 10 Pennsylvania counties without 
a LHD (Cardelle 2004).   

The data shows that in counties without a LHD, public health 
services are offered by a varied set of institutions (Cardelle 2004) (Figure 
1, above). In counties without LHDs, state health centers report offering 
less than a quarter of the public health services in the area.  Private sector 



10 PUBLIC HEALTH SHORTAGE AREAS IN PENNSYLVANIA:  A BARRIER TO  
 HEALTH INFORMATION 
 

 

 

entities such as nonprofits and hospitals offer more public health services 
than the state health centers.  As Figure 2 (below) shows, when asked to 
identify what other institutions in the county offered public health 
services, the majority of the organizations delivering public health 
services were either not aware of who delivered essential public health 
services or indicated that the services were not offered in their area 
(Cardelle 2004).  Fifty percent of the respondents could not identify who 
inspected recreational facilities and an additional 20% reported that those 
services were not offered in their area. Close to 80% of the respondents 
were not aware of a provider who provided vector control and 50% could 
not identify providers in their area that offered HIV testing, hazardous 
material control or carried out epidemiologic surveillance (Figure 2).  
These are all services that existing LHDs deliver or coordinate in areas 
with LHDs. 

 
Figure 2 

 

 

 

 

 

 

 

 

 

 

The data from this previous assessment (Cardelle 2004) shows 
evidence that even though essential public health services may be being 
offered in the areas of Pennsylvania not covered by a LHD, they are 
being offered through a decentralized structure that is influencing their 
visibility. Without a LHD to serve a coordinating role, the residents of 
the locale have greater obstacles to knowing what services are offered 

0% 10% 20% 30% 40% 50% 60% 70% 80%

Hazardous Material Control

Vector Control

Epidemiology

Not Aware of Service Provider No Service Available



Alberto J.F. Cardelle and Deidre Holland        11 
 
 
 

 

and where they are being offered. Therefore, since one of the essential 
public health functions is to “inform, educate, and empower people about 
health issues” (Public Health Functions Steering Committee 1994), this 
paper postulates that the decentralized structure present in locales 
without a LHD would be a barrier to individuals gaining health 
information that help them make informed decisions about healthy living 
and lifestyle choices. 

 

Methodology 

This assessment compared the ease of accessing basic public health 
information, and the accuracy of that information in counties and 
municipalities without LHDs3 as compared to counties and 
municipalities served by a LHD. A sample of 10 counties and 18 
municipalities that were similar in population and income to the counties 
and municipalities with LHD were selected. The non-LHD locales falling 
within a two standard deviation range of the mean population size and 
per capita income of the nine LHD locales (Philadelphia was not 
included) were sampled.  

Trained college students (health education majors) adhering to the 
following protocol collected the data: 

1. The beginning of an inquiry was a call to the main municipality or 
county telephone number.4  

2. The students introduced themselves as a college student and asked if 
there was someone or some office in the municipality or county that 
could answer one of the following six questions (there was only one 
question asked per phone call): 

a. Does this municipality/county have high incidence levels of 
Lyme disease? If I found a tick on me, what should I do? 

                                                 
3For the purpose of this study, LHDs are defined as health departments established and 
managed by local governments (county or municipal).  
4There are many examples in the literature of using mystery patients or clients to measure 
performance of health care institutions (Van den Borne 2006; Borfitz 2001). More 
recently, a similar methodology was used to measure performance of disease reporting 
systems (Dausey et al. 2008).  



12 PUBLIC HEALTH SHORTAGE AREAS IN PENNSYLVANIA:  A BARRIER TO  
 HEALTH INFORMATION 
 

 

 

b. Do I need to get a meningitis vaccine at my age? 

c. Where can someone get checked for Chlamydia?   

d. Who is at risk for West Nile Virus? 

e. Why should I get my house tested for radon? How do I get 
my house tested for radon? 

f. Who is at risk for lead poisoning? What are the signs of lead 
poisoning? 

3. The response from this initial call was intended to result in a referral; 
however, if the response was a negative (no, do not know, I am not 
sure, etc.) the student then asked if there was a local health 
department or a public health director. If the answer was no, then 
they asked if the person could suggest somewhere else that the caller 
could call to get the information.   

4. If referred to call somewhere else, the student collected information 
and called the next destination, then repeated the process. 

5. The student kept calling referrals until receiving useful information 
(defined as information that they as health educators would 
determine allow a prudent layperson to make an informed decision),5 
no longer referred, or referred five times.   

6. All questions were kept consistent and calls were made within 
approximately the same time of the day (afternoon).    

For each locale, written records of the dates, times, responses, and 
referrals for each call were maintained.    

 

                                                 
5The responses to what the callers indicated was useful were compared when the protocol 
and the methodology was tested on three counties (not part of the study) using Cohen’s 
Kapa measure, which measures the degree of agreement between raters on categorical 
assessments (Is this information useful?). The Kappa score for inter-rate reliability on 
whether an answer was useful was 0.69, which the literature defines as signifying 
substantial agreement (Landis and Kich 1977).    



Alberto J.F. Cardelle and Deidre Holland        13 
 
 
 

 

Results 

A total of 181 calls were made to 15 counties and 22 municipalities. 
There were 43 calls made to the counties and 138 to municipalities. Of 
181 calls, 142 were made to locales without LHDs and 39 were made to 
locales with LHDs. An average of 30 calls were made per question.   

In order to receive useful information, an average of 2.43 calls were 
required. Eighty percent of the inquiries required three or fewer 
telephone calls.  Sixty-four percent of the inquiries resulted in the caller 
receiving useful information.  

 
Table 1 

 
Mean number 

of calls1 

Percent of the inquiries 
receiving useful 

information2 

Percent of inquiries 
that required two or 

fewer calls 2 

Locale with LHD 2.15 100% 84% 

Locale without LHD 2.71 55% 46% 
 

1Difference in means sig <.05. 
2Difference is sig <.05. The percentage of calls not receiving useful information 
means that the either the caller was referred five times without getting useful 
information or they failed to get the last caller to refer them to another number. 

 
Inquiries made to locales served by LHDs required fewer calls in 

order to receive useful information, and useful information was attained 
100% of the time (Table 1, above).  The difference between the numbers 
of calls between the two groups is statistically significant at the .05 level 
(sig < .05)6. Locales with LHDs were significantly more likely to provide 
useful information in fewer calls across all themes (Table 2, following 
page). Locales without LHDs required almost three calls for the 
questions on Lyme disease and STD testing and over 2.5 calls on the 
question about immunizations, radon and lead poisoning. In addition, 
non-LHDs had a lower percentage of inquiries resulting in useful 
information. While 80% of the inquiries made about STDs resulted in 

                                                 
6Statistically significant at .05 (p < .05) means that the probability of that relationship 
occurring by chance is less than 5%.  



14 PUBLIC HEALTH SHORTAGE AREAS IN PENNSYLVANIA:  A BARRIER TO  
 HEALTH INFORMATION 
 

 

 

useful information, only 26% of the inquiries on radon resulted in useful 
information and only 32% of the inquiries on West Nile Virus resulted in 
useful information.  

 
Table 2 

 
Mean number of calls 

Percent of the inquiries 
receiving useful information 

 LHD Non-LHD LHD Non-LHD 

Lyme Disease 2.14 2.96 100% 60% 

Immunization 2.14 2.57 100% 50% 

STD Testing 2.14 2.96 100% 80% 

West Nile Virus 2.17 2.38 100% 32% 

Radon 2.17 2.71 100% 26.3% 

Lead Poisoning 2.17 2.64 100% 75% 

 
The locales with no LHD required more phone calls to complete the 

inquiry.  One hundred percent of the inquiries made to locales with a 
LHD were completed with three phone calls. Only 85% of the inquiries 
made to locales without a LHD were completed with three or fewer calls 
(Figure 3, page 15) (sig < .05).    

Analysis of the relationship between the number of calls and receipt 
of useful information showed a statistically significant relationship 
between the two variables.  Inquiries that resulted in useful information 
had a mean number of calls of 2.45, while inquiries not resulting in 
useful information had a mean number of calls of 2.87. Figure 4 (next 
page) demonstrates that 92% of the inquiries which resulted in useful 
information were responded with fewer than three calls, while the 
inquiries not resulting in useful information required more calls.   

 



Alberto J.F. Cardelle and Deidre Holland        15 
 
 
 

 

Figure 3 

0

10

20

30

40

50

60

70

80

90

LHD Locales Non-LHD Locales

P
er

ce
n

t 
of

 I
n

q
u

ir
ie

s 
C

om
p

le
te

d
2 Calls

3 Calls

4 Calls

5 Calls

 

 
Figure 4 

0

10

20

30

40

50

60

70

Useful Information Received Useful Information Not
Received

P
er

ec
en

t 
of

 t
he

 I
nq

ui
ri

es
 C

om
p

le
te

d

2 Calls

3 Calls

4 Calls

5 Calls

 



16 PUBLIC HEALTH SHORTAGE AREAS IN PENNSYLVANIA:  A BARRIER TO  
 HEALTH INFORMATION 
 

 

 

A third major issue identified in the difference between locales is the 
dispersed types of referrals made during the inquiry. While locales with 
LHDs referred the caller to the LHD sometime during the first three calls 
100% of the time, inquiries made to locales without LHDs were referred 
to other government agencies 55% of the time, after the initial call. In 
these locales, other government agencies were the most common referral 
after the first and second referrals; only during the third referral are non-
LHD locales likely to refer the inquiries to the Pennsylvania Department 
of Health (either state level or county level). These delayed referrals 
could exhaust callers and keep them from continuing their inquiries. The 
callers in this study had a protocol that required them to continue to call, 
but members of the public requesting information would likely not be so 
persistent (see Table 3, below).  

 

Table 3 

 Referral 1 Referral 2 Referral 3 

 LHD 
Non-
LHD 

LHD 
Non-
LHD 

LHD 
Non-
LHD 

PA DOH (state or county) 0% 9.9% 0% 24.8% 0% 40% 

Private Provider1 0% 11.3% 0% 15.8% 0% 14.5% 

State Health Center 0% 8.5% 0% 22.8% 0% 18.5% 

Other Government Health 
Agency2 

0% 2.1% 0% 2% 0% 1.8% 

Other Government Office3 11% 55.6% 0% 28.7% 0% 18.2% 

Penn State Extension Office 0% 2.8% 0% 1% 0% 3.6% 

LHD 89% 0% 100% 0% 0% 0% 

Other4  9.8%  4.9%  3.4% 
 

1Includes hospitals, private physicians, non-profit agencies 
2Local Health Board, Department of Environmental Protection, Department of 
Agriculture, Vector Control 
3Housing Authority, Assistance Office, hotline, commissioner’s office 
4Library, courthouse, college professor 



Alberto J.F. Cardelle and Deidre Holland        17 
 
 
 

 

Overall, as compared to callers in locales with LHDs, callers within 
locales without LHDs tend to run a 93% greater risk of being referred to 
non-public health entity during the first referral and 60% greater risk 
during the second referral.  In addition, callers in locales without a LHD 
have a 50% greater risk of not receiving useful information (Table 4). As 
Table 5 demonstrates, the individuals responding to the calls are trying to 
provide some type of useful information, but lack the adequate 
information. The callers did not report an unwillingness to provide 
service, but instead a genuine unawareness of the information. The 
referrals are also problematic because some, such as having the caller 
search the web or go to the library, may result in erroneous information. 

  

Table 4 

 Relative 
Risk 

Confidence 
Intervals 

Inquiries to non-LHD locales not resulting in useful 
information  1.51 1.32 1.70 

Referrals made to non-public health entities in non-LHD 
locales after first call.  1.93 1.56 2.37 

Referrals made to non-public health entities in non-LHD 
locales after second call.  1.60 1.37 1.85 

 

 

Table 5 

Sample Responses 

Q LHD Locale Non-LHD Locale 

1 “Try the Health Department, 
hold and I will transfer you.” 

“There is no one in the city that can help you 
with that. … Go into a search engine like Yahoo 
and type in the words Lyme disease…” 

2 “I’ll transfer you to the Health 
Bureau.” 

“I have no idea; maybe you could try your 
personal physician.” 

3 “We have a walk-in clinic.” “Since there is not a LHD, we (the city) have 
contracted with Family Health Counseling.” 



18 PUBLIC HEALTH SHORTAGE AREAS IN PENNSYLVANIA:  A BARRIER TO  
 HEALTH INFORMATION 
 

 

 

4 “I will transfer you to a nurse at 
the Health Department.” 

“I would try the library for that information. I do 
not know if we have a local health department - 
that is a good question. Sorry I can’t help you.”   

5 “You need to do what everybody 
else does and go to Home Depot 
and buy a test kit.” 

“I will transfer you to our building inspector.” 

6 “Call our Lead Program.” “We don’t have a local health department but    
we have a board of health.” Transferred to a  
physician’s office … The receptionist told me to   
“call the state Health Department.” 

 

Discussion 

This assessment shows that the potential for callers to receive useful 
public health information in LHDs and non-LHDs was impacted by both 
the higher number of calls that were required and in the diversity of 
places to which callers were referred.     

In locales without LHDs, the caller was much more likely to be 
referred to a non-public health entity, and was statistically less likely to 
get to speak to a public health professional early on in their inquiry.  This 
delay or diversion increases the chance of getting information that is not 
useful.  It is important to highlight that over 85% of the calls that are 
eventually referred to public health entities such as state department of 
health centers, county vector control programs, and county West Nile 
control programs resulted in useful information. Therefore, it was not 
that the personnel in the network of organizations that provide public 
health services to non-LHD locales could not provide useful and timely 
information, it was that it took the caller a longer time to get to that 
organization.  In both types of locales, once the caller was forwarded to a 
public health entity, the chances of receiving useful information 
improved dramatically. The difference is that in locales without LHDs, 
the caller had to demonstrate a greater determination in following-up on 
the referrals.  

In general, there is an overall uncoordinated response in locales 
without a LHD. In locales with LHDs, over 90% of the initial calls were 
sent to the LHD by government personnel with the job of answering 
phone calls (administrative assistant or receptionist). This shows that in 



Alberto J.F. Cardelle and Deidre Holland        19 
 
 
 

 

LHD locales, frontline government workers were aware of the portal of 
entry into the public health infrastructure. In non-LHD locales, there was 
not a common entity that government employees could consistently 
identify as being in charge of public health. There was no pattern in non-
LHD locales as to where the call was referred; it was sometimes a 
hospital, sometimes another government agency, and sometimes the 
Department of Health. Even in situations in which the caller was told 
there was a local public health entity in non-LHD locales, the caller had a 
greater likelihood of getting messages such as:  

“There is a Health Officer but she is part time, you would 
have more luck calling the state Health Department.” 

“…the health officer is working part time now, but leave a 
message and I will get back to you as soon as possible.” 

“The city health line was closed about two years ago, so you 
need to call the state Health Department or go on the web to 
www.state.pa.us.” 

The danger in these situations, as well as in situations in which it 
took callers four to five calls to get an answer, is that callers would give 
up. It is fair to ask how many calls a college student, who is mildly 
concerned about being tested for an STD, would make before giving up.    

This study did have inherent limitations. The sample of the counties 
and municipalities called are not representative of the rural counties in 
Pennsylvania. The sampled counties have higher per capita incomes and 
are more densely populated. In addition, the determination of what 
constitutes “useful” information is a subjective assessment.  

In addition, since the study was not blind, unconscious subjective 
bias may have been introduced. Although the calls were made at 
different times of the day, there is a possibility that the study did not 
capture a representative sample of persons who answer the calls in 
county offices.  

 

Conclusion 

 The results of these different studies indicate that areas without 
LHDs are lacking the infrastructure with which to provide all the 



20 PUBLIC HEALTH SHORTAGE AREAS IN PENNSYLVANIA:  A BARRIER TO  
 HEALTH INFORMATION 
 

 

 

essential public health functions. In areas across the country that lack a 
LHD, a proposed solution to this problem has been the creation of a 
network of coordinated entities that could provide the 10 essential 
functions of public health. However, a lesson of this assessment is that 
these networks are not recognized as being an entity that is perceived as 
having the charge of providing public health services. This study 
highlights that while private-public partnership may adequately serve the 
public health needs of a population, there is a clear need for public sector 
stewardship of the partnership. Without at least one central entity that has 
the established reputation of being the steward of the public’s health, 
residents in need of information and services will have difficulty in 
solving their public health related problems. The disjointed services 
offered or the paucity of useful information provided may not be a 
reflection of the competence of the personnel in these network-based 
infrastructures, but rather a reflection of the lack of a centralized portal 
of entry into the infrastructure. While it may be possible to provide 
public health services through a decentralized network, it is difficult for 
this network’s structure to establish a clear portal of entry for residents.  
Unlike a fire, a traffic accident, or even a water-main break where what 
is required of local services is clearly evident; responses to a strange 
rash, the identification of a vector, or environmental hazards are less 
clear and linear. As a result, residents of locales without LHDs are more 
likely to be without the essential service of public health needed to make 
informed decisions regarding their health. These differences create a 
public health disparity, a disparity that given the increasing threats to the 
public’s health, SARS, West Nile Virus and bioterrorism, stands to 
negatively affect the long-term health status of the population. Since the 
public entrusts the public sector with its protection, it follows that the 
portal of entry into a public health infrastructure be the responsibility of 
the public sector. 

 

References 

Allukian, M. Jr. 1993. “Forging the Future: The Public Health Imperative.” American 
Journal of Public Health 83: 655-60. 

American Public Health Association. August 2004. “Shift in Preparedness Funds 
Undermines Readiness Efforts.” The Nation’s Health.      



Alberto J.F. Cardelle and Deidre Holland        21 
 
 
 

 

Baker, E.L., and J.P. Koplan. 2002. “Strengthening the Nation’s Public Health 
Infrastructure: Historic Challenge, Unprecedented Opportunity.” Health Affairs 
(Millwood) 6:15-27. 

Baxter, R.J., C.R. Steinberg, and J.R. Shapiro. 2001. “Is the U.S. Public Health System 
Ready For Bioterrorism? An Assessment of the U.S. Public Health Infrastructure and 
Its Capacity for Infectious Disease Surveillance.” Yale Journal of Health Policy Law 
Ethics 1: 1-21. 

Beitsch, Leslie, Robert G. Brooks, Nir Menachemi, and Patrick M. Libbey. 2006. Public 
Health at Center Stage: New Roles, Old Props. Health Affairs 25: 911-922. 

Borfitz, Deborah. “Is a Mystery Shopper Lurking In Your Waiting Room?” Medical 
Economics. May 1, 2001.   

Cardelle, Alberto. 2004. “Assessment of Rural Public Health Infrastructure.” unpublished 
manuscript. 

Center for Disease Control and Prevention. 1994. Public Health Performance Measure 
(Accessed March 1, 2008).  

Center for Rural Pennsylvania. 2004.  Profile of Pennsylvania’s Rural and Urban 
Counties.  http://www.ruralpa.org/county_profiles.html (Accessed March 10, 2007). 

Dausey, David J., Anita Chandra, Agnes G. Schaefer, Ben Bahney, Amelia Haviland, 
Sarah Zakowski, and Nicole Luriee. 2008. “Measuring the Performance of 
Telephone-Based Disease Surveillance Systems in Local Health Departments.” Am J 
Public Health 9:1-6. 

Dreicker, M. 2004. Plans for a Northampton County Health Department. Testimony 
presented to Northampton County Council, April 12, 2004. 

Frist, William. 2002. Public Health And National Security: The Critical Role of Increased 
Federal Support. Health Affairs (Millwood) 6:117-30. 

Gebbie, Kristine. 2000. The Public Health Workforce; Enumeration 2000. Washington 
DC.: Health Research and Services Administration. 

Gebbie, Kristine. 1993.  Rebuilding A Public Health Infrastructure. Journal of Law and 
Medical Ethics 3-4:368-71. 

Gebbie, Kristine. 1999. The Public Health Workforce: Key to Public Health 
Infrastructure. American Journal of Public Health 5:660-1. 

Landis Eric, and Mariane Koch. 1977. Measurement of Measurer Agreement for 
Categorical Data. Biometrics 33:159-174. 

Van Den Borne, Francine. 2007. Using Mystery Clients to Assess Condom Negotiation 
in Malawi: Some Ethical Concerns. Studies in Family Planning 4:322-330. 




