SAMHSA Phase Four Response

Response to OMB Comments and Questions

The National Evaluation of the Comprehensive Community Mental Health Services for Children and their Families Program: Phase Four

Revised Clearance

1. The following response pertains to the Primary Care Provider Study

1. Please describe the lists from where the sample of primary care providers will be drawn and provide information about where they will be contacted for the study (e.g., home or office).

The list of primary care providers will be obtained from American Medical Information, an InfoUSA company. InfoUSA is also the parent company to Macro International. Contact information is included for both physician offices and home addresses. Either or both can be provided. Macro will request both addresses and determine which is the most complete and comprehensive list. Mailings will be conducted accordingly.

Lists are compiled by AMI using the following methods: The basis of the list is derived from 5200 business yellow and white pages, CME (Continuing Medical Education) directories, and industry and state files regarding medical marketing. Each record is hand-keyed and every business is called to confirm accurate contact information. Every month, data are matched with the HSPS National Change of Address (NCOA) zip+4 and Delivery Sequence File to standardize and keep addresses accurate. Lists are updated monthly by region, with the entire list updated 4 times per year. The list is updated when new phonebooks are released.

It is estimated that approximately 97% of all physicians in the U.S. are included in the list. Those not included would primarily be physicians who are hospital-based only, and do not have a separate office address. However, given that this study is targeting office-based primary care physicians, we anticipate the list to be very comprehensive for this group. AMI estimated a deliverable rate (i.e., accurate address) of 92% by direct mail. Another Macro study used a physician list provided by AMI. Out of 500 contacts made, they estimated that less than 1% of phone numbers were inaccurate. Thus we anticipate that the list provided by AMI will be sufficiently comprehensive and no additional tracing work will be necessary prior to mailing the survey.

While we anticipate relatively complete and comprehensive lists, Macro will take additional steps if necessary to ensure that pediatricians are represented on this list. This may include phone calls to a random sample of physicians on the list to verify mailing addresses, and cross-checks with the yellow pages from particular zip codes and regions.

There are no expected regional gaps in the database. Regardless of urbanicity or rurality, if a pediatrician has a phone number listed in a phonebook, it will be included in the database. Similarly, all physicians are required to participate in CME every two years, thus all physicians should be included in this directory. An AMI representative stated that they have not had a problem with regional variation previously.

If office addresses are determined to be appropriate, Macro will make every effort to ensure that the survey is received by the pediatrician. This will include sending the survey by a method requiring a signature. The initial mailing will be sent by FedEx, and the final survey mailing will be sent by registered mail. Both methods will increase the likelihood that the targeted pediatrician will receive and complete the survey.

2. What is the approach to sampling and how will the results be generalized?

The sample of pediatricians will be selected using a stratified random sampling approach. The sampling frame will be constructed from the comprehensive list of pediatricians in the target areas (defined by ZIP Code areas) obtained from InfoUSA. The stratification will be by community. The frame data generate the number of pediatricians in each ZIP Code area and in each community, a number that will be used to allocate the sample. The sample will be proportionally allocated based on the total number of pediatricians in each community, such that a greater number of pediatricians will be sampled from larger communities and a smaller number from smaller communities. Proportional allocation will generate a self-weighting sample of pediatricians, at least approximately. Self-weighting samples—i.e., samples with equal probabilities of selection-- have the advantages of simplicity and efficiency as they minimize unequal weighting effects on sampling variances. The self-weighting feature will be attained only approximately, however, because the allocation will be approximately proportional and non-response adjustments will be applied to the survey weights. Although this sampling approach is used, we expect to generalize only to pediatricians within communities funded through systems of care between 2002-2004. Findings will not be representative of pediatricians as a whole.

3. What is the plan for follow-up with providers in the event of non-response or undelivered surveys and what steps will be taken to achieve the 80 percent response rate?

In the event of an incorrect address, non-response or an undelivered survey, we will conduct a yellow pages or Google address search to obtain correct address information. We will also request updated information from AMI on returned/bad addresses. Respondents will also have an opportunity to fill out an online survey. Recognized strategies to maximize response rates for mail or Internet surveys will be employed for the Primary Care Provider Study. This includes the Dillman method, which will be used for follow-up activities and includes pre-survey notification mailings, survey mailings with explanatory cover letters and incentives, and follow-up postcards, letters and phone calls. Further clarification can be found on page 39 and 83 of the package.

2. The following response pertains to the Family Education and Support Study

1. Where in the OMB package are the scripts and instruments for the Family Education and Support study?

Attachment 4F has the recruitment letter and all instruments associated with the Family Education and Support Study. Consent forms are available in Attachment 3, numbers 10 through 13.

3. The following response pertains to both the Primary Care Provider Study and the Family Education and Support Study

1. How will the issue of confidentiality be addressed in both studies?

OMB has requested that a Certificate of Confidentiality be required before data collection on both the Primary Care Provider Study and Family Education and Support Study can begin. An application for a Certificate of Confidentiality for these two special studies will be made to SAMHSA. No data collection activities will begin on the either of the two proposed special studies included within this Phase IV clearance submission until a Certificate of Confidentiality is obtained through SAMHSA.  This Certificate of Confidentiality secures that data collected in SAMHSA studies, including these proposed studies, will remain confidential within the terms specified within the awarded Certificate.