Oklahoma Marriage

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe and Sampling

The process study of the Oklahoma Marriage Initiative relies primarily on qualitative data for which the study team needs to obtain information from individuals who are most knowledgeable about the initiative. When identifying informants, it is necessary to focus on the potential universe of respondents and determine whether “purposeful identification” of respondents or a representative approach is most appropriate. Purposeful selection is most appropriate when insights and information can come only from individuals with particular roles or knowledge, such as lead staff at public agencies that collaborate with the Oklahoma Marriage Initiative. In contrast, a “representative” approach generally is most relevant for examining responses of workshop leaders and participants when there is a large number in the universe. Table 6 summarizes our approach for the five types of respondents.

In total, we will interview representatives from the five public institutions that collaborate on the OMI and from the community/volunteer sector. Representatives will include public agency staff, workshop leaders, and workshop participants. The majority of data collection is focused on agencies and individuals who are actively involved in OMI services, particularly at the workshop leader and participant level. However, we also plan to interview lead staff at four additional agencies, including some that delivered services in the past, or who intended to deliver services but for various reasons were unable to reach a critical mass.

TABLE 6

SUMMARY OF SAMPLING APPROACH

Respondent Group	Interview Type and Number	Respondents or or Focus Group Participants	Sampling Approach
Lead staff at public agencies	Semi-structured individual or small-group interviews (11)	Education State Career Tech oversight staff (1) Cooperative Extension directors (1) Head Start lead staff (1) Social services TANF agency directors (1) Adoptive/foster parent unit leaders (1) Corrections Prison wardens or unit leaders (3) OAYS unit leaders/directors (1) Health Child guidance unit leaders (1) Military Military base commanders or chaplains (1)	Purposeful Identification: MPR obtains names and contact information from PSI and arranges individual or small group interviews.
Workshop leaders at public agencies	Group interviews (7)	Education High school teachers (1) Social services Within My Reach trainers (1) Adoptive/foster parent staff (1) Corrections Prison chaplains (3) OAYS staff (1)	Representative Approach: For each selected agency, MPR identifies workshop leaders from OMI’s MIS, stratified by activity level. Verifies contact information with public agency. If more than enough leaders for a group interview, MPR randomly selects leaders and over-recruits by 50 percent.
Workshops leaders in community/ volunteer sector	Group Interviews (2)	Community/volunteers Faith leaders/clergy (1) Mental health counselors (1)	Representative Approach: MPR identifies trained workshop leaders through OMI’s MIS, stratified by activity level. MPR over-recruits at twice the desired attendance, ensuring representation from each stratum.
W TABLE 6 (continued) orkshop participants at public agencies	Focus groups (7)	Education High school students (1) Social services TANF recipients (1) Adoptive/foster parents (1) Corrections Prison inmates (3) Families of youth first offenders (1)	Representative Approach: MPR obtains list of participants from agency and randomly orders the list. If agency can provide contact information without compromising confidentiality, MPR recruits for focus groups. If not possible, MPR provides randomly ordered list to agency staff and enlists their assistance in recruiting. Participants over-recruited by 50 percent, and offered a $35 incentive.
Workshop participants in community/volunteer sector	Focus groups (2)	Members of faith community (1) General public (1)	Representative Approach: MPR develops a recruitment flyer for focus groups. MPR and OMI enlist assistance of all trained workshop leaders in OKC and Tulsa to distribute flyers between May and October 2006. Participants call into a toll free number if interested. MPR recruits from this list when time comes for focus groups. Participants recruited at twice the required attendance and offered a $35 incentive.

Lead Public Agency Staff Representing OMI Sectors. Over time the initiative has collaborated with public agency staff in five institutional sectors (Education, Social Services, Corrections, Health, and Military). The number of collaborating public agencies varies within each institutional sector. In total, we plan to engage the public agencies through individual or group semi-structured interviews with lead staff from eleven collaborating agencies, three of which will be adult correctional facilities where Prison PREP is in full operation. For these semi-structured interviews, the study team will use a purposeful approach in selecting respondents, since their roles enable them to contribute unique insights and information. We will identify all relevant staff for semi-structured interviews and schedule appointments for in-person discussions with the assistance of OMI leadership. To learn about the barriers to implementing a marriage education program in the prison setting, we will also conduct telephone interviews with wardens and lead staff at prisons that have so far been less effective in implementing Prison PREP.

Workshop Leaders at Public Agencies. We have selected three of the five institutional sectors that have collaborated with the OMI for further investigation—Education, Social Services, and Corrections—including a particular emphasis on the latter. For each public agency within these three institutions (high schools, TANF agencies, Oklahoma's adoptive/foster care agency, Oklahoma state prison facilities, and the Oklahoma Association of Youth Services), we will identify a sample of workshop leaders for the group interviews. Based on the number of staff who have completed training, provided by PSI, we estimate the universe of public agency workshop leaders to be approximately 800 individuals. From this universe, we will conduct seven group interviews with up to 12 workshop leaders each (84 individuals). Three of the seven group interviews will be held with prison chaplains or other staff functioning as Prison PREP workshop leaders (up to 36 leaders).

Workshop Leaders in the Community/Volunteer Sector. Again using data provided by PSI, we estimate the universe of trained workshop leaders from the community/volunteer sector to be approximately 1,500 individuals. From this universe, we will select up to 12 workshop leaders for each of two group interviews, for a total maximum of 24 individuals.

Workshop Participants at Public Agencies. To the extent data are available, we will identify the total number of individuals who have participated in the OMI workshops at public agencies, from the OMI’s database. Seven focus groups will be conducted with up to 12 workshop participants from the same five public agencies where we convene semi-structured interviews with workshop leaders, for a total maximum of 84 individuals. Three of the seven focus groups with participants will be held with prisoners or ex-offenders from the three correctional facilities we will visit.

Workshop Participants in Community/Volunteer Sector. To the extent that data are available, we will determine the total number of individuals who have participated in OMI workshops in the community sector. We will conduct a focus group with up to 12 individuals who have participated in workshops within the faith community, and another focus group with up to 12 individuals who participated within a mental health or counseling context, for a total maximum of 24 participants.

2. Procedures for the Collection of Information

Table 6 identifies the intended informants for data collection and the number of semi-structured interviews and focus groups to be conducted. Semi-structured interviews or focus groups will be scheduled with staff from public agencies representing OMI sectors, workshop leaders at public agencies and from the community/volunteer sector, and workshop participants at public agencies and from the community/volunteer sector. The planned procedure for collecting data from each type of respondent is described below.

Lead Public Agency Staff Representing OMI Sectors. We will obtain contact information for the relevant lead staff at nine public agencies corresponding to the OMI’s five selected public institutions (Education, Social Services, Corrections, Health, and Military). These interviews will be conducted in person and scheduled as group or individual interviews, based on the topics being covered and the relevant experiences of the individuals being interviewed.

Workshop Leaders at Public Agencies. We will conduct up to seven semi-structured group interviews with workshop leaders at public agencies associated with three of the public institutions, selected because of the extent of activity in each. These include Education, Social Services, and Corrections. To contact institutional workshop leaders, we will first identify the trained leaders included in a database maintained by OMI staff, grouped by the number of workshops they have completed. We will consult with lead public agency staff prior to selecting participants to verify and update information for workshop leaders. If the number of trained workshop leaders who have conducted workshops within a given institutional sector is substantially more than can be accommodated in our data collection schedule, we will randomly sample from the overall list of potential respondents. Once we identify the list of potential respondents, we will work with agency staff to over-recruit by 50 percent (if possible), and schedule the interviews.

Workshop Leaders in the Community/Volunteer Sector. We will conduct two focus groups with the volunteer workshop leaders from areas that have been the most active: the faith area and the mental health/counseling area. Scheduling focus groups with the community/volunteer workshop leaders will require significant creativity due to the volunteer nature of their commitment to the OMI. We plan to obtain lists from the OMI of all trained volunteer workshop leaders, the timing of their training, and the number of workshops they have reportedly conducted. We will stratify the lists based on whether or not workshops have been conducted, with the aim of ensuring that 2/3 of each group is composed of leaders who have conducted workshops. After stratification, we will randomly order the list and recruit the desired number of participants, ensuring representation from each stratum. Because the volunteer workshop leaders are less likely to be connected to a public agency that is a full OMI partner, we will attempt to recruit twice the desired attendance at these focus groups. For example, we will likely aim to conduct a group interview with 12 workshop leaders from the community, where two-thirds (or 8) of the participants have conducted workshops and one-third (or 4) have not. To ensure the desired level of participation at each group, we will recruit 24 participants, including 16 who have provided a workshop and 8 who have not.

During telephone calls to recruit workshop leaders, MPR will conduct a short screening interview so that we can identify any obvious differences between those trained leaders who agree to participate in the focus group and those who do not. This screener will include up-to-date contact information, the agency or organizational affiliation for the leader, whether the leader ever conducted a PREP workshop, and the major obstacles to conducting workshops if none has been conducted.

Workshop Participants at Public Agencies. We will conduct up to seven focus groups with participants in the same public agencies where we will gather information from workshop leaders. In recruiting for these groups, we anticipate that lead agency staff will be able to generate a list of past participants. Depending on the size of the list, we may ask agency staff to compile a list of participants for a specified time period only. After receiving the list, we will order it randomly to ensure that known or readily accessible participants are not targeted. If agency staff can provide contact information for participants without compromising confidentiality, MPR will recruit individuals for the focus groups. However, if agency staff are unable to provide contact information, MPR will seek their assistance to recruit for the group; in these cases, we will provide the list of participants to agency staff in segments to avoid selection of the most readily accessible participants. Recognizing the likelihood of a high no-show rate, we will over-recruit by at least 50 percent.

Workshop Participants in Community/Volunteer Sector. We will conduct two focus groups with participants; one with members of a faith community and one with participants in a mental health or counseling setting. In the community/volunteer sectors, there are no centralized and readily accessible lists of participants and it is likely that workshop leaders do not maintain complete records of participants. Consequently, we plan to identify potential participants in these focus groups by seeking the assistance of all trained leaders in the community/volunteer sector who provide workshops in either the Oklahoma City or Tulsa area. We will work with the OMI to distribute recruitment flyers to all workshop participants by including flyers in the workshop materials sent to leaders by the OMI. The flyer will invite participants to contact MPR at a toll-free phone number to learn about the focus group and will describe incentives for participating. When a participant calls the number, an MPR staff person will thank the individual for his or her interest in the focus group, gather contact information, describe the incentives provided for participating in the focus group, and inform the person about the tentative timeframe for focus groups (fall 2006). When it is time to schedule the focus group, we will use this list to recruit participants. Given the timeframe of several months, this method of recruitment should generate a list sufficient in size for conducting two focus groups with up to 12 participants in each. However, it will require substantial over-recruiting; MPR staff will attempt to recruit twice the desired attendance.

3. Methods to Maximize the Response Rate and to Deal with Nonresponse

Nonresponse is of particular concern for researchers using group interview and focus group methods. To address concerns with potential nonresponse, we will over-recruit participants for each scheduled group interview and focus group. In addition, MPR will provide each recruit with a follow-up letter, thanking them for their interest, and outlining all of the relevant information they will need to attend the group. On the day before each group, telephone calls will be placed to each recruit, politely reminding them of the day and time of the group, and asking them to contact us if they have an emergency and cannot attend. For the data collection to be held state correctional facilities, we will enlist the assistance of the warden and other lead staff to ensure the prisoners are available for focus groups and interviews.

4. Tests of Procedures and Methods to be Undertaken

The instrument to be used during each interview or focus group is semi-structured in design. Consequently, we will not pretest instruments.

5. Individuals Consulted on Statistical Aspects of the Design

This study is being conducted by Mathematica Policy Research, Inc., under contract to the Office of the Assistance Secretary for Planning and Evaluation, U.S. Department of Health and Human Services. The project director is Ms. Robin Dion and the principal investigator is Mr. Alan Hershey, both of MPR. The project team consulted with Dr. Barbara Devaney and Dr. Sameena Salvucci of MPR about the sampling approach for the process evaluation. The analysis will not employ statistical methods, since the data are qualitative in nature and collected through the semi-structured interviews and focus groups.

APPENDIX A

AUTHORIZING LEGISLATION

Section 301 of the Public Health Service Act (42 U.S.C.241)

TITLE 42 > CHAPTER 6A > SUBCHAPTER II > Part A > § 241

§ 241. Research and investigations generally

(a) Authority of Secretary

The Secretary shall conduct in the Service, and encourage, cooperate with, and render assistance to other appropriate public authorities, scientific institutions, and scientists in the conduct of, and promote the coordination of, research, investigations, experiments, demonstrations, and studies relating to the causes, diagnosis, treatment, control, and prevention of physical and mental diseases and impairments of man, including water purification, sewage treatment, and pollution of lakes and streams. In carrying out the foregoing the Secretary is authorized to—

(1) collect and make available through publications and other appropriate means, information as to, and the practical application of, such research and other activities;

(2) make available research facilities of the Service to appropriate public authorities, and to health officials and scientists engaged in special study;

(3) make grants-in-aid to universities, hospitals, laboratories, and other public or private institutions, and to individuals for such research projects as are recommended by the advisory council to the entity of the Department supporting such projects and make, upon recommendation of the advisory council to the appropriate entity of the Department, grants-in-aid to public or nonprofit universities, hospitals, laboratories, and other institutions for the general support of their research;

(4) secure from time to time and for such periods as he deems advisable, the assistance and advice of experts, scholars, and consultants from the United States or abroad;

(5) for purposes of study, admit and treat at institutions, hospitals, and stations of the Service, persons not otherwise eligible for such treatment;

(6) make available, to health officials, scientists, and appropriate public and other nonprofit institutions and organizations, technical advice and assistance on the application of statistical methods to experiments, studies, and surveys in health and medical fields;

(7) enter into contracts, including contracts for research in accordance with and subject to the provisions of law applicable to contracts entered into by the military departments under sections 2353 and 2354 of title 10, except that determination, approval, and certification required thereby shall be by the Secretary of Health and Human Services; and

(8) adopt, upon recommendations of the advisory councils to the appropriate entities of the Department or, with respect to mental health, the National Advisory Mental Health Council, such additional means as the Secretary considers necessary or appropriate to carry out the purposes of this section.

The Secretary may make available to individuals and entities, for biomedical and behavioral research, substances and living organisms. Such substances and organisms shall be made available under such terms and conditions (including payment for them) as the Secretary determines appropriate.

(b) Testing for carcinogenicity, teratogenicity, mutagenicity, and other harmful biological effects; consultation

(1) The Secretary shall conduct and may support through grants and contracts studies and testing of substances for carcinogenicity, teratogenicity, mutagenicity, and other harmful biological effects. In carrying out this paragraph, the Secretary shall consult with entities of the Federal Government, outside of the Department of Health and Human Services, engaged in comparable activities. The Secretary, upon request of such an entity and under appropriate arrangements for the payment of expenses, may conduct for such entity studies and testing of substances for carcinogenicity, teratogenicity, mutagenicity, and other harmful biological effects.

(2)

(A) The Secretary shall establish a comprehensive program of research into the biological effects of low-level ionizing radiation under which program the Secretary shall conduct such research and may support such research by others through grants and contracts.

(B) The Secretary shall conduct a comprehensive review of Federal programs of research on the biological effects of ionizing radiation.

(3) The Secretary shall conduct and may support through grants and contracts research and studies on human nutrition, with particular emphasis on the role of nutrition in the prevention and treatment of disease and on the maintenance and promotion of health, and programs for the dissemination of information respecting human nutrition to health professionals and the public. In carrying out activities under this paragraph, the Secretary shall provide for the coordination of such of these activities as are performed by the different divisions within the Department of Health and Human Services and shall consult with entities of the Federal Government, outside of the Department of Health and Human Services, engaged in comparable activities. The Secretary, upon request of such an entity and under appropriate arrangements for the payment of expenses, may conduct and support such activities for such entity.

(4) The Secretary shall publish a biennial report which contains—

(A) a list of all substances

(i) which either are known to be carcinogens or may reasonably be anticipated to be carcinogens and

(ii) to which a significant number of persons residing in the United States are exposed;

(B) information concerning the nature of such exposure and the estimated number of persons exposed to such substances;

(C) a statement identifying

(i) each substance contained in the list under subparagraph (A) for which no effluent, ambient, or exposure standard has been established by a Federal agency, and

(ii) for each effluent, ambient, or exposure standard established by a Federal agency with respect to a substance contained in the list under subparagraph (A), the extent to which, on the basis of available medical, scientific, or other data, such standard, and the implementation of such standard by the agency, decreases the risk to public health from exposure to the substance; and

(D) a description of (i) each request received during the year involved—

                (I) from a Federal agency outside the Department of Health and Human Services for the Secretary, or

                (II) from an entity within the Department of Health and Human Services to any other entity within the Department,

to conduct research into, or testing for, the carcinogenicity of substances or to provide information described in clause (ii) of subparagraph (C), and (ii) how the Secretary and each such other entity, respectively, have responded to each such request.

(5) The authority of the Secretary to enter into any contract for the conduct of any study, testing, program, research, or review, or assessment under this subsection shall be effective for any fiscal year only to such extent or in such amounts as are provided in advance in appropriation Acts.

(c) Diseases not significantly occurring in United States
The Secretary may conduct biomedical research, directly or through grants or contracts, for the identification, control, treatment, and prevention of diseases (including tropical diseases) which do not occur to a significant extent in the United States.

(d) Protection of privacy of individuals who are research subjects
The Secretary may authorize persons engaged in biomedical, behavioral, clinical, or other research (including research on mental health, including research on the use and effect of alcohol and other psychoactive drugs) to protect the privacy of individuals who are the subject of such research by withholding from all persons not connected with the conduct of such research the names or other identifying characteristics of such individuals. Persons so authorized to protect the privacy of such individuals may not be compelled in any Federal, State, or local civil, criminal, administrative, legislative, or other proceedings to identify such individuals.

APPENDIX B

GUIDE FOR SEMI-STRUCTURED INTERVIEWS WITH LEAD STAFF

AT PUBLIC AGENCIES

Oklahoma Marriage Initiative Process Evaluation

Guide for semi-structured interviews with
lead staff at public agenicies

This protocol will be used to guide semi-structured individual or small-group interviews with current and/or former staff at public agencies in selected locations that sponsor or considered sponsoring OMI relationship skills workshops and related activities, either currently or in the past, or who have integrated OMI-related content into existing curricula. Each interview is expected to last about 90 minutes. Only relevant sections of the protocol will be administered to each group of respondents. Interviews will be conducted with the relevant lead staff at the following Oklahoma public agencies:

• Department of Corrections

• Department of Human Services Local TANF agencies

• Department of Human Services Adoptive/Foster Parent units

• Department of Education, Division of Career Technology

• Oklahoma Association of Youth Services First Time Offenders Program

• Local military installations

• Oklahoma State University, Cooperative Extension Services

• Department of Health Child Guidance division

• Community Action or Head Start agencies

A. learning of the omi

1. How did you or your agency first learn about the OMI? Who approached your agency to solicit your interest in OMI, and when?

2. Is your agency currently involved in any way with OMI, such as providing referrals or currently using OMI curriculum materials or messages?

1. If so, what is the nature of you or your agency’s current involvement with the OMI or its programs and services, or your use of the PREP curriculum?

2. How long have you been involved with the OMI in this way?

3. Initially, what appealed to you about the OMI or PREP-related services? What concerned you? How were thee concerns addressed?

4. How do the goals of the OMI fit with your agency’s mission?

B. reasons AGENCY not ACTIVELY INVOLVED IN OMI (IF APPLICABLE)

1. What types of activities or programs related to OMI or the PREP curriculum have you considered or been asked to implement?

2. What was your agency’s process for exploring or considering these activities or programs?

3. What issues or factors have you considered in deciding whether or how to become involved in OMI? Which were decisive, and why?

1. Relevance or appropriateness of addressing marriage with agency clients

2. Access to interested participants

3. “Fit” with existing mission, program, or services

4. Availability or lack of resources

4. Do you currently support OMI-related activities in other ways?

5. Do you think your agency may take on any type of role or activity related to OMI in the future? What types of roles or activities would fit well with your agency’s agenda and structure?

c. background

1. Briefly describe the OMI-related program or activity(ies) that you oversee or that you helped design or implement, including their purpose, content, structure, and locations.

2. Do you coordinate with any other agency, non-profit, or other entities to provide this program or service (other than PSI)?

3. Why and how did your agency decide to offer this special program or service? Who initiated the idea of creating the program?

4. Why was your agency interested in the OMI or in providing OMI-related services?

   1. What benefits did you anticipate for your agency or clients?

   2. What concerns, if any, did you have?

5. To whom is the OMI-related program or activity offered or targeted?

   1. Are there specific eligibility criteria that clients must meet to participate in the program or activity?

   2. Why is this group a focus of the program or activities?

   3. Is participation voluntary or mandatory?

6. What was the process for planning and implementing the OMI activity or service?

7. Does your agency have any formal or informal agreements with PSI or DHS regarding your OMI-related activity?

8. Can clients at your agency receive other services related to marriage and relationships besides the PREP workshops? If so, please describe these other services.

9. Does the agency provide OMI-related services at multiple locations? If so, please specify the number of locations.

   1. How many total locations does this agency have in Oklahoma (e.g., number of prisons, high schools, TANF agencies)?

10. How many total full-time employees are devoted to operating the OMI-related program at each participating location of your agency?

    1. What proportion of your total staff is this?

11. Who is the staff contact person at PSI with whom you coordinate services or activities?

NOTE TO INTERVIEWER:

Administer Section D to lead staff at agencies that deliver OMI workshops. For agencies that embed PREP material into a broader program or curriculum, skip to Section E.

D. CURRICULUM and training (STAND ALONE WORKSHOPS)

1. What curriculum is used to deliver the OMI workshop? Is this an adaptation of PREP?

   1. If so, how was the need for adaptation determined?

   2. What adaptations were made to work with the targeted population?

   3. Who created these adaptations, and what was the process for creating them?

   4. Did you have a role in the curriculum adaptation? If so, what was that role?

2. What is the frequency and duration of the workshop? How many total hours of curriculum are provided over what period of time?

3. How are workshop leaders identified and recruited?

   1. Are the workshop leaders agency staff? PSI staff? Independent contract workers?

   2. Who performs the recruiting function—PSI staff or agency staff?

4. What kind of background or experience do agency workshop leaders typically have?

5. Who provides the agency’s workshop leaders with training in how to use the curriculum with agency clients?

   1. The curriculum authors? Agency staff? PSI staff?

   2. What happens when new staff come on board and need training?

6. How many agency staff members have been trained to conduct workshops?

   1. How many actively conduct workshops?

   2. Does the agency plan to train more in the future?

7. Is the curriculum delivery supervised or monitored by agency staff, PSI, or others?

   1. If so, who provides this supervision?

   2. What does this supervision entail?

8. Is assistance or support available from others outside of your agency in arranging for or conducting workshops?

   1. If so, who provides this assistance or support?

   2. What is the process for receiving this assistance? Do you initiate the request assistance, or are you contacted on a regular schedule to see if you need assistance? Or both?

   3. Have you ever received this assistance? If so, for what types of issues did you seek help or receive assistance?

9. Is the issue of domestic violence relevant to the program, class, or activity you provide?

   1. How have you addressed it in your planning?

   2. How do you address it in your operations?

E. CURRICULUM and training (broader programs)

1. What overall curriculum is used in your program?

2. How does PREP fit into the purpose of your program, and your existing or previous curriculum?

3. Why did you decide to incorporate PREP into your program or class? How was the decision made, and who made it? How was PSI involved?

4. Do you use an adaptation of PREP, or was your original curriculum adapted to incorporate PREP elements? Or both? If so, why was adaptation needed?

   1. What kinds of adaptations were made? (e.g. deletion of certain segments or elements)

   2. Who created these adaptations, and what was the process?

   3. Did you have a role in the curriculum adaptation? If so, what was that role?

5. What is the frequency and duration of the course or program? How many total hours of curriculum is provided over what period of time?

6. What proportion of the broader course or program is devoted to PREP content (e.g. number of hours or sessions)?

7. Who trains your program or course teachers/facilitators on the PREP component?

8. What kind of background or experience, skills, or credentials do teachers/facilitators have?

9. How many teachers or facilitators have been trained to conduct workshops? How many actively conduct workshops?

10. Is instruction in the curriculum supervised or monitored by agency staff, PSI, or others?

    1. If so, who provides this supervision?

    2. What does this supervision entail?

11. Is assistance or support available from others outside of your agency in arranging for or conducting workshops?

    1. If so, who provides this assistance or support?

    2. What is the process for receiving this assistance? Do you initiate the request assistance, or are you contacted on a regular schedule to see if you need assistance? Or both?

    3. Have you ever received this assistance? If so, for what types of issues did you seek help or receive assistance?

12. Is the issue of domestic violence relevant to the program, class, or activity you provide?

    1. How have you addressed it in your planning?

    2. How do you address it in your operations?

f. recruiting AND enrolling participants

1. Who has primary responsibility for recruiting or enrolling individuals in the OMI-related program or activity?

   1. Have these staff received training in how to identify and recruit participants?

   2. If so, what kind of training do they receive, who provides it, and for how long?

   3. Are classes easy or hard to fill? Is there a waiting list for classes?

2. Are all possible participants informed of the service, or are only specific groups or individuals targeted?

   1. If so, who is targeted and how are the individuals identified?

3. How are individuals recruited or enrolled for the activity?

   1. What is the context or setting of the recruitment effort?

• For example, a one-on-one meeting, a group orientation session, a visit in the home? Is an invitation sent through the mail? Is the program presented to both partners if client is married? To youth and their parents?

1. Is the recruitment “occasion” one that is exclusively for discussing and enrolling the client in the program, or is it integrated into some other purpose for client contact? If integrated, how well does this work?

2. What “messages” are potential participants given about the activity?

• What are they told of the purpose of the activity or program; what they may gain from it; what the level of commitment to participation is?

4. Are the potential recruits offered any incentives or additional agency benefits or credits for participating? If so, please describe these.

g. client participation and reactions

1. How long has your organization been providing the OMI-related program or activity?

2. Does your agency track and maintain program management data beyond what is reported to the OMI?

   1. If so, what types of information are tracked?

   2. How are the records kept (e.g., electronic or paper), and who maintains them?

3. How many curriculum workshops has the agency provided to date?

4. How many individuals at your agency have participated in the workshop, program or activity to date?

5. If participation is open to anyone, who typically enrolls or participates? Couples? Singles? Youth? Girls/Women? Boys/Men? Parents?

6. Do you track referrals to your program? How many total individuals have been referred to your program or workshop? Which organizations are responsible for most referrals?

7. Is activity by workshop leaders tracked by location? Is it recorded in the OMI’s database, or elsewhere?

8. How do participants generally respond to the curriculum workshops?

   1. In what ways does the material interest them, or meet their needs?

   2. What gaps or shortcomings have you observed in the curriculum or activities?

9. What is the general attitude among your client population regarding marriage?

   1. Have you noticed a change in general attitudes since you began offering the OMI-related program or activity?

10. What factors besides marriage and relationship skills and knowledge do you believe affect or will affect the decisions your clients make or will make about marriage and relationships?

h. Systems and Culture Change

1. What was the agency’s culture with respect to marriage and relationships before it began offering OMI-related services?

   1. Did staff discuss relationships or marriage with agency clients?

   2. Were any services offered to help agency clients with their relationships?

   3. Was there any formal or informal policy with respect to addressing the topic of clients’ relationships or marriage?

2. Has the agency specifically sought to create change in its culture regarding marriage?

   1. For example, has it sought to raise awareness of the importance of relationships and marriage for client’s well-being? If so, how was this done?

   2. In what other ways has it sought to create change in agency culture?

3. Has the process of being trained in the OMI-related services affected the attitudes or behavior of staff members who provide referrals or deliver OMI-related materials or workshops?

   1. Have you noted changes in staff attitudes or the way they approach their clients with regard to marriage and relationships?

• For example, changes in attitudes toward working with fathers or couples? About clients’ potential for healthy marriage? About value of marriage education?

1. If not, do you think there will be any such changes in the future?

1. Has the OMI-related program affected the operations or structure of the agency or the location at which the activities are provided? If so, how?

   1. If not, do you think there will be any such changes in the future?

2. Has the content of the program or curriculum affected how members of your organization work together, handle problems, or communicate? If so, how?

i. Wrap-Up

1. What characteristics, elements, or implementation approaches of the OMI-related service changed over time, if any? Why were these changes made?

2. In general, what organizational characteristics or strategies do you think are important in successfully developing and implementing classes, programs, or services related to OMI?

3. What are the greatest challenges, and the most important lessons your agency has learned in making these classes, services, or programs available?

APPENDIX C

GUIDES FOR GROUP INTERVIEWS WITH WORKSHOP LEADERS

• Guide for Group Interviews with Workshop Leaders at Public Agencies

• Guide for Group Interviews with Community-Volunteer Workshop Leaders

Oklahoma Marriage Initiative Process Evaluation

guide for group interviews with public
agencY WORKSHOP LEADERS

This protocol will be used to guide semi-structured small-group interviews with a sample of workshop, course, or program leaders who deliver OMI-related services at public agencies or institutions. Five group interviews will be conducted with up to 12 curriculum instructors each at selected locations. Each interview will last about 90 minutes.

• TANF agencies: Within My Reach trainers

• Child welfare agencies: Adoptive/foster parent staff

• Department of Corrections prison facilities: Prison chaplains

• Oklahoma Association of Youth Services first-time offender program facilitators

• High school Family and Consumer Sciences teachers

A. Introduction (10 min)

1. Introduce self and other MPR Staff

2. Purpose of discussion

• Instruction in relationship skills is being offered by various state agencies in Oklahoma, stimulated by or as a part of the Oklahoma Marriage Initiative. We are interested in learning about your experiences conducting these workshops/classes so that we can understand better how they are delivered in various settings.

• During the discussion today we would like to hear about your experiences delivering OMI workshops/PREP curriculum materials for your agency. Specifically, we would like to hear about the training you received to deliver this material, how your agency recruits or enrolls participants for workshops/classes, your experiences using the curriculum, and whether and how delivering the workshops/classes/materials may have affected participants or yourselves, or resulted in any changes at your agency.

3. Confidentiality and audiotaping

• We never use names in report or associate any name with answers. Reports are written to reflect the combined responses of a lot of different people. For example, “most people felt ….” or “about half the people who participated disagreed with x statement.”

• We will be audiotaping to help us summarize the discussion later, but no one will be given audiotapes outside of MPR

• To help ensure we have a clear tape, we ask that only one person talk at a time

• My colleague may also be taking some notes during the discussion

4. Time restrictions—moderator must move discussion to all topics

5. Offer opinion even if different from others; no right or wrong answers

B. background (10 min)

1. How long have you been involved with providing [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS] for [PUBLIC AGENCY OR INSTITUTION]?

2. Why and how did you first become involved with delivering this workshop/class or using this material?

1. Did you have a choice in whether or not you would deliver this workshop/class or use the material?

2. What initially appealed to you about it? What concerns did you have, if any? How did these change over time?

3. How well did delivering OMI-related instruction fit with your personal goals and beliefs?

3. How often do you present the curriculum material? Do you provide instruction on a regular basis, or intermittently?

4. On average, how much time do you spend preparing for and delivering [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS] in a typical month?

C. curriculum training and ongoing assistance (15 min)

1. Tell me about the training you received to deliver [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS].

   1. Who conducted the training? Where did it occur? How long did it last?

   2. Was the training specifically for your agency’s staff, or did it include people from other agencies or organizations?

   3. Was it tailored specifically for the curriculum adaptation you use, or was the training focused on the standard PREP program?

2. How prepared did you feel to deliver [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS] workshops after this initial training?

1. Did you feel prepared to organize workshops, including where and when you would offer them?

2. Did you feel prepared to recruit participants? Did you know where or how you would recruit them?

3. Have you requested or received any refresher training? Do you think this would be useful?

4. Is your instruction in the curriculum supervised or monitored by agency staff, PSI, or others?

   1. If so, who provides this supervision?

   2. What does this supervision entail?

5. Is assistance or support available from others outside of [AGENCY OR INSTITUTION] in arranging for or conducting workshops?

   1. If so, who provides this assistance or support?

   2. What is the process for receiving this assistance? Do you initiate the request assistance, or are you contacted on a regular schedule to see if you need assistance? Or both?

   3. Have you ever received this assistance? If so, for what types of issues did you seek help or receive assistance?

D. CURRICULUM ADAPTATION and format (15 min)

1. To your knowledge, were any formal adaptations made to the original PREP curriculum for use with the participants served by your agency?

   1. If so, who made these adaptations, what adaptations were made, and why?

   2. Did you or other agency staff have any role in the adaptation process? What was that role? For example, did you or others provide feedback to the curriculum authors, participate in a focus group, permit authors to observe classes?

2. To your knowledge, were any formal adaptations made to your original program or course curriculum when the PREP content was added? (if applicable)

   1. If so, who made these adaptations, what adaptations were made, and why?

   2. Did you or other agency staff have a role in the adaptation process? What was that role? For example, did you or others provide feedback to the curriculum authors, participate in a focus group, or permit authors to observe your classes?

3. Many instructors make minor changes to curriculum content or format during its delivery to make it more relevant to or appropriate for their audience, class, or participants.

   1. Other than the formal adaptations we discussed before, do you typically make other changes when delivering [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]?

   2. If so, what are these changes and why are they needed?

4. Let’s talk about the format you use for providing [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS].

1. How many total hours of instruction are provided, over how many days/weeks/months?

2. Why did you choose this format over other formats?

3. What proportion of the program or class is devoted to the PREP material on relationships and marriage?

E. recruiting and enrolling participants (15 min)

1. Who has primary responsibility for identifying and recruiting participants for the [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]?

• For example, agency staff such as caseworkers, teachers, high school counselors, or chaplains? Or is this the responsibility of the curriculum instructor?

2. Is [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS] mandated for certain clients (required by the agency to fulfill a requirement) or court-ordered?

3. Are there any eligibility criteria that clients must meet to participate in the program? Or is the program open to any agency client that wishes to participate?

   1. What are the eligibility requirements, if any? Why were these requirements identified?

4. Do clients receive special benefits, incentives, or credits for participating in [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]? If so, please describe.

5. Does your agency have trouble filling slots for [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS], or are they usually filled to capacity?

6. What strategies do you or your agency use to identify and recruit participants for [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]?

   1. What strategies appear to be most successful at recruiting participants?

   2. Which strategies do not appear to work well?

   3. What would help your agency recruit and enroll more participants?

7. Have you (or other agency staff) received any training in conducting outreach, marketing the workshop, or identifying and recruiting participants for this program/class?

   1. If so, who provided the training, when, and how long did it last?

F. CLIENT participation and reactions (15 min)

1. Who typically attends the [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]?

1. Couples? Singles? Youth? Women/Girls? Men/Boys? Parents?

2. Do participants usually represent a cross-section of [AGENCY OR INSTITUTION], or do specific types of individuals or couples tend to be attracted?

2. Based on your experiences, how well do you think [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS] meets the needs of your agency’s clients?

1. How interested do clients appear to be in learning the curriculum skills and information?

2. How well do clients seem to understand and apply the curriculum content?

3. What reactions do your agency’s clients have to participating in a group workshop that teaches relationship and marriage skills?

   1. What element(s) of the group workshop do participants appear to enjoy the most?

   2. Are there aspects of the curriculum that participants seem to dislike? What are these aspects?

4. How is attendance at [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]?

   1. Have you experienced frequent absences from individual classes or sessions? If so, to what do you attribute these absences?

5. What level of attrition (dropping out prior to the conclusion of all sessions/classes) do you tend to see?

   1. Can you estimate the percent of participants that tend to complete the workshop?

   2. At what point in the program does attrition tend to occur?

6. Has the issue of domestic violence or dating violence come up as part of providing [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]?

   1. If so, how frequently does it come up? How do you deal with it?

   2. Have you received any special training on recognizing or dealing with domestic or dating violence? How adequate was this training, in your view?

g. Systems and CULTURE CHANGE (15 min)

One of the OMI’s goals is to change the way Oklahomans think about marriage and divorce, both at the level of individual citizens and in the way public and private organizations and agencies deal with individuals.

1. In your opinion, has offering [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS] as one of your agency’s services changed the way staff think about or treat their clients?

1. Before [AGENCY OR INSTITUTION] began offering instruction in relationship skills, what was the general agency culture with respect to marriage and relationships?

2. Have staff attitudes about their clients changed? If so, how?

3. Have staff interactions with their clients changed? If so, how?

2. Has offering [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS] changed the agency’s structure, operations, or policies?

   1. In what ways has your agency’s structure, operations, or policies changed?

3. Has being trained in [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS] changed the way you or other staff work together, work with participants, or handle conflict?

4. Have your own attitudes or behavior regarding relationships and marriage changed as a result of receiving the PREP training and/or conducting workshops?

5. Have you noticed any changes in participants’ attitudes toward marriage or divorce as a result of learning relationship skills in your workshops?

h. Wrap-up and post-group information form (5 min)

1. In your opinion, how well does the collaboration between PSI and your agency work?

   1. What about the collaboration works well and what is not working well?

   2. How could PSI and your agency work more collaboratively?

2. Are there other aspects of your role or experience as an OMI workshop leader for your agency that you would like to share with me?

COLLECT POST-GROUP INFORMATION FORM.

ATTACHMENT D

guide for group interviews with community volunteer

workshop leaders

Oklahoma Marriage Initiative Process Evaluation

guide for group interviews with community volunteer

workshop leaders

This protocol will be used to guide semi-structured small-group interviews with curriculum instructors from local communities who voluntarily provide PREP workshops to their clients or to members of the community. Two group interviews will be conducted with up to 12 workshop leaders; one with instructors in the faith sector; and another with instructors in the mental health, counseling or related fields. Each group interview will last about 90 minutes.

A. Introduction (10 min)

1. Introduce self and other MPR staff

2. Purpose of discussion

• As you know, workshops or classes that provide information on relationship or marriage skills are being offered all across the state of Oklahoma, stimulated by or as a part of the Oklahoma Marriage Initiative. We are interested in learning about your experiences conducting these workshops/classes so that we can understand better how they are delivered in various settings.

• Specifically, we would like to hear about the training you received to deliver this material, how you recruit or enroll participants for workshops/classes, your experiences using the curriculum, and whether and how delivering the workshops or classes may have affected participants or yourselves, or resulted in any changes in your communities.

3. Confidentiality and Taping

• We never use names in report or associate any name with answers. Reports are written to reflect the combined responses of a lot of different people. For example, “most people felt ….” or “about half the people who participated disagreed with x statement.”

• We will be audiotaping to help us summarize the discussion later, but no one will be given tapes outside of MPR

• To help ensure we have a clear tape, we ask that only one person talk at a time

• My colleague may also be taking some notes during the discussion

4. Time restrictions—moderator must move discussion to all topics

5. Offer opinion even if different from others; no right or wrong answers

B. BACKGROUND (5 min)

1. How long have you been involved with providing PREP workshops for the OMI?

2. How and when did you first hear about the OMI and the free PREP training it offers?

3. Why and how did you choose to become involved in delivering this workshop?

   1. What made you interested in participating?

• You thought it would be a valuable addition to other counseling services you offer?

• You saw a need in your community or church for this kind of program?

• You anticipated there would be benefits to your own relationship by learning PREP?

4. How important was the fact that the PREP training was free of charge in your decision to be trained to deliver workshops?

5. What initial concerns did you have about being trained or delivering the workshops? How were these concerns addressed?

C. training and ongoing assistance (15 min)

1. Tell me about the training you received to deliver PREP.

1. Who conducted the training? When and where did it occur? How long did it last? About how many people were in the training with you?

2. What did you like the best about the training? What was least useful, or could use improvement?

2. How prepared did you feel to deliver workshops after this initial training?

1. Did you feel prepared to organize workshops, including where and when you would offer them?

2. Did you feel prepared to recruit participants? Did you know where or how you would recruit them?

3. Have you requested or received any refresher training? Do you think this would be useful?

4. Is your delivery of PREP workshops supervised or monitored by anyone?

1. If so, who provides this supervision?

2. What does this supervision entail?

5. Is assistance or support available from others in arranging for or conducting workshops?

a. If so, who provides this assistance or support?

b. What is the process for receiving this assistance? Do you initiate the request assistance, or are you contacted on a regular schedule to see if you need assistance? Or both?

3. Have you ever received this assistance? If so, for what types of issues did you seek help or receive assistance? Did the assistance meet your needs?

6. Our understanding is that workshop leaders who receive free PREP training are requested to provide 4 workshops in their communities at no cost to participants. Have you conducted any workshops yet? If so, how many so far?

a. What made this obligation easy or difficult to fulfill?

b. Have you continued to provide workshops beyond the 4 required ones? If so, are they free? If not, what is the cost to participants?

7. If you have not yet fulfilled the obligation of providing four free workshops, why not?

a. Not enough time has elapsed since training

b. Enough time has elapsed but it has been difficult to recruit a sufficient number of participants to conduct four workshops

c. Other reasons

D. CURRICULUM ADAPTATION and format (10 min)

1. Many facilitators or instructors make minor changes to a curriculum so that it is more relevant to or appropriate for their audience, class, or participants.

1. What changes or refinements have you made when delivering PREP?

2. Why were these changes or refinements needed?

3. Are the modifications discussed with PSI or the curriculum authors?

2. Let’s talk about the format you use for providing the PREP workshop.

a. How many total hours of instruction are provided, over how many days/weeks/months?

b. Why did you choose this format over other formats?

3. Where have you conducted workshops?

a. Have you had trouble finding suitable locations/facilities?

b. Do you think the location has any effect on attendance (positive or negative)?

E. recruiting and enrolling participants (15 min)

1. To your knowledge, where do most of your participants first hear about your workshop?

1. Your agency? Your work? Your church? Your neighbors and friends? Word-of-mouth? The OMI website? From a flyer, poster or brochure? Networking with public agencies? Community event? Public service announcements? Other?

2. Which of these outreach strategies have you tried so far?

1. Which of them appear to be the most effective in generating interest in the workshop?

2. The least effective? Why?

3. Have you had trouble filling slots for a workshop? Or are workshops usually filled to capacity?

4. If any participants have been referred to you by other organizations, which organizations are they? Why and how did these sources agree to make referrals to you?

5. Have any participants in your workshops learned about you at a Sweetheart’s Weekend or other OMI-sponsored event?

6. Can participants receive or win gifts such as door prizes for participating in a workshop or hearing a presentation about it?

1. If so, what are these gifts and who funds them?

2. What role do you think such gifts or prizes play in encouraging enrollment and ongoing participation?

7. Have you ever received any training in conducting outreach, marketing the workshop, or identifying and recruiting participants for this program/class? If so, who provided the training, when, and how long did it last?

8. How could you be more successful planning workshops or recruiting participants? What kind of support or assistance would be helpful?

F. client participation and reactions (15 min)

1. Who typically attends your workshops?

1. Couples? Singles? Youth? Women/Girls? Men/Boys? Parents?

b. Do participants usually represent a cross-section of your community, or do specific types of individuals or couples tend to be attracted?

2. Based on your experiences, how well do you think the workshop meets the needs of your participants?

   1. How interested do clients appear to be in learning the curriculum skills and information?

   2. How well do clients seem to understand and apply the curriculum content?

3. How do participants in your community tend to react to the PREP workshop?

   1. What element(s) of the workshop do they appear to enjoy the most?

   2. Are there elements of the curriculum that participants seem to dislike? What are these aspects?

4. How is attendance at your workshops?

   1. Have you experienced frequent absences from individual classes or sessions? If so, to what do you attribute these absences?

5. What level of attrition (dropping out prior to the conclusion of all sessions/classes) do you tend to see?

   1. Can you estimate the percent of participants that complete all sessions?

   2. At what point in the program does attrition tend to occur?

6. Has the issue of domestic violence or dating violence come up as part of providing the workshop?

   1. If so, how frequently does this come up? How do you deal with it?

   2. Have you received any special training on recognizing or dealing with domestic or dating violence? How adequate was this training, in your view?

g. community attitudes and norms about marriage (15 min)

One of the OMI’s goals is to change the way Oklahomans think about marriage and divorce.

1. Have your own attitudes or behavior regarding relationships and marriage changed as a result of receiving the PREP training and/or conducting workshops?

2. Have you noticed any changes in participants’ attitudes toward marriage or divorce as a result of learning relationship skills in your workshops?

   1. If so, what kind of attitude changes have you observed?

3. What is the general attitude in your community toward marriage and divorce?

   1. Besides having relationship and marriage skills, what other factors seem to affect people’s attitudes and norms about marriage and relationships in your community?

   2. Do you expect that offering relationship skills classes in a community will gradually change the attitudes and norms about marriage? If so, how? If not, what would be needed to make this kind of community-level change?

h. Wrap-up and post-group information form (5 min)

1. Are there other aspects of your role as a community PREP workshop leader that you would like to share with us?

COLLECT POST-GROUP INFORMATION FORM.

APPENDIX E

GUIDES FOR FOCUS GROUPS WITH PROGRAM PARTICIPANTS

• Guide for Focus Groups with Program Participants at Public Agencies

• Guide for Focus Groups with Program Participants in Community

Oklahoma Marriage Initiative Process Evaluation

guide for focus groups with workshop participants at public agencies

This protocol will be used to guide focus groups with participants who receive OMI workshops as part of the services provided by a public agency or institution. Focus groups will be conducted with up to 12 participants each, at each of the five public agencies identified below. Each focus group will last about 90 minutes.

• TANF recipients who participated in Within My Reach

• Adoptive/foster parents who participated in a PREP-ENRICH retreat

• Prisoners who participated in Prison PREP as part of a reentry program

• High school students who took a Connections-PREP class

• Youth and their parents who received a PREP class as part of a diversion program for first-time offenders

A. INTRODUCTION (5 MIN)

1. Introduce self and other MPR Staff

2. Purpose of Discussion

• Relationship skills workshops or classes are being offered by various state agencies in Oklahoma, in coordination with the Oklahoma Marriage Initiative. We are interested in learning about your experiences participating in [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS] so that we can understand better how the program works.

• During the discussion today we would like to hear about how you came to participate in this program, why you enrolled in it, what you thought of it, how useful it was in your daily life, and whether you think offering this program has changed the way the agency interacts with you.

3. Confidentiality and taping

• We never use names in report or associate any name with answers. Reports are written to reflect the combined responses of a lot of different people. For example, “most people felt ….” or “about half the people who participated disagreed with x statement.”

• We will be audiotaping to help us summarize the discussion later, but no one will be given tapes outside of MPR

• To help ensure we have a clear tape, we ask that only one person talk at a time

• My colleague may also be taking some notes during the discussion

4. Time restrictions—moderator must move discussion to all topics

5. Offer opinion even if different from others; no right or wrong answers

B. PARTICIPANT INTRODUCTIONS (10 MIN)

1. Please introduce yourself by telling us:

• Your first name

• When and where you attended the [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]

C. REFERRAL AND ENROLLMENT [NAME OF PREP ADAPTATION]

(15 MIN)

1. How did you come to participate or enroll in [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]?

   1. Was this class required to fulfill an agency requirement? For example, was it required as a condition of prisoner release, high school graduation, release from a first-offender program, or as a condition for adoption? If so, who first approached you about the class, and how was it described to you?

   2. If the class was not required, was it recommended to you by an agency representative? For example, was it recommended by a school counselor or advisor, a caseworker, a prison chaplain, or a representative of the Oklahoma Marriage Initiative? If so, how was the class described to you?

   3. If the class was not required and you did not hear about it from an agency representative, did you seek out the program as the result of seeing a flyer, poster, or hearing about it from a friend or acquaintance?

2. What were your reasons for participating in [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]?

   1. What interested you? What benefits or knowledge did you expect to get out of it?

   2. What concerns did you have, if any? How were these addressed?

3. Were you eligible to receive any special benefits, like gift cards or door prizes, by participating in the program?

   1. Were you eligible for any other agency benefits or services because of your participation in [NAME OF PREP ADAPTATION, PROGRAM, OR CLASS]?

   2. What effect did these potential benefits have on your motivation to join or continue participating in the program?

NOTE TO INTERVIEWER: Administer Section D to focus groups who participated in an OMI workshop. For focus groups who received PREP material within a broader program or curriculum, skip to Section E. (e.g., high school students and first offenders).

D. EXPERIENCES WITH workshop (20 MIN)

1. Describe the format of a typical workshop/class meeting for me.

   1. How large was your group? How often did you meet, and for how long at a time? Where were the meetings or classes held?

2. What topics did you discuss during your sessions?

   1. Which topics seemed the most useful for you?

   2. The least useful?

3. What kind of activities did you do during a typical class or workshop meeting? Did you listen to lectures? Have group discussions? Do written exercises? Watch videos?

   1. Which activities did you like most?

   2. Least?

4. Did any of your classes or sessions address issues of domestic violence, or dating violence, or other types of violence in relationships?

   1. If so, what was said about violence in relationships?

5. How effective was your workshop/class instructor?

   1. Was he/she able to convey the information clearly and in a way that you could clearly understand?

   2. Was the material presented in a way that kept your interest and attention? Or were you often bored?

   3. Were examples and illustrations relevant to your situation and life?

   4. In what areas might the instructor improve? In what areas did he/she excel?

6. How difficult or easy was it for you to understand and apply the information and skills covered in the class?

7. Did you attend by yourself, with your spouse or partner, with your parents, or with someone else?

8. How many of the workshop/class meetings did you attend?

   1. If you missed any sessions, what were the reasons you missed them?

   2. If you dropped out of the program after some period, please explain when and why you dropped out.

9. Would you recommend this class/program/workshop to your friends and family? Why or why not?

E. EXPERIENCES of PREP material within a broader program (15 MIN) [ONLY FOR PROGRAMS USING PREP AS ONE COMPONENT OF A CLASS OR BROADER CURRICULUM]

1. You are attending this focus group because you participated in a class that involved some discussion about the topic of relationships and relationship skills.

   1. About how much of the overall program was devoted to this general topic?

2. What kinds of topics were discussed during that session or module?

   1. Which seemed to be most useful?

   2. Least useful?

3. What kinds of relationships were addressed?

   1. Relationships between you and your parents? Your friends? Boyfriends or girlfriends? Dating partners? Others?

4. Was marriage discussed? What did the program say about marriage?

5. How interested were you in this part of the course or program?

   1. What interested you about it?

   2. Did you find any parts of it boring? If so, why?

6. Were there topics, issues, or problems that you think should have been discussed during this class or module, but were not? If so, what were they?

7. How effective was your teacher or facilitator in presenting this class or module?

   1. Was he/she able to convey the information clearly and in a way that you could clearly understand?

   2. Was the material presented in a way that kept your interest and attention?

   3. Were examples and illustrations relevant to your situation and life?

8. Would you recommend this class or module to your friends? Why or why not?

f. participation and reactions (15 MIN)

1. What is the main thing you learned from this workshop/class?

2. Have you used any of the skills or lessons in your daily life? If so, which skills?

3. What is the current attitude toward marriage and relationships among your friends, or in your neighborhood or community?

4. Have your attitudes or thoughts about relationships or marriage changed as a result of this workshop/class? If so, what were these changes?

5. Do you think the workshops/classes have helped you personally?

   1. If so, how have they helped? How important would you say the workshop/classes have been in your life?

   2. Have the workshop/classes improved your relationship with a significant other or with other family members? If so, how have they improved it?

6. What do you think works well in this program?

7. What changes would you suggest to improve the services?

g. WRAP-UP and post-group information form (10 MIN)

1. If you are in a relationship with a partner or spouse, are your instructors or agency staff, such as caseworkers, aware of that relationship?

   1. Do you feel they are supportive of your relationship?

   2. In what way have they been supportive or not supportive?

2. Is there anything else you’d like to tell me about workshop/class or your experience?

COLLECT POST-GROUP INFORMATION FORM.

PAY PARTICIPANTS AS THEY LEAVE (SIGN RECEIPT LOG)

appendix f

guide for focus groups with community workshop participants

Oklahoma Marriage Initiative Process Evaluation

guide for focus groups with community workshop participants

This protocol will be used to guide focus groups with individuals who have participated in OMI workshops provided by community volunteers. One group will focus on members of a faith community; a second focus group will be composed of individuals served by mental health/counseling providers and possibly other members of the general public. Up to 12 individuals will participate in each focus group, and each will last approximately 90 minutes.

A. INTRODUCTION (5 MIN)

1. Introduce self and other MPR Staff

2. Purpose of discussion

• Relationship skills workshops or classes are being offered by various people all across Oklahoma, in coordination with the Oklahoma Marriage Initiative. We are interested in learning about your experiences participating in the PREP workshop so that we can understand better how the program works.

• During the discussion today we would like to hear about how you came to participate in this program, why you enrolled in it, what you thought of it, how useful it was in your daily life, and whether you think offering this program has changed your relationships.

3. Confidentiality and taping

• We never use names in report or associate any name with answers. Reports are written to reflect the combined responses of a lot of different people. For example, “most people felt ….” or “about half the people who participated disagreed with x statement.”

• We will be audiotaping to help us summarize the discussion later, but no one will be given tapes outside of MPR

• To help ensure we have a clear tape, we ask that only one person talk at a time

• My colleague may also be taking some notes during the discussion

4. Time restrictions—moderator must move discussion to all topics

5. Offer opinion even if different from others; no right or wrong answers

B. PARTICIPANT INTRODUCTIONS (10 MIN)

1. Please introduce yourself by telling us:

• Your first name

• When and where you attended the PREP workshop

C. REFERRAL TO AND ENROLLMENT IN PREP (20 MIN)

1. How did you learn about this workshop?

1. Did you seek out the workshop after seeing a flyer, poster, or brochure, visiting a website, or hearing about it on a public service announcement or at a community event?

2. Did you hear about it from a friend, neighbor, or acquaintance?

3. Did a representative of any organization, such as a health clinic, a church, or the workshop leader recommend the workshop to you? If so, who recommended it?

2. If you first heard about it from an organization representative, what did that person tell you about the workshop before you enrolled?

3. What were your reasons for participating in the program?

1. What interested you about the workshop? How did you think it would benefit you?

2. What concerns did you have, if any? How were those concerns addressed?

4. Were you eligible for any special benefits, like gift cards or door prizes, by participating in the program?

1. Did you receive any other benefits or services because of your participation in the workshop?

2. What effect did these benefits have on your motivation to join or continue participating in the PREP workshop?

D. EXPERIENCES with PREP workshop (20 MIN)

1. Describe a typical PREP class/meeting for me.

   1. How many other people were in your group? How often did you meet, for how long at a time, and at what time of day? Where were the meetings held?

2. Did the program provide childcare during meetings? What about transportation assistance? Were refreshments provided?

3. What kind of activities did you do during a typical workshop meeting? Did you listen to lectures? Have group discussions? Do written exercises? Watch videos?

   1. Which activities did you like best?

   2. Which did you like least?

4. What topics did you discuss during your sessions?

   1. Which topics have been most useful for you?

   2. Least useful?

5. Did any of your classes or sessions address issues of domestic violence, or dating violence, or other types of violence in relationships?

   1. If so, how was it addressed?

6. How effective was your workshop instructor?

   1. Was he/she able to convey the information clearly and in a way that you could readily understand?

   2. Was the material presented in a way that kept your interest and attention? Or were you often bored?

   3. Were examples and illustrations relevant to your situation and life?

   4. In what areas might the instructor improve? In what areas did he/she excel?

7. How difficult or easy was it to understand and apply the information and skills covered in the class?

8. Did you attend by yourself, with a partner, or with someone else?

9. How many of the workshop/class meetings did you attend?

   1. If you did not attend all of the meetings, please explain why you had to miss some of them.

   2. If you dropped out of the program after some period, please explain when and why you dropped out.

E. participation and reactions (20 min)

1. What is the main thing you learned from this workshop/class?

2. Have you used any of the skills or lessons in your daily life? If so, which skills?

3. What is the current attitude toward marriage and relationships among your friends, or in your neighborhood or community?

4. Have your attitudes or thoughts about relationships or marriage changed as a result of this workshop/class? If so, what were these changes?

5. Do you think the workshop has helped you personally?

1. If so, how has it helped? How important would you say the workshop/classes have been in your life?

2. Have the workshop/classes improved your relationship with a significant other or with other family members? If so, how have they improved it?

6. What do you think works well in this workshop?

7. What changes would you suggest to improve the workshop?

8. Would you recommend attending a PREP workshop to your friends or family members? Why or why not?

9. What are the main attitudes about marriage in your community?

1. Do you think this type of workshop can changes people’s attitudes about marriage? In what ways?

F. WRAP UP AND POST-GROUP INFORMATION FORM (5 MIN)

1. Does anyone have any other thoughts or comments about your experiences in the PREP program?

COLLECT POST-GROUP INFORMATION FORM.

PAY PARTICIPANTS AS THEY LEAVE (SIGN RECEIPT LOG)

APPENDIX g

PARTICIPANT INFORMATION FORMS

• Information Form for Workshop Participants

• Information Form for Public Agency Workshop Leaders

• Information Form for Community-Volunteer Workshop Leaders

APPENDIX h

FEDERAL REGISTER NOTICE

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Office of the Assistant Secretary for Planning and Evaluation

Proposed Information Collection Activity

Comment Request

Proposed Project

Title: Oklahoma Marriage Initiative Process Study.

OMB No.: New collection.

1. Description: The Oklahoma Marriage Initiative Process Study (OMIPS) is an opportunity to improve understanding of this initiative and suggest lessons from its implementation to inform similar state and federal efforts. The project will document and analyze the context from which the initiative developed; its overall goals and objectives; and its evolution, organizational structure, partnerships, and service delivery system. The Oklahoma Marriage Initiative (OMI), launched in 1999 and funded by the Oklahoma Department of Human Services (DHS), was designed to reduce divorce through a multi-sector strategy with broad community support and the delivery of marriage and relationship skills education. The process study will document the evolution of the OMI, its program strategy and design, the consequences of OMI implementation approaches, the role of research in shaping implementation strategy, and OMI goals for changing the systems and culture within which it operates. The study team will gather data through focus groups and semi-structured interviews, and from documentary information and administrative records. Semi-structured interviews will be conducted with lead program staff, including representatives from up to 11 public agencies who have participated in the initiative. Seven group interviews will be held with workshop leaders from public agencies; two additional group interviews will be held with workshop leaders in the community/volunteer sector. Seven focus groups will be conducted with participants who have taken workshops at public agencies; two additional focus groups will be held with participants in the community-volunteer sector.

2. Respondents: Respondents for the process study include lead representatives of public agencies in Oklahoma, workshop instructors at public agencies, workshop instructors who provide services in the general community, workshop participants at public agencies, and workshop participants in the community. Five data collection instruments will be used in the process study data collection. This includes a protocol for semi-structured interviews with lead staff of OMI programs at public agencies; two guides for group interviews with workshop leaders at public agencies and in the community; and two guides for focus groups with workshop participants at public agencies and in the community. One to four lead staff at each of nine public agencies are expected to participate in semi-structured interviews, for a total of 11 to 44 respondents. The attendance goal for each group interview and focus group is eight to 12 people in 18 groups, for a total of 144 to 216 respondents.

Annual Burden Estimates

Number of Average burden

Number of responses per hours per Total burden

Instrument Respondents respondent response hours

Lead Staff Public Agencies protocol 44 1 1.50 66

Int. Guide for WS Leaders at Public Agencies 84 1 1.50 126

Int. Guide for Community WS Leaders 24 1 1.50 36

FG Guide for WS Participants at Agencies 84 1 1.50 126

FG Guide for Community WS Participants 24 1 1.50 36

Estimated Total Annual Burden Hours 390

In compliance with the requirements of section 3506(c)(2)(A) of the Paperwork Reduction Act of 1995, the Office of the Assistant Secretary for Planning and Evaluation is soliciting public comment on the specific aspects of the information collection described above. Copies of the proposed collection of information can be obtained and comments may be forwarded by writing to the Office of the Assistant Secretary for Planning and Evaluation, [INSERT ADDRESS] Washington, DC 20447, Attn: ASPE Reports Clearance Officer. All requests should be identified by the title of the information collection. The Department specifically requests comments on: (a) Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility; (b) the accuracy of the agency's estimate of the burden of the proposed collection of information; (c) the quality, utility, and clarity of the information to be collected; and (d) ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology. Consideration will be given to comments and suggestions submitted within 60 days of this publication.

Dated: December __, 2006.

[INSERT NAME]

Reports Clearance Officer

10