51321 Econ Risk OMB SSA_5-18-22_rev2

The purpose of this project is to understand the role of economic risk in public benefit recipients’ decision making about whether to increase earnings. This study will investigate the factors that influence the decisions made by people with low incomes to increase earnings (or not) by presenting beneficiaries with vignettes about other people with the opportunity to increase their earnings and asking them what decision they would make in similar circumstances. The vignettes will vary the risk of an opportunity, the ease of regaining benefits if income later dropped, and the extent of the earnings increase. The job opportunities and associated risks will vary systematically. The choices that study participants recommend will provide insight into the relative importance of these different factors – both alone and in combination. Guided by a better understanding of the role of risk and risk tolerance in families’ decisions about work, the government could design policy levers to reduce risk and thereby support families’ choices to increase earnings and achieve greater economic security.

A.1. Circumstances making the collection of information necessary

Increasing the employment earnings of individuals is central to federal poverty reduction policy for working-age adults. Most social services aim to serve as temporary supports for adults who will eventually return to the workforce. For this reason, it is important to understand how policies can incentivize or disincentivize increased employment. Central to this concern is understanding how to phase out public support for people as their earnings increase without discouraging benefit recipients from pursuing opportunities to improve their circumstances.

A previous ASPE study conducted with working parents who received one or more government benefits found that, when faced with a decision about increasing earnings, these parents considered multiple complex (and at times competing) factors (Chien et al., 2021). In addition to the more obvious consideration of benefit reductions, working parents also considered at least two dimensions of economic risk when deciding whether to increase earnings: (1) the risk of an earnings reduction (for example, from job loss) at a later time; and (2) in the case of an earnings reduction, the risk of being unable to regain benefits lost due to the original earnings increase.

Unfortunately, little is known about how beneficiaries perceive job opportunities that might result in loss of benefits. The research team is aware of simulation studies that illustrate the effects of policy on beneficiaries’ income under different scenarios. The research team is also aware of qualitative research on beneficiaries’ experiences. However, to our knowledge there is no quantitative data on how beneficiaries decide whether to pursue job opportunities. In addition, discourse about benefits cliffs often conflate several different policy challenges that are important and interrelated, but which might have distinct effects on beneficiaries’ perceptions and choices. These challenges include abrupt cliffs, steep cliffs, risks involved with pursuing opportunities to increase earnings, and challenges in regaining benefits should earnings decline.

The most serious cliff effect is a policy that leads social benefits to drop dramatically in response to small increases in earnings, producing a net loss of resources for beneficiaries. For example, eligibility for the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) ends abruptly as soon as household income exceeds a specific threshold. Policymakers and economists largely agree that this kind of benefits cliff is a challenge that prevents beneficiaries, employers, and policymakers from achieving their preferred economic outcomes (National Conference of State Legislators, 2019).

Steep cliffs in which benefits decline at a rate that does not exceed earnings but lead beneficiaries to feel as though they are running in place are also common (Albeda & Carr, 2017). Policymakers are not always aware of steep cliffs, which sometimes emerge from the combined policy decisions for different federal, state, and local programs. It is likely that, if the rate of decline is too steep, the additional earnings might not seem worth it, especially if acquiring these earnings requires taking a job that is more difficult or time consuming than the beneficiary’s current employment. The extent to which steep cliffs influence employment choice depends on how beneficiaries think about job opportunities. For example, do they tend to focus on the net gain in earnings alone, or do they focus on the difference between their net and gross income?

Beneficiaries raise the risk of accepting new employment and the difficulty of regaining benefits as important elements to the decision to accept a new job (Chien et al., 2021). Although the causes of these concerns are distinct from the cause of benefit reductions, they can contribute to cliff effects by lowering the expected value of a job opportunity, which people form by combining the value of each possible outcome (Barberis, 2013). When making this calculation, people might discount the value of increased wages because the job might not work out and, if it does not, they might not be able to regain the benefits they relied upon.

The decision to accept a new job can be complicated and high stakes. It is difficult to know how different elements of a job opportunity work together to motivate or discourage people to act. Understanding whether these features affect decision making alone or in concert, their relative influence on decision making, and whether their impacts vary across subgroups, will provide evidence about which policy levers are most likely to influence behavior. The research team will share these findings with federal agencies that administer benefits programs and will use them to inform decisions about how to encourage increases in employment earnings. Additionally, no legal or administrative requirements necessitate the collection. ASPE is undertaking the collection at the discretion of the agency.

A.2. Purpose and use of the information collection

The goal of this project is to better understand the information beneficiaries use to make decisions about employment opportunities and how this information might be incorporated into choice of whether to accept a new position. More specifically, the research team examines three factors that have direct impacts on the expected value of a job opportunity: 1) the risk of later job loss, 2) the ease of regaining benefits, and 3) the level of earnings increase. The study will examine whether these factors could be important for beneficiaries’ decision making and, if so, estimate the relative magnitude of their impact. The research team expects the information collected to contribute to the body of knowledge on Administration for Children & Families (ACF) programs.

This study will examine the following research questions (RQs):

Exhibit 2.1: Research Questions

RQ 1: How do potential earnings affect job acceptance rates?

RQ 1a: Are beneficiaries more likely to accept jobs with higher gross earnings?

RQ 1b: Are beneficiaries less likely to accept jobs that result in greater benefits loss, holding net earnings constant?

RQ 2: Are beneficiaries less likely to accept high-risk job opportunities—that is, those less likely to result in long- term employment?

RQ 3: Are beneficiaries more likely to accept job opportunities when it is easier to regain benefits if needed?

RQ 3a: Are beneficiaries more likely to accept a job if benefits can be automatically regained if needed compared to if they must reapply for them?

RQ 3b: Are beneficiaries more likely to accept a job if they can retain their benefits than if they can be automatically regained if needed?

RQ 4: Do the factors listed in RQs 1–3 interact with one another? For example, does the decision to take a high-risk job depend on how easy it is to regain benefits if needed?

RQ 5: Do the factors listed in RQs 1–3 depend on the type of benefit that might be lost? For example, is the difficulty of regaining benefits of greater concern for Supplemental Nutrition Assistance Program (SNAP) beneficiaries (who lose access to these benefits during the redetermination process) than for Medicaid beneficiaries.

Study design

This study strives to improve the quality of data collected about how benefit recipients make decisions. To achieve this, we will recruit 2000 benefits recipients and ask them to complete a web survey (Attachment A-1) in which they make decisions about different hypothetical job opportunities.

Participants will complete a one-time survey with several data collection components. Survey respondents will be recruited from NORC’s AmeriSpeak panel. The first section of the survey collects relevant demographic and biographical information. Next, respondents will hear five vignettes describing different job opportunities. The job opportunities will have varied financial benefits (wage increases and benefits loss). The opportunities will also have varied risks that the job opportunity might or might not work out and different levels of ease or difficulty to regain benefits.

The study is exploratory, using a within-subjects design and standardized vignettes to maximize the ability to detect differences between treatment levels. The study will provide some insight into whether each of these factors has an independent effect on beneficiaries’ decision making, or if they are likely to interact with each other.

The characteristics of the job opportunities will be varied simultaneously within a single factorial experiment. A factorial experiment is one that simultaneously examines several factors, each of which can have two or more levels or categories. Using a factorial design makes it possible to test a number of different treatments concurrently and to consider how the effects of each factor might interact with each other. Furthermore, a factorial design will allow the researchers to observe why people make certain decisions about employment opportunities and the availability of benefits. For example, the research team will observe the willingness for people to accept safer vs. riskier job opportunities depending on the ease or difficulty of regaining benefits. Exhibit A.2 lists the factors and treatment levels the research team will study.

Exhibit 2.2. Framework for testing factor combinations and contrasts

Factor and factor level	A (earnings increase) = 1 Smaller net earnings increase		A (earnings increase) = 2 Larger net earnings increase		A (earnings increase) = 3 Small net earnings increase after large benefits reduction
Factor and factor level	B (risk of earnings loss) = 1 Low risk	B (risk of earnings loss) = 2 High risk	B (risk of earnings loss) = 1 Low risk	B (risk of earnings loss) = 2 High risk	B (risk of earnings loss) = 1 Low risk	B (risk of earnings loss) = 2 High risk
C (Loss and recovery of benefits) = 1 (difficult to regain benefits)	A₁B₁C₁	A₁B₂C₁	A₂B₁C₁	A₂B₂C₁	A₃B₁C₁	A₃B₂C₁
C (Loss and recovery of benefits) = 2 (easy to regain benefits)	A₁B₁C₂	A₁B₂C₂	A₂B₁C₂	A₂B₂C₂	A₃B₁C₂	A₃B₂C₂
C (Loss and recovery of benefits) = 3 (no benefits loss)	A₁B₁C₃	A₁B₂C₃	A₂B₁C₃	A₂B₂C₃	A₃B₁C₃	A₃B₂C₃

Note: Factor A addresses RQ 1, Factor B addresses RQ 2, Factor C addresses RQ 3

The study design will allow for timely data reporting, with all reporting from this effort aimed to be complete by October 2023. The use of the panel supports and preserves the schedule of a one-month data collection period with analysis and reporting completed by October 2024.

The research team will invite all participants to complete the survey online via the web through the National Opinion Research Center (NORC) AmeriSpeak research panel. NORC recruits this panel through a two-stage, address-based sample design.

A.3. Use of improved information technology and burden reduction

ASPE will use advanced technology to collect and process data to reduce respondents’ burden and make data processing and reporting timelier and more efficient. NORC’s AmeriSpeak survey software system supports an integrated sample management and data collection platform. Using an existing panel also reduces burden, as respondents have already been recruited by NORC, are deemed likely to meet study inclusion criteria based on administrative records and have already agreed to participate in web-based surveys.

The NORC AmeriSpeak survey software system also provides opportunities to participate in a web mode using smartphones; for these panelists, the web-based system renders an optimized presentation of the questions. For all participants, regardless of mode, the AmeriSpeak survey technology includes tailored skip patterns and text fills, which enables respondents to move through the questions more easily and minimizes respondents’ burden.

A.4. Efforts to identify duplication and use of similar information

This study is the first data collection opportunity that will collect data from currently or recently working benefit recipients about the risks of increasing earnings. It is an extension of earlier qualitative interviews (Chien et al., 2021) that were instrumental in determining the design factors included in this study but were not designed to produce quantitative data about their relative influence on decision making. The research team is not aware of any similar studies conducted by other agencies or nongovernment entities. No other research has identified the information the research team will gather, thus necessitating the need for the current study.

A.5. Impact on small businesses or other small entities

This data collection will not affect or involve any small businesses.

A.6. Consequences of collecting the information less frequently

This request is for a one-time data collection. These data have not previously been collected elsewhere.

A.7. Special circumstances relating to the guidelines of 5 CFR 1320.5

There are no special circumstances with this information collection package. This request fully complies with the regulation 5 CFR 1320.5 and will be voluntary.

A.8. Comments in response to the Federal Register Notice and efforts to consult outside the agency

Federal register announcement

A 60-day Federal Register Notice for the Understanding Economic Risk for Low-Income Families: Economic Security, Program Benefits, and Decisions About Work project was published in the Federal Register on January 4, 2022, vol. 87 no. 2, p. 230. (Attachment A-2). ASPE received one public comment requesting a copy of the instrument (Attachment A-3).

Consultations outside the agency

ASPE engaged Mathematica and NORC to assist in developing the questionnaire, study instrument, and screeners. NORC is experienced in managing and conducting projects of this nature and provides expertise on issues such as the availability of data, frequency of collection, clarity of instructions, record keeping, data privacy, disclosure of data, reporting format, and necessary data elements.

Unresolved issues

There are no unresolved issues.

A.9. Explanation of any payment or gift to respondents

Participants have voluntarily registered with the AmeriSpeak panel. As part of the AmeriSpeak model they will be offered survey choice “points” to redeem for rewards. This is common practice for survey panel respondents who complete online surveys. The points are delivered via the online panel provider to respondents who complete the survey. This particular study will award points worth $5 to panelists who complete the survey. Participants recruited through nonprobability sample vendors (see Section B.1.c of Supporting Statement B for detail on nonprobability sampling) will be offered payments or gifts by that sample vendor. These points or gifts are unlikely to exceed the value of points offered to AmeriSpeak panelists.

A.10. Assurance of confidentiality provided to respondents

Personally identifiable information

ASPE will not maintain Information in a paper or electronic system from which it actually or directly retrieved data by an individuals’ personal identifier. During recruitment for the AmeriSpeak research panel (unrelated to this study), NORC collects individual respondent’s personally identifiable information (PII) (for example, name, address, email address, and the names and ages of household members) solely for purposes of conducting its research business, which includes prequalifying members or households for surveys and communicating with panel members. After selecting panel members, NORC uses periodic surveys to collect a range of demographic, social, health, attitudinal, and other information about panel members and their households.

Assurances of privacy

ASPE and all contractors collecting and analyzing data on behalf of ASPE will keep information provided during this data collection private to the extent permitted by law. During the informed consent process, NORC will inform respondents of all planned uses of data, that their participation is voluntary, and that their information will remain private to the extent permitted by law (see Section A11 for institutional review board [IRB] details). As specified in the contract, the contractor will comply with all federal and departmental regulations for private information.

Every AmeriSpeak panelist is provided a privacy statement that outlines the information AmeriSpeak will collect and how it will use the information. Because the survey will ask each panel member to provide key demographic data—such as age, gender, race and ethnicity, state of residence, household income, and more—the privacy statement also tells panel members how they can verify the accuracy of their PII and how they can request that AmeriSpeak delete or update the information. The AmeriSpeak privacy statement includes the following:

A promise to treat all AmeriSpeak panelists and their information with respect.
The assurance that participation in any AmeriSpeak study is completely voluntary and panel members may choose not to answer any questions they do not wish to answer. Furthermore, panel members may withdraw their participation in AmeriSpeak at any time.
Ameri Speak will not share the PII with any clients unless panel members have given explicit permission to do so. AmeriSpeak will share only survey responses with clients.
AmeriSpeak has established security measures to protect the security and confidentiality of its panel members.
Panel members control their personal information and have the right to view their personal information or ask AmeriSpeak to delete it.

A.11. Justification for sensitive questions

The survey will ask participants to provide information regarding their household income and health status, which some participants could perceive as sensitive. This effort will ask no other sensitive questions.

Household income is necessary to ask about because this survey seeks to understand how people with low incomes familiar with federal benefit programs think about economic risk. The survey will screen out people with incomes above $40,000 because they are unlikely to be familiar with SNAP, Medicaid, and/or CCDF.

Health is necessary to ask about because people with poor health, or with family members in poor health, might be more sensitive to the risk of losing benefits, such as Medicaid.

This study’s human subjects’ protection protocols received IRB exemption from ongoing review in accordance with the requirements for the US Code of Federal Regulations for the Protection of Human Subjects, 45 CFR 46.104.

A.12. Estimates of annualized burden hours and costs

The research team estimates each prospective participant will spend a total of 15 minutes participating in the survey. Exhibit 12.1 shows estimated burden and cost information.

Exhibit 12.1. Estimated annualized burden hours and costs to participants

Category of respondent	No. of respondents	Number of responses	Participation time (minutes)	Burden hours	Average hourly wage^a	Total respondent cost
Individuals	2,000	1	15/60	500	$10.25	$5,125
Totals	2,000	1		500	$10.25	$5,125

^a Average hourly wage estimated as the population-weighted average minimum wage in the United States.

Exhibit 12.1 shows the estimated respondent burden for the data collection instrument. The research team estimates 2,000 people will complete the survey. Based on pilot test timings, the research team estimates it will take about 15 minutes to complete this survey over the web, with an estimated 500 burden hours.

A.13. Estimates of annualized burden hours and costs

ASPE expects participants will incur no costs beyond the burden hours required to answer screening questions and participate in the interview.

A.14. Annualized cost to the government

The annualized cost estimates include (1) programming the surveys for web administration, (2) administering the web-based survey, (3) cleaning the data and providing data to the federal government in a machine-readable format, (4) data analysis, and (5) preparing a final report or manuscript. Federal costs are estimated as labor costs for a Social Science Analyst, GS 14 (using the 2022 OPM Salary Table 2022-DCB for Washington-Baltimore-Arlington locality salary schedule). Managing the project will require the expertise of one ASPE staff member.

ASPE expects to contract with Mathematica to perform the bulk of the data cleaning, analysis, and report preparation. Mathematica and ASPE will work in partnership to prepare the final results.

Exhibit 14.1. Annualized cost to the government

Expense type	Expense explanation	Annual costs (dollars)
Cost of ASPE staff time	ASPE staff hours	$6,444
Cost of ASPE staff time	Subtotal, direct costs	$6,444
Contract costs	Professional services	$120,000.00
	Direct costs for NORC panel and survey	$100,000.00
	Subtotal, contract costs	$220,000.00
	Total cost to the government	$226,444.00

A.15. Explanation for program changes or adjustments

This is a new data collection.

A.16. Plans for tabulation and publication and project time schedule

We will answer the research questions mentioned previously by estimating the effect of various treatment factors—those related to earnings increase, risk of earnings loss and loss and recovery of benefits respectively.

We will answer the RQ 1 by estimating the main effect that contrasts Factor A set to 1 (first two columns in the table) versus Factor A set to 2 (middle two columns in the table), versus Factor A set to 3 (last two columns in the table), averaging the overall values of Factors B and C. We will answer the second and third questions by estimating the main effect contrasts for Factors B and C in the same way, respectively.

We will also consider interaction effects in this analysis, which will enable us to answer more nuanced questions (RQ 4, RQ 5). such as, “Does the amount of employment risk people are willing to accept differ when benefits are easier or more difficult to recover?” This approach to deriving the contrasts will be extended to a more general and ambitious set of detailed questions by specifying more factor levels within A, B, and C.

We will answer RQ 5 by estimating the interactions between treatment and subgroup membership. This will enable us to answer questions such as “does making it easier to recover benefits have a larger impact on decisions to increase earnings for parents or non-parents?”

The study team will analyze data from the experiment and produce tables and bullets on key findings that will be delivered in the form of a report. Findings may also be presented at relevant conferences or academic journals. This initial will be delivered in December 2023. Anonymized data and code relevant for the analysis will be deposited in a code/data repository such as data.gov.

Exhibit 16.1. Proposed timeline

Months from project start	Major tasks or milestones
1	Program and test survey instrument
3	Conduct survey
5	Clean data
9	Data analysis
12	Write reports or manuscripts
15	Publish

A.17. Reason(s) display of OMB expiration data is inappropriate

ASPE is not requesting a waiver for the display of the OMB approval number and expiration date. The study will display the OMB expiration date.

A.18. Exceptions to certification for Paperwork Reduction Act submissions

There are no exceptions to the certification. These activities comply with the requirements in 5 CFR 1320.9.

References

Albelda, R., & Carr, M. (2017). “Combining Earnings with Public Supports: Cliff Effects in Massachusetts.” Economics Faculty Publication Series. 42. Available at: http://scholarworks.umb.edu/econ_faculty_pubs/42

Barberis, N.C. (2013). “Thirty years of prospect theory in economics: A review and assessment.” Journal of Economic Perspectives, 27(1), 173-96.

Chien, N., Winston, P., Gaddes, R., & Holzwart, R. (2021) “Risks that Come with Increasing Earnings for Low-Income Workers.” Office of the Assistant Secretary for Planning and Evaluation Report. Available at: https://aspe.hhs.gov/sites/default/files/documents/04efcaa192583caf5f53b98b80802ca6/MTR_Qual_Study_Brief_Risks.pdf

National Conference of State Legislators (2019). “Addressing Benefits Cliffs.” Available at: https://www.ncsl.org/research/human-services/addressing-benefits-cliffs.aspx

Section A — List of Attachments

Attachment A-1: Economic Risk Study Instrument (survey instrument)
Attachment A-2: Federal Register Notice
Attachment A-3: Comment on Federal Register Notice

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Econ Risk OMB SSA A
Author	MATHEMATICA
File Modified	0000-00-00
File Created	2022-05-27

51321 Econ Risk OMB SSA_5-18-22_rev2