Appendix B - Independent Quality Instruments_7

OMB Control No.: 0970-xxxx

Expiration Date: xx/xx/20xx

Appendix B: Independent Quality Instruments

This information collection is voluntary and will be used to learn whether the Head Start Designation Renewal System works as intended as a valid, reliable, and transparent method for identifying high-quality programs eligible for non-competitive five-year grants and as a system that encourages overall quality improvement. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number.  The OMB number for this collection is 0970-XXXX and expires XX/XX/XXX.

This appendix includes the instruments which will be used to independently measure the quality of Head Start programs, in order to assess the validity of the conditions applied through the Head Start Designation Renewal System. These include classroom observation instruments (B1), quality management assessments (B2), health and safety assessments (B3), and secondary data to examine the financial health of organizations (B4).

Appendix B1: Classroom Observation Instruments

The four classroom observational instruments are the Classroom Assessment Scoring System (CLASS), an abbreviated version of the Early Childhood Environmental Rating Scale Revised (ECERS-R), the Early Childhood Environmental Rating Scale Extension (ECERS-E), and classroom management items from the Adapted Teacher Style Rating Scale (Adapted TSRS). As the CLASS, ECERS-R, and ECERS-E are proprietary, and the Adapted TSRS is unpublished, we provide descriptions of the instruments and how they will be used rather than copies of the instruments.

Appendix B1.1: Classroom Assessment Scoring System (CLASS)

The CLASS (Pianta, La Paro, & Hamre, 2008) provides an assessment of the quality of teacher-child interactions. Its ten dimensions are organized into three domains. The Emotional Support domain includes positive climate, negative climate, teacher sensitivity, and regard for student perspectives. The Classroom Organization domain includes behavior management, productivity, and instructional learning formats. The Instructional Support domain includes concept development, quality of feedback, and language modeling. Observers will score each dimension within all three domains on a scale from 1 to 7 with higher scores indicating higher quality. This instrument is proprietary. More information can be found at http://www.teachstone.org/about-the-class/.

Appendix B1.2: Early Childhood Environment Rating Scale – Revised (ECERS-R)

The ECERS-R measures programs’ structure, provisions for learning, and teaching and interactions. The full scale includes subscales in the areas of: (1) Space and Furnishing, (2) Personal Care Routines, (3) Listening and Talking, (4) Activities, (5) Interaction, (6) Program Structure, and (7) Parents and Staff. The current study will use an abbreviated version that includes 22 items across two subscales: Interaction and Space and Furnishing (also used in FACES 2009; http://www.acf.hhs.gov/sites/default/files/opre/study_2009.pdf). Observers will score each item on a scale from 1 to 7 with higher scores indicating higher quality. This instrument is proprietary. More information can be found at http://ers.fpg.unc.edu/.

Appendix B1.3: Early Childhood Environment Rating Scale – Extension (ECERS-E)

The ECERS-E was developed to supplement the environmental rating scales and to measure practices in preschool classrooms specifically related to Literacy, Mathematics, Science and Environment, and Diversity (the four subscales of the measure). The instrument includes 15 items which observers will rate on a scale from 1 to 7 with higher scores indicating higher quality. This instrument is proprietary. More information can be found at http://store.tcpress.com/0807751502.shtml.

Appendix B1.4: Adapted Teacher Style Rating Scale (Adapted TSRS)

Domitrovich and colleagues (2008) used the original TSRS as a compliment to the CLASS in the Head Start REDI program because it focuses on the behavior a specific teacher. An adpated version of the TSRS instrument incorporating additional subscales, which will be used for this study, was created for The Head Start CARES study (Raver et. al., 2012). The Adapted TSRS, used for this evaluation, consists of 45 items which assess teaching style across 15 domains: consistency/routine, preparedness, classroom awareness, positive behavior management, negative behavior management, emotion modeling, emotion expression, emotion regulation, social awareness, social problem solving, provision of interpersonal support, attention/engagement, scaffolding dramatic play, scaffolding peer interaction, and talk aloud (Raver et. al., 2012). The current study will use the domains related to classroom structure and management only. Observers will rate each item on a scale from 1 to 5 with higher scores indicating higher quality.

Appendix B2: Quality Management Assessment

Program Administration Scale (PAS)

The PAS contains 25 items grouped into 10 subscales that measure leadership, management, and administrative practices of center based early childhood programs. The current study will include 20 items across nine subscales: Family Partnership; Child Assessment; Staff Qualifications; Programming, Planning and Evaluation; Center Operations; Human Resources Development; Marketing and Public Relations; Personnel Cost and Allocation; Fiscal Management. Observers will rate each item on a scale from 1 to 7 with higher scores indicating higher quality.

This instrument is proprietary. More information can be found at http://store.tcpress.com/0807752452.shtml

Appendix B3: Health and Safety

Child Care Health and Safety Checklist

Adapted from Healthy Young Children and the CCHP Health and Safety Checklist-Revised

Instructions for Data Collectors: Make observations when in the Head Start center. Request to see multiple rooms/areas, including bathrooms, kitchens/food prep areas, nap rooms, stairs, hallways, and outdoor areas.

Complete the ratings as you observe: Y if present, N if absent, and N/A if it was unable to be observed. Make notes as to the reason that you could not observe. After the observation portion of the assessment, you will speak with the center director to follow up on items that you could not observe or that were not clear in the observation process (see Quality Measures Follow Up Interview: Center Director, Appendix D).

INDOOR

Emergency Prevention/Poisons/Medications	Y	N	N/A	Notes
Emergency telephone numbers are posted for emergency services (for example, 911, poison control, fire department, police and children’s protective services.
Emergency procedures are posted for fire and earthquakes, and other regional natural disasters (tornados, floods, hurricanes) in each classroom.
Emergency lighting equipment works.
If natural disasters occur in a region, a 48-hour supply of food and water is stored for each child and staff member.
An evacuation map is posted in each classroom.
There is a smoke detector system, alarm, or sprinklers in working order in each room or place where children spend time.
Water that may be in direct contact with children is no more than 120F.
Earthquake safety: There are no heavy objects on open shelves and/or cabinets over 3 feet (for example, prevent falling objects).
Earthquake safety: All tall furniture over 4 feet (for example, cubbies, shelves) is bolted to the wall or floor.
First Aid: The program has at least one fully equipped, readily available first aid kit.
First Aid: The program has a separate fully equipped first aid kit for field trips.
Poisons: Cleaning agents are labeled and stored (except bleach solution) in their original containers.
Poisons: Cleaning agents are kept separate from food in cabinets and out of children’s reach.
Cleansers and other poisonous products are stored in their original containers, away from food, and out of children’s reach.
Pesticides are applied only to surfaces that children cannot reach and surfaces not in direct contact with food.
A certified pest control operator applies pesticides while observed by a caregiver.
Air quality: Temperature is between 65F and 75F during the winter months and between 68F and 82F during the summer months.
Air quality: There are no bad odors or fumes (for example, tobacco, mildew, urine/excrement, chemicals, air fresheners).
Are there children with medications at this program? Yes ____ -Continue with question # X –X No ____ -Skip to question #.
Medications are labeled with the child’s name.
Medications are stored out-of-reach of children and they are in their original and childproof container.
Medications are labeled with a date that shows they are not expired.
Medications are labeled with the administration instructions and are refrigerated (if needed).
The names of the children with special dietary needs and a description of their needs (including specific food allergies) are posted in food preparation area(s).
The names of the children with special dietary needs and a description of their needs (including specific food allergies) are posted in food service area(s).

General Building/Classroom Features	Y	N	N/A	Notes
Children’s personal belongings (including clothing and bedding) are stored so they do not touch others’ belongings.
Adult purses/backpacks are out of reach of children
All Heating units/fireplaces in children’s reach are covered with barriers.
All electrical outlets in children’s reach are covered.
Rugs, curtains, pillows, blankets, and cloth toys are flame resistant.
Hinges and joints are covered to prevent small fingers from being pinched or caught.
Handrails are securely mounted at child height.
Handrails are attached to walls for right-hand decent, but preferably are attached to the walls on both right and left sides.
Stairway gates are locked in place when infants or toddlers are nearby. Gates should have openings small enough to prevent a child’s head from fitting through. No accordion-type gates are used.
Doorways to unsupervised or unsafe areas are closed and locked unless the doors are used for emergency exits.
Emergency exit doors have easy-open latches.
Safety glass is used in all areas of potential impacts.
Caregivers can easily monitor all entrances and exits to keep out strangers.
Stairways and hallways are clear of objects that can cause a fall.
Pets are free from disease, are immunized as appropriate, and are maintained in a sanitary manner.
Doors to places that children can enter, such as bathrooms, can be easily opened from the outside by a child or by an adult.
Doors in children’s areas have see-though panes so children are visible to anyone opening the door.
Doors have slow closing devices and/or rubber gaskets on the edges to prevent finger pinching.
Glass doors and full-length windows have decals on them that are at eye levels of both children and adults.
Windows cannot be opened more than 6 inches from the bottom.
All windows have closed, permanent screens.
Bottom windows are lockable.
Walls and ceilings have no peeling paint and no cracked or falling plaster.
Electrical cords are out of children’s reach. Electrical cords are placed away from doorways and traffic paths.
Covers or guards for fans have openings small enough to keep children’s fingers out.
Free standing space heaters are not used.
Nobody smokes or has lighted cigarettes, matches, or lighters around children.
Trash is covered at all times and is stored away from heaters or other heat sources.
Drawers are closed to prevent tripping or bumps.
Sharp furniture edges are cushioned with cotton and masking tape or with commercial corner guards.
Regular lighting is good enough for good visibility in each room.
Enough staff members are always present to exit with children safely and quickly in case of an emergency.
All adults can easily view all areas used by children.

Toys and Equipment	Y	N	N/A	Notes
Number of adults _____ Number of children ______ Indoor staff to child ratios (# adults: # children) _____
Indoor climbing equipment is over approved mats that extend at least 6 feet from the structure.
Indoor climbing equipment is a maximum height (for example, 3 feet for 3 year olds, 4 feet for 4 year olds, 5 feet for 5 year olds).
Guns, projectile toys, darts, and cap pistols are not kept in the child care setting.
Toys and play equipment have no sharp edges or points, small parts, pinch points, chipped paint, splinters, or loose nuts or bolts.
All painted toys are free of lead.
Toys are put away when not in use.
Toys that are mouthed are washed after each use.
Children are not permitted to play with any type of plastic bag, balloon or latex/vinyl gloves.
Toys are too large to fit completely into a child’s mouth and have no small, detachable parts to cause choking. No coins, safety pins, or marbles for children under 4 years of age.
Toy chests have air holes and a lid support or have no lid. A lid that slams shut can cause pinching, head injuries or suffocation.
Shooting or projectile toys are not present
Commercial art materials are stored in their original containers out of children’s reach. The manufacturers label includes a reference to meeting ASTM Standards.
Cots are placed in such a way that walkways are clear for emergencies.
Teaching aids such as projectors are put away when not in use.

Hand washing/Toileting/Oral Health	Y	N	N/A	Notes
Proper hand washing procedures are posted at all hand washing sinks, including food preparation sinks
Children’s hand washing sink(s) are at child’s level or accessible by a safety step.
Staff wash their hands with liquid or foam soap and running water after each toileting/diapering of children over 35 months.
Staff wash their hands with liquid or foam soap and running water before food preparation and/or service.
Children (over 35 months) wash their hands or have their hands washed, with liquid or foam soap and running after each toileting/diapering. Total # of children observed after toileting/diapering ____ # of children observed washing correctly ____
All children wash their hands, or have their hands washed, with liquid or foam soap and running water before eating. Total # of children observed before eating ____ # of children observed washing correctly ____
All children wash their hands, or have their hands washed, with liquid or foam soap and running water after eating. Total # of children observed after eating ____ # of children observed washing correctly ____
All children (over 35 months of age) brush their own teeth or have their teeth brushed at least once a day.
Toothbrushes are labeled with the child’s name and stored safely.
Non-porous gloves are readily available for caregivers in all areas where child care is provided
Stable step stools are available where needed.
Electrical outlets have safety covers or are modified to prevent shock.
Electrical equipment is stored away from water.
Cleaning products and disinfectants are locked in a cabinet out of children’s reach.
Toilet paper is located where children can reach it without having to get up from the toilet.
If potty chairs are used, they are easy to clean with a bleach solution in a utility sink used only for that purpose, if possible.
Potty chairs are not used in the food preparation or dining areas, and potty chairs cannot be reached by children when they are not in use.
There are enough toilet seats so children do not have to stand in line.
Caregivers and children always wash their hands after toileting and diaper changing.
The changing of diapers or soiled underwear is done in a special , separate, area way from food and play
The diapering or changing table has rails to keep the child from rolling off.
Trash cans for diapers , tissues, and other materials that come in contact with body fluids can be opened with a step pedal and are lined with a plastic bag, emptied daily, and kept clean.
Paper towels and liquid soaps are readily available at the sink.
Thermometers are used to check that water temperatures are between 120 and 130 Fahrenheit or lower. The lower the water temperature the safer it is for young children.
Cosmetics are stored out of children’s reach.
Bathtubs have skid proof mats or stickers.
Children take baths only when adults can supervise.
Children are never left alone on a changing table, bed, or any other elevated surface.
Children are never left unsupervised in or near water.

Food Preparation/Eating/Sanitation	Y	N	N/A	Notes
No perishables, including food brought in by children (for example, meat, fish, poultry, milk, bottles of formula/breast milk, eggs) are left out of the refrigerator for more than one hour.
Food preparation areas are separate from the eating and play areas.
Food preparation and eating areas, including counters, tabletops and floors, are cleaned/swept before and after food preparation and meals.
Food preparation areas are separate from the toilet, bathroom and diaper-changing areas.
Refrigerators have a thermometer and are equal to or less than 40° F.
Children under 4 years old do not have food that causes choking, even if brought from home (for example, nuts, popcorn, candy, whole grapes, hot dog rounds, chunks of meat, spoonfuls of peanut butter, carrots).
A current weekly menu of all food and beverages served in the program is posted.
There is a designated area (for example, sink) or container, out of children’s reach, for mouthed or contaminated toys.
Caregivers always wash children’s hands before mealtimes.
Trash is always stored away from food preparation and storage areas.
All perishable foods are stored in covered containers at 40 Fahrenheit or lower.
Hot foods are kept at 140 Fahrenheit or higher until ready to be eaten.
Pest strips are not used.
Nonperishable food is stored in labeled, insect-resistant metal or plastic containers with tight lids.
Five gallon buckets are not accessible to the children.
Refrigerated medicines are kept in closed containers to prevent spills that would contaminate food.
Food preparation surfaces are clean and free of cracks and chips.
Eating utensils and dishes are clean, free of cracks, chips, and lead.
Appliances and sharp or hazardous cooking utensils are stored out of children’s reach.
Pot handles are always turned toward the back of the stove.
An ABC-type fire extinguisher is securely mounted on the wall near the stove.
All caregivers know how to use the fire extinguisher correctly and have seen a demonstration by members of the fire department.
There is a “danger zone” in front of the stove where the children are not allowed to go.
A sanitarian has inspected food preparation and service equipment and procedures within the past year.
Children are taught the meaning of “hot.”
Trash is stored away from the furnace, stove, and hot water heater.
Kitchen area is not available to children without constant adult supervision.
Caregivers do not cook while holding a child.
Hot foods and liquids are kept out of children’s reach.
Stable step stools are used to reach high places.

OUTDOOR & INDOOR PLAY AREAS

Equipment and Play Areas	Y	N	N/A	Notes
Number of adults _____ Number of children ______ Outdoor staff to child ratios (# adults: # children) _____
Adult staff are present in all outdoor areas and provide direct supervision.
All children are always visible by an adult staff member while outside.
Outdoor areas are maintained and are clean and safe; no trash, broken equipment, sharp objects, splinters, glass, animal excrement present.
Helmets are worn by all children when riding bicycles, tricycles or other riding toys/equipment.
Outdoor stationary playground equipment is stable and set in secure concrete settings and equipment does not wobble or tip when shaken.
All pieces of outdoor playground equipment are over 9 or more inches of shock-absorbing surfaces; loose fine sand, wood chips, mulch or approved rubber mats.
Shock-absorbing surfaces (fi ne sand, woodchips, mulch or approved mats) under outdoor playground equipment extend at least 6 feet beyond the perimeter of the equipment.
All fixed outdoor playground equipment has a minimum of 6 feet clearance space from walkways, buildings and other structures that are not used as part of play activities.
The outdoor play area is enclosed with a fence or natural barrier that allows observation of children.
Outdoor fences/barriers have at least two exits.
Outdoor fences/barriers have self-closing gates with self-latching mechanisms that cannot be opened by children.
Openings on outdoor playground equipment, fences and handrails are either less than 31/2 inches or greater than 9 inches wide.
Fall zones (the areas onto which a child falling from or exiting from a piece of play equipment would be expected to land) do not overlap.
Impact absorbing surface material extends at least 6 feet beyond all sides of the equipment.
For to-fro swings: the impact-absorbing surfacing material extends in front and in back of the swings a distance that measures twice the height of the swing beam.
For slides: : the impact-absorbing surfacing material extends at least 6 feet from the end of the slide chute-or-a distance that equals the height of the slide platform + 4 feet, whichever is greater. (It is not necessary for surfacing material to exceed 14 feet.)
The dirt in the play area has been tested and found free of toxic materials, including lead.
There are no toys or objects (including surfacing material) with a diameter small enough to completely fit in a child’s mouth accessible to children who are still placing objects in their mouths.
Measure the highest point that a child can climb to (critical height). For swings, the critical height is measured from the pivot point where the swing is suspended down to the ground. For elevated structures with guard rails, the critical height is measured from the top of guard rail down to the ground. The highest accessible part for platforms with protective barriers is the deck. For all other structures, the critical height is measured from the highest point of the structure down to the ground.
Surfaces underneath indoor and outdoor Play equipment that children can climb are covered with impact-absorbing material according to the CPSC recommendations for critical height.
The following surfacing materials are not in use underneath indoor and outdoor play equipment that children can climb: asphalt, concrete, soil or hard packed dirt, grass, turf, linoleum, or carpeting.
The active play areas offers a wide range if parallel and interactive activities.
Water for drinking and first aid is available near the play area.
A well-stocked first aid kit is accessible to all caregivers during outdoor play
A file is available containing the name and address of the manufacturer of each piece of equipment.
The file also contains records of equipment purchase, installation, inspection, maintenance, and CPSC/ASTM approval.
For old equipment, the file contains documentation of safety provided by an inspector who is certified by the national Playground Safety Institute (703) 858-2148.

Appendix B4: Secondary Data to Assess Financial Health

Tuckman & Chang (1991) Financial Ratios

Each year nonprofit organizations with gross receipts at or above $50,000 file Form 990 EZ or Form 990 with the US Internal Revenue Service which provides information about the revenues, assets, and expenses of nonprofit organizations. The current study will use data from Form 990 to examine the financial vulnerability of the nonprofit grantees participating in the study based on four vulnerability ratios related to equity balances, revenue concentration, administrative expenses, and operating margins (as described by Tuckman & Chang, 1991)¹.

1 Tuckman, Howard and Cyril Chang. (1991). Methodology for Measuring the Financial Vulnerability of Charitable Nonprofit Organization. Nonprofit and Voluntary Sector Quarterly 20 no.4: 445-460.

16