SERVICES |
 |
"DIR provides high quality . . . market reserach at an exceptional value." |
|
| |
|
| |
Research and Evaluation Project Descriptions
____________________________________________________ |
| |
Designs & Methods
National Evaluation of Early Reading First
The U.S. Department of Education's (ED) National Center for Education Evaluation (NCEE) contracted with DIR to conduct a national evaluation of the Early Reading First (ERF) program. For the evaluation, DIR used a regression discontinuity design to determine the extent to which the additional funds and technical assistance given to ERF grantees change instructional content and children's outcomes compared
to the content and outcomes in the absence of ERF. DIR used computer-assisted telephone interviewing to conduct surveys with parents, used self-administered surveys for teachers, made classroom observations, and performed child assessments. In June 2007, ED published DIR's findings as a report to Congress. See http://ies.ed.gov/ncee/pdf/20074007.pdf.
DIR participated in the national evaluation of the 21st Century Community Learning Centers program funded by the U.S. Department of Education. This longitudinal evaluation included both process and impact components, such as measurement of student achievement. The evaluation design included two parts: an elementary-school component that used random assignment and a middle-school component that used a comparison design. DIR participated in establishing the evaluation sample, developing instruments, conducting site visits, and analyzing data.
The National Office of Job Corps (NOJC) contracted with DIR to conduct a multiyear evaluation of the Bridges Learning Systems (Bridges) eXcelerate
program, including its Infusion variant. To measure the impact of the eXcelerate program, DIR used a modified quasi-experimental design that matched eXcelerate and similar non-eXcelerate participants. DIR performed an analysis of quantitative data and examined outcomes in the areas of retention; type of separation; learning gains; on-center completions of vocational training, GED, or high-school diploma, or all three; and placement. We compared the outcomes of these studies and assessed the extent to which the eXcelerate program may have contributed to differences in those outcomes.
DIR conducted a multiphase, organization-wide climate study that combined
the use of employee surveys and a series of focus groups for NEA. DIR conducted three rounds of data collection in which surveys were mailed to all employees for them to complete and return to DIR. NEA employees received the survey at their home address with a cover letter form NEA's executive director requesting them to complete the survey. DIR performed all analysis and presented results to the full staff of the NEA.
At the request of Duke Energy Gas Transmission's diversity consultant, DIR designed and implemented a survey of employees in the Houston and Charlotte locations. DIR researchers developed employee opinion surveys which were self-administered.
The City of Houston's Building Services Department contracted with DIR to assess the human services needs of communities served by multi-service centers operated by the city's Health and Human Services Department. DIR used a comprehensive approach to data collection and analysis, including a telephone survey of 2,500 randomly selected households; interviews and focus groups with key informants; analysis of primary and secondary data from a variety of sources, including the 2000 United States census; periodic meetings with City staff; and a review of previous studies.
Rice University contracted with DIR to conduct an evaluation of its Mathematics Leadership Institute (MLI). The Rice University School Mathematics Project (RUSMP) created the MLI as a result of its collaborative efforts with the directors of mathematics for two Houston-area school districts and Rice University faculty members from the computer and applied mathematics, mathematics, and statistics departments. The MLI partnership is intended to develop mathematics teachers who initiate and sustain meaningful mathematics instruction on their campuses in each of the participating school districts.
The W.K. Kellogg Foundation (WKKF) contracted with DIR to evaluate its
$20 million blanket appropriation for building the organizational capacity and financial sustainability of nonprofit organizations (NPOs). DIR worked with WKKF to develop an evaluation plan and a logic (or conceptual) model that describes how the ideas behind the blanket are translated into measurable program services and outcomes.
Data Analysis
DIR was selected by the U.S. Department of Labor to conduct a 5-year comprehensive evaluation of the Youth Opportunity (YO) grant initiative. The evaluation included two rounds of an area survey of over 18,000 youths to measure labor-market outcomes in YO sites; an ethnographic study to assess community well-being before and after delivery of YO grant services; and a process analysis to document how programs are designed and implemented to meet the employment, training, and educational needs of area youths. All of the data collected throughout the study have been synthesized, analyzed, and reported in one comprehensive report on the impact of the YO grant.
DIR used a system of peer review to conduct an interim evaluation of the Regional Educational Laboratories for the U.S. Department of Education. Each laboratory
was evaluated individually by a 5- or
6-member peer-review panel using standardized criteria for assessing implementation, management, quality, utility, outcomes, and impacts. Four functional areas were analyzed: applied research and development; services to the field; strategic alliances and learning communities; and the laboratories' chosen specialty areas. As a follow-up to the interim evaluation, DIR developed a framework and methods for a market analysis of the laboratories' system.
DIR provided analysis and technical assistance to help the National Office of Job Corps (NOJC) develop policies
to meet requirements of the Workforce Investment Act (WIA). DIR assisted the NOJC to develop a comprehensive strategic plan to restructure its placement services, post-center services, and student tracking systems. DIR performed a comprehensive analysis of the placement component; designed and developed a database and management-information system; performed an analysis and evaluation of pilot projects for 16- and 17-year-old students; and provided technical assistance, training, and staff development on
this and related projects specified by NOJC.
The U.S. Department of Education awarded a contract to DIR to determine the extent to which participants in the Ronald E. McNair Postbaccalaureate Achievement Program complete doctoral studies and obtain faculty or research positions at institutions of higher education. DIR identified McNair program participants, constructed a sampling frame, and designed the follow-up survey. Items in the survey instrument were created with the specific intent of obtaining data to answer the research questions of interest regarding services received, satisfaction with services, educational and employment experiences, and demographic characteristics. DIR used computer-assisted telephone interviewing (CATI) to administer the follow-up survey and obtain information about participants.
The National Center for Education Statistics of the U.S. Department of Education contracted with DIR to assess and analyze the reliability and validity of IPEDS
data that were collected by using Web-based procedures in 2002 and 2003. DIR compared IPEDS data to non-IPEDS sources and analyzed eight IPEDS components: tuition and price, employees by assigned position,enrollment, completions, student financial aid, finance, salaries, and graduation rates.
|
|
