Frequently Asked Questions
Frequently Asked Questions About Cobblestone Applied Research and Evaluation, Inc.
What do you offer that’s unique?
Our staff is well versed in conducting both applied research and
evaluation. Many evaluators lack formal training in evaluation,
but our staff has formal training in evaluation theory, evaluation
planning and management, research methods, and statistics. Additionally, our company is committed to working closely with clients
through all steps of the evaluation process. Our ability to
maintain close communication with our clients is one of the strengths of
our organization. Clients have also been impressed with our willingness
to assist in filing federal reports, as well as local, tailored reports
and presentations
What do you charge?
Evaluation typically cost 8-12% of your project budget, however, budgets
are constructed on a case-by-case basis depending on the scope of the
evaluation work and timeline for the project.
Who oversees your work and holds you responsible?
We are accountable, first and foremost, to you, the client. In our
profession, our reputation hinges on the quality and integrity of the
work we conduct for our clients. We also follow the Guiding
Principles for Evaluators published by the American Evaluation
Association and the Program Evaluation Standards published by the Joint
Committee on Standards for Educational Evaluations.
Will you submit our federal report for us?
Yes, we are pleased to offer assistance in the submission of federal
reports.
Do you assist in writing grant proposals?
Yes. We have experience assisting in the grant writing process and
in joining the project well into the grant funding cycle.
Can you still evaluate my program even if the program is not located in
California?
Cobblestone Applied Research and Evaluation, Inc. has experience
conducting evaluations on a national scale and with clients from across
the United States. We work with the client through regular
communication via phone, email, and will travel when in-person meetings
are necessary.
What can I expect from our working relationship?
Our working relationships with clients are characterized by close
contact and regular communication. We will work closely with your
organization to provide the most useful and effective evaluation
product.
Frequently Asked Questions About Evaluation
What is evaluation?
There are several different definitions of evaluation, but generally
speaking evaluation is the use of social science research methods to
systematically assess the merit, worth, or value of a program or
product.
I hear the terms “formative” and “summative” a lot, what do they mean?
Evaluations tend to fall into two categories: formative or summative. Formative evaluation typically occurs during the development of a
program for the purpose of improving the program. Summative
evaluation typically occurs once the program is stabilized for the
purpose of informing a judgment about whether to continue, expand, or
discontinue the program.
How do I know what type of evaluation I need?
If you are unsure of the type of evaluation you need, Cobblestone
Applied Research and Evaluation, Inc. will work with you to answer that
question. Different clients have different needs and no two
evaluations are exactly the same. Commonly, evaluation needs fit
into the following categories: needs assessment (assessing what needs
exist), a process or implementation evaluation (assessing how a program
is being implemented), an outcome evaluation (assessing program
outcomes), or an impact evaluation (assessing the impact of the program,
which includes detecting, interpreting, and analyzing program effects). Cobblestone Applied Research and Evaluation, Inc. can also help you to
express your program theory.
What is a program theory?
A program theory is the logic that connects your program’s activities to
its intended outcomes. In describing a program theory the implicit
and explicit assumptions related to the program are revealed. A
clear program theory helps you describe your program to others,
including potential funding sources and helps guide evaluation of your
program.
What is the difference between research and evaluation?
Applied research and evaluation are quite similar. The clearest
distinction between the two forms of social inquiry is that evaluation
is client-driven, while applied research is researcher-driven.
Applied research answers research questions that aim to contribute
knowledge to the field, while evaluation answers evaluation questions
that aim to contribute information about the merit, worth, or value of
something. Evaluators are more likely than researchers to focus on
stakeholders (those who have a stake in the object of the evaluation or
research). Applied researchers ground their research in social
science theory, while inclusion of social science theory is not a
necessary component of evaluation. While the production of
research and evaluation knowledge is similar the purpose and use of
applied research predominantly differs from evaluation, although there
is overlap.
Why do I need an evaluator?
You need an evaluator because evaluators are trained and experienced in
designing, managing, and executing evaluation studies in a timely and
cost-effective manner. Additionally, hiring an evaluator who is
external to your organization can increase the objectivity of the
evaluation findings, lending the findings more credibility.
What should I look for in an evaluator?
You want an evaluator who is experienced conducting evaluations and has
a good track record. Look at their previous work and ask for
references. Check to see whether the evaluator has formal training
in evaluation. Is the evaluator a member of relevant professional
associations like the American Evaluation Association? Can the
evaluator articulate their model or philosophy for conducting
evaluation? Does their approach to evaluation fit with your
evaluation needs? Finally, do you get along with the evaluator? Having a positive working relationship with your evaluator makes for a
more enjoyable and productive experience.
How involved do I need to be in the evaluation?
Level of involvement varies according to the evaluation approach used
and your preference. At the start of the evaluation, you and your
evaluator will agree upon the level of involvement that seems
appropriate for that particular evaluation. Involvement can range
from very little to a lot, depending on time and interest.
When is the best time to start an evaluation?
We prefer to start an evaluation at the initial stages of program
development; however, it is rarely too late to conduct an evaluation.
The advantage of starting an evaluation in the early stages of a program
is the ability to construct program goals that are feasible and
measurable and to initiate an ongoing system for tracking program
outcomes.
What kinds of programs can be evaluated?
There are no restrictions on the type of programs that can be evaluated.
What if the evaluation finds negative results?
This is a common fear associated with having your program evaluated.
Rest assure that evaluations provide information about what your program
does well, as well as information about where your program could
improve. In all of our years of conducting evaluation we have
never come across a program that has no redeemable qualities. Most
evaluations are geared toward improving your program and helping your
program accomplish its goals. Often times evaluations can feel
like they are being done to you, but Cobblestone Applied Research and
Evaluation, Inc. will include you in the evaluation process so that
there are no surprises at the conclusion of the evaluation.
Do you share the findings of my program with other people?
The results of evaluations are only shared with the consent of the
client. Prior to starting the evaluation, we draft a dissemination
plan with you that outlines how the results of the evaluation will be
shared. Commonly, we deliver an evaluation report and executive
summary to the client. We can also share findings and make
presentations to stakeholder groups as needed. Sometimes clients
want to pursue writing publishable articles or presenting at
professional conferences, both of which we have experience doing with
our clients.
I have heard that I need a content expert in my field to accurately
evaluate my program. Is this true?
Having a content expert involved in the evaluation is useful, but not
necessary. Commonly, the evaluator provides the evaluation
expertise while the client provides the expertise about the program and
its content. The Cobblestone Applied Research and Evaluation, Inc.
team conducts evaluations to the best of our abilities, utilizing the
content expertise of our team. In a situation where we feel unable to
complete evaluation tasks based on our training, we seek outside
consultants to advise our evaluation processes. For example, in
one of our recent evaluations of a chemistry curriculum we hired a
chemistry professor to craft our chemistry content assessment to ensure
that mastery of the exam would accurately reflect a deep understanding
of chemistry.
