Data sources

Data sources are tools, documents, and locations for information that can be designed to show what happened in your target audience.

• Anecdotes
• Surveys or feedback forms
• Observation or assessment reports
• Participant projects
• Other organization’s records or test information

Examine the strengths and weaknesses of each data source.

	Okay for attitudes, although you may just hear from happy people.  Worst for skills, as people will want to talk about what they’re good at. Good for information on outcomes you hadn’t anticipated. Good for quotable quotes.
	Surveys depend on “self-reporting”—fine for confidence in a skill, or interest in a subject—but not objective measures of skill levels or of behaviors. Best for attitudes because survey questions can include quantifiable rating scales. Weak for behaviors (depending on time between the experience and the survey), and weakest for skills. Surveys are good for important outcomes that you were able to anticipate. Look for standardized surveys (professionally and expertly designed) that cover the outcomes you’re interested in.  This may save time, but some may need to be adapted.
	With observation, an expert watches participant behaviors—mostly during a particular program—and can document objectively whether skills or behaviors have actually changed. Professional assessments include data sources such as teacher checklists of student behavior, portfolio assessment. Best for skills and short-term behaviors because they provide objective assessment by third-party observers. Bad for longer-term behaviors and for attitudes.
	Best for skills and some behaviors. Weakest for attitudes because it is impersonal. Example: Tip sheets created by participants in a bird identification workshop show that they’ve internalized knowledge about bird characteristics enough to articulate them clearly.
	Best for status or condition (Depending on the test or records). Good for knowledge. Weakest for behavior and attitude. Many programs start as a way to improve some indicator that already exists but is lagging: reading scores, grades, graduation rates, attendance, employment. Using these scores is inexpensive because they already exist, but both the organization that creates them and the individual participants need to cooperate to make them available to the program. Remember that your program may play only a small part in the change in scores.