How do I Choose Which Aspects to Evaluate?
As you’ve learned thus far, there are many ways to approach program evaluation. To help you choose, start by thinking about these broad questions:
- Which aspects of your program are currently evaluated?
- Who is involved in gathering information about those aspects?
- Who uses the information and how do they use it?
- Which aspects of your program are not currently evaluated? Do you think they should be? Why or why not?
Clearly no LPD can continuously evaluate all of the aspects of a language program described in the previous section, so thinking about your program’s context can help you prioritize them. Below are some guiding questions to help you decide; click on each one for details and examples.
1. What information do I need to improve my program (a “needs analysis”)?
For example, if you already have plenty of evidence that writing is a weakness for students, you don’t need to prove that again. However, you may need to drill down to figure out which aspects of writing need more support, or you may ask potential employers what kinds of writing are needed in their profession. You should also consider whether any of your stakeholders require information from you and your program to inform their decisions for supporting you.
2. What do I want to do with the information I gather?
You can make use of information in almost any aspect of your multiple roles, from informing curriculum or materials revision to determining priorities for instructor professional development to designing transitional activities for students leaving your program for upper-level courses. It can be helpful to ask yourself which aspects raise concerns or which are most important to your program’s success.
3. What information do I already have (or can I get access to) that would help me choose actions to take to improve my program?
You may already have student performance data through your learning management system that you could mine, for instance, or your chair or institutional research office may have enrollment data that you could request. Strategically evaluate and use that information before collecting new data.
4. Do I have academic or contractual constraints?
As an LPD, you may be required to evaluate teacher performance at specific time points or be prohibited from doing so at other time points. Likewise, your academic calendar may influence when you can change course materials or offer professional development to instructors. Knowing these constraints not only informs your priorities but also clarifies how much capacity you may have for additional assessment efforts.
For more information to help you think through these questions, see Davis & McKay (2018), referenced on the Resources page. The authors provide a “step-by-step guide for evaluation novices within language educational institutions who are responsible for evaluation activities” (p. 4) that includes an example of a needs analysis case, possible indicators, and data collection methods.