Evaluation

In the Evaluation phase of ADDIE, the learning intervention and the degree to which learning outcomes were met are assessed. This evaluation should occur during all phases of the ADDIE process to facilitate an effective pre-implementation and to revise future iterations of the learning intervention. My evaluation plan outlines my approach to assessing student learning within my course, given that there is no instructor, as well as how to collect data to improve my course in the future.

Kirkpatrick Levels

For this course, Kirkpatrick Level 1 (Reaction) and Level 2 (Learning) are most applicable for evaluation. This course is a conceptual project envisioned to be used by First-Year Experience program offices as one of many similar courses on different topics, or as a single lesson in a full First-Year seminar course.

As such, evaluation at Levels 3 (Learner Behavior) and 4 (Organizational Results) would be specific to the program offices or academic departments that would be deploying this course in their student learning efforts.

Level 1 Variables

Engagement – For this first version of the course, it will be important to know which sections were most engaging.

Relevance – Even though it is known based on student success research the relevance of wellness and time management to student success, it would be valuable to evaluate the extent to which learners themselves make this connection as a result of the course.

Learner satisfaction – It would be valuable to evaluate the extent students felt this was an enjoyable way to learn the material and a good use of their time.

Level 2 Variables

Knowledge, Skills, Confidence, and Commitment – The course content covers measurable learning outcomes that tie to both acquiring new knowledge (wellness dimensions and time-blocking) and applying new skills (time-blocking implementation, prioritization). It would be valuable to evaluate the change in knowledge and skills, as well as students’ confidence and commitment in their ability to implement changes in their time-management approach.

Assessment Techniques and Tools

Assessment Techniques

As this will be learner-paced and not instructor-led, this course will not use formal grading. However, with the goal of the course being able to plug in as an individual lesson in a First-Year Seminar course, I do envision each of the learning activities as being able to be graded assignments, most of which will have rubrics embedded in the course by default.

The course will include formative assessment in the form of learning activities taking place within each section. Some sections will include reflective questions to guide students in individual reflection on the material. For example, after completing a dimensions of wellness inventory, learners will be asked to reflect on their results and consider which areas they may want to give more time and attention to.

To build in opportunities for social learning, the course will also include discussion boards to facilitate student interaction with the material and each other. For example, one discussion board question will ask learners to name activities they engage in for each of the 6 Dimensions of Wellness covered in the section. This will also serve as a declarative assessment activity, giving learners the opportunity to categorize activities within the framework.

The summative assessment for the course will be the completion of an action plan that ties in the elements of the entire course. This reflective action plan will guide them in naming the time management tools they plan to use (digital/tactile/combination), in naming their priorities for the academic term using the wellness framework, in crafting a time blocked calendar template they can use each week, and in considering other time management skills they may want to learn more about. The action plan will serve as a set of procedural assessment exercises, guiding students in practicing the step-by-step process of effective time-blocking, prioritization, and scheduling.

The assessment tools and activities presented to learners will be primarily qualitative. Given that the content in the course focuses on soft time management skills and making meaning of the dimensions of wellness, there aren’t rigid right or wrong answers. Students will be presented opportunities to learn from the qualitative responses of their peers and make meaning of the material together and individually.

To achieve reliability and validity of the assessments for this course, I will include rubrics that are specific in what is asked of learners, and ensure the rubrics and activities connect back to the learning outcomes. These rubrics, as well as the social discourse and engagement from their peers, will serve as the two primary feedback methods in the course.

Assessment Tools

Discussion Board Questions

“What tasks and activities do you think successful college students spend the most time doing? Which activities are most important?” – Padlet Link

“What’s an activity you spend time doing that helps you thrive in each area? Why is that activity helpful for you? Or if you are struggling in an area, what’s an activity you want to try and why?” – Padlet Link

Reflection Questions

Reflection questions will be embedded into the module using a plug-in that will allow learners to write free responses and download them as a PDF to save their work and include as artifacts in their action plan.

Wellness Reflection Questions

After completing a Wellness Wheel Self-Assessment, learners will be asked to reflect on their results by answering the following questions:

1. 1. Which of the categories ranked the highest in your results? Lowest
   2. What do you like about your wheel? What would you like to change?
   3. Based on your results, which areas would you like to prioritize more time and energy towards during this academic term?

Time-Blocking Reflection Questions

To prepare for practicing time-blocking strategies, learners will be asked to answer the following questions:

1. 1. Which types of tools (digital and/or tactile) do you think you’ll use in your time management approach?
   2. To prepare for time-blocking, jot down below all the commitments/tasks you can think of that you may need to complete in the coming week.
   3. When is the best time for you to do certain tasks? When are you most alert and focused for deep work?

Rubrics

As a way to provide feedback and guide learners with their reflection question responses, I will include rubrics in the form of checklists. Learners will be guided to refer to the checklists as they are drafting their responses as a way to self-review their work.

End of Course Questionnaire

1. I found the topic to be relevant to my success as a student (Strongly Agree, Agree, Neutral, Disagree, Strongly Disagree)
2. I found the course and activities to be engaging (Strongly Agree, Agree, Neutral, Disagree, Strongly Disagree)
3. I would recommend this course to others (Strongly Agree, Agree, Neutral, Disagree, Strongly Disagree)
4. I learned a new time-management strategy or tool that I am going to try this academic term (Strongly Agree, Agree, Neutral, Disagree, Strongly Disagree)
5. How likely are you to try one or more new activities or habits as a result of the wellness lesson? (Definitely Yes, Yes, No, Definitely No)
6. Any suggestions for how we could improve this course? (Open-ended)