The Evaluation step in the ADDIE Model has been a large focus of my team (and stakeholders) attention as they are research faculty and they use this information to publish and further research based on our training activities and outcomes. Due to this we have pre and post assessment, pre and post belief survey, brief experience survey at the end of each course throughout our eLearning courses, and a final Training Satisfaction Survey at the end. We also include Knowledge Checks throughout the courses for formative assessment. I have voiced concern that we stay aware of learner survey/assessment overwhelm that could affect responses. Due to our eLearning courses being new to our training offerings, we are gathering large amounts of data to review, revise, and improve our training this first year.

E-learning Goals

The goal of this eLearning course is to prepare our learners on navigating this new program tool to assist with their CM Program fidelity and regulatory compliance. This course is the foundational introduction and step by step procedural information for using the REDCap platform and customizing a project for their site. We hold monthly coaching calls to further guide and assist learners on this tool application with their clients and organization. The three case studies at the end of the course is a summative assessment in the REDCap tool that provides a real world scenario by having the learner enter client data, recording visit information, reward distribution, test results. The instructor will grade these case studies, provide feedback, and use this data for coaching call training agenda items.

Kirkpatrick Levels

[List what your specific e-learning is evaluating at each level.]

Level 1 Reaction: This course is within a series of eLearning courses for our Implementation Training, which is Phase II of training. On our LMS, we have pre and post assessments, belief survey, course experience survey at the end of every course, and an over Training Satisfaction Survey at the end of Phase I. We are currently in development of Phase II having rolled out the majority of courses so we do not yet have a Training Satisfaction Survey at the end of this phase at this time. I do think it would be helpful to have a survey at the beginning of this specific course, REDCap Reward Tracker, to get an idea of the level of existing knowledge the learner has on the REDCap platform, if any. Again, I am working with the stakeholders and would need to get their support to do so.

Level 2 Learning: For most of our learners, the REDCap platform is new. This course is designed to assist learners with logging in to the platform (to include a module on Logging-in Troubleshooting Tips), navigating the platform, creating records, entering CM visit data, and the importance of tracking drug test results and rewards given in order to align with regulatory compliance. This course, within our series of course throughout Phase I-Education, and Phase II-Implementation, provides the information and tools to learn and deliver a new behavioral approach treatment to their clients. The three case studies at the end of the course will support learners knowledge acquisition in a real world scenario by using the REDCap platform.

Level 3 Behavior: This course is intended for foundational procedural knowledge on the REDCap platform. The learners also receive access to download a manual with all of the screenshots and narration for reference, along with unlimited access to the course. The REDCap Reward Tracker is an vital tool for implementation of the CM Program in terms of CM Program fidelity and recording for regulatory compliance. We continue training after this course through monthly coaching calls to assist learners with developing and implementing the CM Program at their site (which includes using this tracking tool). We created this tool after several years of finding other means of record keeping (i.e. Excel) to be unreliable and not efficient for recording all the information needed and calculating the rewards at each visit based on the drug testing results. We have found treatment sites really enjoy and excel at using this tool once they know how to navigate the platform.

Level 4 Results: As this course was just created and rolled out a month ago, we do not have the data yet on the newer cohorts performance on the platform.We currently have a couple of learners who have submitted their case study assignments and the instructor is in the process of grading. The instructor plans to use the data collected from the assignments to guide the coaching call training agendas to guide where common continued support is needed. We are excited to see the results of this course in terms of helping learners expedite implementation of their CM Program and have a more secure and efficient tool to use for the CM visits.

Assessment Techniques and Tools/ Evaluation Plan

ELID 530.Wk8.Summative Assessment.Vogel

REDCap Reward Tracker.Assessment Checklist

Questionnaire

1. Did you find the screenshots for the REDCap platform to be helpful when describing the steps?
2. Will you use reference the course with screenshots as you work through learning how to enter CM visits, or the manual, or neither?
3. What other information or delivery would you find helpful in learning this platform?