A learning assessment, in which we compared pre-test and post-test results.
A usability test, in which we conducted a cognitive walk-through with participants to get feedback on users' experience with the module. During the walk-through, we asked participants to talk us through their thought processes. We also asked questions to keep the flow of conversation as users completed the module.
Usability tests were conducted over Zoom or in-person. Pre-and post-tests were built into the module. Participants accessed the module from a Moodle course shell we set up for the occasion.
This project allowed us to gather valuable feedback and lessons learned to carry into Capstone development in Term 4.
Project Output: Evaluation Report
Select the button to view the summary report, which also includes detailed assessment data.
What we wanted to find out: Did learning place as evidenced by posttest scores being higher than pretest scores? In science terms, this was stated as:
H0 (Null Hypothesis): Mean Pretest Score will be equal to or greater than the Mean Posttest Scores; the module does not increase post-test scores.
H1: Mean Pretest Score will be less than the Mean Posttest Scores; the module increases post-test scores.
Eleven learners were tested before and after going through a Community Market Shopping Floor orientation module. The research team believes the module will enhance learners' knowledge about how to follow Community Market Shopping Floor procedures. The significance level is set to the conventional 0.5. T-test and effect size were calculated to determine the statistical and practical significance of the results.
Conclusion We discovered learning did take place, as the difference between pre-test and post-test results was meaningful and significant. This was expressed as: The t Stat absolute value is greater than the one-tail critical value; therefore, we reject the null hypothesis and conclude the tutorial is effective in teaching the learners about procedures when using the Community Market Shopping Floor. The effect size is greater than 0.8, and therefore, the result is practically significant as well. (|t|=6.38>1.81, p<.05, d=2.12)
This chart compares the number of missed questions on the prettest versus the posttest.
Usability Testing
We evaluated the original Captivate version of the module, which you can see here.
Following this project, we made several changes based on usability feedback (and we re-built the module using Storyline, which you can see here).