2012 Learning Management System (LMS) Evaluation

Goal

The purpose of the Learning Management System Evaluation was to determine the best online learning environment for PCC’s students and faculty. Currently the College is using Blackboard Learn 9.1 SP7.

The evaluation was prompted by the following reasons:

      • Our Blackboard contract renewal was fast approaching. The contract ends June 30, 2012.
      • A college-wide survey indicated sufficient dissatisfaction with Blackboard, our current LMS, to warrant investigation of alternatives.
      • Other, potentially more effective, systems are available at similar or lesser cost.
      • New developments in online learning methods and technology suggest a change may be appropriate.
      • We need to make the right decision for PCC’s future.

 

Process

The LMS Evaluation Steering Group was formed to systematically evaluate the two highest rated systems in the faculty survey (Canvas and Moodle) plus Blackboard (SP8). Faculty across divisions were invited to participate in this evaluation team, in addition to PCC students, administrators and staff. Thirty-one people accepted the task to represent their divisions/areas.

During the Spring 2012 semester, the group evaluated vendor presentations, migrated courses in each system, and examined additional resources to determine which LMS will best serve student and faculty needs.

The Decision

All processes and voting procedures were agreed upon by the Steering Group:

VOTING PROCEDURE
  • Anonymous voting using the clicker
  • Decision would be based on a majority vote
  • Everyone would support the group’s final decision
VOTING OPTIONS
  • OPTION 1: STAY with Blackboard for one more year and extend the LMS Evaluation process through Fall, 2012 with a final decision to be made by December, 2012
  • OPTION 2: RENEW Blackboard to start July 1, 2012 (3-year contract)
  • OPTION 3: SELECT Canvas to start July 1, 2012
  • OPTION 4: SELECT Moodlerooms to start July 1, 2012
FINAL VOTING RESULT Select Canvas to start July 1, 2012

Full Steering Group Evaluation Results

Overall Ratings (On a scale of 1-10)

Canvas — 7.8 ;  Moodlerooms — 5.5 ;  Blackboard —  4.5

Individual Features, Functions & Capabilities. Blackboard: Meets 508 Compliance needs, slightly meets User Interface needs, slightly meets Customization needs, slightly meets Calendar needs, meets Communication Tool needs, meets Chat/Interactive Tool needs, slighly meets Discussion Board/Forum needs, slightly meets E-Portfolio needs, slightly meets Course Content needs, slightly meets Multi-Media Support needs, slightly meets Adaptive Release needs, slightly meets Assignment Submission needs, slightly meets Submitted Assignments needs, slightly meets Anit-Plagiarism needs, meets Groups needs, meets Blogs/Wikis nees, meets Tests/Quizzes needs, barely meets Gradebook/Grade Center needs, barely meets Rubrics needs, slightly meets Self & Peer Assessment needs, barely meets Student Learning needs, barely meets Evaluation Tool needs, meets Syllabus needs, meets Roster needs, slightly meets Mobile Device needs.Moodlerooms: Meets 508 Compliance needs, meets User Interface needs, slightly meets Customization needs, meets Calendar needs, slightly meets Communication Tool needs, meets Chat/Interactive Tool needs, meets Discussion Board/Forum needs, meets E-Portfolio needs, meets Course Content needs, meets Multi-Media Support needs, meets Adaptive Release needs, meets Assignment Submission needs, almost exceeds Submitted Assignments needs, meets Anit-Plagiarism needs, meets Groups needs, meets Blogs/Wikis nees, meets Tests/Quizzes needs, meets Gradebook/Grade Center needs, meets Rubrics needs, meets Self & Peer Assessment needs,  meets Student Learning needs, meets Evaluation Tool needs, meets Syllabus needs, meets Roster needs, slightly meets Mobile Device needs.Canvas: Meets 508 Compliance needs, Exceeds User Interface needs, nearly exceeds Customization needs, exceeds Calendar needs, exceeds meets Communication Tool needs, exceeds Chat/Interactive Tool needs, exceeds Discussion Board/Forum needs, meets E-Portfolio needs, exceeds Course Content needs, exceeds Multi-Media Support needs, exceeds Adaptive Release needs, exceeds Assignment Submission needs, exceeds Submitted Assignments needs, exceeds Anit-Plagiarism needs, nearly exceeds Groups needs, meets Blogs/Wikis nees, exceeds Tests/Quizzes needs, exceeds Gradebook/Grade Center needs, exceeds Rubrics needs, exceeds Self & Peer Assessment needs,  exceeds Student Learning needs, exceeds Evaluation Tool needs, nearly exceeds Syllabus needs, exceeds Roster needs, exceeds Mobile Device needs.
Administrative & Support Functions. Blackboard: barely meets Migration needs, barely meets Section Merge needs, meets E-Pacs/Content Import needs, meets Multi-platform/Browser needs, barely meets Performance/Reliablility needs, doesn't meet Course Copy needs, meets Course Archiving needs, barely meets Training Options/Help needs.Moodlerooms: barely meets Migration needs, meets Section Merge needs, meets E-Pacs/Content Import needs, meets Multi-platform/Browser needs, meets Performance/Reliablility needs, meet Course Copy needs, meets Course Archiving needs, meets Training Options/Help needs.Canvas: meets Migration needs, nearly exceeds Section Merge needs, nearly exceeds E-Pacs/Content Import needs, nearly exceeds Multi-platform/Browser needs, exceeds Performance/Reliablility needs, nearly exceeds Course Copy needs, nearly exceeds Course Archiving needs, nearly exceeds Training Options/Help needs.

Presentations

All three vendors visited PCC to present their products. Faculty were asked to evaluate these presentations and their responses were used to inform the steering group’s final decision.  The presentation evaluations were not comparative.The results represent the initial response to the product after each presentation.

Presentation Evaluation Results

Average Rating: Blackboard: 6.0 for User Interface, 4.5 for Course Calendar, 5.9 for Comm. Capability, 5.9 for Course Content, 5.8 for Disc. Board/Forum, 7.1 for Student Learning, 6.8 for Grades, 6.0 for Mobile Devices, 6.0 for Other Areas, 6.1 for Overall Rating, 6.0 for Average of all Areas. Moodlerooms: 7.2 for User Interface, 7.1 for Course Calendar, 7.0 for Comm. Capability, 7.3 for Course Content, 7.8 for Disc. Board/Forum, 7.1 for Student Learning, 7.3 for Grades, 7.3 for Mobile Devices, 6.2 for Other Areas, 7.3 for Overall Rating, 7.1 for Average of all Areas. Canvas: 9.0 for User Interface, 9.2 for Course Calendar, 9.3 for Comm. Capability, 9.0 for Course Content, 9.0 for Disc. Board/Forum, 9.1 for Student Learning, 9.1 for Grades, 9.0 for Mobile Devices, 9.5 for Other Areas, 9.1 for Overall Rating, 9.1 for Average of all Areas.

Last Updated on: October 8, 2012 @ 6:09 pm