Problem Database

	A	B	C	D	E	F
1		Problem	Heuristic #	Course (FS = Fire Service, MG = Master Gardner)	Severity (1 = Minor, 5 = Major)	Note: Three researchers evaluated the severity of the problems. Either the consensus is listed (based on consensus of 2 out of 3 raters) or the range of the severity levels is listed if the raters did not agree.
2	1	Learning objectives are not clearly stated and are not located at an identifiable place.	15	FS	5
3	2	It is unclear whether the learning objectives are appropriately designed for the level of the course as they are written as learning outcomes.	15	FS	3
4	3	Homepage has no direction in form of text description or hypertext link to lead to the syllabus. It is unclear how students will navigate to the syllabus to locate the learning objectives and goals	15	FS	4
5	4	The objectives/ goals of the course and each module are absent and it is unclear how learners know what objectives/ goals they can achieve in the course.	15	FS	5
6	5	Objectives are not present in the syllabus or homepage for the course or individual modules.. It is unclear if the instructor has developed the objectives and plan of learning based on Bloom's taxonomy.	15	FS	4
7	6	Course competencies are not clear and mistaken for course activities.	15	FS	4
8	7	Course was not provided with a clear and detailed course schedule within the syllabus.	15	FS	4
9	8	Syllabus does not contain information regarding instructor presence and response time on assignments on the syllabus.	14	FS	3
10	9	Syllabus does not contain information regarding ‘Drop/withdraw/return’ policy.	14	FS	4
11	10	Homepage does provide the course summary. However, the summary does not provide a picture of how many modules there are in the course, what each module is about, and when each module starts and ends.	14	FS	3
12	11	This section in the syllabus is named Course Competencies and Assessment but there is no Assessment activities listed in this section.	14	FS	4
13	12	There is no description of what content will be covered in each week.	14	FS	4
14	13	The summary does not have titles for each week’s content.	14	FS	3
15	14	Homepage does not include a brief description of the course. The syllabus contains course overview but the homepage has no directions for the students to understand where course syllabus and overview is located.	14	FS	5
16	15	Syllabus information is not present on the homepage of the course and no direction is available for the students to reach/access the course syllabus section. Instead, Syllabus is included in the modules, however, no information is provided to lead the students to the modules to find the syllabus of the course.	16	FS	5
17	16	From the front-page it is confusing how to start the course. I could go back to the list and click the first link or from the module select the first. Users usually expect to see a ‘Next Page” or “Start Here” when they get to the bottom of a page.	16	FS	5
18	17	In case of technical or instructional difficulty, contact info is easily accessible.	16	FS	3
19	18	Live link to email instructors is not present.	16	FS	3
20	19	How to “get started’ is not present in the Homepage.	16	FS	4
21	20	The course does not provide help documentation for issues learners might encounter like they do not know how to find grades.	16	FS	3
22	21	Students can get technical assistant from the instructors of the course but the course lacks a FAQ system.	16	FS	4
23	22	The course lack flexibility in type of material and the whole course is asynchronous.	16	FS	4
24	23	The assignment instructions have the format like an overview of a module. Also, guides for the presentation are mistaken with learning outcomes. Criteria to evaluate performance is not clearly articulated.	2	FS	4
25	24	Each module starts with tasks after reading rather than provide learners with the objectives of the module and what will be covered.	2	FS	3
26	25	It is unclear in the notetaking guide regarding what information learners are supposed to fill in the note taking guide when the note taking guide also provides them with the summary of each chapter.	2	FS	4
27	26	How each element rounding up the final grade was not provided.	2	FS	3
28	27	Self-introduction of the instructor online was not provided.	13	FS	4
29	28	The first page of the module started with an assignment without any “Welcome” note or introduction. It can confuse the users if they are on the right page to start learning.	1	FS	3
30	29	Content is spread out and there is no summary page/window. Students may not remember where they left off.	1	FS	4
31	30	It might be hard for learners to locate their grades and other elements like files, quizzes, etc. in the course.	3	FS	4
32	31	General course and institutional policies were missing, link to the policies was not provided.	3	FS	5
33	32	Images are not provided with any alternative text or caption.	8	FS	5
34	33	The course does not provide accessibility statements.	8	FS	5
35	34	The course does not provide Alt-text for the pictures.	8	FS	3/4/5
36	35	There is no information about accessibility for technologies and materials required in the course.	8	FS	5
37	36	Content in the Menu is not grouped and organized by type; some elements are missing. For example, as the image displays that Discussion is not listed as a menu item. Similarly, neither syllabus is not listed as a menu item, nor Grades.	8	FS	5
38	37	Course Summary on homepage and Schedule in syllabus look similar but are confusing because of different terminologies and timeline format used for the same content (whether both sections mean the same).	4	FS	4
39	38	The design and presentation of the layout of youtube videos are not consistent. Size of media is not the same all throughout the course.	4	FS	2
40	39	The links are not recognizable from non-links. Even hovering the cursor on the section does not yield to a meaningful result.	4	FS	3
41	40	A long list without break and all capital letters decreases the visibility of the text. All Capital words should be used with a specific purpose that is usually drawing attention	4	FS	4
42	41	The font size of the slides is expectantly small. Also, the space in the page has not been properly utilized.	4	FS	2
43	42	there is lack of instruction of what do with various hyperlink.	4	FS	2
44	43	Information sections in the homepage are inappropriately spaced. The announcement takes up significant space in the Homepage. Learners have to scroll down to see what the course content is about.	4	FS	2
45	44	My Media and Media gallery pages are listed in the Navigation menu but not used as part of the course.	4	FS	1
46	45	Colors for different levels of headings and text in the note-taking handouts are not consistently used.	4	FS	2
47	46	Videos are presented with different sizes.	4	FS	2
48	47	Video labels are not consistently placed. In one video, video label is placed in the middle of the page. In the other videos in the same page, their labels are placed at the beginning of the description of what the video is about.	4	FS	2
49	48	The font size regarding contact information is not consistent with the rest of the course.	4	FS	2
50	49	Some pages are very wordy and content is not partitioned in block.	4	FS	2
51	50	Assignments, quizzes, discussions, syllabus, grades, and files are not present in the main navigation menu, which makes it hard for learners to quickly access these elements.	4	FS	4
52	51	Resources are not listed in the main navigation menu.	5	FS	4
53	52	Actions buttons are too close to each other.	5	FS	3
54	53	The order of content for each module in the summary does not match with the order in the module.	6	FS	4
55	54	Feedback is not consistent. Feedback for each quiz is different. Quiz shows the submission details with correct answers hidden, precisely no label as ‘correct’ for the submitted quiz questions. Even for the full scored quiz feedback is not consistent as submission details were hidden totally, not showing any submitted quiz questions in one quiz and showing all the details in another quiz.	6	FS	4
56	55	the videos in the page are extremely long and exceeds the attention span of a regular learner.	7	FS	5
57	56	The content is presented without emotion and soulless	9	FS	3
58	57	no instruction was provided how to initiate a conversation or post a comment.	9	FS	5
59	58	Information is not well grouped. Course resources, course project resources, syllabus, module content etc. are all grouped under Modules.	10	FS	4
60	59	The assessments and assignments in this course are not diverse and there is an imbalance in learning content and activities between module 1 and the other modules.	10	FS	4
61	60	There are 3 technology tools used in this course: discussion boards, chat rooms, and videos as technology tools. However, chat rooms are not frequently used by learners. Only two students posted their questions there but got no response from other students or from the instructors.	10	FS	3
62	61	Content in the summary is not logically and intuitively presented. Discussion of content comes before readings.	10	FS	4
63	62	Most of the materials are just information presentation and did not utilize interactivity of online technologies.	11	FS	3
64	63	Almost all the video are professionally created by the external parties and was accessible in YouTube. Length of videos were not concerning; however, there was not statement of acknowledgment. Occasionally, I felt this course is amalgamations of YouTube videos and the authority of the course developers are minimal.	12	FS	4
65	64	none of videos that are used in the course is interactive.	12	FS	5
66	65	Personality test is not related to the course content.	12	FS	4
67	66	The interactions learners have with the learning materials and content mainly lies in reading long texts then taking notes, watching videos, and answering quizzes.	12	FS	3
68	67	Dates are not mentioned in the course schedule and not are clearly stated at what time due (Day before or day after).	12	FS	3/4/5
69	68	The course competencies and assessment do not clearly describe the learning objectives of the course.	15	FS	5
70	69	The power point sides that where converted to PDF files are almost impossible to read, even the page is maximized.	11	FS	5
71	70	Some important sections that helps with more efficient navigation such as Discussions, Assignment and Syllabus is deactivated which makes the navigation harder that it should be.	7	FS	5
72	71	It does not seem that the aesthetics are ignored. The overall impression is that the course of amalgamation of slides and videos and assignments. The front page lack any graphics, pictures or icons. And modules started gracelessly.	4	FS	2
73	72	In appropriate use of space in the homepage of the course.	4	FS	3
74	73	The course lack annotations and citations of sources in the videos.	11	FS	3/4/5
75	74	The heading and subheading and content text does not support text to read software.	8	FS	4
76	75	PDF slides with embedded pictures does not meet the accessibilities requirements in terms of font size and readability.	8	FS	4
77	76	There are not downloadable transcriptions for the videos.	8	FS	3
78	77	did not know how many tasks were involved in each activity and the activity lacked clear instructions on how to complete it.	16/15/16	MG	4
79	78	Learners cannot bookmark pages with videos	7	MG	3
80	79	the video does not fit the screen so participants have to scroll down to watch in full screen	3/11/3	MG	4
81	80	difficulties watching the video in full size.	3/11/3	MG	4
82	81	felt like discussions were not led by instructors, "I feel like we're just talking to each other. And there's not somebody leading the discussion."	16	MG	5
83	82	overwhelmed with the amount of information	11/5/10	MG	4	Raters did not agree on one heuristic but 3 raters assigned the problem to 3 different heuristics.
84	83	Two of four participants said the content of the firsttwoweekswas not what they anticipated	14	MG	4
85	84	The questions in the quiz were not opiniated and were basically a copy-paste strategy.	12	MG	4
86	85	learners cannot go forward with the videos, had to go back and watch the whole video up to the point that they wanted to see	3	MG	5
87	86	Each video is accompanied by a PDF presentation however, the PDF presentation might not cover all the main points listed in the videoand they are not the transcripts of the videos.	11	MG	4
88	87	Student interactions with peers are limited to discussion posts.	1	MG	4
89	88	there are additional readings but there is no link to those readings	5	MG	4
90	89	The video each week is made by a different content expert which might make the presentation style, length of video, and content intensity inconsistent.	12	MG	3
91	90	Participants sounded that they struggled navigating through the course in the beginning but gradually, they found their way around the course structure in the canvas	6	MG	2
92	91	clicking Submit button after finishing a video, learners might not be taken to the next page	7	MG	4
93	92	Videos are simply narrativePowerPoint slides or scripts which might not help engage learners’ attention.	12	MG	3
94	93	Videos do not have accompanied transcripts.	9/11/9	MG	3
95	94	did not know what Canvas was for a couple of weeks.	16	MG	5
96	95	difficulty in locating the ‘next button’at times tomove to the next content.	6	MG	3
97	96	the numbers of videos per module and the length of videos are not consistent	10	MG	3
98	97	Videos are script-readand not presented professionally	12	MG	3
99	98	Participants felt that the quizzes were tooshort to cover the entire module	13	MG	4
100	99	There is a lack of instructions of how to complete activities or assignments in the course	16	MG	5