CSUF Online Learning

• ​The client is CSUF’s Online Learning Staff located in Fullerton, CA. The research focused on determining which platform, Canvas or Titanium, do students prefer over the other as well as cultivating a plan for students’ preferences for either asynchronous or synchronous classes.
• ​
• ​

• May 14, 2021

• Prepared by:�Michael Oakland
• Max Nuremberg
• Ben Wright
• Matt Young
• Myles Rodriguez
• ​

Client: CSUF Online

Learning Staff

1

|

Table of Contents

4

​

7

​

​

9

​

11

13

page

page

page

page

page

19

26

28

29

30

page

page

page

page

page

2

|

Table of Research Results

20

21

22

23

24

page

page

page

page

page

25

page

3

|

Executive Summary

4

|

Background and Objectives

Method Used to obtain Information

With Covid-19 changing so much in today’s society, the education system vastly changed as well. Cal State Fullerton was forced to place all students in online classes right as the pandemic began.

​

Prior to the pandemic, Cal State Fullerton used the platform “Titanium”, and when they made the switch to all online all students had to adapt to their new platform (Canvas).

​

Our main objective was to find which platform student’s preferred, and specific details on why.

To obtain this information we sent out a survey to over 100 students at California State Fullerton, due to the pandemic word of mouth was not possible, so all contact was through email/social media.

5

|

What resources in titanium and canvas make the student prefer one over the other?

Questions Asked?

Do virtual lectures or asynchronous settings offer more value to the student

Should group work be prioritized in an online setting with the tools available to students?

Should there be stricter requirements such as participance and attendance when it is over a virtual setting?

Is the software offered by CSUF for download tailored to all departments, or do we see digital arts being prioritized?

Has Canvas been problematic when it comes to learning tools professors have their students use?

6

|

Background

7

|

�Background

• Canvas in an online learning management system for schools, districts, and universities. Educators use Canvas to manage their classrooms by taking attendance, uploading assignments and class files, and posting grades while students log on to check and complete assignments, view course materials, and monitor their grades. Canvas is available for purchase at an annual subscription rate.

​

• Titanium is an online learning platform that does similar tasks Canvas does but they are run by an open source learning platform known as Moodle. Moodle is a learning platform designed to provide educators, administrators and learners with a single robust, secure and integrated system to create personalized learning environments.

​

• CSUF announced its plans to end its contract with Moodle, which provides the Titanium system, in April 2020, with plans to transition to the more widely used Canvas by the start of fall 2021.

​

​

• Despite many student complaints over splitting their time between two systems, the plan was always to have a transition period for professors to move over to Canvas, said Berhanu Tadesse, the associate vice president of technology services.

8

|

Competition

​

The competition is between the online learning platform Canvas, and the online learning platform known as Titanium. Two separate online class systems that both students and faculty have used or are currently using at the University of Cal State Fullerton. We will get into how both platforms have competitive advantage in their own unique way on the next slide.

​

​

9

|

Canvas: From our qualtrics survey and further analysis, we determined that 77.1% of students prefer Canvas over Titanium. The fact that Canvas is already well received by the majority of the students at CSUF is a competitive advantage for the online learning staff. Also, 62.33% of students reported that they encounter more problems with Titanium then they do with Canvas. Even our VP of technology services stated that —-->

​

Titanium: “It is the fourth day of class, and I'm only 70% sure I am doing all the work for my classes. Canvas is a labyrinth of a website with multiple different tabs and links,” one student wrote on the CSUF Reddit page. “Thank God, only two of my professors are using Titanium. The difference is night and day!”

“One of our concerns about Moodle is if there was any experience with downtime, would we be able to recover in time with the limited support through sources we have on campus,” Tadesse said. “Canvas is definitely a better-maintained system than Titanium.”

​

​

​

10

|

Research Questions

11

|

Research Questions

Does Canvas offer more beneficial feedback to students compared to Titanium?

​

Do Professors feel more connected to the student while using the feedback tools through Canvas?

After hearing all the key results, which online learning platform would you prefer?

​

Even though in Q4 65.05% prefer asynchronous classes, is there going to be a constant change in attendance moving forward with the school system?

Do you believe the right research questions were asked in order to receive sufficient responses on which online learning platform is ultimately better?

​

12

|

Methods

Development of Questionnaire

Description of Questionnaire

Data Collection Method

Sampling Method

Descriptive of Sample Obtained

13

|

Questionnaire Development

• Development and Description of Questionnaire

​

Pretest Procedure

​

Questionnaire Physical Description

• ​

• The questionnaire was used to answer the following research objective of identifying how CSUF students compare Titanium (previous generation online learning platform) to Canvas. This questionnaire also helps answer what tools and valuable resources can be used to engage students in a more direct learning experience.
• The questionnaire was developed with consistency and carefully crafted wording for the purposes of making the questionnaire easily understandable and engaging enough for the highest completion rate possible. These questions were designed to answer the research questions.

• A pretest for the questionnaire was given via online due to the COVID-19 Pandemic. Nonetheless, a small sample of willing classmates that consisted of current CSUF students were targeted for testing any errors such as formatting, grammar, quality, and overall clarity for each questionnaire question.
• The questionnaire was given through CSUF Qualtrics, an online creation tool for survey distribution.

14

|

Questionnaire Content

• Overview of Questionnaire Content

​

​

​

​

• ​

• The overview of the questionnaire helped to analyze the research objective further in identifying how CSUF students compare Titanium to Canvas. Also, the questionnaire answers students’ preferences in synchronous (online-meeting) vs. asynchronous (online non-meeting) classes. A total of 17 questions were asked in the survey containing all levels of measurement: nominal, ordinal, interval, and ratio.
• The questions that had the most importance include those that determined students’ preferences in terms of likeability and frequency of usage for asynchronous vs. synchronous classes, preferred platform properties for both Canvas and Titanium, and potential problematic experiences for each platform.

• Demographic questions included age, income, household size, gender, major, marital status, and academic year at CSUF (multiple CSUF graduates appeared as survey respondents despite being the fewer academic bracket).
• ​
• 3 screener questions were devised in order to understand if the respondent qualifies under multiple criteria–students must have experience with Canvas and Titanium and both online class types so that our survey had a large enough response rate.

15

|

Sampling Method

• Data Collection Method and Sampling Plan

​

​

​

​

• ​

• Due to the COVID-19 Pandemic, the team administered a survey online through Qualtrics CSUF. CSUF did not have an open campus at the time of the administered survey, so distributing through Qualtrics was a reasonable option to partake in.
• ​

• On March 6, 2021, the survey was distributed and officially published for online survey respondents to partake in the questionnaire for data and statistics purposes. CSUF Students and Alumni were informed via a privacy statement that the survey underheld no data selling and ensured anonymity.

16

|

Sample Characteristics

• Description of Sample Obtained

​

​

​

​

• ​

• This survey was distributed via email as a pretest, then re-published after the final editing phase for the N (103) respondents. The duration of the sampling took about 2-3 weeks as additional CSUF students were needed after the 2nd week of official publishment. Once this extra sample was obtained, our survey completion expedited to 103 within those 3 weeks.
• ​

• Due to the anonymity of the survey, locations were not disclosed. However, the survey “Data and Analysis” tab gave recorded responses of 154 and the Results tab gave a completion of 103 respondents. This tells the team that about 51 respondents did not complete the survey. It is important to note that during the 2-3 time period it took to gather the results, the goal was set out to be 100. Since only 54 completed responses were recorded, the team distributed their survey link to current CSUF business students at the time.
• ​

17

|

Sample Characteristics

• Demographics

​

​

​

​

• ​

• The demographics of the target sample were men, women, and non-binary / third gender. All genders encompassed all class levels from Freshman to Graduates. As shown on the graph below, the survey produced 27 senior males. No non-binary juniors or graduates were recorded, and no freshman preferred not to disclose their gender.
• ​

• ​

18

|

Results

19

|

Do live lectures or asynchronous settings offer more value to the student?

Data collected from questionnaire questions 6 and 7 respectively

Q6 - Students on average took about 2 online synchronous (meeting) classes compared to 1.8 online asynchronous (non-meeting) classes per person. Q7 - 65.05% of students prefer asynchronous classes. We have found that most students prefer asynchronous even though students on average have taken more synchronous classes. (This might suggest that due to COVID, people prefer more asynchronous online classes).

20

Statistical Test - Percent

Do men and women view their online group experience differently?

21

Data collected from questionnaire question #5

Mean: 59.03- Hence, students are just above neutral in how they view their online group experience. We are 95% sure that students in Mihaylo business school will rate between 54.4 and 63.68.

On top of this, females view their group experience with slightly higher contentment than males from this sample. Since male/female view on their experience is substantially similar, it may be difficult to gather enough confidence for the entire population of college students in the US to determine that females rate higher than males.

​

Statistical Test - Mean Comparison

(Mean 0-100) - Interest

What resources in titanium and canvas make the student prefer one over the other?

Data collected from questionnaire question #7

​

77.1% of students prefer Canvas platform features overall over Titanium. The bar graph on the right shows that the “user friendly experience” was attributed most to Canvas, with feedback and grading right behind.

22

Statistical Test - Percent

How do men and women differ in terms of how they view participation and attendance when it is over a virtual setting?

Data collected from questionnaire question #4

Mean: Students view stricter attendance as just above neutral in contributing to their online learning experience with a 62.96 (0-100) average. Females value their online attendance/participation 8.3% higher than males.

Note* Standard Deviation: The variation is large based on the variability of accepted numerical answers

23

Statistical Test - Mean Comparison

(Mean 0-100) Significance

percentage

Data collected from questionnaire question #2

On average, 64.22% of students felt like they had more than enough online software to get them by with their online classes. Suggesting more software for students will most likely be a loss for CSUF budgeting allocations.

24

Did you have enough student software to fully optimize your learning experience?

Statistical Test - Percent

Has Canvas been problematic when it comes to learning tools professors have their students use?

Data collected from questionnaire question #6

Mean comparison: 62.33% of students believe that overall, titanium has more problems in terms of online platform features. More specifically, the grading property for Titanium had the most problems according to students, with approx. =67/108 (62%) students allocating at least one bad experience with this feature.

25

Statistical Test - Percent

Limitations

26

|

• Our survey could only be sent to California State Fullerton students, therefore making it much harder to reach 100 randomized responses across other colleges and other organizations.
• Our survey only concerned two platforms, therefore leaving not much variety when it comes to responses.
• Students are only a select age range/social demographic, therefore our pool of survey takers were slim.
• With school being online, the only way to reachout to survey respondents was via online as well, therefore word of mouth through answers and qualitative data was not effective.

​

27

|

Key Findings and Recommendations

28

|

Key Findings

Recommendations for making a Survey

After our survey was complete, we discovered Canvas had such a bias over Titanium…

​

“77.1% of students prefer Canvas platform properties holistically over Titanium. The bar graph on the right shows that the “user friendly experience” took the cake for Canvas, with feedback and grading right behind”.

​

• We suggest that someone formulating a survey should place priority on ratio and interval questions, therefore allowing for a larger variety of answers.
• Make sure the target demographic is tech savvy if the survey is ultimately all online.
• Find different ways to share the survey, whether this be through social media or text.
• Put significant time into each question making sure it is focused and concise.
• Make sure the target demographic is large enough to achieve the target amount of surveys.

29

|

Appendix

30

|

31

|

32

|

33

|

34

|

35

|

36

|

37

|

38

|

39

|

40

|

41

|

42

|

43

|

44

|

45

|

46

|

47

|

48

|

49

|

50

|

51

|

52

|

53

|

54

|

55

|

56

|

57

|

58

|

59

|