Winter 2016 – EDUC 208B – Curriculum Construction |

Celine Zhang, Lisa Jiang, Lucas Longo, Mohamad Hasan

MOVING TO BLENDED:

DATA & DECISIONS AT THE STANFORD GRADUATE SCHOOL OF BUSINESS

TABLE OF CONTENTS

TABLE OF CONTENTS

RATIONALE & CONTEXT

GOALS

LEARNING ACTIVITIES

ASSESSMENT

APPENDIX A: NEW SYLLABUS

APPENDIX B: ORIGINAL SYLLABUS

APPENDIX C: SAMPLE LESSON PLANS

LESSON PLAN FOR SESSION 2

LESSON PLAN FOR SESSION 17

APPENDIX D: CAPSTONE PROJECT

APPENDIX E: REFERENCES

RATIONALE & CONTEXT

Our site, the Operations, Information & Technology (OIT) Department at the Stanford Graduate School of Business (GSB) has the mission to redesign the Data & Decisions (T265) Course Curriculum. The course’s goal is to enable students to use data (probability, statistics, and regression analysis) to inform decision making in complex business situations. It was originally designed as a support course, or prerequisite, for other courses such as Finance and Accounting. Since then, specific topics have been added or subtracted from the curriculum to be more focused on data analysis than the calculation of probability and statistical procedures.

The course has been taught in the same way for the past 15-20 years, in a lecture-based format with limited hands-on applications or discussions during class time. The course has a capstone-project that is intended to engage students in a real-world scenario where they can apply the concepts learned in class. The students work with real clients who have real data and decision needs, yet only start on this project towards the end of the course.

Their desire is to introduce online content delivery and assessment to their Base course level as a pilot, then extend it to other course levels. The theoretical and conceptual components of the course would go online freeing up class time for engaging with exercises, clarifying questions and discussing/reflecting upon the content.  The currently assigned reading materials would also be supplied but not required readings.

The learners are first year MBA students at the GSB and come from a wide range of background knowledge and experiences. For this matter, there are three levels the Data and Decisions course: Base, Intermediate, and Advanced. While most learners were exposed to some part of the content in high school or college, many of them may not have been exposed to or have applied the concepts at work. They also have different computer skills, which will influence how well they deal with the relatively heavy use of Excel and R, a programming language and software environment for statistical computing and graphics.

The learners are in the process of getting a wider business degree, and we can assume that they are more interested in the business applications of the content as opposed to the details of the formulas and their derivation. For example, it may be more beneficial to know the idea behind variance, how it’s calculated in Excel or R and its use as opposed to knowing the exact formula.

It is fair to assume that the MBA1s are also busy with many social and academic events, which means that there attention and dedication to the course will be spread thin. MBA students, in particularly, care deeply about ‘authentic’ learning experiences and will only devote their time and energy to topics that they perceive as having direct connections to their professional pursuits. We also assume that all MBA students have the appropriate technology affordances for online learning - access to high quality internet access and up- to-date computers to stream videos and run statistical programs.

Given the target student audience, moving the lecture-based, one-sided content to an online environment has several benefits:

This shift towards using the class time for content engagement instead of content delivery is central to the current redesign of the curriculum. The goal is to shift from the ‘teaching of formulas’ to doing problem sets, discussions, and application of core concepts. Given this shift in focus, we believe that the teaching of formulas and procedures tend to be more linear and repetitive and thus are great candidates for online content, instead of using valuable classroom time. Students will not only learn methods of using data but, more importantly, should be able to build models and critique them. The hope is that students will become intelligent consumers of data who can look at it and interpret it.

Online content also corrects for student’s previous knowledge and pace. Problem sets can be personalized for each student’s level of understanding, thus ensuring everyone's preparedness for the course’s learning progression. Discussion forums and peer-review mechanisms can also provide different learning opportunities for those who have different learning styles and prefer more collaboration or explanations in different ways. It can also serve as a great formative assessment for teachers to identify common misconceptions and course correct. The implementation of these features and what technological platform will be used remains undecided.

The site would also like to be able to create adaptive assessments that would provide immediate feedback, hints, or harder or simpler questions for the students. It would also provide real-time and historical data for the teachers to evaluate how students are progressing through the material.

Allison O’Hair is leading this initiative at the site and has experience in designing and implementing online content for MIT Sloan and is knowledgeable on the medium of online education. It is interesting to note that although the OIT Department is leading this initiative, the course is technically under the Economics Department at the GSB.

In terms of educational ideology, our curriculum would be undergirded primarily by the Dewey’s concept of progressivism in that we hope to design learning experiences that drive students’ innate desire to learn. According to Dewey, the educator’s role is to set up the right conditions for transfer, rather than teach lessons in isolation. The sign of a mature learner is then someone capable of both identifying and solving their own problems. This focus on “transfer” to enable students to apply their learnings beyond the classroom is a key learning goal in this curriculum, with ample class time devoted to discussing practical applications of concepts and a capstone real-world project for assessment, presented as the main driver of the learning progression.

In addition, Dewey placed great emphasis on the interaction between internal and objective conditions for curriculum design. He contended that curriculum construction was always contextual, and that

“The trouble with traditional education was not that it emphasized the external conditions that enter into the control of the experiences but that it paid so little attention to the internal factors which also decide what kind of experience is had”. (Dewey, 1997, p.42)

Personalizing the learning experiences for students based on their “internal” conditions, such as their backgrounds, prior knowledge, social-emotional skills and so forth, would be key to effective learning. Ideally, these learning experiences should help prepare students for later experiences and drive continued learning. Selecting and creating learning experiences - be it direct instruction, class discussions or assessments - that are tailored to students’ backgrounds will very much be a core consideration in this curriculum design. Our ultimate goal would be to equip our students with both the desire and skill to continue enhancing their understandings of probability, statistics and regression application in a business context.

In a similar vein to Dewey’s progressivism, Bruner’s theory of emphasizing structure, transfer, and students’ readiness for learning would also underpin our curriculum design. We agree with Bruner that the ultimate goal of education is to help students “learn how to learn” and facilitate transfer. The emphasis on structure, then, is critical because a deep understanding of structure is also a deep understanding of how things are related, and therefore permitting transfer. The implication for us would be to delineate the key concepts to be covered in the course and sequencing them in such a way that

“If earlier learning renders later learning easier, it must do so by providing a general picture in terms of which the relations between things encountered earlier and later are made as clear as possible”. (Bruner, 1960, p.12).

In terms of readiness to learn, we are cognizant that the learners in our course may come equipped with different levels of understanding and aptitudes, and it is our hope to create a curriculum that provides satisfying learning experiences for students regardless of their existing preparedness for the course. We want our curriculum to result in a class that stimulates our students’ desires to learn.

On the assessment side, this curriculum follows Coffey’s (2003) principles on involving students in “everyday assessment” which is a “dynamic classroom activity that includes the ongoing interactions among teachers and students as well as more scheduled events, such as weekly quizzes and unit tests.” Time will also be dedicated for students/groups to present the progress of their project work as well as key understandings or struggles they faced in the task. Finally, the online assessments/exercises followed by class time dedicated for explanations will also provide opportunities for learning, and not only testing memorized knowledge.

GOALS

For the overall course, the enduring understandings for students would include understanding that:

Upon completing the course, first-year MBA students will be able to:

LEARNING ACTIVITIES

The learning activities proposed in our curriculum will include multiple formats of delivery and engagement designed specifically for each desired learning goal. These activities will include:

  1. In class topic introduction
  1. Video based lectures
  1. Classroom and online group discussions

Additional learning activities are discussed under “Assessment”. For more details, refer to Appendix A for the Syllabus featuring more details on the learning activities.

ASSESSMENT

For the course, there will be four main forms of student assessments:

  1. Attendance and class participation (15% of grade)

Since the offline component of the course is focused on class discussions of concepts and project work, it is participative in nature. As a result, student attendance and active participation in class are included in assessment to highlight their importance. While in the past student attendance at lectures may have been less critical because certain classes cover material that can be self-taught, the new version of the course would move that theoretical content online. Attendance at the offline classes is then comparatively important since the offline component primarily serves to deepen students’ understanding of material through in person discussions and group work. Furthermore, class participation – in the form of asking questions, being involved in group discussions, and communicating with the instructor(s) – are important forms of formative assessment. Instructors are able to glean from what students share during class participation the areas in which students have a good grasp, versus areas that require more clarification. It should be noted that class participation should encourage wide participation and not serve primarily evaluative purposes. What this means is that class participation should welcome clarifying questions and different opinions from students with an emphasis on “there are no stupid questions”, instead of intimidating students with trick questions and displeasure at lack of understanding. As long as students are engaged in class, their class participation should be viewed as sufficient. In addition, since the in-class environment is more effective as a forum for students to ask questions and clarify concepts, the online platform would be employed mainly for content delivery and formative assessment through problem sets. Discussion groups would be kept as an in-class component, rather than being migrated online.

  1. Problem sets and pretests (15% of grade)

For the new version of the course, there will be weekly online problem sets associated with the topic covered. The problem sets are meant to be practice for students to deepen their understanding of the material. The problem sets will employ a mix of selected and constructed response formats using an online platform such as Open edX. It should be noted that questions with selected-response formats, such as multiple choice, should be limited because students can be led to focus on knowing “facts” (or even engage in guessing) rather than understanding and applying knowledge (McTighe & Ferrara, 1993). Selected response questions are included largely for the purpose of being able to offer students feedback when their responses were incorrect, a functionality yet to be available for constructed response questions. For example, if a student gets a particular question wrong, the online problem set program would be able to point out immediately that the student’s response was incorrect, provide a recap of the associated concept and offer hints for solving the question. Furthermore, students are allowed to revise and resubmit the online problem sets as many times as they wish, with a focus on allowing students to uncover areas of weaker understanding and learning through online feedback on their problem sets. In this manner, the problem sets themselves become embedded assessments, as suggested by Treagust et al, by becoming a teaching activity in itself (Treagust, Jacobowitz, Gallagher & Parker, 2003). The problem sets would also be completed prior to the in-class session and the instructors would be able to glean from student responses what common misconceptions to address during class time.

In addition, there will be three pretests included for the in-class sessions. The pretests are not meant to be evaluative so much as formative and are included to  “identify students’ personal conceptions, misconceptions, and problems in understanding the topic” (Treagust, Jacobowitz, Gallagher & Parker, 2003). Students will be given 10 minutes at the start of a session to complete the pretest. The instructor would then go through the answers and leave room for clarification based on the most salient or common misconceptions. Students would also exchange their pretests with a classmate to allow for peer teaching and learning as well, as students are able to compare notes on their understandings of class concepts. These pretests would be graded based solely on a completion basis, since they serve to assess students’ existing knowledge and areas needing more work, rather than serving as summative assessment. This would also likely have a positive impact on class attendance since students would need to attend the in-class sessions to take the pretests.

Again, as the problem sets and pretests only constitute 15% of the final grade, they serve mainly formative purposes rather than evaluative, and should be perceived as tools in helping students learn.

  1. Capstone project (50% of grade)

A key change to the course is the inclusion of a capstone project that runs throughout the duration of the course. Students will form groups of 4-5 and work with a company to apply the material they learn in class. The company will provide a dataset at the beginning of the course, and students will conduct various forms of analyses on the data based on concepts and techniques covered as the course progresses. The project should be informed by hypotheses the company wishes to test on the data set. Students will ultimately submit a write-up and present their findings to classmates and the company that provided the data, to give them a clear sense of real-world applicability of their work. There will also be feedback on the project from the instructor and, ideally, from the company itself. As a key learning goal of the course is for students to grasp the underlying concepts of formulas and theories and know how to apply them in a practical business context, this capstone project would serve as a real-world study for students to apply their conceptual learning.

For the capstone project, there will be two key deliverables for students: a group write up of the analysis (40% of grade), as well as an individual critique of another group’s project (10% of grade). With the group write up, there will be periodic check points over the course of the class. For example, each group should have a write up of the project hypothesis and scope of the data set at the start of the quarter. As the course progresses through different analytical techniques, groups should provide a short report on their initial findings. These drafts should be shared with another group in the class for peer discussion and assessment, as this can be an effective way of creating “shared meanings of quality work” as recommended by Coffey (Coffey, 2003). Students can often learn tremendously from feedback and suggestions from peers, and the process of reflecting and assessing another group’s work deepens understanding in itself. The drafts should serve as formative assessment opportunities for students and not be graded. With the final write-up, students will be expected to not only provide a summary of their findings, but also reflect on the strengths and weaknesses of their own analyses. The final write-up would serve as a summative assessment of students’ learning outcomes. It should also ideally be an authentic assessment that “engage students in applying knowledge and skills in ways that they are used in the real world” (McTighe & Ferrara, 1993).

With the individual critique, students will pair up with another classmate to take a critical look at each other’s group project write up. Students should deliver a one-page critique of the strengths and limitations of their partner’s group project, especially around the validity of findings. As a learning goal of the course is to inspire students to be critical of data and ask key questions around data presented to them, it is important for students to understand the constraints of their analyses. The report critiques serve the purpose of having students become critical consumers of data, beyond performing data analysis. The fact that group project reports would be evaluated by classmates through the individual critiques would also potentially create more accountability for students to deliver a well thought-out project report, since they would possibly not want to let their classmates down with shoddy work.

  1. Midterm exam (20% of grade)

A midterm exam will be continue to be included for the course primarily for summative assessment of individual students, as well as to focus on conceptual understanding. As the capstone project is completed in groups, the instructor should still strive to gain an understanding of individual students’ learning outcomes. It should be noted that while the midterm is summative in nature, it can still deepen student understanding by having them review the course content and make connections amongst various concepts presented. The midterm itself should also feature open-ended questions that focus more on students’ thinking process rather than regurgitation of knowledge. For example, questions can ask students to provide a critique on a piece of analysis, instead of having them solve a linear regression. The midterm should be set at a level where students with a good grasp of the concepts and their applications in the real world would be able to do well, without being excessively easy or challenging. Students’ performances on problem sets should ideally signal to the instructors the suitable level of difficulty for the midterm.

APPENDIX A: NEW SYLLABUS

PHASE 1: GETTING EVERYONE ON THE SAME PAGE

Week One

Session 1: Introduction to Probability and Statistics

Topics:

Watch:

Practice:

In-class:

Session 2: Binomial and Normal Random Variables

Topics:

Watch:

Practice:

In-class:

Week Two

Session 3: Sample Statistics: Estimation and Distribution

Topics:

Watch:

Practice:

In-class:

Session 4: Sampling, Surveys and Confidence Intervals

Topics:

Watch:

Practice:

In-class:

Week Three

Session 5: Hypothesis Testing: Foundations

Topics:

Watch:

Practice:

In-class:

Session 6: Hypothesis Testing: Refinements and Applications

Topics:

Watch:

Practice:

In-class:

PHASE 2: APPLICATION (all done in Lab)

Week Four

Session 7: Introduction to Data and Regression Analysis Part 1

Topics:

Watch:

In Lab:

Session 8: Introduction to Data and Regression Analysis Part 2

Topics:

Watch:

In Lab:

Week Five

Session 9: Statistical Inference and Regression Part 1

Topics:

Watch:

In Lab:

Session 10: Statistical Inference and Regression Part 2

Topics:

Watch:

In Lab:

Week Six

Session 11: Multiple Regression

Topics:

Watch:

In Lab:

Session 12: Regression Model Building

Topics:

Watch:

In Lab:

Week Seven

Session 13: Linear Probability Model

Topics:

Watch:

Practice:

In-class:

Session 14: Prediction / Big Data

Topics:

Watch/Read:

Assignment:

In-Class:

Week Eight

Session 15: A/B Testing

Topics:

Watch/Read:

Assignment:

In-Class:

Session 16: Differences in Differences

Topics:

Watch/Read:

Assignment:

In-Class:

PHASE 3: CRITIQUE

Week Nine

Session 17: Data Critique, Part 1 (see sample lesson plan in Appendix C)

Topics:

Read:

In-Class:

Session 18: Data Critique, Part 2

Topics:

Watch:

Assignment:

In-Class:

APPENDIX B: ORIGINAL SYLLABUS

  1. Week 1
  1. Probabilities and Events
  1. Readings
  1. Optional Readings
  1. Class Topics
  1. Probability Tables and Trees
  1. Readings
  1. Optional Readings
  1. Class Topics
  1. Week 2
  1. Properties of Random Variables
  1. Readings
  1. Optional Readings
  1. Class Topics
  1. Prepare for class
  1. Assignment 1
  1.         Binomial Random Variables
  1. Readings
  1. Optional Readings
  1. Class Topics
  1. Week 3
  1. Normal Random Variables
  1. Readings
  1. Class Topics
  1. Assignment 2
  1. Sample Statistics and Estimation
  1. Readings
  1. Class Topics
  1. Prepare for Class
  1. Week 4
  1. Distributions of Sample Statistics
  1. Readings
  1. Optional Readings
  1. Class Topics
  1. Assignment 3
  1. Sampling, Surveys, and Confidence Intervals
  1. Readings
  2. Optional Readings
  3. Class Lecture
  1. Week 5
  1. Hypothesis Testing: Foundations
  1. Readings
  1. Optional Readings
  1. Class Topics
  1. Assignment 4
  1. Midterm Examination
  1. Week 6
  1. Hypothesis Testing: Refinements and Applications
  1. Readings
  1. Optional Readings
  1. Class Topics
  1. Assignment 5
  1.         Introduction to Regression Analysis
  1. Readings
  1. Class Topics
  1. Week 7
  1. Statistical Inference in Regression
  1. Readings
  1. Class Topics
  1. Prepare for Class
  1. Multiple Regression
  1. Readings
  1. Class Topics
  1. Prepare for Class
  1. Week 8
  1. Regression Model Building
  1. Readings
  1. Class Topics
  1.         Prepare for class
  1.         Assignment 6
  2. Regression project proposal
  1. Linear Probability Model
  1. Readings
  1. Class Topics
  1. Week 9
  1. Prediction/Big Data
  1. Readings
  1. Class Topics
  1. Prepare for Class
  1. A/B Testing
  1. Readings
  1. Class Topics
  1. Week 10
  1. Differences in Differences
  1. Readings
  1. Class Topics
  1. Regression Project Due
  1. (Optional) In Class Review Session
  1. Week 11
  1. Final Examination

APPENDIX C: SAMPLE LESSON PLANS

LESSON PLAN FOR SESSION 2

BINOMIAL AND NORMAL RANDOM VARIABLES

Enduring Understandings (EUs): To be able to practically apply the concepts of Binomial and Normal probability to solve common problems.

Essential Questions (EQs):

  • What are examples of real world situations that can be modeled as binomial/normal distributions?
  • How can you calculate mean/variance of binomial/normal distribution?
  • Can you explain what mean/variance mean conceptually in a binomial/normal distribution?
  • Can you explain the concept of the Central Limit Theorem and how that applies to normal distribution (do you grasp that the sum of NON-normal distributions sum up into a Normal distribution)?
  • Why and how do we standardize Normal distributions?

 

Preparation/Resources/Instructional Materials Needed:

  • Make sure that summaries of all the project clients and their data needs are ready and distributed to all students before class begins, ideally at least 2 days before.
  • The data from student answers to the online problem sets should be compiled and used to identify the problems that students had the most trouble with and the teacher should re-arrange his/her slides to focus the lecture part of the in-class time on these (or similar) problems within the allocated time.
  • Make sure that the class knows that groups should be formed and finalized by now. The students freely form their own groups and they add their names to a list on the learning system (Canvas or otherwise) or on a shared spreadsheet (using Google Docs for example).
  • You need a projector for any slides to explain content, and a whiteboard to be able to solve through questions in front of entire class if needed.
  • Prepare at least 3 sample problems for group work to be presented on the projector. Sample problems must have solutions that will also be presented after groups get a chance to solve them together.

//ONLINE

TIME

INSTRUCTIONAL ACTIVITY

NOTES

10 Min

Calculating Binomial probabilities

 

Check out example video here: http://bit.ly/1OnkLej

 

10 Min

Calculating expected value and variance of Binomial random variables

 

 

Check out example video here: http://bit.ly/1OMD50R

 

 10 Min

Calculating Normal probabilities

 

 

Check out example video here:  http://bit.ly/1pRc2Ga

The way that Khan Academy (KA) explains Normal distribution using a spreadsheet is very effective, especially for users who will use Excel in their lives. We recommend this, however this could be done in a more concise fashion than how KA did it, and focus on using Excel formulas. Look at min 11:30 onwards.

 10 Min

Sums of Normal random variables and standardizing

 

For sum of Normal random variables example:  http://bit.ly/1TP8pDX

For standardizing: http://bit.ly/1UwTf5A

For the first video: there is a bit of detail in the middle of this video that goes deep into the formulas that isn't necessary for the class, however this video ties the binomial distribution to the normal distribution through the central limit theorem very nicely and we like that it culminates in pulling everything together.

For the second video, we particularly like the use of a worked example.

 n/a

Online problem sets

There should be ~5 questions after each of the four video sections above that test the student’s understanding. The question type will be constructed response (i.e. with a correct/incorrect answer) and the student can try to solve the questions as many times as the want. There is no time limit on solving the questions. The EQs and the EU should be kept in mind when designing the questions. The system will be tracking the answers and the attempts of the students in order to provide feedback to teachers on what to concentrate on during face-to-face class time. The notes show some sample questions that can be used to understand what kind of skills are to be assessed.

 

Video 1: Calculating binomial probabilities

1. What is the probability of obtaining exactly eight heads in ten tosses of a fair coin?

2. Experience has shown that 1/200 of all CDs produced by a certain machine are defective. If a quality control technician randomly tests twenty CDs, compute each of the following probabilities:

(a) P(exactly one is defective)

(b) P(half are defective)

(c) P( no more than two are defective)

 

Video 2: Calculating expected value and variance of binomial random variables

1. A roulette wheel has 38 slots, 18 are red, 18 are black, and 2 are green. You play five games and always bet on red. How many games can you expect to win?

2. Suppose that 5% of videos rented at Campus Video incur a late rental fee. If 700 videos were rented last week, the number that will incur a late rental fee should be around __________ give or take __________ or so.

 

Video 3: Calculating normal probabilities

1. For a certain type of computer, the length of time between charges of the battery is normally distributed with a mean of 50 hours and a standard deviation of 15 hours. What is the probability that the length of time between charges will be between 50 and 70 hours?

2. X is a normally distributed variable with mean μ = 30 and standard deviation σ = 4. Find:

(a) P(x < 40)

(b) P(x > 21)

(c) P(30 < x < 35)

 

Video 4: Sums of Normal random variables and standardizing

1. In the spaces below, write down the probability of getting 0 to 20 heads in a 20-trial coin flip of a biased coin that has a probability of heads of 65%. You can use Excel for calculation if you wish.

2. Suppose the height of adult American males is approximately normally distributed with a mean of 182.1cm and a standard deviation of 8.3cm. Standardize this normal distribution and then calculate the 90th percentile height of American males.

 

//IN CLASS

 

TIME

INSTRUCTIONAL ACTIVITY

NOTES

 

5 min

Brief overview of the selection process of clients for the group project, and how this will be based on a stacked-rank algorithm to try and help everyone get one of their top choices. Instruct groups to get together and discuss their top 3 project clients.

Groups should have been formed and added online. Mention that anyone who doesn’t have a group yet should email the teacher and the teacher will add them to groups that can handle the extra person(s).

10 min

Groups discuss their preferences and culminate in one person from the group entering preferences into the online system.

 

 15 min

Cold-call and/or general questions asking students to reflect on what they learned from the online classes. The teacher could ask questions like:

  • Can anyone tell me what the Binomial/Normal distribution is in a few sentences?
  • Can you give me an example of how the Binomial/Normal distribution can be used in everyday life?
  • What is the significance of mean/variance?

Other strategies could be to ask the class if they are struggling with understanding anything in particular and reflect those questions back onto the class to see if there is anyone from the class who can answer the questions.

Cold-calling is a method that some teachers like to keep students on their toes. We are suggesting using it with the students that have scored the highest in the online assessment and ask them to answer some of the questions by putting them on the spot (i.e. without volunteering).

The EQs from this session could be used as guiding questions for this part.

 20 min

Standard slide-based lecture on a projector. These slides should not be a “content dump” or the theory, rather they should be “worked examples”.  Although this is a flipped classroom model, this lecture acts as a “crutch” of sorts that focuses on the content that was difficult for most people. The lecture slides may act as a repetition of some content that was presented online, but offers students the opportunity to “cement” the theory by either confirming that they actually understood the material correctly or by asking clarification questions or hearing the material explained one more time. Again, this is a short presentation and the teacher should focus the content on the topics that most students had trouble with. We recommend that the worked examples be similar to the problems in the online assessment that had the lowest score or the most repeated trials. The teacher has the option to either have the problems pre-solved on the slides or just use the slide to display the problem and then go through the solution on the whiteboard.

Target content based on student needs (i.e. focus on “worked examples” with the lowest scores and/or the highest repeated trials). Prepare for 12-15 min of talking and interrupted questions of 5-8 minutes.

10 min

x 3

Instruct students to get in groups and prepare to work on examples. Problems are shown on the projector and students are expected to work through the problem together. Emphasize that everyone needs to know how to solve it and that students who can solve it are encouraged to explain it to the other members of the group. Repeat this as many as 3 times (time permitting). This exercise runs until the end of the class.

Problems should be similar to the online problem set questions. We envision the online system to have a “bank” of such questions and the teacher can choose “bank” questions that weren’t presented to the students during the online portion of the class.

 ~1 min

Dismiss class and remind students to watch the next group of online videos before next session.

 

Note: The number of group problems can be increased or decreased as time permits. If there is less lecture material or if there are less questions and the lecture material gets done quicker, the teacher has the option to do more group problems. In addition, group problem solving may take more or less time. 

Additional Comments:

  • Problem sets should be turned in at least 24 hours before the start of class (announced digitally).
  • It is preferable, but not necessary, to break groups up before hand based on their results in the problem sets in order to have mixed-skill groups emerge.
  • If the teacher wants to employ the cold-calling method, then it would help to identify the top students based on the problem sets in order to have them explain the concepts learned in the online lecture. This goes well with the first EQ and gives students the ability to hear multiple interpretations of the distributions in plain English.

LESSON PLAN FOR SESSION 7:

INTRODUCTION TO DATA & REGRESSION ANALYSIS, PART 1

Enduring Understandings:

  • Raw data can come with limitations and require workaround methods to enable reliable analyses
  • Data can be fit into linear models for future predictions
  • Data analysis in real world situations often involve the use of statistical tools and softwares to handle large amounts of information

Essential Questions:

  • What are common types of data constraints that influence analysis?
  • What are strategies for handling limitations in data quality?
  • What is a linear regression model and why is it useful?
  • What are common mistakes to note in building a linear regression model?
  • How does one work with large amount of data?

 

Preparation/Resources/Instructional Materials Needed:

  • Select relevant readings / videos on data cleansing and linear regressions
  • Record online videos for instruction on the use of statistical software “R” and make sure content is updated based on most current version of software
  • Check to ensure instructions for the use of “R” apply to students using different operating systems, including both PCs and Macs
  • Ensure students understand that they should have “R” installed on their personal laptops before attending the in-class session in a lab
  • Make sure all relevant videos are online and accessible to students at least a week before class
  • Prepare all relevant datasets and materials for online exercises and ensure accessibility to students
  • Check that all computers in the lab are installed with the most updated version of “R”
  • Prepare presentations on data cleansing and linear regressions
  • Consider common misconceptions and effective methods to address them in class, for example through asking students to explain their understandings
  • Have white boards and projectors in lab for presentation of content, especially in troubleshooting “R” installation and use with students
  • Prepare “cheatsheet” for “R” functions to be handed out to students in class
  • Prepare sample problems for students to work through in class

//BEFORE CLASS

 

TIME

INSTRUCTIONAL ACTIVITY

NOTES

15

Min

Go through short readings on the importance of and methods for data cleansing

Check out sample readings here:

http://www.iaca.net/Resources/Articles/datacleaning.pdf

http://db.cs.berkeley.edu/jmh/papers/cleaning-unece.pdf (Part 1 only)

5

Min

Watch video for overview of “R” and its functionalities

Check out example video here: https://www.youtube.com/watch?v=QJUsRUDsv3U

5

Min

Watch video for initial set up and navigation of “R” and complete associated exercise

Check out example video here: https://www.youtube.com/watch?v=clB5Ic87Fp8

5

Min

Watch video on basic calculations and setting up of variables in “R” and complete associated exercise

Check out example video here:

https://www.youtube.com/watch?v=DKIEzhJ_Fpo

5

Min

Watch video to understand the use of “help” in R” and complete associated exercise

Check out example video here:

https://www.youtube.com/watch?v=L76Z1mrcbTk

5

Min

Watch video on different data types and loading data in “R”, including associated exercises

Check out example videos here: https://www.youtube.com/watch?v=9U312osgcgk

5

Min

Watch video on handling missing data values and complete associated exercise

Check out example video here:

https://www.youtube.com/watch?v=HmmNrisJoc8

20

Min

Watch videos on introduction to linear regression

Check out example videos here:

https://www.khanacademy.org/math/probability/regression/regression-correlation/v/correlation-and-causality

https://www.khanacademy.org/math/probability/regression/regression-correlation/v/regression-line-example

 

//IN CLASS

 

TIME

INSTRUCTIONAL ACTIVITY

NOTES

5

min

  • Go through logistics of future lab sessions, including seating arrangements (students in the same team should sit in the same row to facilitate group work) and partnering for computers in lab (students need to share a computer due to number of computers available)
  • This will be the first time students are working together in the computer lab
  • Important to deliver clear instructions on logistics to avoid future confusion

10

min

  • Go through importance of data cleansing using concrete example of how poor data quality can hurt reliability of analysis
  • This will be presented on a few slides, but instructors should focus the attention on involving students in class participation rather than repeating online content in presentation, since students have covered the material previously
  • Ask students to suggest reasons why given data set is poor in quality and potential methods for data cleaning
  • Use of a concrete example helps to drive home concepts covered in online session (refer here for potential inspirations: http://www.bcs.org/upload/pdf/ewrazen-120607.pdf)
  • As students have read materials on “data cleaning 101”, rather than engage in direct instruction, instructors should seek to check student understanding and help them apply learnings on real life situations during in class session

20

 min

  • Focus on tricky linear regression concepts from online material, for example, through asking students to explain the difference between causation and correlation, or meaning of different r-squared values
  • Enlist the class’ help in building a linear regression model step-by-step - instructor should introduce the data set and context, then ask individual students to suggest how the data can be fit into a linear regression model (e.g. the first student could say plot all data points on a graph to check visually for a linear trend)
  • Be willing to push students on their responses to why they would follow certain steps in building a linear regression model, to assess their understanding rather than recall
  • There are common misconceptions around linear regression, such as mistaking correlation for causation, falsely fitting data points to a linear model and so on. Instructors should be well versed with the pedagogical content knowledge involved, for example preparing in advance by reading sample articles like this:
    http://www.isixsigma.com/tools-templates/regression/how-to-avoid-common-mistakes-in-linear-regression/
  • Since students have a basic understanding of how linear regression works, the in class session serves to help them translate the conceptual understanding into actual application by collaborating on building a linear regression model

15

 min

  • Have students turn on lab computers for session on “R”
  • Provide new data set and request students to work through basic “R” functions, e.g finding the mean and standard variation of a data set
  • Walk around the class at the point to help individual students troubleshoot
  • While students have gone through “R” installations, the in class session serves to check that they fully understand the basic functions of the software
  • Instructor will not demonstrate how “R” functions work, since students should have cover these functions in their online preparation, but walk around to address individual concern if necessary.
  • If multiple students run into the same error or misconception, instructor could use this as an opportunity to alert the wider class to potential misunderstandings

10

 min

  • Demonstrate linear regression in “R” using sample data set
  • Go slowly and have students follow each step along the way, since they have not covered these techniques in the online session
  • Students should feel safe in asking questions
  • Leading students through building a linear regression model on “R” is a good way to bridge the concepts on linear regression and the practical applications on “R”
  • Student questions are especially important because this is a potentially challenging topic and getting it right the first time is highly important.

20

min

  • Have students gather in their groups to discuss how they can clean the data and employ linear regression for their group project
  • Walk around the class to address specific questions from students
  • Linking the class concept to practical application is a good way to increase student interest
  • Again, if similar questions pop up, the instructor should make the wider class aware of common misconceptions and address those

Additional Comments:

 

LESSON PLAN FOR SESSION 17

DATA LITERACY - HOW TO BE CRITICAL CONSUMERS OF DATA

Enduring Understandings (EUs): Data is useless without the skills to analyze it. All managers need to be critical consumers of data.

Essential Questions (EQs):

  • What are the key questions to ask when evaluating a data report?
  • How do you turn data analysis into data insights?

 

 

Preparation/Resources/Instructional Materials Needed:

  • Encourage students to watch the video and do the short article readings before class. It serves to give them some common language and motivation for why this topic is important for their future careers in business.
  • Check-in with the guest speaker. Help him/her understand the class objectives (EU and EQ). Understand their personal objectives too.
  • Work with guest to get their presentation aligned with course objective.. Ask to see the Google case again and familiarize yourself with the case.
  • Create a shared Google Doc and make sure that access is open. You will be using the Google Doc to get groups to input ideas for questions to ask for the Google Case Study.
  • Be sure to have the case data report available and printed for every student.

//BEFORE CLASS

 

TIME

INSTRUCTIONAL ACTIVITY

NOTES

 

2 Min

Watch: What is Data Literacy?

https://www.youtube.com/watch?v=qHz_ogTH2p4

 

25 Min

Read:

These are short reads

 

//IN CLASS

TIME

INSTRUCTIONAL ACTIVITY

NOTES

 5 min

Introduce the topic of Data Literacy.

  • Ask students to share things they learned from the video or reading
  • Ask students to share how they have used data in their prior work experience. What was successful or not about those experiences?

Be prepared to introduce the speaker

20 min

Guest Speaker: Google Analytics

Guest will speak on his/her personal background and present a case study to the class with a sample data report they received from their team.

Hand out the data report as guest speaker finishes talking.

20 min

Guided practice:

  • Form groups to review the data report
  • As a group, come up with a series of questions and feedback around the data report.
  • Input responses into a shared Google Doc.

Groups should be the same as their project groups.

Walk around the room to listen in on conversations. Make sure groups are on-task. Offer up suggestions of possible questions.

20 min

Report out and Discuss:

  • Call on each group to share what they added to the Google doc
  • Ask if they see any themes in the questions or feedback (note: you could organize the input around the themes to serve as a graphic organizer)

This portion should be led by instructor but guest should be encouraged to chime in as well.

10 min

 Short lecture:

  • Introduce the Data Credibility Checklist
  • Introduce the fundamental questions:
  • What questions are you trying to answer?
  • How was the data collected?
  • What’s in there to learn?
  • How reliable is the information?

Using the case data, try to show students how you would apply the ‘Data Credibility Checklist’ and Introduce the fundamental questions.

15 min

Case wrap-up and Q&A with Guest

 

Additional Comments: n/a

Screen Shot 2016-03-06 at 11.06.10 AM.pngReference: http://docs.lib.purdue.edu/cgi/viewcontent.cgi?article=1095&context=lib_fsdocs

APPENDIX D: CAPSTONE PROJECT

This note provides detailed information about the final capstone regression project and should be made available to students. Much of the content is incorporated from the current description of the regression project but we have added some details related to timing and theme.

CAPSTONE PROJECT NOTE

The capstone project provides you with the opportunity to analyze a business-related problem of your choice, using the analytical tools that are taught in D&D.

Objectives:

  • Provide a practical experience at regression modeling
  • Create a teamwork experience and project management experience addressing quantitative problems
  • Give students an opportunity to review and critique other’s data reports

Enduring Understandings:

  • Apply probability and statistics to a real world data set and business setting

Essential Questions:

  • How does data turn into insights?
  • What are the limitations of data?
  • How can data inform decisions?

 

Project Details

The capstone project involves four basic steps:

  1. Identify, as a group, a business question that can be answered with regression analysis. Obtain suitable data that can be used to analyze the problem. This may involve data from a recent GSB alumni, the student’s former employer or past project contacts.
  2. Perform the analysis using techniques discussed in class - data cleaning, regression analysis, Rstudio, etc.
  3. Communicate and document your results in a clear, polished report
  4. Review and critique a classmate’s data report.

Details and Logistics

The assignment is to be completed in groups of four to five students; five is probably ideal.  

Grading Criteria

Group Work

Each project group should hand in only one regression proposal and one project report. All project team members will receive the same grade on the project. Besides the usual administrative criteria (e.g., the project meets the deadline), we will use the following criteria:

1. Originality of the problem.

2. Usefulness and practicality of the analysis.

3. Quality and correctness of the analysis.

4. Quality of the exposition, writing, and presentation.

Because this is a D&D project, item 3 receives more weight than the other three criteria do individually. We take into account, on a more subjective basis, unusual effort or findings that are valuable to a real client. We also give credit for entrepreneurial, clever and difficult data collection efforts.

Individual Work

In addition to the group data report, we also ask that each student individually write a short 1-2 page data critique of another group’s data project. This is a form of peer assessment that supports your classmate’s learning and is a way for you to practice being in the seat of a manager reviewing a data report your team produces.  

Data

We ask that you make your data file available to us. In the past, we have found this useful in understanding details of the report. You can do this in several ways. The easiest thing for us is for you to put it on student scratch and tell us where it is located. Alternatively, if your data is sensitive and you feel uncomfortable putting it there, you can either email it to us, or provide a CD instead (if you choose to provide a CD, please label the CD!).

Regarding the quality of data, here are some hints that may help:

APPENDIX E: REFERENCES                

Bruner, J. (1960). The Process of Education. Cambridge: Harvard University Press.

                                        

Coffey, J. (2003). Involving Students in Assessment. In J. Atkin & J. Coffey (Eds.) Everyday Assessment in

the Science Classroom. Arlington, VA: National Science Teachers Association.

Dewey, J. (1938/1997). Experience and Education. New York: Simon & Schuster.

McTighe, J., & Ferrara, S. (1998). Assessing Learning in the Classroom. Washington, DC: National

Education Association.

                                

Treagust, D., Jacobowitz, R., Gallagher, J, & Parker, J. (March 2003). Embed Assessment in Your

Teaching, Science Scope

                        

                

/