TEAMMATES Online Feedback System for Educators 
[System Specification (Spec)]

Preliminaries

Features are organized by user. We document requirements as a hierarchical feature list.

Feature ID : We use short titles instead of numbers to identify features. For example, we can refer to a feature as ‘Instructor:ManageCourses:Create’. This also allows easy insertion of new features into the document.

{text within curly brackets are explanatory notes}

Feature priorities

Feature status: [pending] Annotation is used to mark features not implemented yet but likely to be implemented in the future.

Data definitions: A data field is described under the feature in which it is populated. e.g., What kind of data is allowed as the Course Name should be defined under createCourse feature.

Last updated at: V4.74 (half-updated at V5.72)

Table of Contents

Preliminaries

Table of Contents

--------------------------------------------------------------------------------------------------

TheProduct

Vision

Identity

Users

--------------------------------------------------------------------------------------------------

CommonFeatures

Login

Logout

Website

HomePage

AccountRequestPage

FeaturesPage

AboutPage

TermsOfUsePage

ContactUsPage

--------------------------------------------------------------------------------------------------

InstructorFeatures

Homepage

ShowOnlyRecent

GettingStartedLink[pending]

HelpPage

ManageAccount [pending]

EditAccountDetails

ManageCourses

CRUD

Archive

ManageInstructors

CRUD

AccessControl

ManageStudents

MassEnroll

ViewList

SendInvite [to be removed]

AddStudent [pending]

EditStudent

MassEdit

DeleteStudent

MassDelete[pending]

ChangeTeamStructure

SearchStudents

StudentRecords

CommentOnStudents

EmailStudents

TagStudents [pending]

ManageFeedbackSessions

Create

ViewList

Edit

Delete

Clone

ManageFeedbackQuestions

Create

ViewList

QuestionTypes [partial]

Edit

Delete

Clone [pending]

Upload files[pending]

ManageFeedbackSessionReports

ByRecipient

ByGiver

AsTable

isSeenByStudent[pending]

PrinterFriendlyView

Download

EditSubmissions

Publish

Unpublish

--------------------------------------------------------------------------------------------------

StudentFeatures

HomePage

ShowRecentOnly [pending]

HelpPage

ManageAccount

ResetGoogleId[pending]

ManageCourses

Join

ViewTeams

AutoJoin [pending]

ManageFeedbackSessions

Submit

View

Edit

ViewReport

--------------------------------------------------------------------------------------------------

AdminFeatures

ManageSystem

OnlineOffline

Statistics [pending]

ManageData

PurgeOldData

CreateInstructor

EditInstructor [pending]

DeleteInstructor

FindEntity[pending]

ManageEntity

Monitor

Masquerade

ActivityLog

--------------------------------------------------------------------------------------------------

AutomationFeatures

EmailAlerts

FeedbackSesssionOpeningAlert

FeedbackSesssionClosingAlert

FeedbackSesssionResultsPublishedAlert

Other

AutoCloseSessions

AutoPurgeOldData [pending]

ContextualHelp

--------------------------------------------------------------------------------------------------

SupplementaryRequirements

PointCalculationScheme

Report Format

Scalability

Security

--------------------------------------------------------------------------------------------------

Data Validation Rules

Glossary

--------------------------------------------------------------------------------------------------

TheProduct

Vision

TEAMMATES is a tool to help instructors manage various forms of feedback in a class, including peer evaluations.

Our target users are instructors. Their students are secondary users. i.e., Instructors decide to use the system, not students.

Identity

Name : The name TEAMMATES was chosen because the original focus of the system was managing peer feedback/evaluations in team projects. Since then, our scope has expanded to other forms of feedback. However, we are going to stick with this name for now until we reach a point of 'rebranding' in the future.

Tag line : Manage feedback in your class.

Logo:  

Users

--------------------------------------------------------------------------------------------------

CommonFeatures

These are the features common to all users.

Login

Users use Google ID to log in

Logout

Website

{'Website' refers to static pages describing our product, which is a WebApp }

--------------------------------------------------------------------------------------------------

InstructorFeatures

Homepage

The first page user sees upon login. Contains shortcuts to actions the user is likely to do next.

HelpPage

ManageAccount [pending]

ManageCourses

{See glossary for the specific definition of ‘course’}

ManageInstructors

 

Making changes to instructors in a course is done with the Add/Edit Course forms.

ManageStudents

{The concept is ‘joining’ is to be removed from user-visible features.}

ManageFeedbackSessions

We use ‘FeedbackSessions’ to refer to sessions with customizable questions.

ManageFeedbackQuestions

ManageFeedbackSessionReports

All reports are viewable at any time.

--------------------------------------------------------------------------------------------------

StudentFeatures

HomePage

The home page contains things requiring student's attention e.g., evaluations to submit, reports to view etc.

HelpPage

Contains details such as Getting Started, FAQ etc.

ManageAccount

ManageCourses

{To be replaced by AutoJoin given below}

ManageFeedbackSessions

--------------------------------------------------------------------------------------------------

AdminFeatures

ManageSystem

ManageData

Monitor

--------------------------------------------------------------------------------------------------

AutomationFeatures

EmailAlerts

Other

--------------------------------------------------------------------------------------------------

SupplementaryRequirements

PointCalculationScheme

Students enter contribution estimates for self and team members, using the contribution scale (see glossary).

Based on those values, we try to deduce the student's answer to the following two questions:

(a) In your opinion, what portion of the project did you do?

(b) In your opinion, if your teammates are doing the project by themselves without you, how do they compare against each other in terms of contribution?

In the calculation, we try to avoid (a) affecting (b). We use (b) to calculate the average perceived contribution for each student.

Calculation steps:

  1. calculate normalizedClaimed values. This is required because the total of points entered might not sum up to 100 x (team size).

(normalized value) = (original value) * (normalization factor)

e.g.,

entered values 90 [self], 110,130, N/A (total = 330)

normalization factor: (100*3)/(90+110+130) = 300/330

normalized: 82, 100, 118, N/A

normalized total = 300 (i.e. 100*number of inputs)

This answers the question (a) above. The student thinks he did 'Equal share - 18%' (as indicated by 82).

  1. Calculate peerContributionRatio values by removing self-rating bias

Here, we ignore the self rating and normalize remaining values.

e.g.,

normalized input (from above): 82,100, 118, N/A

Calculating unbiased values:

82 → ignored.

100 → 100*200/(100+118) = 92

118 → 118*200/(100+118) = 108

Unbiased values: [self (ignored)], 92, 108, N/A

Unbiased values total = 200 (100*number of ratings) This answers the question (b) above. In the example above, the student thinks his teammates contribution ratio is 92:108 and is unsure of the third teammate.

  1. Next, we calculate averagePerceivedContributionRatio among team members, independent of (a). This consists of these steps:

i. calculate averagePerceived:

For each student, take the average of peerContributionRatio that others have given him.

ii. calculate normalizedAveragePerceived:

Normalize the averages, similar to how input was normalized.

normalizedAveragePerceived = averagePerceived*normalizationFactor

normalizationFactor = 100 * (number of students with averagePerceived values)/(sum of averagePerceived)

This is the relative work distribution among team members based on unbiased opinions of team members.

iii. calculate normalizedPeerContributionRatio.

Since we normalized the averages (in previous step), we also normalize the value that were averaged in the first place. This is such that average and averaged tallies with each other.

normalizedPeerContributionRatio = peerContributionRatio*normalizationFactor

  1. For each student, denormalize normalizedAveragePerceived. We scale back to match the total of original input by student. That way, student can compare his input (i.e., his opinion of the team’s work distribution) with the team’s opinion. In the example used above, we should use 330/300 as the denormalizing factor for that student. The result could be something like this:

student’s opinion: 90 [self], 110,130, N/A (total = 330)

team’s opinion : 95, 105,125, 115 (total = 440)

Value transformation steps: input (i.e. claimed) → normalizedClaimed → peerContributionRatio → averagePerceived → normalizedAveragePerceived → denormalizedAveragePerceived → normalizedPeerContributionRatio

Student view:

Instructor view:

Note:

Report Format

Some important questions a instructor may want answered (i.e., the system should provide ways of answering these quickly and easily)

Reports have to be highly optimized for space, navigation, and attention.

Scalability

Security

--------------------------------------------------------------------------------------------------

Data Validation Rules

Data validation rules (e.g. max length, allowed characters, etc.) are defined in teammates.common.util.FieldValidator.java

Glossary

Refer to the Glossary in the Developer Manual

--- End of Document ---