User Analytics

Requirements and Team Policies

1 - Team info & policies

Team Members

• Nikhil Sharma
• Patrick Sharp
• Yukai Yan
• Zhennan Zhou
• Richard Jiang

Project Artifacts

• https://github.com/patrick-sharp/UserAnalytics
• https://www.figma.com/file/zm6dGgsUf1aPi1X8LkQ7Ib/User-Analytics?node-id=0%3A1

Communication Channels

• Slack channel (project-user-analytics)

2 - Product Description

UserAnalytics is a Chrome extension that allows users to easily view their Chrome usage trends. Through stylized and interactive visualizations, they should be able to see what domains they visit the most, the volume of internet usage throughout the day by hours, and the time for each category of website. We want UserAnalytics to be easy to use and install, allow user customization in domain tracking, store data securely, and be respectful of users’ privacy.

Four Main features:

1. User can visualize usage for each domain in the timesheet
2. User can visualize usage for each day of the week in hours.
3. User can configure whitelist to turn off tracking on certain domains.
4. Extension categorizes time spent automatically (social media, reading/reference, and productivity)
5. User can visualize time spent per category of websites for each day of the week with a running average.
6. Users can toggle whether or not they want their browsing habits to be tracked.

Stretch Goals:

1. Focus/Relax mode that can be suggested on/off based on user actions (ML) or manually turned on.
2. Block sites at certain times, set time limits for certain sites.
3. ✅ Users can customize the categorization of their browsing habits by associating specific domains with specific categories of website (e.g. associating bitbucket with the Productivity category)
4. ✅ Users can clear their browsing data from our extension’s storage at any time.

3 - Use Cases

1. Case: User wants to check his usage for each domain (Yukai)

1. Actors: users who downloaded our plugin and uses Chrome, can be anyone.
2. Triggers: start our plugin manually in Chrome, or automatically triggered as Chrome started.
3. Preconditions: Use Chrome as a default browser, have downloaded and configured plugins.
4. Postconditions (success scenario): the usage data being recorded in our database upon the termination of Chrome.
5. List of steps (success scenario):

1. The user started Chrome and downloaded the plugins.
2. The user configured plugins to start monitoring usage.
3. The user is free to start surfing at this point.
4. The user can check his usage history on each domain by clicking on the plugins anytime it wants.
5. A detailed analysis with the visual representation will be displayed.
6. The user closes Chrome, the plugin will terminate and data will be kept in the database.

6. Extensions/variations of the success scenario

1. Variation1: the user wants the plugin to automatically record the data.

1. At step ii, the user can set the plugin to run as Chrome started.

2. Variation2: the user does not want data to be saved.

1. At step vi, the plugin will not record data into the database
2. The plugin will not use this data for analysis on domains.

7. Exceptions: failure conditions and scenarios

1. Exception1: Chrome updates

1. Chrome can perform updates automatically and our plugin may no longer be compatible with the newer version of Chrome

2. Exception2: database errors

1. The database may fail to record data, hence discard any usage the user has this time

2. Case: User wants to configure black/white lists for domain tracking (Zhennan)

1. Actors: Users who downloaded and installed the plugin on their Chrome browser.
2. Triggers: launch the plugin manually by pressing the plugin icon
3. Preconditions: Chrome is used as the primary browsing tool. Incognito mode is turned off. The plugin is successfully installed.
4. Postconditions (success scenario): The plugin keeps track of browsing data based on user-configured whitelist/blacklist.  
5. List of steps (success scenario):

1. The user launches Chrome and installs the plugin.
2. The user launches the plugin page by clicking the icon in the Chrome toolbar and choosing going to the dashboard.
3. The user navigates to plugin setting -> black/white list
4. Making changes to lists of domains they want the system to avoid/track.
5. The user saves the changes and closes the page.
6. Plugin page reflects which list(s) are currently in effect.
7. The user quits Chrome and the black/white list are saved to local storage as well.

6. Extensions/variations of the success scenario

1. User wants to modify domain/websites to black/white list without launching the dashboard and changing manually

1. At step ii, choose either adding/removing current webpage to whitelist or blacklist

2. User wants to export and save current black/white list

1. At step iii, output the list(s) by pressing the download button.
2. A file download will be triggered automatically and file(s) will be stored on the user's local machine.

3. Users wants to import existing black/white list

1. At step iv, import the existing list instead of manually recording any domains.

7. Exceptions: failure conditions and scenarios:

1. Exception1: User inputs incorrect domains/websites format

1. At step v, before saving the changes, notify the user any incorrectness.

2. Exception2: User imports incorrect list config files

1. At step iv, notify the user that they need to select the correct file in correct format.

3. Exception3: User forgets to save changes before quitting

1. At step v, show the user an alert, telling them the consequences of any unsaved changes.

3. Case: User wants to check his usage for each day of the week (Richard)

1. Actors: users who downloaded our plugin, can be anyone.
2. Triggers: start our plugin manually in Chrome, or automatically triggered as Chrome started.
3. Preconditions: Use Chrome as a default browser, have downloaded and configured plugins.
4. Postconditions (success scenario): the usage data per day of the week being recorded in our database upon the termination of Chrome.
5. List of steps (success scenario):

1. The user started Chrome and downloaded the plugins.
2. The user configured plugins to start monitoring usage.
3. The user is free to start surfing at this point.
4. The user can check his usage history per day by clicking on the plugins anytime it wants.
5. A detailed breakdown of hours per day will be displayed.
6. The user closes Chrome, the plugin will terminate and data will be kept in the database for up to 3 weeks.

6. Extensions/variations of the success scenario

1. Variation1: the user wants the plugin to automatically record the data.

1. At step ii, the user can set the plugin to run as Chrome started.

2. Variation2: the user does not want data to be saved.

1. At step vi, the plugin will not record data into the database
2. The plugin will only display daily usage for the current day, the data will be lost when Chrome is terminated

7. Exceptions: failure conditions and scenarios

1. Exception1: Chrome updates

1. Chrome can perform updates automatically and our plugin may no longer be compatible with the newer version of Chrome

2. Exception2: database errors

1. The database may fail to record data, hence discard any usage the user has this time

4. Case: User wants to see their browsing habits batched by category (e.g. social media, productivity, entertainment, education) (Patrick)

1. Actors: Any user.
2. Triggers:  After starting our Chrome extension manually, or automatically when Chrome is launched.
3. Preconditions: Use Chrome as a default browser, have our Chrome extension downloaded and enabled, have some existing usage data in our database to show the user.
4. Postconditions (success scenario): The user getting a better understanding of how long they spend doing different kinds of internet browsing.
5. List of steps (success scenario):

1. The user launches Chrome with our extension installed.
2. The user clicks a button in our extension to visualize their browsing habits by category.
3. Our extension displays a detailed breakdown of the user’s browsing habits by category, visualized in an easy-to-understand way. This might be a bar graph or pie chart for example. The extension also displays a list of all websites that make up a category (e.g. youtube, netflix, hulu for entertainment).
4. The user views a visualization of how long they spend on each category of website over the course of a day and gains a greater understanding of their browsing habits.
5. The user clicks outside the visualization window or presses the escape key to terminate the visualization.

6. Extensions/variations of the success scenario

1. At step iv, the user views the breakdown of their browsing habits over the course of a week rather than a day.

1. The user can click a button to change which time period they want to visualize data over.

2. At step iv, the user views the breakdown of their browsing habits over the course of a month rather than a day.

1. The user can click a button to change which time period they want to visualize data over.

7. Exceptions: failure conditions and scenarios:

1. Exception1: Chrome updates

1. Chrome can perform updates automatically and our plugin may no longer be compatible with the newer version of Chrome

2. Exception2: Backend error

1. If the user is loading their data from the backend, the database may not have as much data as they require. We only store the user’s data for the most recent 3 week period, meaning that if a user is loading their data from the backend instead of locally, the backend might not have all of the data they want to visualize.
2. If the user is loading their data from the backend, the backend may fail to respond.

5. Case: User wants to block certain sites or a category of sites during certain times

1. Actors: Any user
2. Triggers: When the user opens the extension
3. Preconditions: Chrome installed, extension installed, extension configured
4. Postconditions (success scenario): The user is not able to access the specified websites
5. List of steps (success scenario):

1. User opens chrome
2. User installs extension
3. User configures websites/category they do not want to visit and sets a timer
4. User confirms and continues browsing
5. User attempts to access a blocked site, and is redirected to page notifying them why the site is blocked and shows the time remaining

6. Extensions/variations of the success scenario

1. User does not visit site before the timer expires

7. Exceptions: failure conditions and scenarios:

1. User does not set specific websites

1. In this case when the blocking mode is on, nothing will change

2. Website is mis-categorized

1. The user turns on blocking mode
2. User tries to visit a site that should not be blocked, but is blocked

3. Subdomains are not treated properly        

1. Users turns on blocking mode
2. User visits a sub-domain of a site and the site should be blocked, but is not blocked

4 - Non-functional Requirements

1. User-Analytics will protect its users' privacy by only storing browsing data for up to 1 week. The user’s data will be securely stored through Chrome sync storage.
2. User-Analytics will correctly collect and represent a user’s data, so as not to mislead them with false visualizations and conclusions.
3. User-Analytics’ will have an intuitive user interface, have minimal impact on performance of Chrome, and have close to real time updates on the visualizations.

5 - External Requirements

1. The product must be robust against errors that can reasonably be expected to occur during the course of normal Chrome usage, such as browser crashes, slow-loading pages, invalid user requirements, null settings, or unparsable user inputs.
2. The product must be installable by a user as a chrome extension and interfaced with by clicking its icon in the chrome extension menu.
3. The product should be available either in the google extension store or in the package that we published in our github repository.
4. The software (all parts, including clients and servers) will be buildable from source by others. The instruction for setting up should be included in our top-level Readme file. The link to the github repository will be made publicly available, enabling new developers to make enhancements.
5. The scope and feature set of our chrome extension should match the complexity that a 5-person team is capable of building.

6 - Team Process

Software Toolset

• Chrome Extension Ecosystem, JavaScript, HTML, etc.

• Chrome extensions are the best way to inference with the chrome applications. Javascript is the only way to develop chrome extensions

• Git

• Project spans from front-end to backend development to UI design. Using git allows each team member to branch out with their individual work without interfering with others or existing work.

• Github

• Github supports issue tracking and pull requests that can help manage the project.
• We use Github’s Actions for Continuous Integration.

Team Member Roles

• Design/UI - Zhennan

• User interaction, graphs, data report.

• Frontend - Richard, Zhennan

• Chrome extension, cooperate with Design/UI.

• Backend - Nikhil, Yukai

• Chrome storage, processing frontend request, running scripts.

• Testing - Patrick

• Writing test code for individual features and functions

Schedule

Five Major Risks

• The functionality of the Chrome API may not allow us to track all of the data we want to track.

• Likelihood of occurring: Low
• Impact if it occurs: High
• Evidence for our estimate: We have already made progress in tracking this data. Currently, we can tell that it is possible to track everything we care about. If we realize we need to track something else later, we might not be able to track it.
• Steps we’re taking to reduce impact or improve estimates: As we get more experience with the Chrome APIs, we’ll get more information about what is and is not possible. If we find out that something is impossible, then all we can do is change the scope of the project.
• Plan for detecting the problem: Keep using the Chrome APIs and search for any gaps in functionality compared to what we need.
• Mitigation plan: Re-scope the project to skirt around missing API functionality.

• Chrome extensions have limited functionality compared to full web applications, so we may run into problems that can’t be solved in the way we’re used to solving them in web development. There may also be performance issues associated with running certain libraries/frameworks inside of chrome extensions.

• Likelihood of occurring: Medium
• Impact if it occurs: Medium
• Evidence for our estimate: There are many ways to do things in web development. If we run into some solution that works in full web apps but not in Chrome, we can likely find a way around it.
• Steps we’re taking to reduce impact or improve estimates: As we get more experience with the Chrome APIs, we’ll get more information about what is and is not possible. If we find out that something is impossible, then all we can do is change the scope of the project.
• Plan for detecting the problem: Keep using the Chrome APIs and search for any gaps in functionality compared to what we need.
• Mitigation plan: Search for alternative solutions to whatever problem we have that can’t be solved using regular web development solutions. If necessary, re-scope the project.

• Chrome has several features (multiple users, incognito mode, limiting user permissions) that may impact our ability to track a user’s internet usage accurately.

• Likelihood of occurring: Low
• Impact if it occurs: High
• Evidence for our estimate: There are existing chrome extensions our team has used that rely on gathering data about user browsing habits. Chrome extensions have also been looked at as computer security concerns. It seems as though the data Chrome allows extensions to capture about user browsing activity is rather extensive.
• Steps we’re taking to reduce impact or improve estimates: As we get more experience with the Chrome APIs, we’ll get more information about what is and is not possible. If we find out that something is impossible, then all we can do is change the scope of the project.
• Plan for detecting the problem: Keep using the Chrome APIs and search for any gaps in functionality compared to what we need.
• Mitigation plan: Re-scope the project to skirt around off-limits functionality.

• There may be certain legal concerns about storing data that we aren’t aware of in different countries around the world. These legal concerns may involve prohibiting certain data collection or requiring bureaucratic procedures to get a license for certain data collection.

• Likelihood of occurring: Medium
• Impact if it occurs: High
• Evidence for our estimate: Privacy legislation such as Europe’s GDPR has lots of arcane and extensive rules about what data companies are allowed to gather. There may be other similar regulations for countries around the world, but lots of extensions / websites / applications seem to get away with a simple terms-of-service agreement somewhere in the application.
• Steps we’re taking to reduce impact or improve estimates: We will implement a user agreement checkbox form that requires users to agree to certain terms before they can use the extension.
• Plan for detecting the problem: We will not deploy the application in any country before verifying the legality of collecting data in that country.
• Mitigation plan: e will implement a user agreement checkbox form that requires users to agree to certain terms before they can use the extension.

• It may not be feasible to run a chrome extension in the background for the lengths of time we want to.

• Likelihood of occurring: Low
• Impact if it occurs: Medium
• Evidence for our estimate: There are lots of chrome extensions that seem to run in the background for long periods of time, or which automatically start when Chrome is launched and pick up where they left off.
• Steps we’re taking to reduce impact or improve estimates: Keep using the Chrome APIs and search for any gaps in functionality compared to what we need.
• Plan for detecting the problem: We’re looking into writing unit tests that simulate the extension running for long periods of time. Other than that, we’ll just have to test running the extension for a long period of time.
• Mitigation plan: We may need to program in functionality that asks the user to re-launch the extension every now and then. This would be frustrating, but not a complete catastrophe.

External Feedback

• External feedback would be most useful while designing the UI and usability. To get feedback, we will ask designers to review and reach out to potential users for their opinions. We reached out to several people through our personal networks over the course of the quarter to get their feedback on our project’s usability.
• In our README, we document the process for giving feedback to the project via opening GitHub issues. Users of the product will be able to give us notes and feedback through GitHub.
• During the Peer Review milestone for this class, we received feedback from other students about the state of our product. The students opened three issues, and we integrated that feedback into the project. The issues can be seen in the “Issues” tab of our project GitHub. All are labeled as “Closed” and contain some notes about bugs and UI fixes.

Test Plan & Bugs

• Unit tests exist as a javascript program run with the Node.js runtime. These tests use a Mock-API for Google Chrome and test the functionality of our middleware functions in that mock environment.

• These tests are run by our CI actions in GitHub.
• This allows our extension to be lightweight and easily installable by users since we don’t use a build / bundle system. Node is only used for running the testing program outside of a browser environment and not for building or running the extension itself.

• Our test-automation infrastructure.

• For CI-triggered unit tests, we have a Javascript file that tests our middleware javascript functions.

• The file ‘test.js’ in the top-level folder ‘test’ runs a series of unit tests on our Javascript code. It uses a mock API that mimics the parts of the Chrome API that we use to store browsing data. It is a simple set of unit tests that triggers a bunch of different test functions and counts how many of them passed. If any of the tests fails, the GitHub action will fail, and the logs will contain the information about exactly which tests failed and any relevant exceptions.

• For UI tests, we use a Javascript file for UI tests (e.g. for making sure certain text appears if you click a button on the dashboard)

• Our UI tests are run by a file called ‘dashboard_test.js’, which has access to the dashboard’s DOM.
• These UI tests are only run if the variable “doRunTests” at the bottom of “dashboard_test.js” is set to true.
• The functionality of this file is limited due to inherent constraints on what actions javascript is capable of triggering at what time on the page. Currently, this file allows us to verify that the DOM loaded correctly and that our dependencies were successfully loaded. It runs very quickly in the background, and can be a useful diagnostic tool for figuring out if any crashes or other unforeseen issues have occurred.

• A brief justification for why we chose that test-automation infrastructure.

• We wanted to avoid using any system for building our extension into a single file. Keeping the code in vanilla Javascript and HTML allows us to take advantage of instant hot-reloading during development without having to wait for a build process to finish.
• We also wanted to make the testing infrastructure as simple as possible.
• We decided to separate tests into our two categories (CI-triggered Javascript unit tests and manually-triggered UI tests).
• The UI tests are not run during CI testing. There isn’t support for running Javascript inside of a Chrome browser environment. Setting up custom infrastructure would likely take at least 2 weeks of work due to the complexity involved.

• How to add a new test to the code base.

• There is a top level folder in our git repository called ‘test’. Within it is a file called ‘test.js’, which contains all of the non-UI unit tests for our javascript code.

• To add a new unit test for the JS, simply append the new test to the ‘test_functions’ array in the ‘test.js’ file.

• For the dashboard, there is a file called ‘dashboard_test.js’. This file is attached by a script tag to the ‘dashboard.html’ file. This javascript file runs tests on the page generated by ‘dashboard.html’ and searches through the DOM to make sure the page displays the right content.

• To add a new dashboard UI test, make a new function that returns true if passes, and returns false if it fails. Put that function inside the ‘test_functions’ array declared in the ‘dashboard_test.js’ file. The code will run that test and report on whether or not it fails in the browser’s developer console.

• Our CI service and how our project repository is linked to it.

• We use GitHub actions for CI. when any code is pushed / pull-requested to the main branch, the CI service triggers and runs the Javascript file described in the next bullet point.
• GitHub will automatically read from YAML files in our codebase and execute the actions described in them to evaluate the pushed / pull-requested code.

• A brief justification for why we chose that CI service.

• It is by far the simplest option for us. Since our code is in GitHub, using GitHub actions requires no additional tooling and is very simple to understand just by looking at the repo’s GitHub page.

• A pros/cons matrix for at least three CI services that we considered.

• Which tests will be executed in a CI build.

• Only the non-UI unit tests for our JS code will be executed in a CI build.
• UI tests will be run manually by loading the extension’s code into chrome and navigating to the dashboard. This is less automated than we would like, but the alternative is incredibly complex (see justification section above). These tests are not run by default, and developers will have to set the `doRunTests` variable to true before they can run these tests.

• Which development actions trigger a CI build.

• Pushing code to the ‘main’ branch of our GitHub repository or opening a pull-request to the ‘main’ branch will trigger a CI build.
• CI testing can also be manually triggered from GitHub’s actions tab.

Documentation Plan

• We will document our code using GitHub readme.md(s). Installation guide will be provided for users along with feature introductions and a video walkthrough. Inside the src/ folder will be another readme detailing the layout of the repository as well as what each file is responsible for. Within each file we have grouped similar functions together and have detailed comments about the use of each function.

7 - Software Architecture

Major Components

• Chrome Background Script

• Listens to tab changes and records timestamps

• Middleware Functions

• Using timestamp information from background script and Chrome storage, calculate time spent on a website to store into Chrome storage

• Chrome Storage

• Key-Value pair storage, used to keep last visited domain and time spent on each domain per day. It also stores preloaded category lists and whitelists.

• Dashboard Processing & UI

• Uses middleware functions to read time spend information from storage and displays in a graph

Interfaces

• Background Script to Middleware

• Script calls function in middleware to update last visited domain, with a timestamp

• Middleware to Chrome storage

• Uses Chromes API to read or write to Chrome’s implementation of javascript localstorage

• Dashboard UI to Middleware

• Dashboard calls function in middleware to grab time spent data to graph

Data Storage in Chrome

Chrome storage is a key-value store.

Assumptions

• We are assuming that Chrome local storage has the space and performance capacity to deal with large amounts of user data
• We are assuming that the user’s Chrome version implements Chrome.runtime, Chrome.tabs, and other Chrome APIs.

Alternatives

• Adding a backend as one of the major components of our software architecture. This backend would use  a database to store user data instead of Chrome localstorage.

• Con: Implementing and hosting a database is more complicated and expensive
• Con: User’s data is now on a remote server which could impact their privacy
• Pro: Could store a lot of more data and data doesn’t have to be Strings
• Pro: User could have access to their data across different devices

• In our dashboard interface, the graphs could update in real time instead of displaying a static snapshot of user data when the page loads.

• Con: More compute power necessary with all the callback functions and redrawing graph regularly
• Pro: More interactive for users

8 - Software Design

Units of Abstraction

• Our code is written in a procedural style. We only have one piece of state, which is the time active on each website. This state is stored in Chrome’s localstorage, so that means we don’t have to define other complicated data structures, class hierarchies, modules, packages, or serialization methods. Our code is mostly made up of functions that populate, manipulate, and display the data we log in localstorage.
• Each component in our software architecture is given its own folder. Within each folder are files that represent some logical piece of that component’s functionality. Each file will contain a number of functions that together implement that file’s piece of the component’s functionality.

Software Component Responsibilities

• Chrome Background Script

• Contained in the file background.js in the background_script folder.
• Composed of different Chrome event listeners and invokes our Middleware tracking functions.

• Middleware Functions

• Contained in the file middleware.js in the middleware folder.
• Middleware functions live in middleware.js. It is composed of different javascript functions that are responsible for reading and writing data to the Chrome local storage.

• handleUrlChange() [called by background.js]
• chromeInactive() [called by background.js]
• chromeActive() [called by background.js]
• getDomainsForDay() [called by processing.js]
• getDomainsForWeek() [called by processing.js]
• getTimeForDay() [called by processing.js]

• Chrome Storage

• Chrome’s local storage is Chrome’s implementation of the window.localstorage feature defined in the ECMAScript specification.
• Chrome storage is represented through provided Chrome APIs, its responsibility is to store and provide data.

• chrome.storage.sync.get() [called by middleware.js]
• chrome.storage.sync.set() [called by middleware.js]

• Dashboard UI

• Contained in dashboard.html, dashboard.js, processing.js, and other css files
• Dashboard is a local webpage with html, js, and css files. It uses javascript to call middleware functions to get user data and store user settings.

• getTotalTime() [called by dashboard.js]
• getMostFrequentTime() [called by dashboard.js]
• getLineChartData() [called by dashboard.js]
• getPolarChartData() [called by dashboard.js]
• getTimesheetData() [called by dashboard.js]

9 - Coding Guidelines

https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode

Our group works in VSCode, so we can use the VSCode extension for the prettier code formatting tool. Prettier formats html, css, and javascript according to a very specific specification. Using prettier as a vscode extension is very convenient and lightweight.

If needed, we can also use the prettier NPM package to format code in exactly the same way. Since prettier is opinionated and automatic, it means all members of the team can format their code to a specific style without lengthy code reviews or editing processes. All team members can make their code conform to the same style guidelines with the press of a button. This will save us time enforcing the style guidelines.

Prettier is widely used in industry to keep a consistent code style.