ALC Tech 2017

This document covers a high level overview of the current tech infrastructure used by Agile Learning Centers with a focus on the website: agilelearningcenters.org 

Recommendations

To begin, I’d like to outline the recommendations for what works should be done on the tech front. Below this section I will go into more detail about each section of the tech infrastructure. These recommendations are ordered in priority, highest coming first.

Member Council

The biggest problem is not a technical one, it is a cultural one. There are so many things that can be worked on I’ve found it to be completely overwhelming over the past few years to ever focus on one. Most recently the “net biz” working group chose to focus on the creation of a membership automation process. From our perspective it seemed like the logical thing to focus on. The many hours spent developing a membership automation process was possibly a mistake. A highly automated process for adding new members probably wasn’t on any member ALC’s list of tech needs.

Thus my first recommendation, and most important one, is for member ALCs to convene a membership council so that their needs can be expressed. The “Spokes Council” model would be best. Each ALC would select a “spoke” to represent their needs in the monthly or quarterly council meeting. Time would be set aside to prioritize tech goals. This document could serve as a menu of sorts.

Maintenance Strategy

There are a number of recurring tasks that are necessary to keep the website running well. These are outlined in the maintenance section below. A strategy to ensure that these tasks are performed needs to be made.

My suggestion

The network should decide how much the tech infrastructure is worth to them and pay someone to maintain it. I do not believe that it is sustainable to rely on volunteer labor to keep such a key piece of infrastructure running.

Development Strategy

See Development Workflow 

If we are going to further develop our tech tools we first need to set up an easy way for people to volunteer time. Currently the system is too complex and messy for anyone to really start digging into. Right now for anyone to do any meaningful work on our system they would need to invest a huge amount of time just figuring out what to do!

With a good development workflow (as outlined below) we could hold hack-a-thons, engage students, and hire developers for discrete tasks and projects that are probably too disorganized or amorphous currently.

ALC Website Support

Each ALC in our network has a website, these sites are a toolbox that can reduce the work an ALC has to do around many processes, from outreach & enrollment to internal processes. I think this is one of the most valuable assets we have because it directly supports the heart of the network, which is the actual members.

There is a LOT that can/should happen, here is a non-exhaustive list, some of which I cover later:

Website refactoring & Big Picture Plan

See The Website

Currently the whole website is built on Commons in a Box, which uses Buddypress to create a social network. Much of these features are not being used and create a lot of bloat and confusion. The whole site should be refactored and, based on the needs of ALC members, a plan should be put forward, budgeted, and money raised to achieve this plan.

I have been advocating for a process of “discovery” to figure out what the users (ALC members) need and create a plan based on that. We have failed over the past 3 years to do this internally.

The major, if not only, selling point of maintaining a network site is to provide a value in having the people in our network interacting on the web platform. This is hardly happening. My fear is that ALCs will begin to look towards 3rd party applications, abandon the website, and the network will fracture and fall apart from a technical side. Once our members have locked themselves into platforms like facebook we will probably never get them back onto a single platform.

Details

A number of feature and bugs can be found on our waffle board (more on that below):

waffle.io/AgileLearningCenters/alc-wordpress-core

WPMU DEV Plugins - $50/month

We use a number of plugins from WPMU DEV, which we currently can’t update unless we get a $50/month membership to their service. Having a plan would help determine what features we actually want. Currently many of these plugins were installed and never really used.

SSL - $90/year?

It is recommended that all traffic goes through HTTPS:// which requires an SSL certificate. It might be possible to do this for free, otherwise it’s about $90/year to secure all of our sites. More info below.

Updating School Theme

The current default ALC theme that most (if not all) school sites use is in need of a major update to get it to a point where we can upgrade it. This is a significant time investment (6-10 hours maybe) because the code has to be tested and the update has to be done in a way that doesn’t break all the existing sites using the theme.

Conclusion

We can not continue to run our tech platform on volunteer labor, with no plan, ignoring the needs of our members. Our members must take responsibility for the platform by taking ownership over this shared resource. This platform was never designed as a service but as a community resource. It’s been brought to the community mostly for free and I can no longer maintain giving an insane amount of my labor to holding it together (poorly).

The Website

Agilelearningcenters.org

Hosted by siteground.com

Core code repository: github.com/AgileLearningCenters/alc-wordpress-core

The ALC Network website (alc.network) is built on the Wordpress Content Management System (CMS). It uses Commons In A Box (CBOX) which is a “plugin” maintained by The City University of New York (CUNY). CBOX provides the basic social networking and “Multi-Site” functionality.

Why Wordpress?

It is no accident that I chose Wordpress to power alc.network. Consider these facts:

Wordpress powers 27% of all websites on the internet[1] and has a nearly 60% market share of the CMS market. It is the platform used by wildly popular organizations including The New York Times.

Each person in the ALC network who interacts with our website is becoming proficient in one of the most popular pieces of web software in the world. The code that powers Wordpress is Free and Open Source meaning that it is co-owned by all of humanity. Thousands of developers, designers, and other professionals maintain this code. Further, the code that runs the ALC Network is available for anyone to use. It is my intention that our platform also be free, open, and accessible so that others can run their own self directed learner network which fulfills our goal of spreading self directed education. The ubiquity of Wordpress also means we have tens of thousands of potential collaborators who can help develop our platform.

Overview

Our Wordpress site is configured to act as a network. Each user has their own site (located at http://username.agilelearningcenters.org) and any user can create a site with a few clicks of a button for any group, project, or ALC.

Note: the term site and blog can be used interchangeably.

Wordpress has two main components: themes and plugins. Themes describe how information is displayed. Plugins extend the functionality of Wordpress.  Themes and plugins are shared by all sites inside the ALC Network.

Themes

There are two main themes, the Network theme which control the look of the main site and the starterkit website. The General theme is (optionally) used by ALCs and many other sites in the network. Both themes are based on a parent theme called Make.

What’s a parent theme? I hear you asking, well it doesn’t have to do with the legal guardian of minors. It is a theme which other themes are based on. So the General theme is a child of the parent theme named Make. Child theme use all the parts of their parent unless they overwrite a file or extend the functionality of the parent. This separation means that Make can be updated without overwriting the child theme.

Plugins

Our site uses a number of 3rd party plugins, meaning the code is maintained by people outside of our organization. I have also developed a few custom plugins to serve the needs of the ALC Network.

All data is stored in a Database on our host (Site Grounds) which is backed up daily. Data represents all content (posts, pages, etc.), user data, site configurations, and nearly all settings across all sites. In the event of a major outage the site can be restored with no more than a 24 hour loss of data.

Buddypress

The site was originally built using Commons in a Box (CBOX) which leverages Buddypress to create a social network. Much of the functionality brought by this tool doesn’t appear to be highly used. The most used feature is the activity feed which surfaces all activity across the site, mostly this comes in the form of student blog posts or updates (much like Facebook updates). The foundational structure of this plugin is causing quite a performance hit across the network, which I go into more detail about later.

Network Theme

Code: https://github.com/AgileLearningCenters/alc-wp-theme-network-make-child

The network theme is separate because the “main website” (agilelearningcenters.org) and other “network sites” such as starterkit.agilelearningcenters.org have unique needs that probably won’t be shared by user created websites (which includes ALC sites). For the most part there isn’t any custom code running on this theme right now so basically this theme is in place in-case network specific alterations need to be made.

For the technically inclined you can see a draft version of custom page templates used to display the custom post type here: https://github.com/AgileLearningCenters/alc-wp-theme-network-make-child/tree/alctemplate

General Theme

Code: https://github.com/AgileLearningCenters/alc-wp-theme-make-child

The general theme is used by nearly all ALCs, many facilitator sites (including my own), and student sites. The main features of this theme are a basic design settings that match the ALC “brand” and a sticky nav. The sticky nav was a feature request from ALC NYC and Mosaic in the early days of the website.

Problems

The code to create the sticky nav effect was hastily put together and presents a significant challenge because in its current state it is preventing me from upgrading the parent theme. What this means is that I can not upgrade the “Make Theme” which this theme that “powers” the general theme.

ALC Template Site

Alc-template.agilelearningcenters.org

This site was created to be a template for new ALCs. The idea here is that we would constantly update this site with the best practices that other ALC websites are using so that each new ALC can clone the template and be way ahead in making their website. The content should be general enough that even if an ALC doesn’t change anything it can still work.

What’s next?

The system should be able to automatically generate a new ALC’s website based on this template. We could even have the site automatically populate ALC specific information into the site, such as contact information or primary email address.

Starter Kit & Membership

Starterkit.agilelearningcenters.org - The Starter kit website makes the first version of the ALC Starter kit available to anyone who gives us their email. Once a user submits their email a number of notifications are sent out to email lists, slack channels, and the data is entered into databases. Emails with information about how to download and share the Starter Kit are also send to the person who requests it. Further, there is even a system for seeing all the people who have downloaded the Starter Kit and contact them via our website.

The detailed documentation about how this works doesn’t really exist.

Membership is managed via agilelearningcenters.org/membership. This process is outlined in this document along with more information in the startups folder of the ALC Network Drive Folder (more on that later)

Custom Plugins

There are a handful of custom plugins that provide ALC specific features to the network. The use of plugins to “upgrade” the network means that they are available no matter what theme a site is using and they are able to be deployed across all sites on the network.

ALC Post Type

Code: https://github.com/AgileLearningCenters/alc-post-type

In Wordpress a single piece of content is called a “post”. Each post has a single “type”. For instance a page on a site is a single “post” with a type of “page”. This can be confusing because blog posts are single “posts” with a type of “blog post”. Each post type contains a number of pieces of data, like a title, body, tags, author, publish date. A “custom post type” is a method of creating a post with custom data.

The ALC Post Type plugin creates a “post type” for ALCs. Which adds custom data fields specific for Agile Learning Centers (you can see an outline of the data here). This allows the website to manage member ALCs and display content about them. Currently this plugin creates the ability to display a map of ALC “posts” (remember a post is simply a single instance of a post type).

Data for ALCs is fed into the custom post type via the membership form (agilelearningcenters.org/membership/join/).

Troubles

This plugin is disorganized and the code is poorly written and needs to be re-formatted. Further the map functionality needs to be separated into its own plugin.

Future

There are improvements to be made in how ALCs are displayed in the dashboard of the site, mainly in the list of ALCs which should display more relevant information. There is also room to further automate the notification of ALC’s payment status, location, and standing within the network.

ALC Tuition Slider

More info: drew.agilelearningcenters.org/introducing-the-alc-tuition-sliding-scale-plugin/

This plugin creates a “shortcode” that can generate a tuition slider on any site (because the plugin is “network activated” meaning it is automatically useable across all sites on the network)

Problems

Currently this plugin is not under version control in the ALC github repository. It exists on the server and is under the core repository. It should be made into it’s own project and added to the core as a “submodule”. Here:

github.com/AgileLearningCenters/alc-wordpress-core/tree/master/wp-content/plugins/alc-tuition-slider

Domain mapping, HTTPS and Dedicated IP

A number of ALCs currently host their website on our server and use “domain mapping” so that their own domain names show up rather than their agilelearningcenters.org domain.

So http://nyc.agilelearningcenters.org becomes http://nycagile.org/.

There is a looming issue here though. We need to be using HTTPS, which encrypts communication from users and the server. This means that if you login to the site at a cafe some hacker can’t sniff the traffic over the public WiFi and see what you are doing.

Siteground offers a reasonable SSL Certificate (this is the thing that allows us to use HTTPS). To use an SSL Certificate one needs a “dedicated IP address”. Currently we have what is called a “shared IP address”. So to get HTTPS we will have to change our IP address, however, the domain mapping of our member ALC sites relies on a setting that “points” their domain name to our current IP address! So the process of setting up HTTPS and a dedicated IP address will also include coordinating with our member ALCs to update their domain name settings to point to our new domain name.

Lucky for us all of these domains are currently listed as “parked domains” within our server cPanel.

What’s Next?

Site Optimization

On the web windows of attention are measured in milliseconds. Our site is woefully unoptimized. Buddypress files are loading uselessly across sub sites. The main website which is mostly static content is talking to the database each time it is visited.

Optimization is taxing our server more than need be and causing load times in the hundreds of milliseconds, which is the network equivalent of minutes!

What’s Next?

Thanks Art Brock for these suggestions

Development Workflow

Any decent web project should follow a development workflow that looks something like this:

Individuals can run the site locally on their own computer to do independent development. Their code is then merged into a “staging” version of the site that exists in “the cloud” (aka on the ALC server) where new features, plugins, or updates are tested. Then once changes are shown to not break anything the staging server is “pushed” to the live website. Part of this process also includes making copies of the live website’s data available to developers without private data such as passwords or private messages between users.

Below is an overview of what of this process exists.

Local Development

Currently there is a process for setting up a version of the ALC website one’s own computer. Instructions can be found in the ALC Wordpress Core readme file. With the right technical knowhow it is possible for someone to run our site on their machine and do development.

What’s Missing

Currently there is no documentation around how to actually submit new code back to the main repository. It is also not possible to download the data from the live site without exposing private information.

What’s Next

Staging Site

Currently there is no staging site. Plugins, Themes, and core additions/updates are done on the live site and I simply pray nothing bad happens. There is no process for reviewing new plugins or even for people to make requests for features. This entire thing needs to be figured out.

Live Site

As I stated above the live site is being managed under the worst possible practices. My basic workflow for keeping the site under version control is as follows:

Unit Tests

The best practice for making updates is to run automatic tests that visit pages, submit forms, and perform other tasks across the site to make sure everything works as expected. It’s unreasonable to visit every page and play with every form or input on the site. For all I know key features on the site could currently be broken and no one has noticed yet (aside from the frustrated users who will probably never tell us they ran into a problem)

Maintenance

All websites require some amount of regular maintenance. Our site requires the following:

There is currently no strategy for doing any of this. Either someone with a level of access accidentally clicks “update wordpress” or I eventually get motivated to do it. This is something that a volunteer could be trained in and do with ease. However it would be best if the above development workflow be implemented so that we could stage and test updates before going live with them.

Keeping the site up-to-date is very important because updates typically fix security vulnerabilities. Wordpress is the most popular CMS on the internet which means it’s also the biggest target for hackers who deploy bots across the internet to test for vulnerabilities in Wordpress sites.

Adding New Plugins

Currently there is no process for members of our community to request new plugins or features for the website. As it stands people ask me for a feature and I either write the code or find a plugin that performs the action they want. A process for reviewing requests and testing solutions needs to be implemented. Because all sites share the same server/plugins/themes it becomes increasingly dangerous to randomly add things that could potentially break the site in unexpected ways.

Imagine a scenario where a new plugin breaks a contact form on a number of ALC websites. It might take months before anyone realizes that they haven’t been receiving enrollment requests. We are literally putting our already struggling ALCs at risk…

Analytics

The ALC Network site and many sub sites are tracked using Google Analytics. There is a wealth of data going back 3 years. To my knowledge no one is using this data. Many of our decisions about what is important website work do not reference any of this analytic data. As we move forward with updates, improvements, and changes to the website it should be kept in mind that this data is available and very valuable for making informed decisions.

This data is even more valuable to ALCs who are looking to enroll or raise money.

Here are the pageviews for the main site over the past 2 years. Any idea where those spikes came from? What we did we do in January 2016 to increase our traffic?

The git Repository

ALC has a github.com organization: https://github.com/AgileLearningCenters

This github repository should be where all the ALC code lives. It could also host foundational documents under version control. Git is a very powerful way to collaboratively edit huge amounts of text while maintaining a record of every single change and who made that change.

Further, this repository could host all kinds of code projects and act as a central hub for ALC community members to collaborate. Further Github offers wikis and issue trackers which allows it to manage documentation and requests of each project.

Waffle.io

This service tracks Github issues from one or more projects and displays them on a Kanban like board. Here is the waffle.io board for ALC:

waffle.io/AgileLearningCenters/alc-wordpress-core

I’ve been tracking bugs, feature requests, and other tasks for the website here. With a decent development cycle this could be the place where community members and volunteers self direct their own involvement in community code projects.

Licensing

Currently the code and much of the assets we have are unlicenced. Lawyers should be consulted to figure out what the best licences are for our content. We want to strike a balance between committing to a core principle of being open and sharing our knowledge and information with the world while also protecting our brand. All code and content should have explicit language around it’s license.

Other Infrastructure

Beyond the website there are other outside processes that make up some important pillars of the ALC tech infrastructure.

Google Suite

ALC Network uses Google Suite to manage emails and documents. Google Suite (or G Suite) is basically a self contained Google for organizations. It provides ALC with gmail style inboxes for emails at our own domain (e.g. when I log into drew@alc.network I am logging into a gmail style inbox managed by ALC, provided by Google)

Email and Groups

The G Suite is where we manage email accounts for members of our organization. Currently there is no process for adding or removing people from our organization. An @alc.network is a powerful sigil that allows those who have it to speak on behalf of the network.

The G Suite also manages organizational email groups, such as alfs@alc.network and info@alc.network, and many others. These email addresses provide the backbone of internal and external communication. For instance, many forms and correspondence from the public come through info@alc.network which is then delivered to a number of people within our network. A full list of email groups and some notes about their use can be found in this spreadsheet.

Google Drive

G Suite provides all users with a semi-shared Google Drive. In practice we use a single Google Drive Folder, named ALC Network, to house much of our shared documentation. The “ALC Network Drive” as it is commonly called has some level of organization, though it is mostly undocumented. There are a few important sub folders:

Event Folders

Contains ALF Summer, and other network “endorsed” event folders.

Member Resources

Resources for new member ALCs. Stuff to help ALCs get started

Projects

Many ALC projects live here

Startups

Lots of detailed organizational information around membership and startup process

There is a whole lot of information within this Google Drive.

Privacy and Security

While most of the information within the drive is harmless there are some items that we probably do not want to have exposed to the public. Currently the whole ALC Network Drive folder is set to Anyone with the link can view. I do not recommend changing this setting. Having the ability to quickly share documents with people from within our drive without having to manage sharing settings is worth the small risk that someone will get their hands on our files.

However, a security audit should be done. Individual files that are too sensitive for public viewing can be set to private one by one. It is much easier to hide the handful of sensitive documents than stifle collaboration by locking down the whole drive.

Editing rights

The ability to edit files in the drive has no clear membrane. The ideal way to manage editing privileges to the drive is to add people individually to documents or folders as need be. If a person should be granted full editing access then they should be added to the email group editors@alc.network. This method provides a clear avenue to contact everyone with editing rights and a single place to manage permissions. (editors@alc.network is given editor privileges to the ALC Folder and those privileges are granted to all members of the group)

Slack

Currently the main place for communication across the network is Slack -  a chat application that contains a number of channels which drive topics of conversation. Read more about how we use Slack in my blog post.

Slack also integrates with services to post messages. For instance, when a starterkit is downloaded, a message is sent to the Slack channel #z-feed-starterkit. When a new member joins, their information is sent to #wg-startups.

Currently there is no clear membrane for who can join Slack. Integrations are not well documented and the onboarding process, community norms, etc. are not documented.

The current channels are also not intuitive. A whole lot of work could go into organizing the system to provide more value to our members. Trainings on how to get the most out of slack would also be very useful as members are seldom using powerful features like pinning posts, threaded conversations, etc.

Trello

Team Hub: https://trello.com/alcfacilitators (you need to be logged in and invited to see it)

ALC makes heavy use of Trello Boards. At the network level most Trello boards which used to be used to manage network projects and processes seem to have fallen out of use.

Board Name

Notes

Last Activity

ALF Community Mastery

Used for the network ∆-Up meeting

Nov 28, 2016

StartupWG

Board for the Startup Working Group

Oct 5, 2016

ALC WORKS!

Outline of working groups and projects

Nov 3, 2016

Facil-Prac Call

Agenda for weekly call

Oct 24, 2016

While these boards still contain very interesting information they no longer seems to be of use. There doesn’t seem to be awareness or interest in these pieces of infrastructure.

Other Projects

The following are projects that have tech components and also relate to existing infrastructure. I thought it would be wise to include them.

Starterkit Gitbook

Link: https://www.gitbook.com/book/artbrock/alc-starter-kit/details (it’s private and won’t work unless Art invited you)

This is the version 2 of the ALC StarterKit hosted on GitBook.com. This platform uses git (as in GitHub) to maintain a collaborative version controlled “book”. It is a much better solution than using Google Documents to create, edit, and share the starter kit. Gitbook allows the project to be provided as an online book as well as in multiple formats (PDF, ebook, etc). It also gives us the ability to theme this and other publications.

This next version is nearly ready to be published

ALC Style GuideScreen Shot 2017-03-04 at 4.13.53 PM.png

The ALC Style guide is a half finished project that I have been working on to create a unified visual language for the ALC Brand. From color pallets to font use.

The project is currently in limbo but most of the work has been put into it. There doesn’t appear to be any real desire from anyone for it so I’ve lost interest. You can see a broken version of the style guide here: https://dhornbein.gitbooks.io/alc-style-guide/content/

Due to a technical glitch the actual styles are not showing up...


[1] w3techs.com