1 of 16

Arch BEES

Arch: mischievous or playful��When someone acts archly,�it might taunt you or�it might make you laugh.

Bees: honeybees��Symbol of hard work,�dedication and�sustainability.

1

2 of 16

Scaling Sustainably

Time to Market

Community and Codebase Size

2

3 of 16

Boundaries

Extensions

Events

Standards

3

4 of 16

Boundaries

4

First step.. (as with many things in life), we need to set boundaries.

Depicted with a box.

Each box has its own set of clear responsibilities.

With boundaries, we know where things are located and where to make changes.

Today, we don’t have this clarity in our codebase.

Who can attest to going into the edx-platform app named “student”. Only to be surprise to find code related to the learner dashboard and user access control?

Similarly, our Support team may have run into the in-cohesive administration of course certificates spread across our multiple tools.

And our Data team has run into issues with conflicting data models across our system.

So one thing we can do is clarify the purpose of each boundary. So you, your future self, other developes know what to include and what NOT to include in each box. Teams can do that for the things you own in each components README file.

Bzzzzz….

What we have begun to do in our platform, is separate our frontend from our backend.

Frontend code runs in the users browser.

Backend code runs on our servers, hosted in the cloud.

Bzzzzz…..

We can’t forget our 2 million LOC beloved pet - our monolith.

Our monolith - doesn’t really look like this - nice and simple large box. It’s very spaghetti-ish inside.

So we cannot easily create physical boundaries inside the monolith.

But we can draw logical boundaries inside.

We can create clear separation of concerns. And document them!

5 of 16

Frontend Plugin Framework

with Javascript APIs & Hooks

Backend Plugin Framework

with Python APIs & Hooks

Extensions

5

Now that we’ve set clear boundaries of what each component does..

We can look to extend them.

Yes, we can extend them, without actually modifying them.

Bzzzz..

For example, I have extended the audio capabilities of my laptop by connecting my own set of headphones to the core laptop experience.

In order to make something like that work, you need a common interface or an API to plug in extensions.

For headphones, one would expect a USB or a headphone jack API.

Bzzzz..

For our platform, we will need plugin interfaces for both the backend and the frontend.

So, if you want to add a new widget to the learner dashboard - present past notes so learners can study for their exams. You could do that with such plugin mechanisms.

Together, with the community, we can build these extension frameworks.

6 of 16

A. Backend� Apps

B. Micro Services

C. Micro Frontends

D. Frontend� Apps

Events

6

Thirdly, an event driven architecture is a an excellent design pattern to inform extensions or other services when something happens - in a very de-coupled way.

The notifications can be in real-time. So whoever’s listening, can know almost immediately that an event occurred.

For example, once a learner completed their exam, your widget on their learner dashboard can stop pestering them to study. And this can happen without the core platform knowing anything about your widget. Your widget simply needs to listen for events that it’s interested in.

Bzzz..

Even other extended services in the system will want to react accordingly, when something happens.

For example, when a learner completes a course in the LMS, our separate credentials service will want to make sure a certificate or badge is properly generated.

Bzzz..

Long-term as we develop this further...

what: decoupled real-time notifications to listeners, including extensions and services

why: maintain/reduce coupling as our platform grows

CTA: look into attending biweekly book-club on building event-driven microservices at Arch-Hour to upskill on this up-and-coming design pattern in our platform

7 of 16

Events

7

8 of 16

Standards

8

9 of 16

Standards

9

10 of 16

Standards

10

11 of 16

Write purpose of components (repos/apps) in their READMEs.

Review OEP-50 and its ADRs; will impact edx-platform dev.

Read and upskill in upcoming event-driven arch book club.

Boundaries

Extensions

Events

Standards

Review LTI Freedom OEP coming with Experience Plugins.

11

12 of 16

Intentionality

12

Finally, we want to ensure we continue an intentional outlook on our platform architecture

So that we do not regress

And that we continue to support the thousand and growing number of engineers.

Here’s a snippet from our Architecture Manifesto. The manifesto reminds us of our basic architecture principles, including the 3 we mentioned just now.

In addition, here are a few other pointers from the manifesto that remind us how to remain deliberate with our efforts.

For example

Work to simplify beats slippery complexity.

Let’s continue to refactor - as needed.

Code does grow in complexity unless we simplify and refactor as we go.

Code does get stale. Code does need to be removed when it’s no longer needed.

A final Taunting from me

Decision Records

Are a means to stop and do timeboxed design and write down tradeoffs

before we jump into coding.

We have Arch Decision Records and Open edX Proposals as tools already in our platform for doing so.

So continue to do those along the way, and we’ll ensure that our platform continues to grow in a scalable and intentional way that we need it to be.

13 of 16

Getting to Extensible Core

Need to build together differently

We cannot scale contributions,� We cannot grow our developer capacity,� Unless we build together an extensible core.

Here’s how we get there:

Boundaries
Extensions
Events
Standards

For more information on Open edX’s B.E.E.S. strategy:

watch video clip 6:22->13:45

13

14 of 16

Monolith�slows us down

	Monolith (Ball of Mud)	Extension of Monolith	Separate Small Service
Lines of code to understand when developing � (cognitive load)	2.5 Million	5-100 Thousand	100 Thousand
Lines of code-mates (in-process) when running on edx.org  (risk of error)	2.5 Million	2.5 Million	100 Thousand
Time it takes to see a change get to edx.org after development (lead time)	2-4 hours	2-4 hours	10-30 mns
Number of other developers to depend on for success �(reduced autonomy)	100+�(many engineers)	100+�(many engineers)	6-8�(owning team)
Examples of features in each situation	Learner Dashboard Import-Export Video XBlock	ORA Coaching Personalized Dates	Registrar Publisher Enterprise Admin

The monolith contains 2.5 million lines of code, organically accumulated over the years.

The lack of separation of concerns in the monolith make it hard to make changes on features within it, requiring high cognitive load.

The first step in taming�the monolith (or migrating �its code to smaller �services) is establishing �Boundaries.

14

15 of 16

Getting to Extensible Core�Valuable code is stuck in the monolith (Ball of Mud).

More valuable code continues�to be added to the monolith,�now reaching 10,000+ files.

We cannot innovate as fast as we did in 2012 on the features and functionalities that are still stuck in the monolith.

15

16 of 16

Getting to Extensible Core

Ball of Mud -> Boundaries of Modularity

Boundaries��High interconnectedness reduces developer autonomy and requires unscalable communication structures. Lack of aligned business taxonomy makes naming hard. ��Action: Establish business-domain names and document our codebase by business concerns. Later, separate the codebase, aligned with organizational boundaries.��Next Step: Document purpose of each component.

Extensions��Today, new features are still built in the monolith since it is easier to do so. Adding new code to the existing tangledness exacerbates problems.��Action: Invest upfront in extensibility frameworks and APIs. Collaborate on developing, advancing, and using these frameworks, with senior engineers and product.��Next Step: Develop with APIs & extensions.

Events��The lack of real-time notifications of user and platform events forces developers to create tightly coupled & fragile connections in our system.��Action: Invest in designing and implementing event-based frameworks. Take the time to learn industry best practices to prevent false starts and long-term issues.��Next Step: Education on event-driven architecture.

Standards��Proprietarily built solutions and APIs prevent us from leveraging industry commodities and technologies in the EdTech space.��Action: Expand usage of existing/new standards integrations in our platform. Leverage our 1-year-old membership in IMS Global to identify and prioritize our certifications.��Next Step: Adopt xAPI/Caliper & ‘LTI Freedom’.

Ask: Deliberateness in investing in BEES when squads implement new features and update old ones we care about.

16