Domain Events - cross-service
WE5.2.6 overview
Outline
How might this help us?
(the why)
MediaWiki data capabilities
Product Velocity
If event data is
Then, product features can
Data Reuse
Broadcast events improve data reusability across the whole of WMF (and beyond).
Decoupling
In-process events reduce coupling between MediaWiki components.
Cross-service events can reduce coupling between MediaWiki and other systems.
2. Receiving event data so that MediaWiki can react to, store and serve data from other services, without polling those services at run time.
MediaWiki Domain Events
(the what)
Domain Events
register
register
dispatch
dispatch
event
listener
listener
In-process MediaWiki PHP emitters and receivers are decoupled.
Code responsible for saving edits is not aware of code responsible for saving RecentChanges, talk page notifications, ResourceLoader module cache, or other downstream product features.
Initial goal: to make MediaWiki and extension development more sustainable
See also: Domain Events for Engineers deck
Domain Events
(the what)
Hypothesis WE5.2.6
If we explore designing an architecture for cross-service events, we can make MediaWiki data more reusable beyond the MediaWiki platform. *
FY2024-2025 Q3
This hypothesis aims to:
Hypothesis WE5.2.6
FY2024-2025 Q3
Outcome:
WE5.2.6 Broadcasting and receiving cross-service events discovery and design document
Detailed exploration and documentation of the technical questions around how to enable cross-service events in MediaWiki
Hypothesis WE5.2.6
FY2024-2025 Q3
cross-service scenario requirements
Can we use the same serialized and broadcast event for all cross-service scenarios?
Should we make DomainEvents WikiAwareEntities?
How might we use WMF’s Event Platform?
How might we account for PII and external usage of events that are broadcast externally?
What about eventual consistency?
How do DomainEvents relate to Dependency Tracking?
(the what, part 2)
Cross-service scenarios
cross-service – scenarios in which an event is received in a different application or service than the emitter.
Each scenario has distinct technical implications.
Cross-service scenarios
Cross-service scenarios
Goals
Learnings
Learnings
Which cross-service scenario support would most benefit WMF?
What capabilities are we lacking?
Learnings
Which cross-service scenarios do we already support, even if poorly?
Recommendations - short term
Improve usability of off-wiki generated data in MediaWiki by enabling MediaWiki to receive events.
Recommendations - short term
Recommendation 1: event bus MediaWiki receiving
Implementing event bus (kafka) MediaWiki receiving would allow non MediaWiki emitters to emit data in a way that is decoupled from the MediaWiki usage of that data.
The emitted event data would be reusable for many use cases, including those inside of MediaWiki.
E.g. PageRevertRiskScore events could be emitted by LiftWing.
Those events could be consumed into the Data Lake, by Enterprise, or by MediaWiki for building products features like moderator tools.
Event bus MediaWiki receiving next steps
Recommendations - short term
Recommendation 2: non MediaWiki job submission
Enable non MediaWiki processes to submit MediaWiki job events.
This solution would not improve data reusability, but is a fast path to enabling usage of generated data in MediaWiki
E.g. LiftWing, would have to emit multiple versions of PageRevertRiskScore event data, for Data Lake, Search and Enterprise, and also as a job into each wiki where the data should be pushed.
Non MediaWiki job submission next steps:
Recommendations - long term
Integrate DomainEvents with Event Platform to build MediaWiki event driven capabilities at WMF.
Ask for leadership
Please consider, recognize and acknowledge the problem:
Please prioritize platform capability initiatives that address this problem holistically, whether it is via cross-process Domain Events, or something else.
Until we address this problem, we will either:
Feedback please!
The discovery and design doc is long and detailed
“How to engage with this document” section provides instructions and shortcuts for reviewers
For higher level review:
Please help us provide examples of how making it easier to share data between MediaWiki and non MediaWiki systems would help you build your products.
Feedback please!
Appendix
Slides that didn’t make it into the main deck above, but might be useful to show.
Domain Events
function handlePageRevisionUpdatedEvent(
PageRevisionUpdatedEvent $event
) {
// (Invoked after the HTTP request is finished.)
// Do whatever you need to do with
// the fact the page that was updated
}
PHP interface example
Domain Events
class CampaignUserRegistrationEvent
implements DomainEvent {
// class that represents when a user registers
// for a Campaign event
}
PHP interface example
Domain Events
// In Campaigns extension handling user registration
$userRegistrationEvent = new CampaignUserRegistrationEvent( ... );
$eventDispatcher->dispatch( $userRegistrationEvent );
// Somewhere else… another extension?
function handleCampaignUserRegistrationEvent(
CampaignUserRegistrationEvent $event
) {
// Do whatever you need to do with
// the fact a user registered for a campaign
}
PHP interface example