Look DATA
Methodology
Phases
(can move in either direction)
Ignition
Liftoff
Orbital flight
Gather
Dispatch
Explore
Energy dip after initial forming
KBW is in the beginning stages of liftoff. We’re revving our engines!
Space station
Systems Change
True?
Why: To set Metric Milestones
It’s awesome that we feel like KBW is a healthy community, but the ambitions of KBW extend beyond being a friendly and inclusive place. Indeed, we’re trying to change the world.
As system conveners and earth-shakers, we need to set ourselves some baselines so that we can set ourselves some metric milestones. With a data-driven approach, we can more accurately observe how KBW is having far-reaching impact.
What: Collecting & Analysing Data
After a year of officially launching the Keep Badges Weird community, we are having a look at the DATA.
In Q4 2022, we are starting data collection of both quantitative and qualitative data.
We will analyse this data and have a look at how the community is progressing through growth, activity, and sense of community. Are we maturing? Let’s find out!
Phases
(can move in either direction)
Ignition
Liftoff
Orbital flight
Gather
Dispatch
Explore
Energy dip after initial forming
KBW is in the beginning stages of liftoff. We’re revving our engines!
Space station
Systems Change
Individual
Engagement
Birthday
Survey
Community
Metrics
We looked at the engagement data from every 5th community member
We created a survey to check in on how people are feeling about KBW
We focused on what can be measured through the platform’s database output
Three different data collections
Individual Engagement
How: Individual Engagement
To select a fair and impartial group of our 317 unique community members to set our engagement baselines, we used systematic sampling. We:
Screenshot of anonymised data
This data was gathered on October 25, 2022 a year after launching the community
46% of our sample were actively engaged. They introduced themselves, completed a course or joined a community call.
Active engagement
The community metric of the KBW and Value Creator badges show that 44% of the whole community earned at least one of these badges. The KBW badge is earned by writing an introduction, the Value Creator is awarded in courses or to people who came to community calls.
The cross-comparison of these individual and community metrics verifies that our sampling method and analysis is solid. Sometimes we surprise ourselves ;)
Of the people who actively engaged, 25.4% completed one task.
Multiple forms of engagement
20.6% engaged in more than one way.
Multiple forms of engagement
Of the people who are actively engaged, 44.1% engage in more than one way.
29% of our sample introduced themselves to the community.
Introductions
The community metric of the KBW badge shows, that 30% of the whole community earned it. The badge is earned by writing an introduction.
Completed a course
20.6% both signed up for and completed a course.
24% of our engaged sample came to a community call!
Birthday
Survey
How: Birthday Survey
Inspired by the book Buzzing Communities by Richard Millington, we created a survey to check in on how people are feeling about KBW. The questionnaire uses statements developed in
McMillan and Chavis’s 1986 paper about the psychological sense of community.
Birthday Survey - Membership
I can trust people in this community.
Most participants answered positively that they have high levels of trust in the KBW community.
Being a member of this community is a part of my identity.
Responses were equally distributed for this question, which suggests that, for some participants, we have some work to do here.
1
2
3
4
Birthday Survey - Membership - Quotes
“I would love to be able to do much more for this community, but I'm also involved in the french speaking community.”
“Perhaps l would say many of the values held in this community are part of my identity.”
“I am new to badging and new to the community. I am still trying to see if it is a fit for my needs and haven't been able to commit much time as of yet.”
“It's an interesting mix between a community (sense of belonging, interdependence) and more of a flexible network. That can be a winning combination. Less of a Wenger & Lave CoP and something closer to a movement.”
“I just re-connected with this community but it feels like returning home.”
“I am getting a first hand experience on being a member of a CoP”
Birthday Survey - Influence
This community can influence other communities.
Almost all participants agreed strongly that the KBW community can influence other communities.
I have influence over what this community is like.
Interestingly, there was less confidence from participants that they have influence over what the community is like.
1
2
3
4
Birthday Survey - Influence - Quotes
“Hard to tell about problem-solving, at this point (at least, for me as a recent member). And unclear to me if influence is important or useful. More about "impact".”
“The core members make me feel welcomed”
Birthday Survey - Fulfilment
1
2
3
4
Community members and I value the same things.
There was strong agreement that the KBW community is one that has a common purpose.
People in this community have similar needs, priorities, and goals.
Although there is a common purpose, participants were slightly less sure that everyone shares the same goals and has the same needs as them.
Birthday Survey - Fulfilment - Quotes
“I think the 'weird' concept doesn't travel well across cultures- I think it works well for members of the group but not as something to refer to externally (at least for most of the audiences I work with). I mean specifically the use of the word weird and some of the associated imagery etc, not what the group represents in terms of using badges to represent a wide range of skills and experience, and convey your personal learning and development journey.”
“Needs and goals not clear”
“Wondering if alignment in priorities could have unwanted consequences. Talking with group members about a question is easy. Haven't tried for actual problems.”
1
2
3
4
Birthday Survey - Connection
It is very important to me to be a part of this community.
Participants skewed towards the positive when asked about the importance of the community to them.
I feel hopeful about the future of this community.
Happily, most people felt quite strongly that they had hope for the future of the community.
Birthday Survey - Connection - Quotes
“I'm a lurker here. I'm interested in the concept and the work being done, and aim to make use of that, but don't have the time available to get more closely involved. Keep up the good work.”
“Thank you to everyone for your work and commitment. You will see more of me next year!”
“It feels like it might morph into something else entirely. And that might be appropriate. Part of the setup is that there are different entities coming together and it's not entirely clear at the onset (for newbies).
KBW, WAO, Participate, different orgs working in badges, events... As for my continued membership... Much of it has to do with the fact that engaging in it is satisfying yet it's not clear that engagement in badging is important for my dayjob. In this sense, it feels a tiny bit like a club or extracurricular. So, I might have to limit my engagement... and encourage more people from my professional learning network to join.”
“I'm not active in the community yet (appears to be mainly focused on educator). That's mainly due to time constraints I have for this year. Nevertheless, I look into it every now and then.”
Community
Metrics
How: Community Metrics
Our participation metrics focus on what can be measured through the platform’s database output. We can accurately see the amount of activity and badges
Oct 2021
Nov
Dec
Jan 2022
Feb
March
April
Mai
June
July
Aug
Sep
Oct 2022
1
2
3
4
5
6
Total number of new discussions per month
Oct 2021
Nov
Dec
Jan 2022
Feb
March
April
Mai
June
July
Aug
Sep
Oct 2022
20
Total number of posts per month
30
40
50
60
70
80
51
38
28
59
82
57
33
38
58
27
46
21
21
603 total posts over the year, including the 44 discussion starts
Oct 2021
Nov
Dec
Jan 2022
Feb
March
April
Mai
June
July
Aug
Sep
Oct 2022
Number of posts per month and new discussions
51
38
28
59
82
57
33
38
58
27
46
21
21
Number of posts
Number of new discussions
Community Metrics: Courses
A total of 59 people started taking one of the 3 courses.
Community Metrics: Badges
KBW issued a total of 315 badges to 129 community members. 41% of the community has earned at least one badge.
We had 19 different badges. For the purposes of this data thing, badges issued fall into three categories:
Community Badges
94
29
17
13
45
0
0
Half as many people that introduced themselves earned the value creator badge.
Stealth Badges
38
6
11
5
10
15
1
So many O.G. badgers are part of the community, so the cultural memory of the community goes back to the beginning of badges.
Badge Wiki Badges
18
2
5
4
1
We’re pleased that since we looked at the data, we’ve issued several more of the 5x badge!
Though KBW has 19 different badges on offer, two have never been issued:
The CoP Coach badge is a stealth badge that has not been issued. The reason for this is, that we’re not deputising new moderators until the launch of the new platform.
The Reflector badge has never been issued. It is linked to an activity that so far nobody submitted. The same is the issue with the Value Creator badge but we issued as we saw fit in people’s activities in the platform and community calls.
Next Steps
Figure out where we’re going