Published using Google Docs
Proposal
Updated automatically every 5 minutes

RCR: A Better Way to Look at the ACT WellBeing Framework

Table of Contents

  1. Executive Summary

1. Problem Statement

2. Proposed Solution

  1. Parameters for Wellbeing Indicator Assessment
  2. Wellbeing Indicator Improvement Strategies
  1. Strategic Fit
  2. Planned outcomes
  3. Recommendations (with examples)
  4. Final Thoughts

Section A: Executive Summary

A.1 Problem Statement

The current ACT Wellbeing Framework, developed through extensive community consultation, aims to reflect the lived experiences and priorities of nearly 3,000 Canberrans, mostly captured through the Living Well in the ACT region survey. However, the data used to measure many of the wellbeing indicators often falls short in capturing the true complexity and diversity of wellbeing in real society. This disconnect arises because the data presented in the dashboard can be incomplete, fail to represent all communities, or fail to capture the nuanced, subjective, or contextual information surrounding the data and provide a misleading view on the current state of wellbeing in the targeted area. As a result, key wellbeing indicators may be misrepresented via the data currently presented on the dashboard and do not provide an accurate or comprehensive picture of the quality of life in Canberra, or provide actionable insights for policymakers, service providers, or the community to effectively address areas of need.

Our solution addresses these gaps by establishing a framework to analyse the relevance, context quality, and representativeness of the data presented on the dashboard and used to measure the wellbeing indicators. We additionally propose strategies to address issues found via our analysis. Among these strategies, we propose adjusting key Living Well in the ACT region survey questions, incorporating external and supplementary data sources, and re-representing existing data to ensure that the indicators align more closely with the actual experiences of diverse communities across the ACT. This will lead to a Wellbeing Framework that better reflects the true state of wellbeing in Canberra at a glance, explaining not only the what’s but the why’s, providing a more robust and informative foundation for policymakers to make targeted decisions to improve wellbeing in the ACT.

A.2 Problem Solution

A.2.1 Parameters for Wellbeing Indicator Assessment

  1. Relevance

Relevance refers to how accurately and meaningfully a data point reflects the wellbeing indicator it is intended to measure. It assesses the degree to which the data captures the key aspects of the indicator and provides insight that is aligned with the intended purpose, ensuring that the data is both applicable and useful for understanding and evaluating wellbeing. It makes sure that the data is not only relevant, but provides insights into actionable areas.

Here are some questions you might consider when evaluating a domain for this parameter:

What key aspects do you believe should be included in this domain?

Is the knowledge in this domain gathered from a survey or questionnaire? If so, does the questionnaire comprehensively address all critical aspects of the domain we aim to understand?

If not, do you think additional questions could be designed to gather valuable insights?

Are the questions in the questionnaire relevant and well-structured for this domain? Could they be reformulated to better capture the necessary information?

Are there objective data sources relevant to this domain based on the aspects that were determined earlier? If not, can you identify potential sources that could provide useful data and enhance our understanding of the domain?

  1. Context Quality

Context quality refers to the extent to which the data points collectively capture the full scope of the wellbeing indicator domain. It evaluates how well the data represents the various dimensions of the indicator by including multiple, complementary data sources. It talks about not only the what’s (can we see the full picture of what is going on?), but the why’s (what is happening that might make people feel this way?).

Here are some questions you might consider when evaluating a domain for this parameter:

What key aspects do you believe should be included in this domain?

Is the knowledge in this domain derived from surveys or questionnaires? If so, is there an objective dataset available that either complements or challenges the findings of the questionnaire?

How can this objective data be integrated to enhance the representation of the domain within the overall well-being framework? This may necessitate partial or complete revisions to the questionnaire to improve its accuracy and relevance.

  1. Representativeness

Representativeness refers to how useful the data is to the context of the factor in how it is represented on the dashboard. It assesses whether the data is appropriately chosen to provide a meaningful and accurate picture of the indicator and whether the voices of people from all backgrounds are included and considered.

Here are some questions you might consider when evaluating a domain for this parameter:

What key aspects do you believe should be included in this domain?

What subgroups or dimensions are covered within this domain or sub-domain?

Has the questionnaire or dataset displayed in the dashboard effectively addressed these relevant subgroups?

Does analyzing the data from different perspectives or slicing it in specific ways offer a more accurate representation of the domain indicator?

A.2.2 Wellbeing Indicator Improvement Strategies

  1. Is this the full picture?

Enhancing existing data points with external data to better represent the wellbeing level in that domain.

We propose rethinking some representative data points if they are irrelevant to the wellbeing domain, unactionable or uninformative, or misleading. We additionally propose integrating additional data sources, whether subjective (like surveys) or objective (such as administrative records), to capture a more accurate and fuller picture of people's lived experiences.

  1. Why is this happening?

Enhancing existing data points with external subjective or objective data to provide more context of the wellbeing domain.

We propose consulting additional data sources to provide more context and depth in the wellbeing indicators, recognizing that relying on a single source or type of data may not fully capture the complexity of a wellbeing domain. By incorporating both subjective and objective data, we can improve the relevance and quality of the insights provided, leading to better-informed decisions. We propose several external datasets that can be used to enhance context quality of certain indicators.

  1. What should we be showing?

Changing the key supporting data shown, or changing the way data is presented on the dashboard.

We propose revising the key supporting data currently displayed or adjusting how it is presented on the dashboard to ensure it better reflects the contexts of the wellbeing domains and gives relevant, actionable information. This concerns data representativeness, and might involve adjusting how data is grouped, and rethinking how data is compared and contrasted, such as by age bracket or by ethnicity.

  1. What should we be asking?

Improving the Living Well in the ACT Region survey.

We propose new and improved questions that either decrease bias, increase coverage or better measure the target wellbeing indicator to enhance all three assessment criteria.

Section B: Strategic Fit

The proposed framework aligns with the broader goals of improving public wellbeing and enhancing data-driven decision-making in government. By refining data representation and improving data collection for wellbeing indicators, the framework directly supports the ACT Government’s commitment to creating a more informed, responsive, and proactive approach to policy development. The improved dashboard will enable quicker identification of problem areas and emerging trends, providing policymakers with more accurate, relevant, and timely insights. This will facilitate more targeted interventions and resource allocation, ensuring that government actions are aligned with the real needs and priorities of the community, ultimately contributing to improved quality of life across the region.

In addition, the framework integrates with existing initiatives to modernise public service delivery by embracing innovative approaches to data management and visualisation, furthering the government’s digital transformation agenda. By improving both the accessibility and the precision of wellbeing data, the framework enhances transparency and accountability in government decision-making processes, reinforcing trust and engagement with the public.

Section C: Planned Outcomes

The planned outcomes of the framework include a more comprehensive and accurate representation of wellbeing indicators, leading to enhanced data quality and relevance. This will enable the wellbeing dashboard to more effectively highlight key problem areas and trends across various domains, such as health, safety, inclusion, and access to services. The improved data collection and representation will provide policymakers with actionable insights, allowing for more evidence-based, timely, and targeted decision-making.

Additionally, the framework will increase the overall effectiveness of government initiatives by ensuring that policy interventions are aligned with real community needs. This will contribute to better resource allocation, improved service delivery, and a higher quality of life for Canberrans. Finally, the framework will enhance public trust in government by promoting transparency and accountability through the clear and accessible presentation of wellbeing data, fostering greater community engagement and participation in policymaking processes.

Section D: Analysis and Recommendations

Social Connection

Factors: Levels of loneliness, Levels of volunteering, Participation in community events and activities, and Sense of social connection

(1)

Problem: Representativeness and Relevance: All indicators are based on the survey responses, which are subjective. In the digital age, many people are trending towards interaction on social media [reference]. Interactions online are also just as diverse and multifaceted as in-person and categorising them into a single ‘local online community group’ category does not sufficiently capture the diversity of online social interaction.

Solution: Update questionnaire: Add questions about social media interaction with family, friends such as Facebook, Instagram and Twitter. Add questions about online activities like multiplayer online games, virtual community participation and online volunteering. Please see a detailed questionnaire here.

Strategy: 4 - enhance survey questions.

(2)

Problem: Context Quality: Survey questions are subjective and often do not paint the full picture of a person’s social network, but there are a variety of tools we can use online to conduct online social network analysis. We also recognise that not all online interaction; especially social media is positive. Therefore, it is important to consider an individual's sentiment on and towards social media when assessing an individual’s degree of social connectedness online.

Solution: Use digital behavioural data: Digital Footprint or Open Source Intelligence to analyze usage time, frequency of interaction, conduct sentiment analysis, or explore the opportunity to collaborate with social media, or online gaming platforms such as Steam.

Strategy: 1, 2 - expand into online interaction and enhance with online user data.

References:

Changes in Modern social styles

https://oosga.com/social-media/aus/

Safety

(1)

Factor: Domestic and family violence – community attitudes on violence against women – community rejection of violence against women

Problem: Context Quality: This factor tracks ACT community’s attitude towards violence against women, which shows more and more people are rejecting domestic and sexual violence against women. However, this

(The data is from https://irp.cdn-website.com/f0688f0c/files/uploaded/NCAS%20State%20and%20Territory%20Report%202023.2.pdf

, which is collected by NCAS via mobile telephone with 19100 Australians aged 16 years or over)

This factor is too subjective which might create bias during the policy making process. People are likely to show they care about family violence during mobile interviews, but their actions may vary. Presenting this data as the sole metric to assess the section ‘Domestic and family violence’ is misleading. To prove this, a comparison can be done with ‘victims of family and domestic violence related offences’ data (collected by Australian Bureau of Statistics, https://www.abs.gov.au/statistics/people/crime-and-justice/recorded-crime-victims/2023#victims-of-family-and-domestic-violence-related-offences), which contains objective data describing number of offences and assault against women, as shown in the following figure:

The figure points out that although people in ACT are getting increasingly confident to reject violence against women, the number of offences remains the same with a slightly increasing trend. Therefore, the subjective statistical data contradicts with objective ground truth.

Solution: To solve this problem, more objective external/internal data should be utilised to calculate the consistency between subjective interview and objective ground truth. In this way, by providing more objective context, we can build people’s confidence on this factor and start to ask questions about why this discrepancy may be occurring.

Strategy: 2 - subjective data contradicts objective data, we require more context.

(2)

Factor: Feeling safe – perception of safety in the neighbourhood at night/during the day – feeling safe at night/during the day

This factor shows how people feel they are safe during day and night. It is collected through questionnaires for people above 18 years, provided by the Australian Government Productivity Commission.

Problem: Context Quality and Relevance: This data point is to represent the indicator ‘feeling safe’ and could be improved to be more relevant and informative. It can additionally be complimented by objective contextual data such as the victims of crime indicator that includes ground truth crime data. The problem of this factor is that the way the question is categorised, which is the sense of safety in the day and night, is inappropriate. It is common sense that night times are perceived to be more dangerous than day time, so it is mostly unhelpful to strategy making. An alternative way is to split data into where people might be during the day and night, such as at home, at school, and at work.

Strategy: 4 – Change survey question, 3 – Re-represent data.

(3)

Factor: Workplace Safety

Problem: Context Quality: The current measure of Workplace Safety is limited to the incidence rate of accepted serious workers’ compensation claims lodged each financial year. We believe this metric is too one-dimensional, as it fails to capture the broader aspects of safety, including employees' psychological and emotional well-being.

Solution: To address this, we recommend implementing a questionnaire or survey that explores various categories of workplace safety, encompassing not only physical safety but also the psychological and emotional experiences of employees.

Questionnaire 

Actionable: This questionnaire would not only provide insights into how safe people feel in the workplace both physically and otherwise which is an important aspect to be taken care of but also offer valuable information on potential risks before incidents occur. The data gathered can help the government identify and target areas that are more vulnerable, enabling proactive measures to improve workplace safety.

Strategy: 4 - Enhance survey questions.

Health

(1)

Factor: Mental Health – Canberrans who self-report very good or excellent mental health

This factor shows the percentage of people who believe they are in good or excellent mental health, provided by ACT General Health Survey. The data is collected through computer-assisted telephone interviewing to collect self-report data.

Factor: Mental Health – Canberrans whose K6 score indicates probable serious mental illness.

The K6 scale measures the probability of getting serious mental illness. The data collection is the same as above, using a computer-assisted telephone interviewing.

Problem: Context Quality: These two factors are too subjective. The survey asks people to judge whether they have mental issues, but people sometimes can not identify their own mental issues, or refuse to give the truth because of privacy or their mental health problems. An objective way of looking at this is compare with the suicide percentage each year (The data is provided by Australian Government Australian Institute of Health and Welfare,https://www.aihw.gov.au/suicide-self-harm-monitoring/data/deaths-by-suicide-in-australia/suicide-deaths-by-state-territories). An illustration is shown in the following diagram:


As the figure shows, with a lower proportion of people report they are in good mental health, the suicide rate increases, which means those two data are consistent. The introduction of suicide data has increased our confidence on people’s self-report data.

Strategy: 2 - Complement subjective data with objective context.

(2)

Factor: Healthy Lifestyle

Problem: Context Quality

ACT Health Questionnaire

https://www.act.gov.au/__data/assets/pdf_file/0018/2313027/ACTGHS-2023-Questionnaire.pdf

The questionnaires rely on self-reported data from citizens reflecting how they perceive themselves. At the current time, the dashboard indicates the percentage of Canberra residents who self-report having a healthy weight. While this subjective data provides valuable insights into personal perceptions and contributes to Canberra's overall well-being framework, it may not offer an objective assessment.

To strengthen the evaluation, it would be beneficial to incorporate more concrete data, such as information from GP records, to complement the self-reported figures.

This approach would yield more accurate data and provide a clearer picture of the actual on-ground reality.

https://www.aihw.gov.au/reports/primary-health-care/practice-incentives-program-measures-2021-22/contents/pipqi-measures/qim-3-height-and-weight-recorded-and-bmi

http://www.aihw.gov.au

Actionable: If the government identifies prevalent health issues like obesity, it should launch targeted public health campaigns, promote community-based wellness programs, and collaborate with healthcare providers to enhance preventive care. Additionally, policy measures such as improving access to physical activity should be implemented. Enhancing public education, supporting workplace wellness initiatives, and investing in research to monitor trends and adjust strategies will also be crucial in addressing the issue effectively.

Strategy: 2 - Complement subjective data with objective context.

Time

(1)

Factor: Work-life balance

Problem: Relevance, Context Quality, Representativeness: The BI dashboard is created from a survey that might not be an objective indicator. Ideally, the insights on the act gov website should use multiple datasets , perhaps focussing on the ones that have a higher sample size.

The survey collects data from residents of ACT, please refer to the table below:

Focusing on the time domain, all of the insights are based on the survey, raising the following questions:

Solution: Include the survey questions in the sources, along with the findings report.

Within the quality of time subdomain, the questions could be

This could look something like:

Strategy: 4 - Enhance the survey questions.

(2)

Factor: Time Spent Travelling Within Canberra

Problem: The current dashboard categorises "Time Spent Commuting" by demographic groups such as gender, age, and LGBTIQ+ status. While this provides some insights, these groupings are insufficient to generate actionable strategies for reducing commuting times. The analysis lacks the granularity needed to inform policy decisions effectively.

Solution: To enhance the Context Quality and Relevance of this metric, we propose re-structuring the data by commuting type (e.g., public transport, private vehicle, cycling, walking). This approach will yield more targeted insights into travel behaviours and infrastructure needs. Additionally, we recommend incorporating external data sources such as the ACT City Services Satisfaction Report and the Canberra Light Rail Customer Satisfaction Survey to provide a more comprehensive view of commuting patterns and pain points. This integration of diverse datasets will support data-driven strategies to optimise commuting efficiency and reduce travel time.

Example graph with suggested representation is as follow:

Current dashboard:

Strategy: 3 – Re-represent data, 2 - Complement survey data with objective measures.

Identity and Belonging

(1)

Factor: Valuing Aboriginal and Torres Strait Islander cultures and recognising our Traditional Custodians

Problem: Context Quality and Relevance: This measure only shows the agreement among Canberrans that:

These measures are problematic as respondents might feel compelled to answer positively due to societal expectations and may misleadingly be correlated with the actual wellbeing of Aboriginal and Torres Strait Islanders. Many people may express pride in Aboriginal and Torres Strait Islander cultures, not because they genuinely engage with or value them deeply, but because it is seen as the "right" or socially acceptable response. This means the survey may overestimate genuine respect and support for these cultures. Additionally, the survey suffers from a lack of actual Aboriginal and Torres Strait Islander voices and the questions regarding attendance of an event with a Welcome to Country and whether respondents can name one or more traditional custodians gives a symbolic and performative perspective on valuing Aboriginal culture.

Solution: We propose to include survey questions that ask how valued do Aboriginal and Torres Strait Islanders feel in the ACT to compliment this factor. The survey should additionally measure how knowledgeable and supportive people are in structural and systemic issues that affect Aboriginal and Torres Strait Islanders such as historical land rights, and engagement with real-world action such as participation in NAIDOC week, Reconciliation week, and Indigenous led community events. Please see a questionnaire here for some proposed example survey questions.

Strategy: 1 - Include voices for Aboriginal and Torres Strait Islanders, 4 - Enhance survey questions

Government and Institutions

(1)

Factor: Feeling that voice and perspectives matter

Problem: The measure that asks respondents how confident they feel about having a say or being heard in decision-making processes is very limited and does not show a complete picture of ‘how do residents feel that their voices are being heard.’

Solution: We propose including objective measurements that assess factors such as voter turnout, accessible via Election 2020 Statistics via the ACT government databases, cultural and ethnic representation in advisory bodies, and surveys such as the ANU Constitutional Referendum Survey, which additionally is relevant for the factor Valuing Aboriginal and Torres Strait Islander cultures and recognising our Traditional Custodians.

Strategy: 2 - Complement survey data with objective measures.

(2)

Factor: Access to Justice and restorative practice

Problem: Currently the measure displayed on the dashboard is the percentage of court matters finalised within 12 months of lodgement including civil and criminal cases. This measure primarily reflects case throughput, but it does not capture the broader context of who can access justice, who is excluded, or how restorative practices are being applied. A LAW Study found that 49% of Australians did not seek professional assistance for legal issues, with reasons among, “they did not know what to do,” “it would be too stressful,” and “it costs too much.” This massive proportion of Australians are excluded from this measure and are a vital and relevant measure to inform how ACT residents feel about their access to justice and restorative practice.

Solution: We propose including insights found through the LAW study or enhancing the survey to include questions to address these access issues.

Strategy: 1 - include the full picture of people who were not able to reach professional legal council, 2 - improve the survey questions.

Economy

1. Cross-domain Analysis: Linking Indicators from Other Domains with the Economy

(1) Equity of Education with Economic performance:

The current educational equity indicators focus on broad measures like performance of students in grades 3, 5, 7, and 9, but this data is too general to establish a concrete relationship between education equity and economic outcomes such as employment rates and income levels.

A more refined analysis should track the employment status of graduates from different education levels over time. This would reveal how educational attainment correlates with job opportunities and income growth

(2)  Other related indicators:

Placing economic indicators alongside related  indicators from other domains (such as Health with Economic performance, Cost of living with Economic performance, and Work-life Balance with Employment) in the same data visualisation provides a clearer and more intuitive view of their interconnections.

2. In-depth Analysis within the Economic Domain

  1. More Detailed Employment Data:

Employment data should differentiate between full-time and part-time work and further track underemployment to assess the size and distribution of workers who desire more hours. This could help to understand whether certain sectors or industries are facing labor surpluses or shortages and how this might influence future employment growth.

  1. Adding Income Indicators:

Current income data should be expanded to include more detailed income inequality metrics such as the Gini coefficient or income quintiles. Tracking the income distribution across different sectors and regions could provide insights into economic mobility and the long-term impacts of income inequality on economic growth.

3. Data Bias and Incompleteness: Income Equality Analysis

The focus on comparing only low-income (bottom 20%) and middle-income groups has several limitations:

  1. Narrow Focus:

Comparing only low and middle-income groups ignores the broader income distribution, particularly high-income earners. This creates a bias, as it doesn't reflect changes in the income of the wealthiest group, who can significantly impact overall income inequality and economic power dynamics.

  1. Need for a Full Income Distribution Analysis:

A comprehensive income analysis should include low, middle, and high-income groups. Examining how income changes across all these groups will provide a clearer picture of income inequality and its effects on the economy.

The economy is the most objective domain among the 12 domains, and it is essential to ensure that the economic data is objective, detailed, and accurate. At the same time, multi-level correlation analysis should be conducted with other domains (especially subjective ones) to support subjective data and improve readability for the audience, enabling a more accurate assessment of well-being.

General notes on Living Well in the ACT Region Survey

We additionally note that the Living Well in the ACT Region survey only provides a binary (male or female) option for gender which presents significant limitations in capturing the diversity of gender identities within the community. This binary approach not only fails to reflect the experiences of people who identify outside of the male-female spectrum, such as non-binary, genderqueer, transgender, and gender-fluid individuals, but also risks alienating and misrepresenting a segment of the population whose wellbeing is just as important to measure.

We propose the expand the gender selection options, which could include a solution with open ended responses, such as:

We also note that the CALD (culturally and linguistically diverse) category having only the options of [Born in Australia, Born overseas - English speaking country, Born overseas - non-English speaking country] may suffer from issues such as over-importance on place of birth, making assumptions about language proficiency based on the place of birth and conflating language and culture. We propose instead to reform the CALD category as based on options such as: a. Cultural background, or b. English speaking proficiency, or others as needed for the context.

Section E: Final Thoughts

Our overall proposal is to provide a helpful framework to help the ACT take a critical lens into the ACT Wellbeing Framework dashboard to identify information gaps, misleading data, and irrelevant data to improve the helpfulness and comprehensiveness of the dashboard in order to build towards a platform that policymakers have confidence in and can inform their decision making. There is still a lot more to be completed, and we recognise the limitations of our analysis and of the current data that we have accessible to, but our hope is that by exposing the current limitations of the dashboard, we can provide a more inclusive and faithful picture of wellbeing in the ACT to better inform progress in wellbeing across all sectors and background of people and lift their quality of life to make sure no-one is left behind.