1 of 9

PII Redaction Experiment using the GC Feedback Tool

​

April 2025

1

2 of 9

Handling of personal information redaction

Current GC Feedback redaction

  • Text only instructions for users to not include personal information
  • Users are not warned before submitting a comment if they have included personal information
  • Common forms of PII are redacted after submission

​

Pilot GC Feedback redaction

  • Still includes text instructions for users to not include personal information
  • Users are alerted via pop-up if personal information is detected
  • Common forms of PII are redacted after submission

2

3 of 9

About the experiment

Findings:�Within those 2 weeks, 28 comments triggered the pop-up. ��It was 2x more common for people to submit as-is than edit their comment.

Page: Contact Us on Canada.ca

Data date range: March 10 - 24, 2025 (2 weeks)

Adobe Analytics showing how many times “Edit” or “”Submit” buttons were pressed within those 2 weeks.

PII redaction pop up window. “Submit” is the main action button.

3

4 of 9

Potential user reasons for continued PII submission

Many of the comments with PII related to users seeking help. It could indicate that users felt the need for immediate assistance and including PII was how they signified this user need.

��

Users could potentially prioritizing speed over correction when the primary user action is submitting the form. �

4

5 of 9

Evaluating Redacted Comments

Data Analysis Method:

  • Evaluated submitted comments that contained “#”

​

  • Tried to determine if the redaction was a:
    • Name
    • Phone number
    • Email address
    • Mailing address
    • Account/Application number
    • Unknown redaction

​

  • Most popular PII types submitted were name, phone and address.
    • Users often format comments like email correspondences, signing with names and including contact details.
    • Some users included phone numbers and treated it like a contact form submission.

​

​

​

​

​

​

Baseline

Experiment

name

​

10

8

phone

4

3

email

5

1

Mailing address

0

1

account

8

2

URL

0

2

unknown

10

16

spam

6

1

Comparison of baseline data and design experiment data

Link to Airtable Data

5

6 of 9

Data comparison before and after experiment

​

Baseline

Experiment

Date range: 2 weeks

February 23 - March 9, 2025

March 10 - 24, 2025

Pop-up warnings

N/A

28

Comments submitted with #

50

35 *

Comments submitted with redactions that contained PII

27

15

Total comments

416

401

% of comments with PII

6.5%

3.7%

* Additional words were being redacted by back-end validator

6

7 of 9

What we learned

​

  • Twice as many people submitted their comment with redactions than go back to edit following established user behavioral patterns �
  • Comments with PII on the Contact Us page on Canada.ca suggest users may be using the feedback tool to seek direct assistance rather than providing feedback. �
  • We are likely over redacting comments for words that are not personal information in the Feedback Viewer such as “CARM”
    • We should review more of the unredacted feedback to see what other false positives are being captured

7

8 of 9

Next steps

​

​

  1. Design iteration: Button order

​

Swap the order of buttons with “Edit” as the primary action to encourage more users edit their comment. Launch date: TBD

​

  • Consider a consistent approach with AI Answers tool

​

Do not allow comments to be submitted if PII is detected

8

9 of 9

Questions?

Contact us:

​

Digital Transformation Office

Canadian Digital Service�

CDS.DTO-BTN.SNC@servicecanada.gc.ca

9