Talk pages project/Usability

A/B Test Findings Published

The findings from the A/B test designed to evaluate the suite of desktop talk page design changes are available for you to review.

A/B Test

On 8 May, the A/B test we mentioned in April started at the following Wikipedias:

• Azerbaijani (azwiki), Bengali (bnwiki), Dutch (nlwiki), German (dewiki), Hebrew (hewiki), Hindi (hiwiki), Indonesian (idwiki), Korean (kowiki), Persian (fawiki), Portuguese (ptwiki), Romanian (rowiki), Thai (thwiki), Turkish (trwiki), Ukrainian (ukwiki), and Vietnamese (viwiki).

On 1 June, we confirmed sufficient data had been gathered to begin an analysis to assess the impact of the suite of changes the Usability portion of the broader Talk Pages Project. In July, we expect results to be ready for the Editing Team to share and for us to collectively make sense of.

The Editing Team is planning an A/B test for Usability features on the desktop site. See phab:T302358 and Talk pages project/Usability/Analysis.

 

Today, the suite of Usability Improvements became available to everyone (logged in and out) by default at the Czech and Hungarian Wikipedias who are accessing talk pages with the Vector (2022) skin enabled.

All Usability features are available to all editors using the mobile site.

Seeking Design Feedback

 

There is a prototype ready that we would value feedback on: https://patchdemo.wmflabs.org/wikis/e22ef06cf7/wiki/Talk:DiscussionTools

The prototype introduces two changes:

1. The "Add topic" button appears in a more prominent location on the page (above the existing page toolbar that contains actions links like "Page," "Discussion," "Read," "Edit," "Edit source," etc.)
2. The "Add topic" button is styled differently

The two changes described above are an effort to make it easier for people to quickly locate and easily "reach for" the button for starting a new discussion topic.

→ Please visit the talk page to share what questions/ideas/concerns this change brings up for you.

Page Frame Features Available at Most Wikis

Next week, people accessing all Wikimedia wikis (except en.wiki) on desktop who have the Show discussion activity setting enabled within Special:Preferences, will see new information on talk pages about the level of activity happening within them.

See the 11 January update for details about the specific changes you can expect.

 

Page Frame Features Available at Partner Wikis

On Tuesday, 17 January, people accessing the Arabic, Czech, and Hungarian Wikipedias on desktop who have the Show discussion activity setting enabled within Special:Preferences, will see the following new features on talk pages:

1. A visual space between the "namespace" and "page" name portion of a talk page's title
2. An indicator that describes, and links to, the latest comment published on a given talk page
3. Information within the table of contents about the number of comments within each section

Usability Improvements in All Talk Namespaces

Next week, people at all Wikipedias who have the Show discussion activity setting enabled within Special:Preferences, will see Usability Improvements when they visit any page within a talk/odd-numbered namespace (e.g. Wikipedia talk, File talk, MediaWiki talk, etc.).

Update to Reply button

	New appearance, used on mobile for all languages. Also used on desktop if the word "Reply" is one or two characters (Arabic, Chinese, Japanese, Northern Thai, Cantonese).
	New appearance, used on desktop for most languages. Visible to people who have the Show discussion activity setting enabled in Special Preferences.
	Oldest appearance. Visible to people who have the Show discussion activity setting disabled in Special Preferences.

The arrow is being added to improve Accessibility .

The default on the desktop site does not include an arrow. The configuration is set globally per language, not per wiki. Follow the process at Requesting wiki configuration changes if you believe your language should include the arrow.

 

 

Clear Affordances (desktop) Deployment

On 27 September, a new appearance for the Reply buttons that appear on desktop talk pages became available to people at the Czech and Hungarian Wikipedias who have the DiscussionTools beta feature enabled.

Tomorrow (18 October), this new Reply button styling will become available at most other Wikipedias. See more in T320683.

Topic Container (desktop) Deployment

 

Today, Topic Containers became available on desktop at the English and German Wikipedias to everyone who has the DiscussionTools beta feature enabled. More details about this deployment can be found in T315625.

In the coming weeks, we plan to:

1. Offer Topic Containers on desktop to volunteers at the Japanese Wikipedia who have the DiscussionTools beta feature enabled. See T318127.
2. Offer a new appearance for the [ reply ] links that appear on desktop talk pages at the Arabic, Czech, and Hungarian Wikipedias. See T315626.

Topic Container (desktop) Deployment

On Tuesday, 30 August, Topic Containers became available on desktop as a Beta Feature at all Wikimedia wikis except for English, German, and Japanese Wikipedias.

Reason being, we are running a central notice campaign at these three wikis to gather feedback about suite of usability improvements. Once these campaigns have finished running and we have understood and responded to the feedback, we will offer desktop Topic Containers as a beta feature at these three Wikipedias.

Design of a new "Add topic" button are ready for you all to review.

 

This change is designed to make it easier for people to notice and access the "Add Topic" button on desktop talk pages where people do not have access to the new site-wide sticky header and the "Add Topic" button that now appears within it.

Please visit the talk page to share feedback about the designs.

The design you see here are inspired by ideas projects like cs.wiki, en.wiki, fi.wiki, and fr.wiki have implemented to make it easier for people to identify and access the "Add topic" / "New section" link, especially on longer pages. See more in T267444.

Making Topic Containers (desktop) available on more wikis

Yesterday (17 August), Topic Containers became available as a Beta Feature on desktop at all Wikimedia wikis except de.wiki, en.wiki, and ja.wiki.

In the coming weeks, following additional prototype testing, we will offer Topic Containers on desktop at the remaining three wikis mentioned above.

Second deployment

The initial desktop Topic Container deployment seems to have gone well. Accordingly, we are planning to make Topic Containers on desktop available as a Beta Feature at all Wikimedia wikis except de.wiki, en.wiki, and ja.wiki before August is over.

If/when this timing changes we will share an update here.

First deployment

On Wednesday, 20 July, Topic Containers (the new talk page section heading design) became available as a Beta Feature at the first three Wikipedias: Arabic (ar.wiki), Czech (cs.wiki), and Hungarian (hu.wiki).

Over the next few weeks, we will offer Topic Containers as a beta feature at all Wikimedia wikis. This timing assumes we do not discover any significant issues.

In the meantime, if you would like to see what a live talk page looks like with Topic Containers enabled, try visiting hu:User talk:Tacsipacsi (make sure the "Discussion tools" feature is enabled in Beta Features).

Design updates

 

 

 

Over the past couple of months, people have shared feedback about the initial designs we published in May.^[1][2][3] This feedback has been helpful in leading us to see the need for making a series of adjustments to the design. You can the latest designs in the screenshots that accompany this update.

Some of the changes we have made in response to the feedback you all have shared include:

• Removing the arrow that previously appeared before each "Reply" link
• Adding the ability for people to click the "Latest comment" indicator and be taken to the most recent comment in the discussion
• Removing the icons that previously accompanied the metadata that is shown beneath each talk page section heading

First deployment

Before July is over, we plan to offer the new talk page section heading design as a Beta Feature at an initial set of wikis.

 

A prototype is ready for you all to try the talk page visual changes.

These changes are designed to make it easier for people to understand and use talk pages on desktop and mobile.

Please visit the talk page to find the information you will need to:

• Try the prototype
• Share feedback about the prototype

 

Mockups are ready for the changes designed to make it easier for people to understand and use talk pages on desktop and mobile.

In the next few weeks, a prototype will be ready for you to try out these changes on a test wiki.

In the meantime, a question for you all: What ideas, questions, or concerns do these mockups bring to your mind?

Knowing this will help the team decide whether adjustments ought to be made to the designs prior to inviting you all, and other volunteers from across the Movement, to share feedback about them.

You can review these designs in more detail and share feedback about them on the talk page.

The Topic Containers prototype we mentioned on 11 March is ready for testing.

To try the desktop prototype, please visit Prototype/Topic Containers where you will find a link to the prototype as well as instructions for trying and sharing feedback about the prototype.

The team is continuing to iterate on the Topic Containers designs for both mobile and desktop talk pages.

 

In the next 1-2 weeks, we will have an initial prototype ready for you all to experiment with and share feedback about. We will also be starting a series of CentralNotice campaigns for volunteers to review and share feedback about the Topic Container prototype which will include:

• New styling for == H2 == section headings
• New metadata about talk page discussions. This will include: the number of people who have commented in a discussion, the last time a new comment was published in the discussion, and the total number of people who have participated in the discussion to-date.

 

Seeking Feedback: Initial "Topic Container" Design Concepts

An initial set of design concepts for Topic Containers is ready for you all to review (see screenshot).

These Topic Containers are an effort to evolve the way talk page section headings ( == H2 ==) appear, and introduce new information within them, to help:

1. Junior Contributors easily recognize talk pages as containing discussions
2. Senior Contributors assess the level of activity on talk pages

📣 We would value you reviewing the design concepts and sharing what you think of them on the talk page, so the team can know what changes to explore in the next set of designs we will share in January (2022).

Design work beginning

Design work on Topic Containers – the first intervention we are working on as part of this phase of the Talk Pages Project – is underway.

In the coming weeks, you can expect to see some preliminary sketches posted here for you all to review and for us to talk about together.

In the meantime, if any questions are on your mind about this part of the project, please post them to the talk page.

We need your help

When looking at a talk page, what do you think would make it easier for you to know what conversations are worth reading? Please share what you think on the talk page.

The thoughts you all share will inform the design of the "topic containers" we will start designing soon. These "topic containers" are intended to help people more easily and quickly recognize talk pages as containing conversations and get a sense for the level of activity happening with them.

Strategy

In June, we published the design strategy and set of interventions we are planning to work on as part of the larger effort to help people, across experience levels, identify and understand the conversations happening on talk pages. You can read more about this strategy.

The team has compiled the research that is informing the approach we will take to help Junior Contributors quickly recognize talk pages as spaces to communicate with other volunteers and locate the tools available to do so and help Senior Contributors can quickly evaluate the level of activity happening on talk pages. More information about this can be found in the Objectives and Background sections below.

In the coming weeks, you can expect this page to be updated with the particular interventions we are planning to work on as part of this phase of the project. Before then, you can review how we are planning and organizing this work in Phabricator at T249579.

The team is planning to begin work on the incremental enhancements needed help people instinctively recognize and use talk pages as spaces to communicate with people on-wiki will begin in earnest in ~3 months.

In the meantime, we will be expanding this this page to include the research that is motivating this work and the impact it is intended to have.

This work to make wikitext talk pages more legible is intended to cause:

1. Junior Contributors to quickly recognize talk pages as spaces to communicate with other volunteers and locate the tools available to do so.^[4][5][6]
2. Senior Contributors to be able to quickly assess what conversations on a given talk page are worth focusing on.

To learn whether the suite of desktop talk page design changes were effective at causing the impacts they were designed to cause, we conducted an A/B test.

At the 15 Wikipedias that participated in the test, 50% of people who were logged in were shown the new talk page design and 50% were shown the existing experience on desktop talk pages.

We then compared how these two groups of people engaged with talk pages in service of making the following decision: Are the set of Usability Improvements on desktop fit to be made available to everyone, at all Wikimedia wikis, by default?

What follows is a summary of what we learned through this A/B test and ultimately, the conclusion we are drawing from it.

You can read the full test report and learn more about the thinking behind this A/B test.

To what extent have the suite of desktop talk page design changes been effective at causing the following?

1. Junior Contributors quickly recognize talk pages as places to communicate with other volunteers and locate the tools available to do so.
2. Senior Contributors are able to quickly assess which conversations on a given talk page are worth focusing on.

This set of desktop talk page design changes resulting in increased engagement and decreased disruption is leading the Editing Team to be confident that offering these Usability Improvements as default-on features would have a net positive impact on wikis and the people who contribute to them.

 

Revert rate.

Defined at the proportion of published talk page edits, of all types, that are reverted within 48 hours.

• Overall: we observed an 11% decrease (-0.4 percentage points) in the revert rate of talk page edits by people shown the suite of desktop talk page design changes across all participating Wikipedias and all editor experience levels.
• Experience level: when we analyzed revert rate by editor experience level, we observed...
  • A 12.5% decrease in the revert rate of talk page edits made by Junior Contributors (<100 cumulative edits)
  • A 7.4% decrease in the revert rate of talk edits made by Senior Contributors (>100 cumulative edits).
• Desktop skin: when we analyzed revert rate by desktop skin type, we observed...
  • A 6.3% increase (3.2% → 3.4%) in the revert rate of edits made by people using the Vector skin
  • A 30.8% decrease (3.9% → 2.7%) in the revert rate of edits made by people using the Vector 2022 skin

Edit completion rate

Defined as the proportion of talk page edits that are started and successfully published (not reverted within 48 hours).

• Overall: People shown the suite of desktop talk page design changes were slightly more likely to complete an edit that they started. We observed a 3.3% increase (+2 percentage points) in edit completion rate across all participating Wikipedias and all editor experience levels.
• Experience level: when we analyzed edit completion rate by experience level, we observed...
  • A 1.6% increase (43.8%→44.5%) in edit completion rate among Junior Contributors (<100 cumulative edits)
  • A 3.6% increase (66.8%→69.2%) in edit completion rate among Senior Contributors (>100 cumulative edits)
• Desktop skin: when we analyzed revert rate by desktop skin type, we observed...
  • A 2.1% increase (62.4% → 63.7%) in the revert rate of edits made by people using the Vector skin
  • A 5.3% decrease (59.9% → 63.1%) in the revert rate of edits made by people using the Vector 2022 skin

 

Block rate

Defined as the proportion of people who publish talk page edits and are subsequently blocked.

• We did not observe any significant changes in the number of users blocked after publishing a desktop talk page edit.
• Overall: 0.5% of users shown the suite of desktop talk page design changes were blocked compared to 0.6% of users who were shown the existing experience across all participating Wikipedias and all editor experience levels.

Talk page engagement

Proportion of Junior Contributors that visited a desktop talk page and made an attempt to edit.

• Of all the visits to desktop talk pages that Junior Contributors made throughout the A/B test, Junior Contributors shown the suite of desktop talk page design changes interacted with the page in 16.7% more talk page views (+1.4 percentage points) than those shown the existing experience.
• We observed an 19% increase (+0.7 percentage point) in the proportion of talk page views by Junior Contributors that included a saved edit.

Timing

This analysis was completed on 27 June 2023 and analyzed how people engaged with desktop talk pages at az.wiki, bn.wiki, de.wiki, fa.wiki, hi.wiki, he.wiki, id.wiki, ko.wiki, nl.wiki, pt.wiki, ro.wiki, th.wiki, tr.wiki, uk.wiki, and vi.wiki from 8 May 2023 through 28 May 2023.

To learn whether the suite of desktop talk page design changes were negatively impacting peoples' experiences using Wikipedia talk pages, we conducted an initial analysis at the three Wikipedias where these features were first made available by default: ar.wiki, cs.wiki, and hu.wiki.

This analysis was meant to help us answer the following question:

Are the set of Usability Improvements negatively impacting other peoples' experiences using Wikipedia talk pages?

The answer to this question was a key factor in helping the team decide whether these new features were fit to be deployed more widely and evaluated through an A/B test.

Timing

This analysis was completed on 12 April 2023 and analyzed engagement data at ar.wiki, cs.wiki, and hu.wiki during the following time periods:

• cs.wiki and hu.wiki: 28 February 2023 - 29 March 2023
• ar.wiki: 05 March 2023 - 3 April 2023

The time periods were selected to collect engagement data two weeks before and two weeks after making the suite of desktop talk page design changes available by default at the participating wikis.

Findings

 

Revert rate

• Overall: We did not observe any changes in the overall desktop talk page edit revert rate pre- and post-deployment across the three participating wikis. The overall revert rate pre- and post- deployment was 4.3%.
• Editor Interface: When we analyzed the revert rate by editor interface type, we observed a -17% percent decrease in the revert rate of desktop talk page edits made with visual editor and a -4% decrease in the revert rate of desktop talk page edits made with wikitext following deployment of the feature.
• Experience level: When we analyzed the revert rate by experience level, we observed a -18% decrease in the revert rate of desktop talk page edits by Junior Contributors (under 100 cumulative edits) following the deployment of the feature; compared to 4.5% increase (+0.1 percentage points) for Senior Contributors (over 100 cumulative edits).

 

Edit completion rate

• Overall: We observed a -4.8% decrease in desktop talk page edit completion rate across all three participating wikis. There were no sudden increases or decreases in daily edit completion rates following deployment of the feature.
• Editor Interface: We observed a slight increase (< 1%) in edit completion rate for desktop talk page edits made with visual editor and an -11% percent decrease for desktop talk page edits made with wikitext following deployment of the feature.
• Experience level: We observed was a 10.2% increase in talk page edit completion rate for Junior Contributors and a slight decrease (-1.3%) in edit completion rate for Senior Contributors.

Block rate

• Overall: We did not observe any significant changes in the number of users blocked after publishing a desktop talk page edit. Less than 1.5% of users were issued a sitewide block pre and post deployment across all three participating wikis.

Opt-out rate

• Overall: A total of 48 distinct users across all three participating wikis turned the feature off.
• Activity level: Less than 1% of the total number of users who made a desktop talk page edit attempt following the deployment of the usability improvements (2,122 people) disabled the feature during the reviewed time period.

Full Report

Usability Improvements Engagement Analysis

We will make incremental and optional changes* to elevate the core talk page components (e.g. topic headings and conversation metadata) and calls to action (e.g. replying to a comment, starting a new discussion, subscribing to a topic, etc.). We will seek changes to improve peoples' ability to:

• Recognize talk pages as space to communicate with other people
• Identify and understand the conversations happening on talk pages
• Know what to do to engage with talk pages

*We are using "optional changes" in this context to mean that new functionality will be developed in ways that ensure people retain the flexibility to turn it off.

We will work to deliver on the strategy above through three phases, the details of which are outlined below.

It is important that volunteers' needs and expectations are heard and incorporated into the design process. To do this, we will publicize the questions and decisions that guide the implementation of each of the phases below and make sure this information is easy to engage with.

 

First, we will evolve how talk page section headings (read: ==H2==) are visually represented and introduce discussion-specific metadata within these headings.

"Discussion-specific metadata" in this context refers to information like: the last time the conversation was edited, the number of comments in the conversation, the number of unique people who have participated in the conversation, etc.

More information can be found in T269950.

Next, we will make the affordances for responding to specific comments and starting new conversations easier for people to identify and access.

More information can be found in T267444 and T255560.

In this last phase, we will introduce new page-level visual elements to help people immediately recognize talk pages as places where editors communicate and for them to assess the activity happening within the talk page they have landed upon.

More information can be found in T269963.

This set of changes will include:

• A visual space between the "namespace" and "page" name portion of a talk page's title
• An indicator that describes, and links to, the latest comment published on a given talk page
• An "Add topic" button that will appear in in the sticky header Vector (2022) introduces (desktop only)
• Information within the new table of contents Vector (2022) introduces about the number of comments within a section

•  
  Proposed design for mobile wikitext talk pages.
•  
  An annotated mockup showing the proposed design for mobile wikitext talk pages (landing page view).
•  
  An annotated mockup showing the proposed design for mobile wikitext talk pages (individual discussion view).
•  
  Proposed design for desktop wikitext talk pages.
•  
  An annotated mockup showing the proposed design for desktop wikitext talk pages.

The Editing team is committed to evolving wikitext talk pages in a way that gives experienced contributors more leverage to coordinate their work and connect with other editors, while making the same communication and coordination practices and capabilities legible and intuitive for newer contributors.

For these wikitext talk pages to be valuable, Junior Contributors need to intuitively recognize them as spaces to communicate with other volunteers and Senior Contributors need to be able to spend minimal effort understanding the conversations happening on a given page.

The trouble is, volunteers across experience levels report the current presentation of wikitext talk pages can make recognizing talk pages, understanding the conversations happening within them, and identifying what they need to click/tap to participate in these conversations unnecessarily difficult.

Specifically, user feedback ^{[8][9][10][11][12][13][14]}, usability tests^[15][16] the Talk Page Consultation (see: #Newcomers, #Confusion, and #Design), and academic research^[7] have highlighted the difficulties people have reading and using talk pages:

Talk pages and their current configuration proved confusing...For [some] participants, it took more than 4-5 minutes to understand the Talk page itself which was “disorganised” making it “difficult to take part in the discussion.”

—Understanding and improving Wikipedia Article Discussion Spaces

I don't know what it means really to start a new discussion. Because none of this looks like a discussion.

—Junior Contributor usability test participant

I often have trouble visually parsing long discussions on the WP:Help desk, for example. Sometimes, a discussion will involve three or more people and/or have five or more posts. Spacing between posts is not standardized, nor are signatures, nor is indentation, nor is the length of posts. Sometimes, people become confused and respond as if one person is the author of what another user has written.

—User:Quisqualis writing in Navigating within long-winded discussions...

I felt I had to scan the page a lot for what i needed, as different elements didn't really stand out. It wasn't really obvious that i was looking at a message from someone...

—Junior Contributor usability test participant

By making the components that comprise talk pages conversations (e.g. conversation topics, conversation boundaries, comments, etc.) and the tools necessary for participating in them easier for people to parse, we are striving to:

1. Increase the likelihood Junior Contributors quickly recognize talk pages as spaces to communicate with other volunteers and locate the tools available to do so and
2. Increase the likelihood Senior Contributors can quickly evaluate the level of activity happening on talk pages.

Many projects, conversations and research have, and are, working to improve contributors' experiences with talk pages. This project is better off for their existence. Some of the projects the team continues to learn from are listed on the main project page and below. If there is a project you think we should be aware of, please boldly add it here.

• https://en.wikiversity.org/wiki/Template_talk:Fig/sandbox
• Custom section calls to action at it.wiki: Wikipedia:Bar/2021 04 14
• People using custom CSS change how talk pages appear: https://global-search.toolforge.org/?namespaces=2%2C4...
• Convenient Discussions
• reply-link
• Flow satisfaction survey
• Moderated Testing, November 2014: talk pages and Flow
• Talk pages consultation

The Talk pages project glossary is intended to help us all communicate about talk pages more effectively by making sure we have a shared understanding about the words we use in our discussions and documentation throughout the project.