Wikimedia Research/Design Research/Reading Team UX Research

Note: these are the preliminary topic areas. More details forthcoming.

• Wikidata description editing: what are some issues users have with completing the task of adding or modifying Wikidata descriptions to article titles?
  • Heuristic evaluation
  • Study plan/results

• Hovercards:
  • Onboarding experience
  • Measuring impact on reading engagement depth
• Branding
• Foundational work for Recommendations

• Table of Contents usability testing
• Diary study: observe participant behavior over a multi-day span of time to see, qualitatively, how they interact with the app. To include regular feedback in the form of surveys. Formerly "iOS Longitudinal Feeds study."
• Foundational Personas research & planning

For this quarter we are focusing on Navigation and continuing with last quarter's research that we did not have enough participants/the right tools to conduct.

• Android workflows: define the top workflows encountered with the Android app. Conduct remote and online usability tests to understand which workflows work well and which don't work as well as they could. Last quarter we tested workflows with two participants, waiting for three more participants to sign up for this test to be completed.
• Reading lists: a prototype for reading lists (a similar feature to Gather, RIP) exists. A usability test will be conducted in Prototype Labs once the prototype is ready. (This is a carry-over from last quarter. We did not have the tools to conduct this test so it was postponed.)
• Feeds: this is a new feature that will benefit from Feeds research being done on iOS, extended either this quarter or next to Android!

Bonus:

• Quantitative research tools: exploring options for future work with our EventLogging data

• Hovercards: this beta feature is up for potentially extending to all wikis.

• Random article browsing feature (guerilla testing, done)
• Longitudinal study: Assess the efficacy of the Feed in promoting ‘engaged browsing' behavior from our users. Currently blocked on implementation of observation tool.
• Nearby feature improvements (low priority): there are a few variations of proposed updates to Nearby features. These may be usability tested.
• Notifications secondary research review: how do others use notifications? What research already exists for the implementation of notifications? What do people like to be notified about?
• Notifications survey: to help us understand users' needs/desires around iOS notifications, a survey will be conducted asking questions around the topic.

• Android workflows: define the top workflows encountered with the Android app. Conduct remote, and potentially online, usability tests to understand which workflows work well and which don't work as well as they could. Still in planning phase, no deadline set yet.
• Reading lists: a prototype for reading lists (a similar feature to Gather, RIP) exists. A usability test will be conducted in Prototype Labs once the prototype is ready. Still in planning phase.

• Hover cards: this beta feature is up for potentially extending to all wikis. First, it needs to have some design issues fixed, and then will be presented to researchers for building a protocol for usability testing. Still in planning phase.
• Language Switching: Some design changes are being made with the language switching functionality. We have run a usability test on usertesting.com.
• Related pages: this feature exists, but it has been flagged by the community as potentially not valuable. Type of testing still being discussed. No plans yet.
• In-article navigation: it has been requested to possibly look in to testing how users interact with mobile web's non-expanded content sections. Type of testing still being discussed. No plans yet.

• Feed: Assess the efficacy of the Feed in promoting ‘engaged browsing' behavior from our users. Still in planning phase.
• Notifications: determine what research already exists on this, and from there decide what we need to do for the app. Still exploring existing research.
• Search bar: based on user feedback from the last release, we are building a more prominent search bar. Still in design phase. Will run usability test when ready. Still in planning phase.
• Text size change/accessibility: based on user feedback from the last release, we will be adding functionality to be able to change text size in the app. We are also working with accessibility experts/engineers to continue audits on the app in this area. We will be making improvements as these results come back, and test the results. Still in planning phase.

At the beginning of each quarter:

The team design researcher individually meets with each team PM to confirm that team's research needs for the quarter, prioritized by desired/expected timelines, but with no promises yet at this stage. We discuss questions that research can help answer, and come up with ideas for testing as needed.

Shortly thereafter:

• Design researcher confirms/negotiates timelines with PM's regarding desired tests.
• Design researcher formulates loose test plans. (Firmer plans are made closer to the time of actual testing.)

Before a test:

Depending on the nature of the test, design researcher builds firm test plans 1-3 weeks prior to implementation. Stakeholders meet to go over the test, and final plans are carried out once everyone signs off on them.

After a test: 

Findings are presented internally, and all relevant data/an internal deck are presented as answers to the research questions.

Note: some of the research is sensitive and not distributable publicly due to privacy policies around user likenesses. Once the information is ready to be posted publicly, it will appear on wiki and linked to this page.

Prioritization to this point has worked like this: After all 1:1 meetings with PMs have taken place, the design researcher puts tickets into the Reading UX Phabricator board and plans the work according to discussed timelines and abilities with recruiting/etc. To date, this has worked out just fine, however, issues with this approach are anticipated and the team is working with TPG now to set up a process for scaling this. Any changes to the process will be updated here.

If you find that you are outside the above outlined process, there are two ways to start conversations about initiating Reading team UX research:

Option 1: get in touch with the Reading UX Research coordinator, currently Sherah Smith.

Option 2: Add a ticket to the Reading UX Phabricator board, and put it in the Backlog column.

Many of our research projects are recorded on video. This helps us to get the most out of our sessions - we can refer to the recordings to:

• Spot-check that the data we recorded in our notetaking is accurate (if we can't quite remember the exact response to a question or situation)
• Trigger new insights by having another look at a session
• Share users' experiences with other researchers/contributors and potentially extract new perspectives from them
• Build empathy with stakeholders and the foundation by allowing anyone to see for themselves how participants respond to the features we're testing

There are certain rules around the sharing of our user research videos. To ensure that we respect the privacy wishes of our participants, we only publicly share out videos that have been signed off on by that participant. Internally, we may share freely; the videos can be found on the internal server under Usability/Reading Team Videos, organized by project.