⚓ T370523 [EPIC] Run UI-less experiment comparing API recommendation quality
Page MenuHomePhabricator
Create Task
Maniphest T370523

[EPIC] Run UI-less experiment comparing API recommendation quality
Open, HighPublicSpike
Actions

Description

Background

  • We would like to run an experiment that compares various APIs recommending content independent of the UI. This task will investigate different way to run this experiment and provide a suggestion for the experiment design

User story

  • As a product team, we would like to compare some of our recommendation based APIs so we can choose the one that will help readers learn the most

Requirements

  • Investigate different ways to run the experiment and provide a proposal for the experiment design

This task was created by Version 1.2.0 of the Web team task template using phabulous

Related Objects
Search...

Event Timeline

ovasileva triaged this task as High priority.Jul 19 2024, 1:30 PM
ovasileva created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 19 2024, 1:30 PM
ovasileva edited projects, added Web-Team-Backlog (FY2024-25 Q1 Sprint 3); removed Web-Team-Backlog.Jul 23 2024, 8:33 AM
ovasileva renamed this task from [Spike?] Design UI-less experiment comparing API recommendation quality to [Spike] Design UI-less experiment comparing API recommendation quality.Jul 23 2024, 8:36 AM
ovasileva added a subtask: T370211: [Design] Generate a recommendation list from multiple APIs for the empty state of search for user testing (UI-less).Jul 23 2024, 12:11 PM
Jdlrobson removed the point value for this task.Jul 26 2024, 1:54 AM
ovasileva renamed this task from [Spike] Design UI-less experiment comparing API recommendation quality to [EPIC] Run UI-less experiment comparing API recommendation quality.Jul 29 2024, 10:16 AM
ovasileva edited projects, added Epic; removed Spike.
ovasileva removed a parent task: T369853: [Spike 2 days] Recommended Content APIs.Jul 29 2024, 10:28 AM
ovasileva added a subtask: T369853: [Spike 2 days] Recommended Content APIs.
ovasileva added a parent task: T370209: [EPIC] Experiment: Display article recommendations in more prominent locations.
NBaca-WMF closed subtask T369853: [Spike 2 days] Recommended Content APIs as Resolved.Jul 29 2024, 5:21 PM
ovasileva added a subtask: T371287: [Spike 1 day] Investigate List Building APIs.Jul 29 2024, 5:39 PM
ovasileva edited projects, added Web-Team-Backlog; removed Web-Team-Backlog (FY2024-25 Q1 Sprint 3).Jul 30 2024, 2:54 PM
ovasileva moved this task from Incoming to Epics/Goals on the Web-Team-Backlog board.
Jdlrobson subscribed.Jul 30 2024, 7:00 PM
ovasileva closed subtask T371287: [Spike 1 day] Investigate List Building APIs as Resolved.Aug 16 2024, 11:47 AM
dr0ptp4kt subscribed.Aug 20 2024, 6:35 PM
ovasileva added a subtask: T373851: Enable UI and non-UI experiments (enable quicksurveys and launch browser extension).Sep 3 2024, 9:35 AM
ovasileva closed subtask T371235: Design structure for UI-less experiment comparing API recommendation quality as Resolved.Sep 3 2024, 4:05 PM
ovasileva closed subtask T373851: Enable UI and non-UI experiments (enable quicksurveys and launch browser extension) as Declined.Sep 3 2024, 5:45 PM
ovasileva closed subtask T373793: Get privacy policy for UI-less experiment as Resolved.Sep 4 2024, 10:04 AM
ovasileva closed subtask T370211: [Design] Generate a recommendation list from multiple APIs for the empty state of search for user testing (UI-less) as Resolved.
ovasileva closed subtask T371238: Report on baseline metrics for recommendations experiments (UI and non-UI experiment) as Resolved.Sep 17 2024, 10:40 AM
ovasileva added a subtask: T374670: Analyze the results of the non-ui surveys.Sep 17 2024, 10:42 AM
ovasileva added a subtask: T374669: {GOAL] Run the non-ui survey with new recommendations.
ovasileva moved this task from Backlog to Recommendations - APIs on the FY2024-25 KR 3.1 Content Discovery board.Oct 10 2024, 9:08 PM
Prototyperspective subscribed.Oct 21 2024, 9:31 PM
ovasileva closed subtask T374670: Analyze the results of the non-ui surveys as Resolved.Oct 25 2024, 4:17 PM
Jdlrobson moved this task from Epics/Goals to Groomed on the Web-Team-Backlog board.Wed, Nov 13, 11:16 PM
Jdlrobson edited projects, added Web-Team; removed Web-Team-Backlog.Wed, Nov 13, 11:35 PM
Jdlrobson moved this task from Product Backlog to Legacy backlog on the Web-Team board.Wed, Nov 13, 11:45 PM
ovasileva edited projects, added Web-Team-Roadmap; removed Web-Team.Thu, Nov 14, 5:09 PM
Prototyperspective added a comment.Thu, Nov 14, 5:12 PM

https://github.com/geohci/list-building-interface/issues/18 has some suggestions for improving the suggestions (don't know if this is only about comparing different ones or partly also about improving things or building something new to test here). One could also compare different variants of that "list-building" tool (I also suggested renaming it to e.g. "Related articles finder").

NBaca-WMF closed subtask T374669: {GOAL] Run the non-ui survey with new recommendations as Resolved.Thu, Nov 21, 5:13 PM
ovasileva reopened subtask T374669: {GOAL] Run the non-ui survey with new recommendations as Open.Tue, Nov 26, 5:58 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits