Maniphest T231980

not reachable
Closed, ResolvedPublicBUG REPORT
Actions

Assigned To

Authored By

	Lobey76
	Sep 4 2019, 11:15 AM

Description

Steps to Reproduce:
The Discernatron tool for labeling Wikipedia search results for relevance testing used to be available at https://discernatron.wmflabs.org

Actual Results:
Instead of the login/webpage I get a "502 Bad Gateway" now

Maybe the service moved or was shut down intentionally? Just curious because I linked the tool as an example of a good, labeled search relevance dataset in an Elasticsearch blog posts from last year (https://www.elastic.co/de/blog/made-to-measure-how-to-use-the-ranking-evaluation-api-in-elasticsearch). Just curious where it went.

Details

	Subject	Repo	Branch	Lines +/-
	Unconditionally add D modifier to regexps ending in $ check	mediawiki/services/parsoid	master	+200 -198

Customize query in gerrit

Related Objects

Mentioned In: T148784: full introduction in results of discernatron
T144509: Discernatron demo: front-end work
T150481: Generate a precision / recall curve in relevance forge
T148787: add an about page in discernatron
T149121: Go over discernatron data to get an idea of where we need to improve
T150030: [UI] Add "reason for skipping" to Discernatron skip button

Event Timeline

Lobey76 created this task.Sep 4 2019, 11:15 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 4 2019, 11:15 AM

We'll take a look...

Change 535251 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/services/parsoid@master] Unconditionally add D modifier to regexps ending in $ check

https://gerrit.wikimedia.org/r/535251

gerritbot added a project: Patch-For-Review.Sep 9 2019, 5:58 PM

Change 535251 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Unconditionally add D modifier to regexps ending in $ check

https://gerrit.wikimedia.org/r/535251

Maintenance_bot removed a project: Patch-For-Review.Sep 9 2019, 10:10 PM

For whatever reason the container stopped, I started it back up again. This probably needs to move to our more managed tools collection rather than a custom container on a cloud instance.

At a higher level, we didn't have much success collecting labels from people outside the team, which has led to disuse of this tool. Overall the labeling process is, as expected, fairly tedious. In terms of the dataset it's available under CC-Zero, feel free to host a snapshot of the data to ensure continued availability.

EBernhardson claimed this task.Sep 12 2019, 8:15 PM

EBernhardson moved this task from needs triage to Current work on the Discovery-Search board.

EBernhardson edited projects, added Discovery-Search (Current work); removed Discovery-Search.

EBernhardson moved this task from Incoming to Needs Reporting on the Discovery-Search (Current work) board.Sep 12 2019, 8:18 PM

debt closed this task as Resolved.Sep 20 2019, 4:34 PM

Aklapper mentioned this in T150030: [UI] Add "reason for skipping" to Discernatron skip button.Jul 23 2022, 1:57 AM

Aklapper mentioned this in T149121: Go over discernatron data to get an idea of where we need to improve.

Aklapper mentioned this in T148787: add an about page in discernatron.

Aklapper mentioned this in T150481: Generate a precision / recall curve in relevance forge.

Aklapper mentioned this in T144509: Discernatron demo: front-end work.

Aklapper mentioned this in T148784: full introduction in results of discernatron.