⚓ T324102 Attribute Phonos audio to file or text-to-speech
Page MenuHomePhabricator

Attribute Phonos audio to file or text-to-speech
Open, Needs TriagePublic

Assigned To
None
Authored By
Nardog
Nov 30 2022, 1:15 PM
Referenced Files
F36966399: attribution_icon_plain.png
Apr 28 2023, 10:07 AM
F36966405: attribution_icon_label.png
Apr 28 2023, 10:07 AM
F36954149: Info.png
Apr 17 2023, 9:05 AM
F36952069: Screenshot from 2023-04-14 15-48-39.png
Apr 14 2023, 7:52 PM
F36922821: image.png
Mar 22 2023, 3:02 PM
F35840838: audio-auto-icon.png
Dec 6 2022, 11:08 AM

Description

There should be some communication to the reader (popup or notification) to the effect of "This audio was automatically generated. (Learn more)" when the audio is played. Otherwise I expect misplaced accusations of inaccuracy/corner-cutting hurled at volunteers.

For the scope of this feature/project, though, I think the intention is to give casual readers/users a very rough idea of how a word sorta sounds for a native speaker, knowing that it's not perfect. I think of TTS with IPA as kind of like building a MIDI player for sheet music: sheet music is already an approximation of a song, and rendering it via MIDI results in a super lossy rendition. For an average user, this should hopefully give them enough of an idea of the melody to recognize the song.

If this is in line with the expectation for this feature, I wonder if there is some way of having a disclaimer that IPA renderings are meant to be approximations, and in some cases could be inaccurate enough to be actually just outright incorrect.

And there should be a link (e.g. an "i" button like VideoJS) to the file description for attribution if the tag uses file="".

Event Timeline

MusikAnimal subscribed.

With everything we know now, I think it's highly unlikely we'll do any sort of large-scale rollout by adding Phonos to Template:IPA, for instance. Instead, we'll instruct wikis to either create a new template, or add an opt-in parameter. They can then piecemeal add it to articles after verifying the audio sounds right. Since the burden would be on editors to do the verification, the disclaimer can instead go on the template/parameter documentation. TemplateData allows us to expose this to visual editors and those adding new templates with TemplateWizard, but it is true plain wikitext editors won't know what they're messing with. My inclination is except for the linguists such as yourself, most won't bother changing the IPA, at least if all variants are represented (and desired).

I just don't feel like readers should be exposed to a message that is likely frequently going to be seen, is all. As we've said, the article lead is often very cluttered as-is. I'd hate to introduce another little icon next to each IPA instance.

And there should be a link (e.g. an "i" button like VideoJS) to the file description for attribution if the tag uses file="".

I believe this is a legal requirement that we haven't considered! Good catch, but it probably deserves its own task as the implementation will differ.

I just don't feel like readers should be exposed to a message that is likely frequently going to be seen, is all. As we've said, the article lead is often very cluttered as-is. I'd hate to introduce another little icon next to each IPA instance.

I was mainly thinking of a popup (like the one for errors) that appears when you click on it. As long as it's not shown until initiated by the user, I doubt it'll be so ubiquitous that it's a problem. (Also, if file= is used alongside, it'll be all the more important that it's made clear to the reader when it's auto-generated versus when it's not.)

I'm not so sure if communities will adopt it as an opt-in on each page. Even if they did, given it'll presumably be in high demand (the reason you're building it!), they would likely have to turn it on on lots of pages, and they would not have the time to listen to and verify each and every one of them (and that is if they have the the knowledge required to do so in the first place—not a sure thing for certain languages). Readers deserve to know when they're presented with something even the editors themselves can't vouch for the accuracy of.

It would be nice if the content of the popup could be specified in a parameter in <phonos> (or in MediaWiki: at the very least), esp. if it came with an ability to place links, e.g. to a forum to report Phonos-related issues.

Also, who's to say attribution isn't required for TTS? Like, can you re-upload it on Commons? If so, under what license, and who would be the author? Either way, it would be mixed in with CC-BY-SA/GFDL content, so we should absolutely ascertain and indicate attribution and license for generated audio as well, like we do for all embedded media.

We talked to WMF Legal about the copyright of the generated audio a few months ago, and the advice was that it's public domain (or CC0) and so doesn't need any notice. We definitely didn't want to be adding content to wiki pages that is not compatible with their licenses! But I'm not able to find a link for that discussion… @JMcLeod_WMF do you know where we documented that?

Of course, that doesn't mean that we wouldn't still want to have something that says that the audio is generated and also that it's reusable.

Good to know! But we do attribute most PD works anyway (to show the provenance that demonstrates the PD status—and to show that the attribution to contributors specifically does not extend to the work), and of course how and where the audio was produced is of great interest to readers, so one popup could kill two birds (clarification of the source and warning of the artificial nature).

I do think that regardless of copyright requirements, having an indication of provenance is valuable (similar to how it would be nice if Google surfaced where it gets its scraped facts from). I don't consider it critical, but it does seem something that morally could be expected.

TheresNoTime renamed this task from Indicate the audio is machine-generated (or link to description in case of file=) to Attribute Phonos audio to file or text-to-speech.Mar 22 2023, 2:51 PM

(Hopefully that task rename makes sense)

And just noting some comments from a recent meeting:

  • By default when a file is provided to Phonos, an "i icon" (or similar) should be shown. Clicking this icon will take the user to the relevant File page for attribution purposes.
  • When ipa is passed (or wikibase, where IPA is rendered), the "i icon" should instead indicate the audio was machine-generated.
  • A parameter could be added to Phonos (i.e. no-info?) which would hide this "i icon", allowing editors to implement their own — this would be useful where Phonos is used in a template, and the template provides its own attribution link method.

@JSengupta-WMF could you review this from a design perspective? A smaller version of the VideoJS icon (below) might look okay? Alternatively, at the 2021 Inline Audio-Player for pronunciations and other usages proposal, a "small superscript i link" was suggested.

image.png (24×24 px, 731 B)

As per our discussion in today's engineering+design sync up, the superscript "i" works fine according to me. Editors can apply CSS to make the "i" Bold/Bigger if they'd like to increase the visibility of it.

We also talked about tracking the number of clicks on the (i) icon. This can easily be added as we're already tracking clicks on other Phonos elements. This is not critical data, but it'd be interesting to see!

Please let it be a button. The superscript "i" is so small you can barely click it and enwp did away with it a long time ago.

When ipa is passed (or wikibase, where IPA is rendered), the "i icon" should instead indicate the audio was machine-generated.

Is this to be done in a popup (PopupButtonWidget)? If so it would be nice if the editor could customize the message in the tag, such as "Report problems here" linking to a noticeboard. (If it's just a link instead of a popup, at least make the destination customizable.)

Change 904564 had a related patch set uploaded (by Samtar; author: Samtar):

[mediawiki/extensions/Phonos@master] [WIP] PhonosButton: Add attribution link to Phonos file instances

https://gerrit.wikimedia.org/r/904564

Again, "🔊ⁱ" is far too small to even notice or click and could pose an accessibility issue. The button is already relatively short (the height of the text) so anything shorter would be too short. And since both the 🔊 icon and the text following it make up one button, a superscript inside (or after) it is invariably awkward. I bet use of any text is problematic given it'll be embedded within prose (how are screen readers going to read it?).

That said, I can see sandwitching the text between icons could also be awkward. Perhaps the link/message should show up in the popup (which is currently used only for errors) on hover/focus.

A few random thoughts (looking at the above patch):

  • Should the attribution be available without JS (because the audio link is)?
  • If the link is constructed in PHP, then it can get all the stuff of LinkRenderer for free.
  • If a file doesn't exist, normal links link to Special:Upload; would it make sense to do the same for the attribution link?

Regarding the 'i' superscript, would it be suitable to use the 🛈 character, Circled Information Source (U+1F6C8, &#x1F6C8;)? That's a bit bigger. Or maybe a bigger button that only appears when hovering?

Or maybe a bigger button that only appears when hovering?

I've thought of that, and although it's an enticing idea, I can't think of a neat way. Anything that reflows the rest of the paragraph would be disruptive (if it's at the end of a visual line, it could wrap to the next, making it never clickable), and position:absolute (or something akin to it) could stick out of the content. So AFAICS the only thing that's satisfactory both visually and UX-wise is something that pops up vertically.

(just an update — I am looking at this, I've just wrecked my xdebug setup locally somehow so bear with me...)

@Samwilson without testing, I'm guessing you're thinking something like

if ( $file ) {
    $buttonConfig['data']['file'] = $file->getTitle()->getText();
    $buttonConfig['data']['attributionLink'] = $this->linkRenderer->makeLink(
        $file->getTitle(),
        "&#x1F6C8;" // This seems wrong.
    );
} [...]

in Phonos.php, and passing that to PhonosButton.js when JS and do something in PhonosButton.php when not JS?

@TheresNoTime that would work, but no I was thinking that the HTML output in PHP would be both links (button and attribution). Something like this (simplified):

<span class="ext-phonos">
    <span class="PhonosButton"><a href=".mp3">Foo</a></span>
    <sup><a href="attribution-url">🛈</a></sup>
</span>

It'd happen outside of PhonosButton, I think, because that's all just the actual audio player button. We could make a new OOUI widget that contains a button and the attribution link, perhaps extending ButtonGroupWidget (although I'm not sure if fighting the CSS for that is worth it).

Change 907894 had a related patch set uploaded (by Samtar; author: Samtar):

[mediawiki/extensions/Phonos@master] [WIP] PhonosButton: Add attribution link to Phonos file instances

https://gerrit.wikimedia.org/r/907894

Change 904564 abandoned by Samtar:

[mediawiki/extensions/Phonos@master] [WIP] PhonosButton: Add attribution link to Phonos file instances

Reason:

I277db6cc859334a46964603476dd58189db65eb1

https://gerrit.wikimedia.org/r/904564

Noting that on my brand new ThinkPad with Ubuntu 20.04.1, &#x1F6C8; and 🛈 do not appear correctly:

Screenshot from 2023-04-14 15-48-39.png (48×155 px, 2 KB)

Is it just me (I only did a minimal install of Ubuntu so maybe I'm missing some fonts or something), or should we use something different, maybe even an image?

The spacing seems a bit too far off too, but that might be because the character isn't showing for me.

We can use the content/info icon from Codex/OOUI for this.

Info.png (20×20 px, 488 B)

To answer one of the previous comments, no it won't be a pop up. The link to the destination can be customizable.

Change 907894 merged by jenkins-bot:

[mediawiki/extensions/Phonos@master] PhonosButton: Add attribution link to Phonos file instances

https://gerrit.wikimedia.org/r/907894

dom_walden subscribed.

Here is how the icon looks like on enwiki beta in English:

attribution_icon_label.png (35×298 px, 3 KB)

attribution_icon_plain.png (42×228 px, 3 KB)

(Examples from https://en.wikipedia.beta.wmflabs.org/wiki/Phonos_Natural4 and https://en.wikipedia.beta.wmflabs.org/wiki/Phonos_Natural3)

The text of the icon is set in the translation files so can be set per-language, including to 🛈 if they want or to something more appropriate for their language. It is currently i in English.

It is a link to the file page (e.g. https://en.wikipedia.beta.wmflabs.org/wiki/File:En-us-Kentucky.ogg).

If the file passed to the file parameter does not exist, the link will vary depending on a couple of different config variables ($wgUploadMissingFileUrl and $wgUploadNavigationUrl).

I tested in RTL and raised T335559.

I tested with a screenreader and raised T335562.

I briefly tested all 5 skins (Vector 2010, Vector 2022, Minerva, MonoBook and Timeless) and with different levels of zoom. I didn't see any obvious problems.

I also raised T335561.

Test environment: https://en.wikipedia.beta.wmflabs.org and https://en-rtl.wikipedia.beta.wmflabs.org Phonos 0.1.0 (3d582f8) 21:33, 26 April 2023.
Test browsers: Mainly Firefox 102. Briefly on Chromium 112 and Safari 16.3.

attribution_icon_label.png (35×298 px, 3 KB)

attribution_icon_plain.png (42×228 px, 3 KB)

(Examples from https://en.wikipedia.beta.wmflabs.org/wiki/Phonos_Natural4 and https://en.wikipedia.beta.wmflabs.org/wiki/Phonos_Natural3)

Do you all seriously think that is an okay thing to ship it with...? The link is literally less than 3px wide in my environment.

And are you not adding it to buttons without file=? Readers deserve to be able to know what it even is that they're hearing. (And since that was the primary thing this task was asking for, it should not be closed until it is achieved.)

Attribution to text-to-speech has not been implemented. You may choose not to work on it but it's certainly not resolved.