NEW [WIP] Add localization support with jekyll-simple-i18n #1509
1 Participants
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: privacyguides/privacytools.io#1509
Loading…
x
Reference in New Issue
Block a user
No description provided.
Delete Branch "i18n-simple"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Supersedes #1458. This brings changes from #1503 into the main repo for development.
@djoate:
Preview of index.html translation: https://deploy-preview-1509--privacytools-io.netlify.com/es/
English site: https://deploy-preview-1509--privacytools-io.netlify.com
Sample of partially translated pages:
Every other page has not been translated (pages without
translate: truein front matter are not generated)This is not meant to be a full-fledged PR but rather a proof of concept for a better solution for localizing the site. If this is an acceptable solution, please make this into a branch in this repository so that we can start localizing using this plugin.
With the jekyll-simple-i18n plugin (MIT licensed), this makes translating the site easier to manage. You should visit the plugin's GitHub repository and read up on it, but here are the features of this plugin from their README:
It's based on Transifex, but it can be used with a different service such as Weblate. I've made some modifications to the plugin (e.g. renaming transifex to weblate and handling null source text).
This PR includes an example of index.html and a part of card.html (The "Learn More" button text) being translated. The plugin did not seem to work with the github-pages gem, so github-pages was switched with jekyll gem (which is what the current i18n branch does anyway). You can go to https://deploy-preview-1509--privacytools-io.netlify.com/es/ (or build locally) to see the following:
https://deploy-preview-1509--privacytools-io.netlify.com gives the original English site.
Here is a snippet of the source code for index.html:
A snippet of resources.html:
Instead of using keys and two different files, you just wrap the original text around with
{% t ... %}tags, and the plugin will automatically key that string (with its own ID) intoweblate-source-file.yml. If you are trying to translate things inside of a card, you have to do the same thing as before with capturing text.The source YAML is generated on build into the root folder of the repo. This source file can then be copied into
_data/languages/and then renamed into one of the languages in the language map to set up a translation. This seems much easier to maintain compared to cross referencing between two different files.The plugin will also not create multiple keys for duplicates of the exact same string. For example,
{% t Worth Mentioning %}will have one key associated with it, and there will only be one key to translate. All other pages that use{% t Worth Mentioning %}will share the same key (however, I've modified it so that, for instance,{% t Worth mentioning %}and{% t Worth Mentioning! %}would have distinct keys)We would have to replace local links with something like this in order to get the right pages (and I believe external links can be wrapped in
translatetags without a problem):A porition of the source file,
weblate-source-file.yml, looks like this:It's a different format when compared to what is currently in the i18n branch now, i.e. it has the format
rather than
If this format doesn't work with Weblate, we can change the plugin so that it generates the latter format.
A sample translation into Spanish (using deepl.com) can be found in
_data/languages/es.yml:Known issues
https://privacytools.io/es/donatewill have to stay as/es/donatefor now. Update: See comment below since this is actually not an issue.jekyll-sitemapplugin also had to be explicitly added in order for the site to compile.To reiterate, this is a proof of concept for a better i18n solution. Feel free to add this as a branch if this seems like an acceptable solution.
Deploy preview for privacytools-io ready!
Built with commit
e5cfd44980https://deploy-preview-1509--privacytools-io.netlify.com
I wonder if there is a way to incorporate this into the
card.htmlcode instead of using captures on every page. Will have to investigate that.@JonahAragon This is done in https://github.com/privacytoolsIO/privacytools.io/pull/1510
The PR description should link to https://deploy-preview-1509--privacytools-io.netlify.com instead of https://deploy-preview-1503--privacytools-io.netlify.com
List of some issues:
translate: true8b5226d388)Keying for parts 1 to 4 have been done to key the entire site. Almost all of the content has been keyed as of part 4. Part 5 is to merge master into the i18n development branch to fix conflicts, and to clean up any loose ends such as breadcrumbs and the language select.
The final check after part 5 is merged should be done on this PR where we have a preview for the entire
i18n-simplebranch and all of the parts.Card/cardv2 content, navbar text, misc, and everything under Providers in the navbar.
Everything under Browsers in the navbar
Everything under Software in the navbar
Everything under OS in the navbar
Merging
masterintoi18n-simpleto fix conflicts, and fixing loose endsThis may not be the case, and is breaking Weblate. I'm seeing this 4 times in
weblate-source-file.yml, for example::(
https://gist.github.com/JonahAragon/79cc67b9bbdb30cb73ccbfa2641b17e2
Edit: lol, it's the question mark. You're importing the strings into the regex filter without escaping the characters, hmm...
This branch is working on Weblate at https://weblate.nablahost.com/projects/privacytoolsio/website/, and you can register at https://weblate.nablahost.com/accounts/register/ to test it out.
Also, the site doesn't build if an empty translation YAML file exists, they need at least 1 translated key. That's probably a bug that could be fixed.Edit: This doesn't appear to be the case anymore. I think it only failed when every language file was blank, because with empty and non-empty files it appears to build as expected.A proposed solution for this is in PR https://github.com/privacytoolsIO/privacytools.io/pull/1524 to encode punctuation (and string length)
To keep track, some things that haven't been keyed or implemented (yet):
The privacy statement (https://github.com/privacytoolsIO/privacytools.io/pull/1525#pullrequestreview-322699154)done ine53f3f5f9cWeblate seems to be down
@djoate I know, I'm having issues with it not sending emails :(
As a side note, would it be possible to serve Weblate with privacytools.io as the root domain?
Not at this time AFAIK.
I'm not sure what's going on with Weblate, it might be a known issue (https://github.com/WeblateOrg/weblate/issues/3231) or I might just need to look at it tomorrow with fresh eyes. In either case, it allows for anonymous suggestions and it works otherwise, so I'll get registrations figured out at a later time.
Seems like Weblate thinks that there's a newline at the end of every string
I've noticed that for regenerating the source file, it seems the best thing to do is to make sure Jekyll isn't serving the site when the file is regenerated.
e.g.,
{% t string %}and save it while the site is being served, the source file will only add that new string to the end of the file rather than nearby other strings (affects the Weblate nearby strings view) until you stop serving the site and reserve/rebuild the site.I have noticed that as well. Maybe it would be better if we did add the file to .gitignore and have a bot of some kind build the source file and push it to the repo when changes are made. But for now at least I think we can handle it easily manually.
I saw that but didn't get a chance to look further. The issue, I believe, is this space between each key:
In YAML, the
|means "all text until the next key" I believe, so technically the source file also has newlines at the end of each string, it just isn't as noticeable in this format, but becomes noticeable when Weblate converts it into a single-line format.What I don't know is whether or not the
\nactually affects anything. I didn't see it change anything on the site itself (and can't imagine when it would make a difference, since newlines are generally ignored in HTML). But if it does affect things removing that space between keys in the source would probably fix it.The plugin itself also does
.stripon all rendered tag text. Regardless, I think leaving the newline for every string may be confusing for translators and that it would be better to try to get rid of itAt a second glance with Weblate, I'd agree:
I've "solved" the email problem temporarily by enabling Sign in with GitHub and GitLab. So feel free to register an account, just don't sign up with email :)
@JonahAragon I looked it up (https://yaml.org/YAML_for_ruby.html#three_trailing_newlines_in_literals). Apparently
|would give a final new line while|-strips all newlines@JonahAragon What's the status of this PR? There was some people looking to localize the site recently (https://www.reddit.com/r/privacytoolsIO/comments/enui17/arabic_version_of_privacytoolsio/).
For things to do before this can be considered ready,
Hi.
Would it be possible to reopen discussions around translations?
Could we put it as a main priority for the community to do every changes needed to make translations a thing with Weblate?
This issue struggles since a long time but is, IMHO, one of the most important one.
Actually, most translations of the website are aged and people should not totally rely on it.
@Booteille My thoughts on the state of translations progress is given at https://github.com/privacytools/privacytools.io/issues/1106#issuecomment-743725471