i18n: Proposal to use jekyll-simple-i18n plugin instead of jekyll-multiple-languages #1503

jonah · 2019-11-18T01:45:57Z

djoate commented

2019-11-18 01:45:57 +00:00

(Migrated from github.com)

Related to issue https://github.com/privacytoolsIO/privacytools.io/issues/1106.
Tagging @JonahAragon since i18n was your project.

Preview of index.html translation: https://deploy-preview-1503--privacytools-io.netlify.com/es/
Original site: https://deploy-preview-1503--privacytools-io.netlify.com

Sample of partially translated pages:

https://deploy-preview-1503--privacytools-io.netlify.com/es/
https://deploy-preview-1503--privacytools-io.netlify.com/es/providers/ (which you can get to by pressing the green Aprenda Más button under Proveedores on the homepage)
https://deploy-preview-1503--privacytools-io.netlify.com/es/providers/cloud-storage/ (which you can get to by pressing the Servidor Cloud link in /es/providers/)

Every other page has not been translated (pages without translate: true in front matter are not generated)

This is not meant to be a full-fledged PR but rather a proof of concept for a better solution for localizing the site. If this is an acceptable solution, please make this into a branch in this repository so that we can start localizing using this plugin.

With the jekyll-simple-i18n plugin (MIT licensed), this makes translating the site easier to manage. You should visit the plugin's GitHub repository and read up on it, but here are the features of this plugin from their README:

No external dependencies. Plugins utilize existing Jekyll features.
Source strings and page titles can be placed directly in templates for seamless editing and readability.
A ready-to-translate YAML file that includes all of the canonical source strings is generated every time the site is built.
It's easy to add new languages. Just create a single file that contains the translated strings. Everything else happens automatically.
Custom front matter is added to translated pages that can be used within your Liquid templates.
Optional Transifex integration.
Built-in support for hreflang tags.

It's based on Transifex, but it can be used with a different service such as Weblate. I've made some modifications to the plugin (e.g. renaming transifex to weblate and handling null source text).

This PR includes an example of index.html and a part of card.html (The "Learn More" button text) being translated. The plugin did not seem to work with the github-pages gem, so github-pages was switched with jekyll gem (which is what the current i18n branch does anyway). You can go to https://deploy-preview-1503--privacytools-io.netlify.com/es/ (or build locally) to see the following:

https://deploy-preview-1503--privacytools-io.netlify.com gives the original English site.

Here is a snippet of the source code for index.html:

<h1 id="sponsors" class="anchor"><a href="#sponsors"><i class="fas fa-link anchor-icon"></i></a> {% t Sponsors%}</h1>

<div class="alert alert-success" role="alert">
  <strong>{% t New!%}</strong> {% t Showcase your brand as a sponsor of PrivacyTools here and support our mission of creating a world free of mass surveillance!%} <a href="/{% if page.language %}{{ page.language }}/{% endif %}sponsors/" class="alert-link">{% t Learn more...%}</a>
</div>

A snippet of resources.html:

<p><a href="/{% if page.language %}{{ page.language }}/{% endif %}classic/"><i class="fas fa-info-circle"></i> {% t Prefer the classic site? View a single-page layout.%}</a></p>

<div class="row">

  {% capture providers_title %}{% t Providers %}{% endcapture %}
  {% capture providers_page %}/{% if page.language %}{{ page.language }}/{% endif %}providers/{% endcapture %}
  {% capture providers_description %}{% t Discover privacy-centric online services, including email providers, VPN operators, DNS administrators, and more!%}{% endcapture %}

  {% include card.html color="success"
  title=providers_title
  icon="fas fa-server"
  iconcolor="dark"
  page=providers_page
  description=providers_description
  %}

Instead of using keys and two different files, you just wrap the original text around with {% t ... %} tags, and the plugin will automatically key that string (with its own ID) into weblate-source-file.yml. If you are trying to translate things inside of a card, you have to do the same thing as before with capturing text.

The source YAML is generated on build into the root folder of the repo. This source file can then be copied into _data/languages/ and then renamed into one of the languages in the language map to set up a translation. This seems much easier to maintain compared to cross referencing between two different files.

The plugin will also not create multiple keys for duplicates of the exact same string. For example, {% t Worth Mentioning %} will have one key associated with it, and there will only be one key to translate. All other pages that use {% t Worth Mentioning %} will share the same key (however, I've modified it so that, for instance, {% t Worth mentioning %} and {% t Worth Mentioning! %} would have distinct keys)

We would have to replace local links with something like this in order to get the right pages (and I believe external links can be wrapped in translate tags without a problem):

<a href="/{% if page.language %}{{ page.language }}/{% endif %}sponsors/" class="alert-link">{% t Learn more...%}</a>

A porition of the source file, weblate-source-file.yml, looks like this:

---
Prefer_the_classic_site?_View_a_singlepage_layout.: |
  Prefer the classic site? View a single-page layout.  

Providers: |
  Providers  

Discover_privacycentric_online_services_including_email_providers_VPN_operators_DNS_administrator: |
  Discover privacy-centric online services, including email providers, VPN operators, DNS administrators, and more!  

Learn_More: |
  Learn More

It's a different format when compared to what is currently in the i18n branch now, i.e. it has the format

string_key_id: |
  This is a source string from the site

rather than

"string_key_id": "This is a source string from the site"

If this format doesn't work with Weblate, we can change the plugin so that it generates the latter format.

A sample translation into Spanish (using deepl.com) can be found in _data/languages/es.yml:

---
Prefer_the_classic_site?_View_a_singlepage_layout.: |
  ¿Prefieres el sitio clásico? Ver un diseño de una sola página.  

Providers: |
  Proveedores  

Discover_privacycentric_online_services_including_email_providers_VPN_operators_DNS_administrator: |
  Descubra servicios en línea centrados en la privacidad, incluyendo proveedores de correo electrónico, operadores de VPN, administradores de DNS y mucho más!  

Learn_More: |
  Aprenda Más

Known issues

I don't know of a way to make the plugin translate the strings of permalinks. This means that pages such as https://privacytools.io/es/donate will have to stay as /es/donate for now. Update: See comment below since this is actually not an issue.
Because the plugin uses the actual string as the translation ID/key, there may be collisions (e.g. "Learn more..." and "Learn more!" will have the same ID). To help remedy this, I've modified the plugin so that the IDs preserve capitalization and can also contain periods, exclamation marks, and question marks. I've also modified the plugin so that the max length for an ID is 100 characters.
I've had to modify the Gemfile to not use github-pages gem, so github-pages was switched with jekyll gem (which is what the current i18n branch does anyway). The jekyll-sitemap plugin also had to be explicitly added in order for the site to compile.
Breadcrumbs: I'm not a Ruby programmer, so I'm not going to try to make a solution for the breadcrumbs.

To reiterate, this is a proof of concept for a better i18n solution. Feel free to add this as a branch if this seems like an acceptable solution.

Related to issue https://github.com/privacytoolsIO/privacytools.io/issues/1106. Tagging @JonahAragon since i18n was your project. Preview of index.html translation: https://deploy-preview-1503--privacytools-io.netlify.com/es/ Original site: https://deploy-preview-1503--privacytools-io.netlify.com Sample of partially translated pages: - https://deploy-preview-1503--privacytools-io.netlify.com/es/ - https://deploy-preview-1503--privacytools-io.netlify.com/es/providers/ (which you can get to by pressing the green Aprenda Más button under Proveedores on the homepage) - https://deploy-preview-1503--privacytools-io.netlify.com/es/providers/cloud-storage/ (which you can get to by pressing the Servidor Cloud link in /es/providers/) Every other page has not been translated (pages without `translate: true` in front matter are not generated) This is not meant to be a full-fledged PR but rather a proof of concept for a better solution for localizing the site. If this is an acceptable solution, please make this into a branch in this repository so that we can start localizing using this plugin. With the [jekyll-simple-i18n](https://github.com/signalapp/jekyll-simple-i18n) plugin (MIT licensed), this makes translating the site easier to manage. You should visit the plugin's GitHub repository and read up on it, but here are the features of this plugin from their README: * No external dependencies. Plugins utilize existing Jekyll features. * Source strings and page titles can be placed directly in templates for seamless editing and readability. * A ready-to-translate YAML file that includes all of the canonical source strings is generated every time the site is built. * It's easy to add new languages. Just create a single file that contains the translated strings. Everything else happens automatically. * Custom [front matter](https://jekyllrb.com/docs/front-matter/) is added to translated pages that can be used within your Liquid templates. * Optional [Transifex](https://www.transifex.com/) integration. * Built-in support for [hreflang tags](https://en.wikipedia.org/wiki/Hreflang). It's based on Transifex, but it can be used with a different service such as Weblate. I've made some modifications to the plugin (e.g. renaming transifex to weblate and handling null source text). This PR includes an example of index.html and a part of card.html (The "Learn More" button text) being translated. The plugin did not seem to work with the github-pages gem, so github-pages was switched with jekyll gem ([which is what the current i18n branch does anyway](https://github.com/privacytoolsIO/privacytools.io/commit/82b94ec597d36ee5dff0d20726f13c0e7e3d026d#diff-8b7db4d5cc4b8f6dc8feb7030baa2478)). You can go to https://deploy-preview-1503--privacytools-io.netlify.com/es/ (or build locally) to see the following: ![indextransl](https://user-images.githubusercontent.com/56777051/69017959-e2f03880-09a1-11ea-95e5-110613a8eda4.png) https://deploy-preview-1503--privacytools-io.netlify.com gives the original English site. Here is a snippet of the source code for index.html: ```html <h1 id="sponsors" class="anchor"><a href="#sponsors"><i class="fas fa-link anchor-icon"></i></a> {% t Sponsors%}</h1> <div class="alert alert-success" role="alert"> <strong>{% t New!%}</strong> {% t Showcase your brand as a sponsor of PrivacyTools here and support our mission of creating a world free of mass surveillance!%} <a href="/{% if page.language %}{{ page.language }}/{% endif %}sponsors/" class="alert-link">{% t Learn more...%}</a> </div> ``` A snippet of resources.html: ```html <p><a href="/{% if page.language %}{{ page.language }}/{% endif %}classic/"><i class="fas fa-info-circle"></i> {% t Prefer the classic site? View a single-page layout.%}</a></p> <div class="row"> {% capture providers_title %}{% t Providers %}{% endcapture %} {% capture providers_page %}/{% if page.language %}{{ page.language }}/{% endif %}providers/{% endcapture %} {% capture providers_description %}{% t Discover privacy-centric online services, including email providers, VPN operators, DNS administrators, and more!%}{% endcapture %} {% include card.html color="success" title=providers_title icon="fas fa-server" iconcolor="dark" page=providers_page description=providers_description %} ``` Instead of using keys and two different files, you just wrap the original text around with `{% t ... %}` tags, and the plugin will automatically key that string (with its own ID) into `weblate-source-file.yml`. If you are trying to translate things inside of a card, you have to do the same thing as before with capturing text. The source YAML is generated on build into the root folder of the repo. This source file can then be copied into `_data/languages/` and then renamed into one of the languages in the language map to set up a translation. This seems much easier to maintain compared to cross referencing between two different files. The plugin will also not create multiple keys for duplicates of the exact same string. For example, `{% t Worth Mentioning %}` will have one key associated with it, and there will only be one key to translate. All other pages that use `{% t Worth Mentioning %}` will share the same key (however, I've modified it so that, for instance, `{% t Worth mentioning %}` and `{% t Worth Mentioning! %}` would have distinct keys) We would have to replace local links with something like this in order to get the right pages (and I believe external links can be wrapped in `translate` tags without a problem): ```html <a href="/{% if page.language %}{{ page.language }}/{% endif %}sponsors/" class="alert-link">{% t Learn more...%}</a> ``` A porition of the source file, `weblate-source-file.yml`, looks like this: ```yml --- Prefer_the_classic_site?_View_a_singlepage_layout.: | Prefer the classic site? View a single-page layout. Providers: | Providers Discover_privacycentric_online_services_including_email_providers_VPN_operators_DNS_administrator: | Discover privacy-centric online services, including email providers, VPN operators, DNS administrators, and more! Learn_More: | Learn More ``` It's a different format when compared to what is currently in the i18n branch now, i.e. it has the format ```yml string_key_id: | This is a source string from the site ``` rather than ```yml "string_key_id": "This is a source string from the site" ``` If this format doesn't work with Weblate, we can change the plugin so that it generates the latter format. A sample translation into Spanish (using deepl.com) can be found in `_data/languages/es.yml`: ```yml --- Prefer_the_classic_site?_View_a_singlepage_layout.: | ¿Prefieres el sitio clásico? Ver un diseño de una sola página. Providers: | Proveedores Discover_privacycentric_online_services_including_email_providers_VPN_operators_DNS_administrator: | Descubra servicios en línea centrados en la privacidad, incluyendo proveedores de correo electrónico, operadores de VPN, administradores de DNS y mucho más! Learn_More: | Aprenda Más ``` ### Known issues - I don't know of a way to make the plugin translate the strings of permalinks. This means that pages such as `https://privacytools.io/es/donate` will have to stay as `/es/donate` for now. _Update: See [comment](https://github.com/privacytoolsIO/privacytools.io/pull/1503#issuecomment-554822362) below since this is actually not an issue._ - Because the plugin uses the actual string as the translation ID/key, there may be collisions (e.g. "Learn more..." and "Learn more!" will have the same ID). To help remedy this, I've modified the plugin so that the IDs preserve capitalization and can also contain periods, exclamation marks, and question marks. I've also modified the plugin so that the max length for an ID is 100 characters. - I've had to modify the Gemfile to not use github-pages gem, so github-pages was switched with jekyll gem ([which is what the current i18n branch does anyway](https://github.com/privacytoolsIO/privacytools.io/commit/82b94ec597d36ee5dff0d20726f13c0e7e3d026d#diff-8b7db4d5cc4b8f6dc8feb7030baa2478)). The `jekyll-sitemap` plugin also had to be explicitly added in order for the site to compile. - Breadcrumbs: I'm not a Ruby programmer, so I'm not going to try to make a solution for the breadcrumbs. To reiterate, this is a proof of concept for a better i18n solution. Feel free to add this as a branch if this seems like an acceptable solution.

❤️ 1

netlify[bot] commented

2019-11-18 01:46:55 +00:00

(Migrated from github.com)

Deploy preview for privacytools-io ready!

Built with commit 1472c2760c

https://deploy-preview-1503--privacytools-io.netlify.com

Deploy preview for *privacytools-io* ready! Built with commit 1472c2760ce5af634edbdeda3ef6d14381a0fe1f https://deploy-preview-1503--privacytools-io.netlify.com

dawidpotocki commented

2019-11-18 02:20:03 +00:00

(Migrated from github.com)

Instead of using keys and two different files, you just wrap the
original text around with {% translate ... %} tags, and the plugin
will automatically key that string (with its own ID) into
weblate-source-file.yml. If you are trying to translate things
inside of a card, you have to do the same thing as before with
capturing text.

That's actually a thing I wanted to build :P.

I don't know of a way to make the plugin translate the strings of
permalinks.

That's good, it's more predictable and easier to implement (like for
example if user would switch language while being on some subpage).

> Instead of using keys and two different files, you just wrap the > original text around with `{% translate ... %}` tags, and the plugin > will automatically key that string (with its own ID) into > `weblate-source-file.yml`. If you are trying to translate things > inside of a card, you have to do the same thing as before with > capturing text. That's actually a thing I wanted to build :P. > - I don't know of a way to make the plugin translate the strings of > permalinks. That's good, it's more predictable and easier to implement (like for example if user would switch language while being on some subpage).

👍 1

jonah commented

2019-11-22 20:07:36 +00:00

Okay. This looks good to me. A saner solution, I think. We can't use Transifex. If this works with Weblate, which it should, we're fine.

We don't want to add weblate-source-file.yml to .gitignore, do we? I think the file would need to be in the repo for Weblate to read it.

Okay. This looks good to me. A saner solution, I think. We can't use Transifex. If this works with Weblate, which it should, we're fine. We don't want to add `weblate-source-file.yml` to .gitignore, do we? I think the file would need to be in the repo for Weblate to read it.

This repo is archived. You cannot comment on pull requests.

No reviewers