Enable sitemap.xml generation & reintroduce robots.txt #669

Merged

Mikaela merged 1 commits from sitemap-robotstxt into master 2018-12-19 20:39:35 +00:00

Conversation 7 Commits 1 Files Changed 2 +7 -1

Mikaela commented 2018-12-19 10:36:01 +00:00 (Migrated from github.com)

Copy Link

I disagree with the decision to remove robots.txt in #668. I think it may at least remove 404 errors from logs and sitemap.xml can only help with SEO or won't be harmful.

However I am not sure if privacytools.io has more than one page and I was unable to get Jekyll working locally on this PC.

I disagree with the decision to remove robots.txt in #668. I think it may at least remove 404 errors from logs and sitemap.xml can only help with SEO or won't be harmful. However I am not sure if privacytools.io has more than one page and I was unable to get Jekyll working locally on this PC.

ghost commented 2018-12-19 12:05:47 +00:00 (Migrated from github.com)

Copy Link

What's the point of having robots.txt when it simply says "all pages allowed"?

I see the point with a sitemap, though.

What's the point of having robots.txt when it simply says "all pages allowed"? I see the point with a sitemap, though.

ghost commented 2018-12-19 12:16:13 +00:00 (Migrated from github.com)

Copy Link

Though as I expected, the sitemap is pretty useless since we have only two pages:

Though as I expected, the sitemap is pretty useless since we have only two pages: 

Vincevrp commented 2018-12-19 12:22:07 +00:00 (Migrated from github.com)

Copy Link

Can you verify that the jekyll-sitemap plugin is supported by Github Pages?

Can you verify that the `jekyll-sitemap` plugin is supported by Github Pages?

ghost commented 2018-12-19 12:25:31 +00:00 (Migrated from github.com)

Copy Link

Good point. Though we don't really need a sitemap.

Regarding robots,

What's the point of having robots.txt when it simply says "all pages allowed"?

Good point. Though we don't really need a sitemap. Regarding robots, > What's the point of having robots.txt when it simply says "all pages allowed"?

Mikaela commented 2018-12-19 12:27:20 +00:00 (Migrated from github.com)

Copy Link

What's the point of having robots.txt when it simply says "all pages allowed"?

Not having it can spam error.log of the HTTP server, but as you use GitHub pages through CloudFlare, I don't know if anyone would ever see them.

Though as I expected, the sitemap is pretty useless since we have only two pages:

Would there be any harm in case there would be more pages in the future? I don't know about your plans though.

Can you verify that the jekyll-sitemap plugin is supported by Github Pages?

Yes, see https://pages.github.com/versions/

> What's the point of having robots.txt when it simply says "all pages allowed"? Not having it can spam error.log of the HTTP server, but as you use GitHub pages through CloudFlare, I don't know if anyone would ever see them. > Though as I expected, the sitemap is pretty useless since we have only two pages: Would there be any harm in case there would be more pages in the future? I don't know about your plans though. > Can you verify that the jekyll-sitemap plugin is supported by Github Pages? Yes, see https://pages.github.com/versions/

👍 1

Vincevrp commented 2018-12-19 12:29:12 +00:00 (Migrated from github.com)

Copy Link

I think it may at least remove 404 errors from logs and sitemap.xml can only help with SEO or won't be harmful.

I agree with this. Having it in place doesn't hurt and it can only benefit SEO.

> I think it may at least remove 404 errors from logs and sitemap.xml can only help with SEO or won't be harmful. I agree with this. Having it in place doesn't hurt and it can only benefit SEO.

jonah approved these changes 2018-12-19 12:31:01 +00:00

jonah left a comment

Would there be any harm in case there would be more pages in the future? I don't know about your plans though.

I guess this can be useful when we have links like /it/index.html, /de/index.html etc.

> Would there be any harm in case there would be more pages in the future? I don't know about your plans though. I guess this can be useful when we have links like `/it/index.html`, `/de/index.html` etc.

This repo is archived. You cannot comment on pull requests.

Reviewers

No Reviewers

jonah

Labels

Clear labels

:mag:🤖 Search Engines

approved

approved, waiting for a PR

dependencies

Pull requests that update a dependency file

duplicate

feedback wanted

high priority

I2P

The Invisible Internet Project (I2P)

iOS

low priority

OS

Operating Systems

Self-contained networks

Social media

stale

A label for stalebot if it gets added

streaming

Anything related to media streaming.

todo

Tor

Anything covering the Tor network

WIP

active work in progress, do not merge or PR (yet)!

wontfix

Issues or bugs that will not be fixed and/or do not have significant impact on the project.

XMPP

Extensible Messaging and Presence Protocol

[m]

Matrix protocol

₿ cryptocurrency

ℹ️ help wanted

↔️ file sharing

⚙️ web extensions

Browser Extension related issues

✨ enhancement

❌ software removal

💬 discussion

🤖 Android

🐛 bug

💢 conflicting

📝 correction

Correction of content on the website

🆘 critical

📧 email

🔒 file encryption

📁 file storage

🦊 Firefox

Firefox & forks, about:config etc.

💻 hardware

🌐 hosting

🏠 housekeeping

Anything primarily related to site cleanup.

🔐 password managers

🧰 productivity tools

🔎 research required

🌐 Social News Aggregators

🆕 software suggestion

👥 team chat

🔒 VPN

Virtual Private Network

🌐 website issue

*Technical* issues with the website.

🚫 Windows

👁️ browsers

🖊️ digital notebooks

🗄️ DNS

Domain Name System

🗨️ instant messaging (im)

🇦🇶 translations

Anything covering a translated version of the site

No Label

Milestone

No items

No Milestone

Assignees

Clear assignees

jonah

No Assignees

1 Participants

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: privacyguides/privacytools.io#669