Generating feeds with files not ending in xml/html

[seam345]

Bug Report

Environment

Zola version: zola 0.19.2

Expected Behavior

I can put any file extension in the templates folder and it be counted as a template, or failing that entering it as a feed_filename should convert it to a template for rendering feeds

Current Behavior

sean@little-my ~/G/G/R/website (master)> zola serve
Building site...
Checking all internal links with anchors.
> Successfully checked 0 internal link(s) with anchors.
-> Creating 6 pages (0 orphan) and 3 sections
Error: Failed to serve the site
Error: Tried to render `cal.ics` but the template wasn't found
sean@little-my ~/G/G/R/website (master) [1]> ls -l templates/cal.ics
-rw-r--r-- 1 sean users 716 Jan 13 13:37 templates/cal.ics

Step to reproduce

1. change feed filenames to feed_filenames = [ "atom.xml", "cal.ics"]
2. create a file in templates named cal.ics
3. zola serve will fail with the above error

Optionally clone and zola serve https://gitlab.com/rust-manchester/website

Additionally I forked zola to see if what I was attempting was possible and got a working version using
seam345@0cb8148
This was very hacked together though and only works for .ics extensions I imagine there are many more plain text formats that could be used to generate feeds in the same way

Did work though :D https://rustmanchester.co.uk/events/cal.ics

[seam345]

#2768 is related but a much more narrow use case

[Keats]

For feeds I prefer to keep the control a bit tight around it. I think adding calendar support would be cool though

[seam345]

oki, are you thinking about having it as a builtin then?

If so I can work on improving the template for a PR

if someone else with more zola experience worked on the code side that would be ace, otherwise I'll try properly develop my hacked solution after finalising the ics template

[seam345]

my current working template is

{# NOTE: lines need to be terminated with CRLF, use unix2dos if on linux #}
{# for now I have ignored timezones, I didn't really understand them #}
{%- if feed_url is containing("events") -%}
BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//zola/{{ config.title }}

{%- for page in pages %}
BEGIN:VEVENT
UID:{{ page.permalink | safe }}

{#- DTSTAMP, when the calender entry was made #}
{#- I pondered page.date, but I wanted to set the page date as the date of the event... it made sense for current rss
feed and for prepending file name to know the dates of the page events, but then doesnt work here for when the event was published
  however it probably wouldnt break anything if we go back to page.date#}
DTSTAMP:{{ page.extra.cal_published | date(format="%Y%m%dT%H%M%S") }}
LAST-MODIFIED:{{ page.updated | default(value=page.date) | date(format="%Y%m%dT%H%M%S") }}

{%- if page.extra.start_date %}
DTSTART:{{ page.extra.start_date | date(format="%Y%m%dT%H%M%S") }}
{%- endif %}

{%- if page.extra.end_date %}
DTEND:{{ page.extra.end_date | date(format="%Y%m%dT%H%M%S") }}
{%- endif %}

{#- make breakline #}
SUMMARY:{{ page.title }}

{%- if page.summary %}
DESCRIPTION:{{ page.summary }}
{% else %}
DESCRIPTION:{% for char in page.content | linebreaksbr | replace(from="<br>", to="\n") | striptags -%}
{{ char }}
{#- cap lines at 75 chars (really wants 75 octets, so this works for english but different unicodes will likely break this)
 tested with japanese kanji and validating with https://icalendar.org/validator.html didn't give any errors -#}
{%- if loop.index / 73 < 1 -%}
{%- if loop.index % 60 == 0 %}
 {% endif %}
{%- else -%}
{%- if loop.index % 73 == 60 %}
 {% endif %}
{%- endif %}
{%- endfor %}
{%- endif %}

{%- if page.extra.geo %}
GEO:{{ page.extra.geo }}
{%- endif %}
END:VEVENT
{%- endfor %}
END:VCALENDAR
{%- endif %}

DTSTAMP, I was quite unsure what to set this too I guess ideally it would be page.date, but I quite like having the page date the event date can be convinced thats a bad idea though.

Variables I added were:

• page.extra.geo to set location info
• page.extra.start_date
• page.extra.end_date
• page.extra.cal_published for DTSTAMP but this maybe better as page.date as mentioned above

[Keats]

I think it would be fine to add but I don't know anything about .ics so I wouldn't really know what we need

[seam345]

I think it would be fine to add but I don't know anything about .ics so I wouldn't really know what we need

At the moment the above template validates against an online validator https://icalendar.org/validator.html?url=https://rustmanchester.co.uk/events/cal.ics

The only new things that are needed are a start and end date, geo is optional and the other things are mostly the same as atom feed like description and title.

[Keats]

Yeah the issue is how to handle the optional things that need to be put in the [extra] section. If it's well documented maybe it's fine

[seam345]

For feeds I prefer to keep the control a bit tight around it. I think adding calendar support would be cool though

Would this all be simpler to add a config to extend templates that render e.g.

additional_templates=["cal.ics", "another_template_file"]

This way it's very explicit that the user is adding a template, unlike my original suggestion of implicitly adding templates if its in feeds making it a less noticeable side effect.

For my config to work I would add cal.ics to feed_filenames and to additional_templates

feed_filenames = [ "atom.xml", "cal.ics"]
additional_templates=["cal.ics"]

bonus points we could change this error Error: Tried to render "cal.ics" but the template wasn't found to let people know about the additional_templates config

[Keats]

Sorry I don't really get what you mean there with the additional_templates.
I think the feed_filenames name is confusing in general, you need to know a filename to say what you want for feed.

[seam345]

A no worries will try explain better,

By default zola/tera only runs templating on a subset of filename extensions (I'm not sure on the complete list).

• We see in Zola doesn't load .txt files as templates in Tera #2768 that the .txt files were not seen as template files.
• Similarly the root cause of my issue for generating a calendar was because .ics files were not seen as template files.

Both #2768 and this issue, result in template wasn't found despite whatever template file existing under the templates folder

My new proposal was to add additional_templates as a config option to expand the templates zola/tera can interpret, in the interest of security and maybe easier implementation instead of wildcard extensions like *.txt or *.ical we can require the full filename be used so other.txt for #2768 or cal.ics my calendar. The new files would still have to exist in the templates folder

With the addition of additional_templates instead of manually adding .txt #2769 to fix #2768 we could have edited the config like

additional_templates = ["other.txt"]

---

Documentation for this config would probably be along the lines of ...

By default Zola only treats *.txt/*.html/*.xml" files as templates, if you have a template ending in another file extensions you will need to add the full filename to additional_templates otherwise you will end up with a template wasn't found error
e.g. if we have a template file for generating a calendar feed templates/cal.ics we need to add cal.ics to additional_templates so additional_templates=["cal.ics"]

todo get full list of template extensions Zola treats as valid

[Keats]

Ah ok I see what you mean. It might be something needed indeed but there might be other options (like eg a feeds folder where we load everything in there etc)

[renanglr]

Supporting feeds with arbitrary extensions would also be useful for adding JSON Feed support (#311)

[seam345]

Happy with any of the solutions suggested, as they all fit my use case :)

• Just adding a calendar feed built in fixes just my issue
• Feeds folder fixes my issue and JSON Feed Option #311
• additional_templates would fix my issue / JSON Feed Option #311 and would have fixed Zola doesn't load .txt files as templates in Tera #2768

if we come to a decision I may get some time next month to work on it :)

---

Speaking with almost no experience I think the additional_templates would be simpler to implement than the feeds folder and keeps the control a lot tighter

[Keats]

Hmm maybe we should just fold the additional_templates into feed_filenames and allow arbitrary names (not globs) and render those?