Description is not correctly parsed

Can you try adding debug="2" to the shortcode and see what the debugger is showing for the descriptions on those events for the ICS Parser Data? I want to find out if it’s the ICS Parser library itself that is cutting off these descriptions or if it’s something my plugin is doing to them after they’re parsed.

Hi,
here you can download the debug-data. It is too long to copy/paste it here directly:
https://greendroid.de/uploads/ICSCalender-DebugData.txt
Thanks and best regards
Greendroid

Thanks… it looks like the ICS Parser library itself is cutting off the descriptions. I’ll investigate whether or not there’s anything I can do about that.

Hi,
Thank you, I am curious to see what comes out of it.
Best regards
Greendroid

I’m attempting to implement a workaround for this issue, based on this:

https://www.stevefenton.co.uk/2010/11/Adding-Multiple-Lines-To-Description-In-ICalendar-Files/

Have a look at it though… I think the way the line breaks are handled in the DESCRIPTION fields in your feed is non-standard. Are you using this plugin to generate the source feed?

https://www.ads-software.com/plugins/wp-caldav2ics/

If so, you may want to open a support ticket with the developer of that plugin to see if there are any known issues with line breaks in the DESCRIPTION field in the feeds it generates.

OK, I was able to put together a special function that seems to resolve these hard line breaks in the DESCRIPTION fields in your feed. I’ve added it to the plugin with a new shortcode parameter option: linebreakfix="true"

Once you’ve updated to version 6.4.2.2, add that parameter to your shortcode, and it will do a bit of extra processing of the feed to remove those hard line breaks before sending the data to the ICS Parser library.

I tested this change using your feed on my test site, and it seems to be working properly.

Hi,
thanks for your support!
I have already spoken with the developer of the plugin. He is mentioning, that those line breaks are allowed according to the ICS-standard and are created by the CALdav server. His plugin is not influencing this.
I am using Hetzner Webmail Horde to generate my ics files. Even if I use the built-in export function in Hetzner Webmail Horde, these strange line breaks are included.

I have already spoken with the Hetzner support but they are also just including the Webmail service to their webhosting platform and cannot change the library code.
According to the ICS validator website these line breaks seems to be allowed… Strange…

I have checked your update. Thanks for this!
Unfortunately I still have some issues:

• Is it possible, that these “unwanted” line breaks are now replaced by a space? This is not working properly, because now some links or text is delimited with a space. But all in all, the information are shown now! ??
• One of our calendar is not showing up at all if the linebreakfix=”true” is included.

I have built up the following on our webpage:
First calendar: With linebreakfix=”true”
Second calendar: Without linebreakfix=”true”

As you can see, the “Mondkalender” is not shown in the first calendar anymore.
As you can see, in the appointment on 1st Feb “Alte Blütenst?nde schneiden” the space is include in the link. This is the same on the 1st March “Pflanzenvermehrung”. In the 3rd line there is a space in the word.

Thanks for your support and kind regards
Greendroid

• This reply was modified 4 years, 1 month ago by Greendroid. Reason: Typo

Line breaks are allowed in the iCalendar spec, but they have to be “folded,” and it appears your feed is not doing that properly:

https://icalendar.org/iCalendar-RFC-5545/3-1-content-lines.html

I think I may be able to modify yesterday’s change to insert this “folding” rather than what it’s doing right now, and that may resolve the issue. I’ll have another look at it. But I think it’s important to note that something is introducing incorrect formatting into your feed. This workaround I’m doing should not be necessary.

Hi,
thanks. Now I got what you where mentioning.
I will forward this to the Hetzner Horde Webmail support, because they are causing the issue. Unfortunately this is really the root of my data, so it is depending on my webhoster to fix this issue. Lets see what happens…
Kind regards
Greendroid

I’ve just spent some time reworking the function I introduced yesterday. Version 6.4.2.3 with this new fix should be available now.

The approach I took yesterday was a bit clumsy. This time around I’ve modified it to, I think, more accurately simulate the proper line folding that’s missing. It seems to be working now with your feed on my test site.

The trick was catching when the line split is in the middle of a URL, vs. a word break. I think I’ve got this issue resolved, but it may need a bit of additional refinement, especially since technically according to the spec, a line break can happen in the middle of a word.

The main problem we have is that your feed is missing the space characters at the beginning of the “folded” lines. In cases where there should be a space between words at the same point, the line should have two blank spaces at the beginning. I’ve tried to accommodate for that, while also not inserting spaces in the middle of URLs.

It seems to be working, but like I said, it may need some additional refinements, so if you notice any descriptions that are still not displaying properly, let me know what the day/event is and I will have a look.

One thing I wanted to point out: As I was testing I discovered what I thought was an error in my code, but it appears you may have a space in the middle of one of your URLs in an event: “Pfirsichb?ume schneiden” on April 1.

• This reply was modified 4 years, 1 month ago by room34.

Hi,
thank you very much for the workaround. Now it is working fine. I could not find any description which is not shown correctly anymore. Wow!

I have already written an detailed report to my webhoster, I hope, that they will include a fix as well.

Please apologize the space in “Pfirsichb?ume schneiden”. This was not intended, sorry. I hope you have not spent too much time in error searching for this one…

But one thing is still strange. As already mentioned two posts above, one calendar is not shown anymore if I include the linebreakfix in the shortcode:

I have built up the following on our webpage:
First calendar: With linebreakfix=”true”
Second calendar: Without linebreakfix=”true”
As you can see, the “Mondkalender” is not shown in the first calendar anymore.

Can you take a look why this is not showing up anymore? The calendar behind is: https://www.greendroid.de/uploads/Greendroid_Mondkalender.ics

Thanks in advance and best regards
Greendroid

• This reply was modified 4 years, 1 month ago by Greendroid.

I think this may be an issue with the calendar itself. I had a look at the data in the raw ICS feed, and almost all of the events in it have DTSTART dates in the distant past. (From 1916 to 1996!) The only event in the feed that has a date in this century is in May 2021, and I do see that one showing up when I load the calendar in a test page on my server.

Hi,
Thanks for further investigating the issue. I could narrow it more:
The “Mondkalender” calendar is the calendar which was created at the very beginning. This is also the only calendar which includes specific times and therefore also a timezone. All the other calendars are only using full-day events.
For testing I have created the following file:
https://www.greendroid.de/uploads/Greendroid_Mondkalender.ics

There you can see the first events with a timezone (Europe/Berlin):

BEGIN:VEVENT
DTSTART;TZID=Europe/Berlin:20210101T000500
DTEND;TZID=Europe/Berlin:20210102T043000
DTSTAMP:20210210T161757Z
CREATED:20210121T165933Z
LAST-MODIFIED:20210201T205908Z
SUMMARY:?? Pflanzzeit
DESCRIPTION:https://greendroid.de/blog/mondkalender
CLASS:PUBLIC
STATUS:CONFIRMED
TRANSP:OPAQUE
END:VEVENT

After this event the timezone description is coming (don’t know why they are so many, but ok…). After these 170 lines of timezone description the rest of the events are following. They all have a valid timestamp in 2021.

All the events are shown perfectly fine when I do not enable the linebreakfix in the shortcode. Please see the second calendar on my webpage.
If I now enable the linebreakfix=”true” in the shortcode (see the first calendar on my webpage) all events with a timezone marker in the DTSTART are not shown anymore.

For more visibility I have changed the timezone from the event on the 16th May and all events from the 22nd November to 31st of December to Standard and therefore removed the Europe/Berlin. Now they are showing up again.

This let me trace the error back to your linebreakfix=”true” shortcode option. I think if this additional parsing is enabled all events with a timezone are not parsed correctly anymore.

As I would like to have the Timezone in these events, could you take a look why these events are not shown when the linebreakfix=”true” is enabled?

Thanks and kind regards
Greendroid

• This reply was modified 4 years, 1 month ago by Greendroid.

Thanks… this tells me exactly what the issue is. The linebreakfix workaround is built on the assumption that all of the field definitions start with an all-uppercase text string followed by a colon. These DTSTART and DTEND fields with a semicolon and the timezone are breaking that rule.

I think it would suffice for me to change the rule to look for lines that start with an all-uppercase string followed by a colon or a semicolon.

However… I think you may not need this at all anymore; it looks to me like your source feed for the other calendar (the one I created this setting for) has been fixed and now includes the required spaces at the beginnings of the “folded” lines.

Try your calendars without the linebreakfix setting now and they should be correct.

(I’m still going to work on this though since I’ve put the time into it… it may come up again for someone else.)

Hi,
again: Thank you very very much for the support and your workaround.
The file you had for testing yesterday was a little modified one and exported via Thunderbird. It was not created directly from Hetzner or created automatically via the WP-CalDav2ICS plugin.
If I use Hetzner or the plugin I still need the linebreakfix.
I am still in discussions with the Hetzner support to show them the issue we have here.
Very strange why they are not handling it according to the standard.
Now I have included the automatic-generated ICS files and they show still the same CalDav parsing issue from Hetzner.
But the linebreakfix is working pretty good now. I just could find one single problem:
In the following file:
https://www.greendroid.de/uploads/Greendroid_Gartenarbeiten.ics
on the date: 1st March in event “Einj?hrige auss?en” in the description there is a missing space “WeiterführendeInformationen”.

But thats all what I have found.
Hopefully I can get a proper fix by my webhoster to get rid of that issue at all. But I really appreciate your work here. Thanks a lot.
Kind regards
Greendroid