Black diamonds are back in the RSS feed widget

KOM

The funny characters that looks like a question mark inside a black diamond are back in the RSS widget in 2.2-RELEASE.

phil.davis

I don't think the cause/fix was ever found in 2.2-RC. The "diamonds" occur when various escape whitespace sequences occur in the posted data that feeds through to the RSS display. It depends on how the original HTML in the blog post/s has been formatted.

KOM

Oh, ha. I remember you saying that you were going to look into it, and then a build or two later they disappeared. I just assumed that you found the issue and fixed it. There is no way to sanitize the input to remove these funny escape chars? 2.1.5 never had this issue, and I would assume it's using the same RSS data.

phil.davis

I hunted around for a while looking for differences in the 2.2 code that reads and parses the RSS fed content compared to 2.1.5 and nothing stood out. I put a few 2.1.5 PHP files as-is onto a test 2.2-RC system to just see if any would fix it, but I got no joy. It uses /etc/inc/simplepie.inc to do a bunch of RSS feed gathering and parsing.
I guess something in simplepie combined with a newer PHP version has resulted in different behavior.
At that point I decided the "diamonds" could stay a "feature" and got on with debugging other widgets.

If someone else wants to try and track this down, then feel free :-\

KOM

Thanks for your efforts anyway. Aren't computers fun? And by 'fun', I mean "mind-numbingly, hair-tearingly frustrating!!!!"

Gertjan

For what its worth:
https://blog.pfsense.org/ is outputting UTF-8
The RRS widget using SimplePie, content coding is set to "latin-1".
Somehow, SimpePie doesn't convert well.

Open /usr/local/www/widgets/widgets/rss.widget.php
Change line 160 to
// $feed->set_output_encoding('latin-1');
Save.

Bye bye diamonds ;)

Btw: I only tested this with https://blog.pfsense.org/ - other RSS feeds might give 'other' results.

Extra info:
The PHP version that pfSense is using right now, 5.5.19, has full 'mbstring' (mb_convert_encoding())support.
Forcing the output to 'latin-1' doesn't seem to be needed.

phil.davis

Thanks, another piece of crud fixed. I made a pull request for that https://github.com/pfsense/pfsense/pull/1465 - and credited you with the fix, hope that is OK.

Gertjan

… and it's merged :)

I was actually still trying to 'understand' how SimplePie converts between 'input' and 'output'.

But, true, the 'fix' works for me

phil.davis

@Gertjan:

… and it's merged :)

I was actually still trying to 'understand' how SimplePie converts between 'input' and 'output'.

But, true, the 'fix' works for me

It works fine for the default pfSense blog feed, so that is the main use of it, I guess.
If others set other feeds in there and find a problem they can report it.

Gertjan

Saw this : Widget "Captive Portal Status" small bug ?
Another one line visual bug ;)