FeedWordPress 2009.0613: minor UI glitches fixed and improved diagnostics for feed problems

indi,

Thanks for posting this. As Zemalf mentioned below, you can actually edit link_rss from within the advanced settings on the link editor. That's a good general work-around for any case in which the feed-finder breaks on some feed that you know to be valid.

On the specific issue of the HTTP request failure errors related to WP_Http_Fsockopen, that issue has now fortunately been solved in the most recent release of FeedWordPress; if you install the upgrade, this work-around should hopefully no longer be necessary.

Zemalf,

Thanks for posting this.That's a good general work-around for any case in which the feed-finder breaks on some feed that you know to be valid.

On the specific issue of the HTTP request failure errors related to WP_Http_Fsockopen, that issue has now fortunately been solved in the most recent release of FeedWordPress; if you install the upgrade, this work-around should hopefully no longer be necessary.

John,

If FeedWordPress fails to pick up any posts from a feed even after you manually set up a subscription using Zemalf's work-around method, then it's possible that there is a problem with the feed which is preventing MagpieRSS from parsing the feed. (In particular, MagpieRSS cannot at present handle feeds with ill-formed XML, which can result from unbalanced tags, invalid character entities, or character encoding problems.) Could you e-mail me the URL of the feed that you are having trouble syndicating so that I can run some tests on it? Or, alternatively, you might run the URL through http://feedvalidator.org/ yourself to check it for well-formedness.

Patrix,

This upgrade does not yet provide an off-the-shelf method for stripping out categories. I have plans for adding a category-mapping feature which will allow you to set things up like that, but that had to be deferred until the next public release. (Which, unless there are emergency bug fixes that need to be pushed out, you can probably expect sometime in August.)

In the meantime, it is possible for you to accomplish what you want to accomplish by installing a custom filter for FeedWordPress, written in PHP. I'll contact you by e-mail with the details of how to do this.

Lowell,

The problem is not with MagpieRSS; unfortunately, the problem is, as far as I can tell, in the WordPress HTTP API. The best way for you to fix this is going to depend in part on details about your web hosting arrangement. Are you able to install new PHP modules on your web server? If you're not able to do it directly, is there someone at your web hosting provider who would be willing to do it for you on request? Let me know whether either of these things is true, and I'll try to offer some advice on possible solutions, given the details about your situation.

Lowell,

As an update to my previous follow-up: it turns out that the problem was partly with the WordPress HTTP API and partly with some of the code that shipped in the MagpieRSS module (although not with the Magpie parser itself). In any case, it should now be more or less fixed in the most recent version of FeedWordPress (2009.0618).

Hope this helps!

William,

Thanks for the report. The diagnostic information that you and other users sent in has been very useful in tracking down the source of the problem.

I believe that you should be able to eliminate this problem by upgrading to the new release of FeedWordPress that I put out yesterday: http://projects.radgeek.com/2009/06/18/feedwordpress-20090618/.

Hope this helps.

Davion,

Unfortunately, the problem here is, as far as I can tell, in the WordPress HTTP API. The best way for you to fix this is going to depend in part on details about your web hosting arrangement. Are you able to install new PHP modules on your web server? If you're not able to do it directly, is there someone at your web hosting provider who would be willing to do it for you on request? Let me know whether either of these things is true, and I'll try to offer some advice on possible solutions, given the details about your situation.

Davion,

O.K., change of plan. I did some more investigating and figured out a solution to the problem which should work regardless of the HTTP transport modules that you have installed. You should be able to eliminate this problem by upgrading to yesterday's new release of FeedWordPress. (Make sure to re-upgrade MagpieRSS; the fix is in some code in rss.php, so you need to replace the older MagpieRSS files in order for it to take effect.)

Hope this helps!

epercpetion,

If FeedWordPress fails to pick up any posts from a feed even after you manually set up a subscription using the work-around method, then it’s possible that there is a problem with the feeds which is preventing MagpieRSS from parsing the feed. (In particular, MagpieRSS cannot at present handle feeds with ill-formed XML, which can result from unbalanced tags, invalid character entities, or character encoding problems.) Could you e-mail me the URL of the feed or feeds that you are having trouble syndicating so that I can run some tests on it? Or, alternatively, you might run the URL through http://feedvalidator.org/ yourself to check it for well-formedness.

Karsten,

FeedWordPress allows for you to implement filters on post content, or on the entire set of feed elements, by coding PHP functions to hook on to specific filter hooks that FeedWordPress triggers. (In particular, the hooks you want are syndicated_item_content, which operates on a string of text, or just syndicated_item, which operates on an array in the format used by MagpieRSS.

For a simple content filter, you would need to create a PHP file formatted as a WordPress plugin. So it would look something like this:

    <?php
    /*
    Plugin Name: Content and item filters
    Plugin URI: [uniquely identifying URL]
    Description: [whatever]
    Version: xx.yy
    Author: [whatever]
    Author URI: [whatever]
    License: GPL
    */

/**
 * Filter content of incoming posts.
 *
 * This manipulates the content of an incoming post and returns the
 * filtered value to be stored.
 *
 * @param string $content Content of the syndicated post.
 *  May have been manipulated already by upstream filters.
 * @param object $postObject of class SyndicatedPost.
 *  Can be used to get information about other elements of the item to be
 *  syndicated, or about the feed it is being syndicated from.
 * @return string The content, appropriately filtered.
 */
function syndicatedItemContentFilter ($content, $postObject) {
    // Insert some PHP code here to manipulate the value of $content
    // according to your whims. What you return will be stored as
    // the content of the syndicated post.

    return $content;
} /* syndicatedItemContentFilter () */

/**
 * Filter one or more elements in a feed item.
 *
 * @param array $item Representation of a single item on the feed,
 *  as represented by MagpieRSS.
 * @param object $postObject of class SyndicatedPost.
 *  Can be used to get more information about the item or
 *  information about feed this item is being syndicated from.
 * @return mixed An array with modified elements will be treated as
 *  the new official representation of the item to be syndicated.
 *  Use NULL to force FeedWordPress to drop that particular item
 *  without syndicating it. (However, if the item has already once
 *  been syndicated, returning NULL here will NOT cause it to be
 *  deleted.)
 */
 function syndicatedItemFilter ($item, $postObject) {
     // MagpieRSS stores elements in the array according to name,
     // namespace, and hierarchy.
     // $item['title'] = the contents of the <title>...</title> element.
     // $item['link'] = the contents of the <link>...</link> element
     // $item['author_name'] = the contents of the
     //     <author><name>...</name></author> child element
     // $item['dc']['creator'] = the contents of the namespaced
     //     <dc:creator>...</dc:creator> element
     //
     // The atom:content element has been forcibly renamed in order to
     // prevent collisions with the namespace element for content:encoded:
     // $item['atom_content'] = the value of the <content>...</content>
     //     element in an Atom entry
     //
     // Elements that can occur multiple times, such as <category>, are
     // stored using a counter in the element name.
     // $item['category#'] = the number of <category> elements on this item
     // $item['category'] = the contents of the first
     //     <category>...</category> element on this item
     // $item['category#2'] = the contents of the second
     //     <category>...</category> element on this item
     //
     // Element attributes can be accessed using the @ operator.
     // $item['category@'] = a comma-separated list of all the
     //     attributes on the first <category>...</category> element on this item
     // $item['category@term'] = the value of the <category term="..." />
     //     attribute on the first <category>...</category> element on this item
     // $item['category#2@term'] = the value of the <category term="..." />
     //     attribute on the second <category>...</category> element on this item.
     // $item['media']['content@width'] = the value of the <media:content width="..." />
     //     attribute on the first <media:content>...</media:content>
     //     element on this item.
     //
     // MagpieRSS normalizes several elements so that both the RSS
     // element name and its Atom equivalent (or vice versa) are
     // present regardless of the original format of the feed. So
     // most element names will work either using RSS names or their
     // Atom equivalents. However, note that some content elements,
     // like RSS 0.9x, 1.0 and 2.0 <description>...</description>
     // elements, are NOT normalized, because they have importantly
     // different semantis from <content:encoded> or <atom:content>.

     $item['atom_content'] = 'A new and fascinating filtered content.';
     $item['title'] = 'A new and fascinating title.';

     return $item;
 } /* syndicatedItemFilter() */

 add_filter (
     /*hook=*/ 'syndicated_item_content',
     /*function=*/ 'syndicatedItemContentFilter',
     /*priority=*/ 10,
     /*arguments=*/ 2
);
add_filter (
    /*hook=*/ 'syndicated_item',
    /*function=*/ 'syndicatedItemFilter',
    /*priority=*/ 10,
    /*arguments=*/ 2
);
?>

Put this in a plain text file, fill in legitimate values for the meta-data at the top, modify what needs to be modified to do what needs to get done in the code, drop it into your WordPress plugins directory, and when ready, activate it like a normal plugin.

Hope this helps. If you need any help on squirrely details of how the filters work, feel free to drop me a line.

Mark,

Thanks for the report. The diagnostic information that you and other users sent in has been very useful in tracking down the source of the problem.

I believe that you should be able to eliminate this problem by upgrading to the new release of FeedWordPress that I put out yesterday: http://projects.radgeek.com/2009/06/18/feedwordpress-20090618/.

Hope this helps.

Joss, "Mismatched tag" indicates an XML error in the feed, which is causing the feed not to validate as well-formed XML. Unfortunately MagpieRSS cannot parse ill-formed feeds.

Try running the feed URL through http://feedvalidator.org/ to get a more detailed report on what's going wrong and where.

Nicholas,

FeedWordPress syndicates whatever is available on the feed, but most feed producers only put something like the 10-25 most recent items on the feed. If the feed producer does not provide a source to get older posts, FeedWordPress doesn't have any good way of getting at that data.

Some feed producers allow you to get a full feed that exports everything that matches the selection criteria, rather than just the N newest posts that do; you'd have to look around the site that you're trying to syndicate data from and see if they produce any feeds that give you that option. If so, my suggestion would be to syndicate once from the full feed, so as to fill in all the old articles, and then turn off updates for that full feed, and rely on the 10-newest or 25-newest or whatever feed to bring in new articles -- you don't want FeedWordPress having to run through hundreds of posts on some monster feed every time it tries to do an update!

Mike,

On the HTTP request failure: thanks for the report. The diagnostic information that you and other users sent in has been very useful in tracking down the source of the problem.

I believe that you should be able to eliminate this problem by upgrading to the new release of FeedWordPress that I put out yesterday: http://projects.radgeek.com/2009/06/18/feedwordpress-20090618/.

On the blank Syndicated Sites page: it looks to me like I failed to notice a problem with the AJAX code that handles the interface boxes. I am investigating how to fix this. In the meantime, though, you should be able to clear out the problem and restore the boxes to their default display by issuing the following query directly to your MySQL database:

    DELETE FROM wp_usermeta WHERE LOCATE('box', meta_key) AND
    LOCATE('feedwordpress', meta_key);

Try that out and let me know how it works for you. In the meantime, I'll research a more permanent solution.

Hope this helps.

Mike and mjcason,

By way of follow-up: the AJAX issue that was causing the blank syndication.php page has now been permanently (I hope) fixed in the most recent release of FeedWordPress (2009.0707), which I made available for download last night. For details, see: http://projects.radgeek.com/2009/07/07/feedwordpress-20090707/.

Hope this helps!

-C