Skip navigation.
Home

FeedWordPress

Project

Download

FeedWordPress is an open-source Atom/RSS aggregator for the WordPress weblog software. You set up feeds that you choose, and FeedWordPress syndicates posts from those sources into your WordPress posts table, where they can be displayed by your WordPress templates like any other post -- but with additional meta-data about the source of each syndicated post, so that your templates can properly attribute the post to its source.

FeedWordPress was originally developed because I needed a more flexible replacement for Planet to use at Feminist Blogs. You can use FeedWordPress to create aggregator (Planet) sites that bring together posts from many different sources, using the WordPress templating engine to display posts from all around the web. Or you can use it to bring together all your online activity from your blog, Twitter, Flickr, or other online services, into a Lifestream with all your online activity in one place.

FeedWordPress is designed with flexibility, ease of use, and ease of configuration in mind. You'll need a working installation of WordPress or WordPress MU (version 3.0 or later), and also FTP or SFTP access to your web host. You don't need to tweak any plain-text configuration files and you don't need access to shell or crontab on your web host to make it work. (Although, I should mention, web hosts that don't offer shell access and cron jobs are bad web hosts.)

FeedWordPress 2010.0905: bug fixed; Categories and Tags now correctly assigned

Note. As with the previous release, before you download or install, please note that this version of FeedWordPress requires WordPress 3 or higher to operate correctly. Do not install the upgrade unless or until you have made the upgrade to WP3.

FeedWordPress 2010.0905 is now available for download.

I've pushed out this release quickly because it contains fixes to an important issue that y'all helped me discover in early user reports back about FeedWordPress 2010.0903. As I mentioned in the release notes, the new release of FeedWordPress has some extensive internal changes to allow for a great deal more flexibility in how it handles Categories and Tags (up to and including support for custom taxonomies provided by plugins). Unfortunately, the changes introduced a bug, due to the fact that the new method used to insert Categories and Tags in WordPress 3 screens the terms to be inserted by the current logged-in user's capabilities. Which is fine for normal operations within the WordPress administrative interface; but since automatic updates and updates from FeedWordPress cron jobs are typically carried out without any user logged in whose credentials could be checked, this meant that several users noticed posts being inserted with only the category Uncategorized, and with no Tags. Which presents a problem.

Fortunately, the problem has been fixed in today's quick upgrade:

  • BUGFIX: CATEGORIES AND TAGS CORRECTLY ASSIGNED IN AUTOMATIC UPDATES. Version 2010.0903 switched over to a new way of assigning categories and tags as part of its support for handling custom taxonomies. Unfortunately, the method that it uses is subjected to some checks of the current user's capabilities, which creates problems for posts that are being inserted into the WordPress database when there is no current user logged in (as, for example, when an update is being carried out from a cron job or automatic update). The result was that posts from cron jobs and automatic updates ended up with no Categories and no Tags being assigned. This bug has now been fixed: in 2010.0905, Tags and Categories should be correctly assigned to all posts, regardless of whether they were added from manual updates, cron jobs, or automatic updates.

All you need to do to apply the fix is download the newest release and drop it into place in your WordPress plugins directory. As always, if you have any issues with the release, or any questions I can help answer, or if there is anything that you would like to see included in a future release, please use the comments form or drop me a line to let me know about it. If you have an issue to report, please be sure to tell me what version of FeedWordPress you're using, what version of WordPress you're using it with, which web browser you are using to view the FeedWordPress user interface, and try to tell me, as clearly as possible, (1) what you were trying to do, (2) what the circumstances were, (3) what you expected to see, and (4) what you ended up seeing instead.

Download and enjoy!.

FeedWordPress 2010.0903: bug fixes, interface improvements, lower memory load & big new features for convenience & uber-geekery

I am happy to announce that FeedWordPress 2010.0903 is now available for download..

Now stop just a minute. Before you proceed, be advised: FeedWordPress 2010.0903 has been thoroughly tested for compatibility with WordPress 3.0. It also requires WordPress 3.0. If you do not have WordPress 3.0 installed, and aren't yet ready to make the upgrade, you should not attempt to upgrade to this version of FeedWordPress, until you are ready to make the upgrade to WP3.

This release includes a big passle of improvements, fixes, and new features. Some of them have been a long time coming; others are new and fancy. For the fixes, you may be most interested in a fix which apparently solves a really vexing vanishing act that Syndicated Sources performed for some users on various versions of Internet Explorer -- where a big chunk of the user interface would suddenly disappear as soon as you added your first feed. You may also notice some of the new features, like subscribing to multiple feeds or the reductions in memory consumption. Many of the best changes, though you may not even notice -- a great deal of the improvements are in low-level interface work, and if I've done my job right it will make your life a few minutes more pleasant every day, without you hardly even noticing it. Many of them are improvements under the hood, which are mainly intended for uber-geeks, hotrodders, or -- and here is where it matters for the rest of you -- Add-On and Filter Developers, who will have that much more of a chance to enrich the FeedWordPress ecosystem. But there's also plenty that should be right out on the surface, for anyone to see, and I hope you enjoy it.

Here's a breakdown of some of the major changes since the previous release:

  • WORDPRESS 3 REQUIRED: Please note that this release of FeedWordPress requires WordPress 3.0 or later. If you are currently using a 2.x branch of WordPress, you will need to upgrade to WordPress 3 before you can successfully upgrade FeedWordPress.

  • BUGFIX: NO MORE DISAPPEARING "SYNDICATED SOURCES" PANEL; INTERNET EXPLORER UI GLITCH APPARENTLY FIXED: Several users independently reported a problem with FWP 2010.0623 and various versions of IE. A problem with the HTML markup caused IE (but not Firefox or Chrome) to completely hide the Syndicated Sources administration panel (the main list of currently-syndicated sources, and the main location for adding new sources, under the Syndication menu item) when a user added their first syndicated feed. Maddeningly, the glitch seemed to affect some IE users and not others: I was never able to reproduce the problem for myself on my own machines. However, the markup of Syndicated Sources has undergone significant changes and corrections since 2010.0623, and two independent sources who had been having this problem confirm that they no longer encounter it with the updated version. For the time being, I am going to declare this bug squashed.

  • BUGFIX: MORE PROTECTION AGAINST FATAL ERRORS FROM PLUGGABLE VERSIONS OF SimplePie: FeedWordPress now takes some precautions that should help to better avoid conflicts for users who have installed pluggable versions of SimplePie for another plugin or theme. (You may not know that you have done this; but if you've been encountering fatal errors indicating that you cannot redeclare class SimplePie, or something along those lines, there is now a better chance that those fatal errors will be eliminated.

  • PERFORMANCE: SIGNIFICANTLY REDUCED MEMORY CONSUMPTION FOR LARGE UPDATES: FeedWordPress is still a memory-hungry little module, especially when you are dealing with very large feeds. However, users should notice a significant reduction in memory overloads, especially if they update a large number of feeds at once.

  • USER INTERFACE IMPROVEMENTS: Nothing is radically different, but there's been a fair amount of extra spit and polish added, including a convenient new Dashboard widget that may save you a trip to the Syndication menu, a lot of effort to make the relationship between global and feed-by-feed settings more obvious to the user and more easily controllable, to make navigation between settings pages easier, to sand off a few rough edges, and to make other improvements on the margins. I hope you'll like how it looks.

  • ADDING MULTIPLE FEEDS: FeedWordPress now provides a convenient mode for adding multiple feeds at once, using either a copy-and-pasted list, or else an OPML file. Go to Syndication --> Syndicated Sources and check out the two new buttons underneath the New Source input box. When you have to add a number of feeds at once, this can save you considerable time and trouble.

  • IMPROVED HANDLING OF AUTHORS WITH DUPLICATE E-MAIL ADDRESS AND AUTHORS WITH NAMES WRITTEN IN FOREIGN SCRIPTS: WordPress 3 is increasingly picky about what it will accept for new author accounts, and some of the conditions it imposes can cause error conditions that prevent posts from being properly syndicated, or properly attributed, if authors happen to have identical e-mail addresses, or if users are given usernames that are written in non-Western scripts. FeedWordPress now handles these much better, and systematically works to avoid clashes between syndicated authors' account names or in their e-mail addresses, which should result in significantly better results in mapping author names to WordPress user accounts.

  • MAPPING CATEGORIES ON SYNDICATED POSTS TO TAGS NOW BETTER SUPPORTED: In previous versions, the only way for the Categories provided by a syndicated feed to be mapped into Post Tags was to instruct FWP to create new tags, rather than new categories, for unfamiliar categories from the feed. This works fine if you want tags to be the default; but if you want only a specific set of tags, there was no way to get them without getting most or all other categories imported as tags. You can now do this by creating a tag (under Posts ==> Post Tags) before importing the post; when the syndicated category matches a pre-existing tag, the incoming post will be tagged with that tag, without creating a local Post Category.

  • REL-TAG MICROFORMAT SUPPORT FOR INLINE TAGS: Syndicated posts that contain inline tags, marked up using the Rel-Tag microformat http://microformats.org/wiki/rel-tag, are now tagged with the tags provided by Rel-Tag format links.

  • MUCH GREATER CONTROL OVER CATEGORY AND TAG MAPPING: This is partly the result of building in support for a potentially endless set of custom taxonomies (see below), but in general there has been a great deal of effort towards giving you more control over how categories and tags provided by the feed are mapped into terms on the local blog. In particular, you can now force FeedWordPress to create only categories from categories and tags provided by the feed; or to create only tags; or to search both categories and tags for a match; or you can simply force it to drop all of the categories provided by the feed and use only categories or tags that you explicitly provide. In addition, you can now also choose whether to override global categories settings with a local, feed-specific setting; or whether to add together both the global categories and the local feed-specific categories -- depending on whatever your use-case may demand.

  • CUSTOM POST TYPES AND TAXONOMY SUPPORTS: This is mainly for the super-geeky, but if you use other plugins or themes that make significant use of WordPress's support for custom post types and custom taxonomies, you may be pleased to find that FeedWordPress now allows you to feed incoming posts into any custom feed type that you wish, and to map categories and tags from the feed to custom taxonomies as well as to the standard Category and Tag taxonomies.

  • STORING NAMESPACED CUSTOM FEED ELEMENTS IN POST CUSTOM FIELDS: If you would like to use FeedWordPress's support for storing custom meta-data from feed elements in the custom fields for a post (for example, to store geolocation data or iTunes media meta-data), you'll find that it's now much easier for you to access these namespaced elements. You always could access them, but in previous versions you might have to write something ugly like $(/{http://www.w3.org/2003/01/geo/wgs84_pos#}lat) just to get at the value of a <geo:lat> tag. Now, as long as you use the same mnemonic codes that the feed producer used, you should always be able to write a nice, simple expression like $(/geo:lat) to get the value of a <geo:lat> tag. Huzzah!

  • CUSTOM DIRECTORY STRUCTURE SUPPORT: if you poke at it enough, WordPress is relatively flexible about where it should store admin interface code, uploaded content, plugins, and a number of other things that occupy an important place in the WordPress directory structure. Previous versions of FeedWordPress encountered serious errors or broke entirely when used with directory structures other than the default. This should now be fixed: FWP now supports custom directory structures wherever WordPress allows them to be customized, rather than depending on the default locations. Enjoy your freedom!

  • MANY NEW FILTERS AND API UTILITY FUNCTIONS FOR ADD-ON PROGRAMMERS: There have been too many improvements to list them all in this ChangeLog, but it means that much more power and ease for folks who are customizing FeedWordPress through PHP filters or add-on modules. Fuller documentation will be put up at the Wiki at feedwordpress.radgeek.org soon.

Download and enjoy! If you have any issues with the release, or any questions I can help answer, or if there is anything that you would like to see included in a future release, please use the comments form or drop me a line to let me know about it. If you have an issue to report, please be sure to tell me what version of FeedWordPress you're using, what version of WordPress you're using it with, which web browser you are using to view the FeedWordPress user interface, and try to tell me, as clearly as possible,

One of the remarkable fact about this release is the number of the new features in this release -- including improved namespace support, subscription to multiple feeds, and custom post type and taxonomy support -- that I was able to make due to the generous support of FeedWordPress users, whose gifts made it possible for me to devote a several solid days to some major new conveniences and some significant internal overhauls that have dramatically improved FeedWordPres's flexibility and power. I'd like to thank all the FWP users who have helped make ongoing development possible. It's really flattering, y'all, and I'm incredibly grateful that you've made it possible for me to devote the needed time to this project.

As always, please remember that your generous gifts to the project tip jar make ongoing development, quick fixes and timely support for FeedWordPress possible.

Now get out there and enjoy FeedWordPress 2010.0903!

FeedWordPress 2010.0623: fixes some update and UI issues, improves performance

FeedWordPress 2010.0623 is now available for download.

This release addresses some performance issues related to a bug fix that was introduced in FeedWordPress 2010.0528; it also includes fixes to a couple of issues introduced by the transition to SimplePie (related to a frequent error condition in feed updates) and WordPress 3.0 (related to the author mapping interface in Syndication --> Authors). In addition, some users related an odd fatal error related to the Relative_URI class. I am not sure what was causing this error, but whatever it was, it should no longer be a problem, because this release eliminates the dependency on the Relative_URI class to resolve relative URIs (instead, FeedWordPress now uses a perfectly adequate substitute which is built in to the SimplePie module).

Here's a breakdown of some of the major changes since the previous release:

  • WORDPRESS 3.0 COMPATIBILITY / AUTHOR MAPPING INTERFACE ISSUES: I resolved a couple of outstanding issues with the author mapping interface (Syndication --> Authors), which were preventing new users from being created correctly and author mapping rules from being set up correctly. These partly had to do with new restrictions on user account creation introduced in WordPress 3.0; anyway, they should now be fixed.

  • MORE EFFICIENT SYNDICATED URL LOOKUPS: Several users noticed that the bug fix introduced in 2010.0528 for compatibility with post-listing plugins caused a lot more queries to the database in order to look up numerical post IDs from the URL provided to the filter. This shouldn't cause any major problems, but it is not as efficient as it could be; the code now takes advantage of a more efficient way of doing things, which usually will not require any additional database queries.

  • SIMPLEPIE FEED UPDATE ISSUES: If you have been having significant problems with getting feeds to update correctly, this may be the result of some bugs in the implementation of SimplePie caching that ships with WordPress (as of version 3.0). (You would most commonly experience this error if you continually saw errors such as "No feed found at <...>" in your updates.) Fortunately, SimplePie allows for a great deal of extensibility and this allows me to work around the problem; these error conditions should now be mostly eliminated when the underlying feed is valid.

  • UI: SHOW INACTIVE SOURCES: When you use the default unsubscribe option -- which turns off the subscription to a feed while preserving the posts from it and the syndication-related meta-data for the feed -- the unsubscribed feed can now easily be viewed in a special "Inactive" section of the Syndicated Sources page. (As a side benefit, if you've accidentally, or only temporarily, turned off the subscription to a feed, it is now much easier to restore the feed to being active, or to delete it permanently, if you prefer.

  • UI: FEED FINDER / SWITCH FEED INTERFACE IMPROVEMENTS: changes to styling and options for the feed finder / switch feed, which should now make it easier, in some cases, to find alternative feeds, and make interface options more clearly visible.

  • FILTERS: syndicated_item_published and syndicated_item_updated NOW PROPERLY AFFECT THE DATING OF POSTS. These filters used to affect some date-related settings, but not others -- and, most importantly, not the final date that is set for a post's publication or last-modified date in the WordPress database. Now, they do affect that, as they should. (Filters should receive, and return, a long integer, representing a Unix epoch relative timestamp.)

  • MAGIC URL TO CLEAR THE CACHE: Suppose that you need to clear the feed cache, for whatever reason; suppose, even, that you need to clear it on a regular basis. One way you might do this is by logging into the FeedWordPress administrative interface and going to Syndication --> Performance. Another way you might do it, now, is to simply send an HTTP request to a magic URL provided by FeedWordPress: if your blog is at example.com, the URL would be http://example.com/?clear_cache=1

Download and enjoy! If you have any issues with the release, or any questions I can help answer, or if there is anything that you would like to see included in a future release, please use the comments form or drop me a line to let me know about it. If you have an issue to report, please be sure to tell me what version of FeedWordPress you're using, what version of WordPress you're using it with, and try to tell me, as clearly as possible,

Please remember that your generous gifts to the project tip jar make ongoing development, quick fixes and timely support for FeedWordPress possible.

FeedWordPress 2010.0602: more fixes rolled up, and some important changes under the hood

FeedWordPress 2010.0602 is now available for download.

Happy June!

I know a lot of you have questions related to the past couple of releases of FeedWordPress that I haven't yet gotten back to you about. I'm sorry it's taking a while, but I have been reading all the questions and reports coming in, and have been working on resolving a number of issues reported, while also doing a lot of work under the hood in order to improve FeedWordPress's extensibility and to allow add-on modules to more thoroughly extend FeedWordPress's user interface and features. I don't have much to say about the under-the-hood changes just yet (I'll have more to say about them as they prove themselves in practical uses in the future). But for now, I thought it would be important to push out another public release which will wrap up some of the improvements and fixes made in response to user questions and reports. There are several questions and issues still outstanding that I will be working on soon, but my wedding anniversary is coming up and I'll be out of town and away from work for the next few days, so I wanted to make sure that these fixes get out before I leave town. The rest are things that I'll be getting to when I get back this weekend.

Here's a quick breakdown of the changes you might notice between the previous release and FeedWordPress 2010.0602:

  • CATEGORY BOX INTERFACE ELEMENT FIXED FOR WP 3.0: Stylesheet changes between WordPress 2.9.x and the WordPress 3.0 RC caused the Categories box under Syndication --> Categories & Tags to malfunction. This has been fixed.

  • LINK CATEGORY SELECTION BOX IN SYNDICATION ==> FEEDS FIXED FOR WP 2.8 AND 2.9: A WP 3.0 compatibility change introduced in 2010.0531 inadvertently broke the Syndicated Link Category selector under Syndication --> Feeds & Updates in WP 2.8 and WP 2.9, causing the post categories to be displayed in the selector rather than the link categories. This should now be fixed so that the selector will work correctly under both the current versions of WordPress and the 3.0 RC.

  • MORE PERMISSIVE HANDLING OF FEEDS WITH BAD CONTENT-TYPE HEADERS: One of the small advantages that MagpieRSS had over SimplePie is that it was more tolerant about parsing well-formed feeds that the remote web server happened to deliver with weird or incorrect HTTP Content-type headers. In feeds affected by this problem, the new SimplePie parser would simply fail to find a feed, due to its being led astray by the contents of the Content-type header. This version includes an extension to SimplePie's content-type sniffer that offers more permissive handling of the HTTP headers.

  • MORE FULL-TEXT "EXCERPTS" NOW PROPERLY SHORTENED. Version 2010.0528 introduced code to control for cases in which elements intended for item summaries are (ill-advisedly) used to carry the full text of posts; past versions of FeedWordPress would simply include the full text of the post in the excerpt field, but newer versions now attempt to detect this condition when it arises and to head it off, by blanking out the excerpt field and filling it with an automatically generated short, plain text excerpt from the full content. This release broadens the test conditions that indicate when an excerpt field is treated as identical to the full text of the post, and should therefore improve the handling of some feeds (such as Google Reader feeds) where the full text of each post was still appearing in the excerpt field.

  • FILTERS: syndicated_item_published AND syndicated_item_updated FILTERS NOW ALLOW FILTER AUTHORS TO CHANGE POST TIMESTAMPS. You can now use the syndicated_item_published and syndicated_item_updated filter hooks to write filters or add-ons which directly change the post date and most-recently-updated timestamps on incoming syndicated posts. Props to niska for pointing out where the filters needed to be applied in order to change WordPress's internal timestamps for incoming posts.

Download and enjoy! If you have any issues with the release, or any questions I can help answer, or if there is anything that you would like to see included in a future release, please use the comments form or drop me a line to let me know about it. If you have an issue to report, please be sure to tell me what version of FeedWordPress you're using, what version of WordPress you're using it with, and try to tell me, as clearly as possible, what you're trying to accomplish, what you expect to see, and what you see instead when you try to accomplish it (including any error messages you may be seeing). Also, as I mentioned, I will be out of town for the next few days. Please don't let this stop you from sending in your questions -- I'll be reading through everything that came in while I was away. Just be aware that I'll be getting back to you sometime around early next week.

Please remember that your generous gifts to the project tip jar make ongoing development, quick fixes and timely support for FeedWordPress possible.

FeedWordPress 2010.0528: a couple of important fixes to the previous big upgrade

FeedWordPress 2010.0531 is now available for download.

I put out this update quckly because it contains fixes to a couple of important issues that y'all helped me to discover in early user reports back about FeedWordPress 2010.0528. Most of the issues that you reported are issues that are linked, in one way or another, to memory consumption, and to running up against the memory constraints provided by PHP. I will be working on some changes over the next several days to FeedWordPress's internal handling of feed data, and the new MagpieRSS compatibility layer, which I hope will help to alleviate the problems with memory consumption, but I wanted to make sure that this bugfix release was issued as quickly as possible in order to help avert a couple of other problems -- database performance problems during updates, and problems with certain syndicated items not having the syndication meta-data properly attached. (This would result in posts appearing as if they were authored locally, without meta-data about syndication source, with permalinks pointing back to the aggregator website, etc.) Both of these issues were actually caused by a common problem, which I believe has been resolved by the changes in this release. I apologize for the difficulties, and want to thank the users who sent me the detailed, informative bug reports that helped me identify the part of the code that needed to be tested, track down the issue, and eliminate the bug. As a result of one part of this fix, it will be much more difficult in the future for any error condition to result in syndicated posts to lose their custom fields or to end up being improperly sourced in the future.

Here's a quick breakdown of the major changes since the release of FeedWordPress 2010.0531:

  • PERMALINK / CUSTOM FIELDS PROBLEM RESOLVED: An issue in 2010.0528 caused some significant performance problems when updating feeds, and caused some posts to be imported without the proper syndication-related meta-data being attached (thus causing permalinks to point back to the aggregator website rather than to the source website, among other problems). This problem has been resolved (and a fix has been applied which will resolve the problem for any posts affected by this problem, if the original post is recent enough to still be available on the feed).

  • UI: The "Back End" section has been split into two separate sections -- "Performance" (dealing with caching, database index, and other performance tweaks), and "Diagnostics" (dealing with debug mode, update logging, and a number of new diagnostic tests which I will be rolling out over the next few releases).

  • Several minor interface bugs and PHP warning notices have also been eliminated.

Download and enjoy! If you have any issues with the release, or any questions I can help answer, or if there is anything that you would like to see included in a future release, please use the comments form or drop me a line to let me know about it. If you have an issue to report, please be sure to tell me what version of FeedWordPress you're using, what version of WordPress you're using it with, and try to tell me, as clearly as possible, what you're trying to accomplish, what you expect to see, and what you see instead when you try to accomplish it (including any error messages you may be seeing). See also Troubleshooting for more details.

Please remember that your generous gifts to the project tip jar make ongoing development, quick fixes and timely support for FeedWordPress possible.

FeedWordPress 2010.0528: Like four updates in one!

FeedWordPress 2010.0528 is now available for download.

This is a major release with many changes, fixes, and new features. Actually, this post will be covering a lot more than just what went into the most recent public release -- if you compared the version numbers in the Download box to the blog posts, you may have noticed that I put out a good three public releases that didn't have corresponding announcements on the blog. (Versions 2009.1111, 2009.1112, and 2010.0127. The reasons, basically, had to do with major releases being postponed for a couple months while I worked on some major internal overhauls, and then ending up with some important stuff to push out, but not having time to get the package's ChangeLog up to date. Since the announcement posts are based on the ChangeLog, the announcement posts didn't go up either.

So, here is my big, fat list of changes since FeedWordPress 2009.0707, the last version that I wrote a blog post about. Some of the most important changes are not obviously visible to end-users, but will make a very big difference: in particular, FeedWordPress is no longer depends on MagpieRSS for feed parsing; it now uses the SimplePie module included with versions of WordPress since version 2.8. SimplePie is much more up to date than MagpieRSS, handles modern feed formats just fine, and is designed with a great deal of flexibility in mind, so the transition should eliminate some common causes of parsing and update errors; it also means that I no longer have to worry about maintaining an unauthorized upgrade to WordPress core modules, and this should eliminate one of the most common causes of sudden unexpected feed errors (the Upgrade Downgrade problem will no longer exist). SimplePie's greater extensibility will also allow many more options in the future for developing extensions or alternatives for users who prefer different arrangements with regard to feed caching or other tasks related to the parser.

(Don't worry, if you use existing filters and add-ons that depend on MagpieRSS's format for feed data. This version includes a compatibility layer that will mock up Magpie's data format from SimplePie objects, and ensure that existing add-ons and filters do not break.)

Some other important new features include syntax for storing feed meta-data in custom fields for syndicated posts; allowing you to protect manual edits to syndicated posts from being overwritten by updates from the feed; settings to help you control update scheduling; more resilient handling of dates (so that feed formats with unusual or broken date formats should no longer result in posts being dated back to 1969 or 1970), and a number of other interface tweaks, compatibility fixes, and some bug fixes around the edges. Here's a breakdown of the major changes since the release of FeedWordPress 2009.0707.

  • SIMPLEPIE IS NOW USED TO PARSE FEEDS; NO MORE MAGPIERSS UPGRADES NEEDED: One of the biggest changes in this release is that FeedWordPress no longer depends on MagpieRSS to parse feeds, and has switched to the much more up-to-date and flexible SimplePie feed parser, which is included as a standard part of WordPress versions 2.8 and later. Using SimplePie will hopefully allow for better handling of feeds going further, and will allow me greater flexibility in determining how exactly the feed parser will operate. It also means that FeedWordPress no longer requires special upgrades to the WordPress core MagpieRSS files, and should eliminate quite a bit of complexity.

  • MAGPIERSS COMPATIBILITY LAYER FOR EXISTING FILTERS AND ADD-ONS: However, I have also implemented a compatibility layer to ensure that existing filters and add-ons for FeedWordPress which depended on the MagpieRSS data format should not be broken by the switch to SimplePie. Going forward, I recommend that new filters and add-ons be written to take advantage of the SimplePie object representations of items, feeds, etc., rather than the MagpieRSS arrays, but the MagpieRSS arrays will still be available and older filters should continue to work as they have in the past.

  • COMPATIBILITY WITH WORDPRESS 2.9.x and 3.0: This release has been tested for the existing WordPress 2.9.x branch and with the upcoming release of WordPress 3.0. Changes in the user interface JavaScript between WordPress 2.8.x and WordPress 2.9 caused the tag box interface element to break in the Syndication --> Categories & Tags settings page; changes in the API functions for adding new authors caused fatal errors under certain conditions in WordPress 3.0. These breakages have been fixed.

  • DROPPED LEGACY SUPPORT FOR WORDPRESS PRIOR TO 2.8: Because SimplePie is not included with versions of WordPress prior to 2.8, I have chosen to drop legacy support for WordPress versions 1.5 through 2.7. If you are using FeedWordPress with a version of WordPress before 2.8, you will have to upgrade your installation of WordPress in order to take advantage of this release.

  • PHP 5.3 COMPATIBILITY: A couple of compatibility issues, which were causing fatal errors amd ugly warnings for users of PHP 5.3, have been eliminated.

Features and Processing

  • INTERFACE REORGANIZATION: The interface restructuring, began with Version 2009.0612, has been completed. Catch-all settings pages have been eliminated entirely for pages that cover each aspect of handling a feed: Feeds & Updates, Posts & Links, Authors, Categories & Tags, and Back End handling of the database and diagnostic information. Extensive new interface hooks allow add-on modules to significantly change or extend the FeedWordPress admin interface and workflow.

  • STORING INFORMATION FROM THE FEED IN CUSTOM FIELDS: Many users have written to request the ability to store information from elements in the feed in a custom field on each post. (So that, for example, if post includes a itunes:duration element, you could store the contents in a Custom Field called duration on the post (for a Theme to access later). The Custom Post Settings under Syndication --> Posts & Links now allow you to access any item or feed tag, using a syntax similar to a much-simplified version of XPath. See Posts & Links settings for details.

  • UPDATE-FREEZING ON MANUALLY EDITED POSTS: FeedWordPress now allows you to mark posts that have been manually edited, so that the changes you make will not be overwritten by later updates from the feed. If you make manual edits to a particular post, just check the "Manual editing" checkbox in order to protect your changes from being overwritten. If you want to block all posts from being updated after they are imported for the first time, a new "Updated Posts" setting in Posts & Links allows you to freeze all posts from a particular feed, or all syndicated posts.

  • SETTING: FEED-BY-FEED SETTINGS FOR WHERE PERMALINKS POINT TO: You've always been able to tell FeedWordPress whether permalinks for posts should point to the original source of the story or the local copy. Now you can choose different policies for different feeds, instead of one global policy for all feeds. (Of course, you can still use a global default if you prefer.)

  • SETTING: USER CONTROL OVER TIMING BASIS. You can now determine the schedule on which feeds are considered ready to poll for updates -- by default feeds become ready for polling after about 1 hour. You can now increase or decrease the time window under Syndication --> Feeds & Updates. (However, please pay CAREFUL ATTENTION to the recommendations and DO NOT set the scheduling lower than 60 minutes unless you are ABSOLUTELY SURE that you have specific permission from webmaster who provides that specific feed to poll more frequently than that. If you set this too low (and about 60 minutes is the polite minimum if you haven't been given a different figure), most webmasters will consider the frequent hits on their server as rude, or even downright abusive.

  • OTHER SETTINGS: New settings also include the ability to stop FWP from resolving relative URLs within syndicated content, and the ability to choose whether FeedWordPress should indicate the comment feed from the original source, or the local comment feed, when providing the comment feed URL for a syndicated post.

Parsing

  • BETTER DATE HANDLING -- FEWER FLASHBACKS TO 1969 and 1970: FeedWordPress has made some bugfixes and some improvements in the logic for parsing dates. This should allow FeedWordPress to correctly parse more dates in more feeds; and, in the last resort, when FeedWordPress fails to correctly parse a date, to fall back to a more intelligent default. This should hopefully avoid most or all error conditions that have resulted in articles being erroneously dated to the dawn of the Unix epoch (31 December 1969 or 1 January 1970).

  • FULL-TEXT "EXCERPTS" NOW PROPERLY SHORTENED. Based on a straightforward reading of the existing RSS specs, it's reasonable for the rss:description element to be read as a plaintext summary or excerpt for the item containing the description -- with the full text of the item, if available, in another, better-suited element, such as the de facto standard content:encoded extension element. The problem is that uses of RSS rarely have much to do with anything like a straightforward reading of the specs. As a result, many actual RSS producers in the wild put the full text of the article in a description element. But since FeedWordPress has treated this text as a summary, this produces aggregated posts with lengthy "excerpts" containing the full text of the article. This release of FeedWordPress fixes the problem by doing a little digging before treating rss:description as a summary: if the description element is used properly as a plain text summary, then FeedWordPress will take the summary provided by the feed, rather than recreating its own excerpt from the full text; but if an RSS item has no full-text element other than description, FeedWordPress will treat the description element as the full text of the article, and generate a shortened excerpt automatically from that text.

API

  • TEMPLATE API: new template tags get_local_permalink() and the_local_permalink() allow you to access the permalink for a post on your aggregator site, even when FeedWordPress is rewriting permalinks to point to the original source site.

  • NEW HOOKS FOR ADD-ONS AND FILTERS: I have added a number of new hooks which allow add-on modules to filter more precisely, gather information at more points, and to enhance the FeedWordPress admin interface. For a list of new hooks and documentation, see the FeedWordPress documentation wiki at http://feedwordpress.radgeek.com/wiki/add-ons-and-filters

  • FILTER API: A number of new utility methods have been added to the SyndicatedPost class to make it easier for filters and add-ons to

  • FILTER API: Globals $fwp_channel and $fwp_feedmeta DEPRECATED. These global variables, originally introduced to allow filters access to information about the source feed in syndicated_item filters (which were passed in through global variables rather than as parameters because of a bug in WP 1.5 which was then fixed in 1.5.1) have been DEPRECATED. If you have any filters or add-ons which still depend on these global variables, you should see about fixing them to access data about the source feed using the SyndicatedPost::link element instead. For documentation, see the FeedWordPress documentation wiki at http://feedwordpress.radgeek.com/wiki/syndicatedpost and http://feedwordpress.radgeek.com/wiki/syndicatedlink.

  • DIAGNOSTICS: I've included a number of new diagnostic options and messages, which should allow an experienced user to better investigate any problems that may crop up.

Bug Fixes

  • BUGFIX: & IN PERMALINKS NO LONGER CAUSING ATOM OR HTML VALIDATION ERRORS: Many users reported an issue in which syndicating a feed with special XML characters in the URLs (& was the most common, since it is used to separate HTTP GET parameters) would cause the aggregator's feeds to produce invalid (malformed) XML. This update addresses the issue in Atom feeds. Unfortunately, it has not been technically possible to address the problem in RSS 2.0 feeds, due to limitations on WordPress's internal templates for RSS feeds.

  • BUGFIX: BROKEN URLS IN "POPULAR POSTS" AND SIMILAR PLUGINS SHOULD NO LONGER BE BROKEN. A number of users noticed an issue where plugins and templates that listed posts in locations outside of the post loop (for example, "Popular Posts"-style plugins that listed posts in the sidebar), often produced the wrong URL for post links. (Typically, all the posts listed would get the same wrong URL.) This should now be fixed. Thanks to Björn for sending in a quick fix!

  • MINOR BUGFIXES: This release includes a number of fixes to minor bugs and compatibility issues, including: silent failures of the "Syndicate" button, "Illegal Offset Type" error messages from MagpieRSS, and others.

Enjoy! As always, if you have any issues with the release, or any questions I can help answer, or if there is anything that you would like to see included in a future release, please use the comments form or drop me a line to let me know about it.

I want to thank everyone who has contributed to the project tip jar over the past several months. Your generous gifts made it possible for me to take a lot of time out from freelance work in order to push through some really major changes to FeedWordPress over the past few months, and your ongoing support, besides being very flattering, goes a long way to help make development, support, and timely fixes possible. Thank you all!

FeedWordPress 2009.0707: bug fix eliminates blank "Syndicated Sites" problem and fixes tagging widget

FeedWordPress 2009.0707 is now available for download.

This is a release to fix two bugs, one small and slightly irritating, and the other potentially very irritating, which resulted from changes to the AJAX interface elements between WordPress 2.7 and 2.8.

Let's start with the more irritating problem which you should be able to eliminate by upgrading to the most recent release:

  • BUGFIX: WORDPRESS 2.8 AJAX COMPATIBILITY ISSUES RESOLVED (blank or truncated "Syndicated Sites" administration page): Due to changes in the AJAX interface elements between WordPress 2.7 and WordPress 2.8, several FeedWordPress users encountered an issue where the front "Syndication" page in the FeedWordPress administrative interface would come up blank, without the normal "Syndicated Sites" list and "Update" control, or sometimes wth the boxes visible but one or both of them truncated, with only the title bar. This issue should now be resolved: with the new version of FeedWordPress, the compatibility issue that caused the disappearance should be eliminated, and if boxes are shown with only their handle visible, you should once again be able to drop down the rest of the box by clicking once on its title bar.

Thank you to the many users who brought this issue to my attention, and supplied some very useful screen shots which helped me reproduce and track down the origin of the problem.

The second problem was with the Tags widget (found under Categories & Tags, where you can set tags to be applied to all syndicated posts, or all posts syndicated from a particular feed). This was also the result of some rather capricious changes to the AJAX interface framework in WordPress 2.8.

  • BUGFIX: TAG SETTING WIDGET FIXED. Due to changes in interface elements between WordPress 2.7 and WordPress 2.8, people using FeedWordPress with WordPress 2.8 found that the widget for setting tags to be applied to all syndicated posts, or all syndicated posts from a particular feed, no longer displayed "Add" and "Remove" buttons for individual tags. This issue has now been fixed, and the tagging widget should once again work more or less exactly like the tagging widget for individual posts in the normal WordPress admin interface.

Download and enjoy! As always, you have any issues with the release, or if there is anything that you would like to see included in a future release, please use the comments form or drop me a line to let me know about it.

Now that these bug fixes are out of the way, I have some ambitious plans for the next release of FeedWordPress, which I hope to be able to release by early August at the latest. Besides ongoing testing for compatibility with new releases in the 2.8.x and upcoming 2.9 branches of WordPress, I would like to implement a Category Mapping system, similar to the existing Author Mapping system, which would provide users with much more fine-grained, powerful, and sophisticated tools for controlling how categories or tags from feeds are imported into the WordPress database. As I have time to do so, I would also like to include some commonly-requested features, such as allowing users more control over how excerpts are generated and more precise control over the timing of automatic updates. And eventually I hope to be able to replace the aging MagpieRSS feed parser with the more robust, reliable, and flexible SimplePie feed parser that is now shipping with new releases of WordPress.

If you have benefited from the bug fixes that went into this most recent release, or if you are interested in making sure that powerful new features like Category Mapping can be developed for an upcoming release, please remember that it's your generous gifts to the project tip jar that make ongoing development and support for FeedWordPress possible.

FeedWordPress 2009.0618: bug fix eliminates HTTP request failures when FeedFinder uses WP_Http_Fsockopen

FeedWordPress 2009.0618 is now available for download.

This is a bug fix release. Other than a few mainly cosmetic code clean-ups, there is only one change between this release and the previous release, 2009.0613. But it's an important one.

After upgrading to WordPress 2.8, many users reported encountering the following error whenever they attempted to add a new feed through FeedWordPress's feed-finder interface:

HTTP request failure

:

HTTP Transports available:

  1. WP_Http_Fsockopen

I reported earlier that this error was the result of some changes in the HTTP request functions provided by FeedWordPress (changes to the order in which different HTTP transports are used; changes which I think were rather ill-considered). After managing to replicate the problem and investigating more deeply, I found out that it was actually the result of a combination of two factors -- the changes in WordPress 2.8 (which I still think were a mistake) and also a subtle bug in FeedWordPress's feed-finder code, which the changes in the HTTP transport code ended up exposing. In any case, the subtle bug has now been fixed, and with it, the source of most of these HTTP request errors should be eliminated.

So, if you've been encountering this problem, be sure to download the most recent FeedWordPress. Then, after it's installed, make sure you also upgrade your MagpieRSS to the version included with the most recent FeedWordPress. The fix is included in the MagpieRSS code, and will not be applied unless and until you upgrade your MagpieRSS to version 2009.0618.

Download and enjoy! As always, you have any issues with the release, or if there is anything that you would like to see included in a future release, please use the comments form or drop me a line to let me know about it.

Please remember that your generous gifts to the project tip jar make ongoing development and support for FeedWordPress possible.

FeedWordPress 2009.0613: minor UI glitches fixed and improved diagnostics for feed problems

FeedWordPress version 2009.0613 is now available for download.

This is a very minor update, which I guess should not be too much of a surprise, if you take into account the fact that I made a major release just yesterday. However, due to some difficulties that some people have been having, I thought it would be a good idea to push out a release with better diagnostics.

The basic issue is this: until WordPress version 2.7, FeedWordPress used the Snoopy HTTP library (which is included in WordPress) to fetch feeds for updating, and also for the feed finder that helps you subscribe to new feeds in the administrative interface. With WordPress 2.7, however, the WordPress team decided to deprecate Snoopy -- so it's still included with WordPress, but not used internally, and it may be dropped from future releases -- and to introduce a new custom API for HTTP requests (the WP_Http class and some wrapper functions that make use of it). So, if you're using WordPress 2.7.x or 2.8, FeedWordPress uses the new functions rather than the deprecated Snoopy module. One of the advantages of the new code is that it is supposed to be able to make use of any of several different HTTP transport APIs which may be available, depending on your PHP set-up (for example libcurl, fsockopen, URL support in fopen, etc.). But I've been noticing problems that many users have had that tie back to problems with the underlying HTTP transports used by WordPress's new code, and, for reasons that are unclear to me, the WordPress development team decided to make some changes in WordPress 2.8 which make these problems even more likely to occur and even harder to get around. In any case, this is almost certainly the underlying issue if you, like others, are encountering something like this when you try to syndicate a new feed:

Error: I couldn’t find any feeds at <http://example.com/> [HTTP request error: :]. Try another URL

The best solution for you will depend on details about your own hosting situation -- in particular, what version of WordPress you are using FeedWordPress with, and whether or not you are able to install new PHP modules on your web host, or, if not, whether or not you have someone who is willing to install new PHP modules for you. Unfortunately it's not likely to be something that FeedWordPress itself can fix. But I have made some updates so that FeedWordPress will, at least, provide some more useful diagnostic information -- which may help you figure out what needs to be done, or which will hep me help you figure it out, if you send the diagnostic information to me along with your support request.

So, anyway, all that said, that's why I'm pushing out a new release today. (There are also a couple other minor changes included in the release, but I would probably not have bothered with a public release just yet except for the number of support requests I've gotten since yesterday's release, which the diagnostics would help with.) Other than some under-the-hood re-arranging of the code, here are the significant changes between 2009.0613 and the previous releas, 2009.0612:

  • INTERFACE/BUGFIX: WORDPRESS 2.8 CATEGORY BOX FIX. Thanks to a subtle change in class names between the WordPress 2.7 and 2.8 stylesheets, category boxes in the FeedWordPress settings interface tended to overflow and have a lot of messy-looking overlapping text under WordPress 2.8. This has now been fixed.

  • FeedFinder FAILURE DIAGNOSTICS: When FWP's FeedFinder fails to find any feeds at a given URL (for example, when you are trying to add a subscription through the administrative interface and you run into an error message), FeedWordPress now provides more diagnostic information for the reasons behind the failure. If that helps you, great; if not, it should help me respond more intelligently to your support request..

Download and enjoy! As always, you have any issues with the release, or if there is anything that you would like to see included in a future release, please use the comments form or drop me a line to let me know about it.

Please remember that your generous gifts to the project tip jar make ongoing development, quick fixes and timely support for FeedWordPress possible.