FeedWordPress 2017.1020. New Boilerplate / Credits feature rolled into main settings interface, code modernization, PHP compatibility fixes, and more.

To-day I released a new version of FeedWordPress, version 2017.1020. This version includes one major visible new feature in the plugin, and a lot of smaller changes, some of them bug fixes and others compatibility tweaks. Download, test it out, and enjoy.

This release includes a major new feature, in a certain sense: you can now set up Boilerplate / Credit attribution text for syndicated posts directly from the FeedWordPress admin interface, without having to touch your template or theme files. This functionality has been available for a long time by installing a separate “experimental” add-on, FWP+: Add Attribution; but after several years of recommending the same add-on to multiple correspondents, with its feature set remaining largely unchanged, I figured it was long since time to incorporate the “experiment” into the off-the-shelf code.)

FeedWordPress is now an extremely … “mature” project, at over 12 years old. I have been working on it on and off as time, academic obligations, and the rest of life allowed, but I regret that I haven’t been doing a very good job here of documenting the changes as they’ve been made, or more actively enhancing and improving the product above and beyond responding to relatively simple bug reports. In the next few months, I expect to have significantly more time and mind space to recommit to keeping this project not just functional, but useful, enjoyable and up to date with ongoing development in the WordPress world. Part of that is going to be making an effort to keep up more with documenting the progress here on this website, rather than simply syndicating in batches of geek-talk from the changes I make behind the scenes at Github or in the WordPress plugins repository SVN. So, in the interest of catching up a bit on that, here’s some of the significant changes that have occurred over the past year in FeedWordPress:

2017.1020

  • ADD BOILERPLATE / CREDITS FEATURE AVAILABLE IN POSTS & LINKS SETTINGS PANEL. I have added a new settings panel to the off-the-shelf features of FeedWordPress, under Syndication > Posts & Links (or under the Posts settings page for any individual feed), which allows you to define boilerplate text that should appear in connection with every syndicated post, or with every post syndicated from a particular feed. So, for example, if you want each syndicated post to include a byline reading “This is a syndicated post, reprinted from (LINK TO ORIGINAL SOURCE WEBSITE).”, you could set up this byline from within the FeedWordPress settings interface, by going to the Boilerplate / Credits panel, and adding a line to appear BEFORE the CONTENT of each syndicated post, using the text and shortcode “This is a syndicated post, reprinted from [1].” For those of you who have corresponded with me about this feature before, you may be familiar with it from the long-standing “experimental” add-on, FWP+: Add Attribution; I’ve decided that it’s been enough years, and I’ve had enough requests, that the Add Attribution feature may as well be included in the main FeedWordPress code.

    Back when FeedWordPress was first created, the assumption was that a well-behaved aggregator would include boilerplate text to indicate the source of syndicated posts, but that the best way to do this was to provide a set of syndication-specific template tags so that the administrator setting up the site could edit their Theme template files with constructs like:

    <?php if (is_syndicated()) : ?>
    <p class="byline">This post by <?php the_author(); ?> originally
    appeared at <a href="<?php the_syndication_source_link(); ?>"><?php
    the_syndication_source(); ?></a>.</p>
    <?php else : ?>
    <p class="byline">By <?php the_author(); ?>.</p>
    <?php endif; ?>
    

    You can still do this, of course, and for maximum expressive power and flexibility, it is certainly the best way to do it. Template Tags are documented here: http://feedwordpress.radgeek.com/wiki/templates/ However, (1) it requires writing PHP code, which not everyone is comfortable doing; and (2) it requires altering template files within your Theme, which is not always possible, especially given the increasing role that prepackaged commercial themes have come to play in the WordPress ecosystem. So, now, you can get some basic features for adding boilerplate text and attribution credits even without touching your template files, and without having to add custom add-ons for FeedWordPress. Enjoy!

  • MINOR CODE MODERNIZATION, PHP 7.1 COMPATIBILITY AND BUG FIXES. This project is now 12+ years old (good lord), and there are still some places where code was written at a time when PHP was a very different language from what it is now. Props to @david-robinson-practiceweb for pointing out and sending a pull request to fix some instances where obsolete PHP reference notation (&$q on parameters and so on) created a compatibility problem for PHP 7.1. Props to an email correspondent for pointing out a place in SyndicatedPost where excerpts should be generated from post content using encoding-aware mb_substr(), instead of naively running them through substr(). I’ve begun making some efforts throughout to begin auditing some of the creakiest old code in the project, to update what needs updating and improve documentation throughout.

2017.0913

  • PARTIAL FIX FOR 2X DUPLICATE POSTS APPEARING ON DUAL HTTP/HTTPS SITES: Some users reported an issue in which their FeedWordPress sites, which are over both insecure HTTP and over HTTPS, would pick up exactly 2 copies of every post or almost every post from certain feeds, and where the guids for each of the pair of duplicate posts would look exactly alike, except for a difference in the protocol, for example:

    http://www.example.com/?guid=c1cd28da39e8d7babcf6499983aca545 
    https://www.example.com/?guid=c1cd28da39e8d7babcf6499983aca545
    

    … where www.example.com is the server that your own copy of FeedWordPress is installed. This release of FeedWordPress normalizes post guid prefixes so as to avoid or limit the scope of this problem.

  • PHP 7 Compatibility: eliminate remaining sources of PHP 7 compatibility-check failures — remove the use of depreciated mysql_error() function, and make sure all classes make use of __construct() convention for constructors.

  • AVOID “PHP Warning: shellexec() has been disabled for security reasons in [2]/feedwordpress/feeds-page.php on line 197″: FeedWordPress uses the PHP shellexec() function in a very narrowly limited way for information gathering, trying to find the real path to curl or wget on your system, so that it can give as realistic as possible a recommendation for the sample crontab line displayed in Syndication > Feeds & Updates. Some web hosting environments disable shellexec for security reasons (since it could in theory be used to do a lot more stuff than the very limited information gathering FWP uses it for); in which case, this part of the code in FeedWordPress could spit out a nasty-looking and potentially worrisome-looking error message. So, now this code is fenced with checks to make sure that shellexec is available, before FWP attempts to make use of it.

2016.1213

  • WORDPRESS BACKWARD COMPATIBILITY FOR VERSIONS 4.7: This change fixes a fatal PHP error (on some web server configurations you’d see the message “Fatal error: require_once(): Failed opening required ‘[3]/wp-includes/class-wp-feed-cache.php'” on others, you might just see an HTTP 500 Internal Server Error or a blank page) when using FeedWordPress with versions of WordPress before 4.7. A change that I introduced to avoid a code module that had been deprecated in version 4.7 ended up relying on code modules that were only introduced as of version 4.7; so now, instead, FeedWordPress attempts to detect which modules the current version of the WordPress core makes available, and load the right modules depending on your WordPress version.

    In theory, up to this point, FeedWordPress supported any version of WordPress from version 3.0 onward. In practice, version 3.0 was released over 6 years ago, and I can realistically commit only to testing out new releases of FeedWordPress with a few prior versions of WordPress; so I’ve updated the “Requires at least” field to version 4.5, the first major release issued in 2016. If you’ve really got to use FeedWordPress with older versions of WordPress, it will probably still work with any moderately modern release of WordPress, but I won’t promise to keep it working with releases of WordPress that are more than about a year old.

2016.1211

  • WORDPRESS COMPATIBILITY: Tested with new versions of WordPress up to 4.7.

  • PHP WARNINGS UNDER WP 4.7: Eliminated cause of a PHP warning under WP 4.7 “Parameter 1 to FeedWordPressHTTPAuthenticator::setauthoptions expected to be reference” Warnings were due to a change in how httpapicurl hook is sometimes called in WP 4.7; so I changed the signature of the event handling method to avoid the notice. Props to @cogdog, @froomkin, @gwynethllewelyn et al. for flagging the issue and @garymarkfuller for suggesting a preliminary fix to the issue that was fairly similar to the solution I ended up adopting.

  • PHP 7 and PHP Strict Standards compatibility changes: @alexiskulash @daidais and @zoul0813 all sent pull requests through Github to fix some issues from a very old code base that has made its way from PHP 3.x through 5.x to the roll-out of PHP 7. Class methods should now fare better under modern versions of PHP and generate fewer “Deprecated” notices.

  • IMPROVEMENTS TO SCHEDULED AND AUTOMATIC UPDATES: use wploaded hook to check for magic URL parameters and to execute updates, to do pageload-based automatic updates, etc. Ensures that anything plugins or themes need to do in init to set up custom post types, taxonomies, etc. will be done before the updatefeedwordpress updates are attempted. If you saw posts not getting put into the correct custom post type or custom taxonomies or similar problems when performing scheduled updates, but the problem seemed to go away when you manually performed updates through the wp-admin interface, then you might be able to solve those problems with this update.

As always

If you notice any problems, have any questions, need any help, or just want to say Hi, don’t hesitate to drop me a line via e-mail or through the comment form. If you have a specific problem that you need help with, please try to describe the circumstances and the problem you are seeing in as much detail as possible — what you expected to happen, what you see happening instead, what you are doing (if anything) when the error comes up. If the problem has to do with one or more particular feeds, it helps a lot to include the URL(s) of the feed or feeds that you’re seeing the problem with. If the problem has to do with an error message appearing that you do not understand, a screenshot of the error message would help a lot.

Now get on out there and check out the new release! Download and enjoy!

Version 2016.0420, my dudes.

So development on the public release of FeedWordPress has been slowed down for a while now, thanks to a combination of life stuff and academic obligations. But the Spring semester is over, I have the summer off of school, and I am trying to get caught up. Most immediately:

  1. I want to make sure that some changes that have been percolating up through the Github development branch of FeedWordPress over the last few months get committed to the official release branch of FeedWordPress in the WordPress plugins repository;

  2. I want to make sure that the official release branch of FeedWordPress in the WordPress plugins repository
    has been tested and rated for compatibility with the most recent releases of WordPress, up through the Version 4.5.2 security release;

  3. And I’m trying to get a little less backlogged on my support requests at the WordPress.org Forums, on issue reports at Github, and via email. This will take a while, but please be patient — I am making progress, and if you have a request pending but you haven’t heard from me yet one way or another, you probably will within a matter of days, even if it’s just a quick check-in to the effect of So, are you still having this problem?</q.

  4. Besides which, I also have this gorgeous new banner for the plugin that Wayne Drescher made for me, and besides giving props for a generous deed, I also have self-interested reasons for wanting to make the plugin’s repository page prettier.

banner-772x250

In any case, then, here’s a new public release of FeedWordPress, Version 2016.0420.

Some quick notes from the ChangeLog:

Changes in 2016.0420

  • WORDPRESS COMPATIBILITY: Tested with new versions of WordPress up to 4.5.2. If you see any breakages that I missed, please make sure to contact me via e-mail with as many details as you can offer on (1) what problem you’re seeing, (2) where you’re seeing it, (3) what you expected to see, and (4) what you’re seeing instead. If there’s something wrong with the handling of a particular feed, the URL of that feed would be extremely helpful for my ability to help you out with troubleshooting.

  • USE XPATH SQUARE-BRACKET SYNTAX FOR BETTER SELECTION OF CUSTOM XML ELEMENTS FROM FEED: If you use the special XPath-like syntax in Syndication > Posts & Links > Custom Post Settings to select custom XML elements from your feed and put them into a Custom Field on the imported post, — or if you use SyndicatedPost::query() to select elements in an add-on or filter — then your ability to pick out particular elements from the feed has just been improved a bit. FeedWordPress only implements a small subset of XPath syntax — so, for example, $(author/name) will select the author child element of the current post, and then the name child element under author, and replace the $(...) expression with the text content of that name element. And $(category/@term) will select all of the category elements, and get the contents of the term attribute from each one. But most of the fancy things you can do to select elements with XPath, you currently can’t do with this syntax. However, if an item has multiple child elements, you should now be able to select one particular element using XPath’s square-bracket syntax — so $(category[1]/@term) now will allow you to select only the first category element, and then get its term attribute; $(category[2]/@term) allows you to select term attribute of the second, and so on. Props to Wayne Drescher for generously sponsoring the development of this feature.

  • FILTERS AND ADD-ONS: Allow filters and add-ons to filter terms and taxonomy (categories, tags, custom taxonomies, etc.) more thoroughly and more fine-grainedly using syndicated_post_terms_match, syndicated_post_terms_match_{taxonomy}, syndicated_post_terms_unfamiliar, syndicated_post_terms_mapping, syndicated_item_feed_terms, and syndicated_item_preset_terms filters.

  • FILTERS AND ADD-ONS: Globals $fwp_channel and $fwp_feedmeta REMOVED. These global variables, originally introduced to allow filters access to information about the source feed in syndicated_item filters were deprecated 6+ years ago. If you have any filters or add-ons which still depend on these global variables, you’ve been using obsolete techniques and you should see about fixing them to access data about the source feed using the SyndicatedPost::link element instead. For documentation, see the FeedWordPress documentation wiki at http://feedwordpress.radgeek.com/wiki/syndicatedpost and http://feedwordpress.radgeek.com/wiki/syndicatedlink.

  • BUGFIX: The Syndication > Diagnostics HTTP diagnostic test widget was broken due to a dumb error on my part. Sorry about that. It’s now fixed.

  • SMALL CODING CHANGES: Lots of small changes to code organization, incorporation of some PHP 5.x coding conventions, etc.

More to come. I have a lot of smaller and larger items that I’m tinkering on, and I hope to follow this up with another public release, most likely to come out late in May or early in June.

As always

If you notice any problems, have any questions, need any help, or just want to say Hi, don’t hesitate to drop me a line via e-mail or through the comment form. If you have a specific problem that you need help with, please try to describe the circumstances and the problem you are seeing in as much detail as possible — what you expected to happen, what you see happening instead, what you are doing (if anything) when the error comes up. If the problem has to do with one or more particular feeds, it helps a lot to include the URL(s) of the feed or feeds that you’re seeing the problem with. If the problem has to do with an error message appearing that you do not understand, a screenshot of the error message would help a lot.

Now get on out there and check out the new release! Download and enjoy!

FeedWordPress 0.99 is hereby released; enjoy WP 2.2 and 2.3 compatibility, bug fixes, major new features, updates without cron

Update 2007-11-21: FeedWordPress 0.99 is now out of date. You can download the latest release — 0.991 at the time of this writing — from the project homepage.

The public (non-beta) release of FeedWordPress version 0.99 is now available for download.

There have been changes to the way that FeedWordPress’s code is organized since version 0.98. If you successfully installed either of the beta releases, you don’t need to do anything special to install the current release. However, if you are upgrading from version 0.98 or before, be sure to see the installation instructions below.

Changes since version 0.98

This release provides compatibility with WordPress 2.2 and 2.3. It has been extensively tested against WordPress version 2.2.3 and the version 2.3 release candidate. I think that all the compatibility issues have been hammered out; of course, if you notice any problems, please let me know and I’ll get on a bugfix as soon as possible.

Version 0.99 also includes an overhaul to the user interface, some significant new features, and a number of bug fixes:

  • AUTOMATIC UPDATES WITHOUT CRON: FeedWordPress now allows you to
    automatically schedule checks for new posts without using external task
    scheduling tools such as cron. In order to enable automatic updates, go
    to Syndication –> Options and set “Check for new posts” to
    “automatically.” When this option is turned on, FeedWordPress will check for new posts
    automatically (1) when someone views your page, (2) if it has been ten minutes (or
    whatever interval you set) since the last time someone viewed your page. This offers a
    way to keep FeedWordPress up-to-date without having to schedule a cron script. It also
    simplifies the process of updating if you do choose to use a cron script — just have curl
    fetch your home page on a fixed schedule (so, for example, I would execute
    curl http://feministblogs.org/ every 15 minutes to keep Feminist Blogs up-to-date).
    Note that this is not the same thing as precisely scheduled updates — at a minimum,
    FeedWordPress will not check for new posts unless and until the next time somebody
    views your page. But for practical purposes it does allow you to keep your aggregator
    updated without having to run cron, and it is as close to precisely scheduled updates as
    you can get without using real scheduling tools such as cron.

    An important side-effect of the changes to the update system is that if
    you were previously using the cron job and the update-feeds.php script
    to schedule updates, you need to change your cron set-up. The old
    update-feeds.php script no longer exists. Instead, if you wish to use
    a cron job to guarantee updates on a particular schedule, you should
    have the cron job fetch the front page of your blog (for example, by
    using curl http://www.zyx.com/blog/ > /dev/null) instead of activating
    the update-feeds.php script. If automatic updates have been enabled,
    fetching the front page will automatically trigger the update process.

  • INTERFACE REORGANIZATION: All FeedWordPress functions are now located
    under a top-level “Syndication” menu in the WordPress Dashboard. To
    manage the list of syndicated sites, manually check for new posts on
    one or more feeds, or syndicate a new site, you should use the main page
    under Syndication. To change global settings for FeedWordPress,
    you should use Syndication –> Options.

  • FILE STRUCTURE REORGANIZATION: Due to a combination of changing styles
    for FeedWordPress plugins and lingering bugs in the FeedWordPress admin
    menu code, the code for FeedWordPress is now contained in two different
    PHP files, which should be installed together in a subdirectory of your
    plugins directory named feedwordpress. (See README.text for
    installation and upgrade instructions relating to the change.)

  • MULTIPLE CATEGORIES SETTING: Some feeds use non-standard methods to
    indicate multiple categories within a single category element. (The most
    popular site to do this is del.icio.us, which separates tags with a
    space.) FeedWordPress now allows you to set an optional setting, for any
    feed which does this, indicating the character or characters used to
    divide multiple categories, using a Perl-compatible regular expression.
    (In the case of del.icio.us feeds, FeedWordPress will automatically use
    \s for the pattern without your having to do any further configuration.)
    To turn this setting on, simply use the “Edit” link for the feed that
    you want to turn it on for.

  • REGULAR EXPRESSION BUG FIXED: Eliminated a minor bug in the regular
    expressions for e-mail addresses (used in parsing RSS author
    elements), which could produce unsightly error messages for some users
    parsing RSS 2.0 feeds.

  • DATE / UPDATE BUG FIXED: A bug in date handling was eliminated that may
    have caused problems if any of (1) WordPress, or (2) PHP, or (3) your
    web server, or (4) your MySQL server, has been set to use a different
    time zone from the one that any of the others is set to use. If
    FeedWordPress has not been properly updating updated posts, or has been
    updating posts when there shouldn’t be any changes for the update, this
    release may solve that problem.

  • GOOGLE READER BUGS FIXED: A couple of bugs that made it difficult for
    FeedWordPress to interact with Google Reader public feeds have been
    fixed. Firstly, if you encountered an error message reading “There was a
    problem adding the newsfeed. ” when you tried to add the feed,
    the cause of this error has been fixed. Secondly, if you succeeded in
    getting FeedWordPress to check a Google Reader feed, only to find that
    the title of posts had junk squashed on to the end of them, that bug
    has been fixed too. To fix this bug, you must install the newest version
    of the optional MagpieRSS upgrade.

  • FILTER PARAMETERS: Due to an old, old bug in WordPress 1.5.0 (which was
    what was available back when I first wrote the filter interface),
    FeedWordPress has traditionally only passed one parameter to
    syndicateditem and syndicatedpost filters functions — an array
    containing either the Magpie representation of a syndicated item from
    the feed, or the database representation of a post about to be inserted
    into the WordPress database. If you needed information about the feed
    that the item came from, this was accessible only through a pair of
    global variables, $fwpchannel and $fwpfeedmeta.

    Since it’s been a pretty long time since WordPress 1.5.0 was in
    widespread usage, I have gone ahead and added an optional second
    parameter to the invocation of the syndicateditem and syndicatedpost
    filters. If you have written a filter for FeedWordPress that uses either
    of these hooks, you can now register that filter to accept 2 parameters.
    If you do so, the second parameter will be a SyndicatedPost object,
    which, among other things, allows you to access information about the
    feed from which an item is syndicated using the $post->feed and the
    $post->feedmeta elements (where $post is the name of the second
    parameter).

    NOTE THAT THE OLD GLOBAL VARIABLES ARE STILL AVAILABLE, for the time
    being at least, so existing filters will not break with the upgrade.
    They should be considered deprecated, however, and may be eliminated in
    the future.

  • FILTER CHANGE / BUGFIX: the array that is passed as the first argument
    syndicatedpost filters no longer is no longer backslash-escaped for
    MySQL when filters are called. This was originally a bug, or an
    oversight; the contents of the array should only be escaped for the
    database after they have gone through all filters. IF YOU HAVE WRITTEN
    ANY syndicated
    post FILTERS THAT PRESUME THE OLD BEHAVIOR OF PASSING IN
    STRINGS THAT ARE ALREADY BACKSLASH-ESCAPED, UPDATE YOUR FILTERS
    ACCORDINGLY.

  • OTHER MINOR BUGFIXES AND INTERNAL CHANGES: The internal architecture of
    FeedWordPress has been significantly changed to make the code more
    modular and clean; hopefully this should help reduce the number of
    compatibility updates that are needed, and make them easier and quicker
    when they are needed.

Installation instructions

To upgrade an existing installation of FeedWordPress to version 0.99:

  1. Download the FeedWordPress archive in zip or gzipped tar format and
    extract the files on your computer.

  2. If you are upgrading from version 0.98 or earlier, then you need to
    create a new directory named feedwordpress in the wp-content/plugins
    directory of your WordPress installation, and you also need to delete
    your existing wp-content/update-feeds.php and
    wp-content/plugins/feedwordpress.php files. The file structure for
    FeedWordPress has changed and the files from your old version will not
    be overwritten, which could cause conflicts if you leave them in place.

  3. Upload the new PHP files to wp-content/plugins/feedwordpress,
    overwriting any existing FeedWordPress files that are there. Also be
    sure to upgrade wp-includes/rss.php and
    wp-includes/rss-functions.php if you use the optional MagpieRSS
    upgrade, or don’t use it yet but do want to syndicate Atom 1.0 feeds.

  4. If you are upgrading from version 0.96 or earlier, immediately log
    in to the WordPress Dashboard, and go to Options –> Syndicated. Follow
    the directions to launch the database upgrade procedure. The new
    versions of FeedWordPress incorporate some long-needed improvements, but
    old meta-data needs to be updated to prevent duplicate posts and other
    possible maladies. If you’re upgrading an existing installation, updates
    and FeedWordPress template functions will not work until you’ve done
    the upgrade. Then take a coffee break while the upgrade runs. It should,
    hopefully, finish within a few minutes even on relatively large
    databases.

  5. If you are upgrading from version 0.98 or earlier, note that the old
    update-feeds.php has been eliminated in favor of a (hopefully) more
    humane method for automatic updating. If you used a cron job for
    scheduled updates, it will not work anymore, but there is another,
    simpler method which will. See Setting Up Feed Updates to get
    scheduled updates back on track.

  6. Enjoy your new installation of FeedWordPress.