Wp Content Crawler – Get Content From Almost Any Site Automatically

WP Content Crawler is a WordPress plugin that enables automatic content aggregation and import from external websites into your WordPress blog. It allows users to scrape content from virtually any website and automatically publish it to their WordPress site, with support for custom scheduling and content transformation.

Key Features

• Universal Web Scraping - Extract content from almost any website regardless of structure or layout using flexible targeting options
• Automatic Publishing - Schedule automatic content imports at specified intervals to keep your site updated without manual intervention
• CSS Selectors - Use CSS selectors to precisely target and extract specific elements from source pages
• XPath Support - Advanced XPath expressions allow complex content extraction from structured HTML documents
• Content Transformation - Modify, filter, and transform scraped content before publishing to your WordPress site
• Multiple Source Management - Create and manage multiple content sources simultaneously with individual settings for each
• Post Template System - Define custom templates for how imported content is formatted and displayed as WordPress posts
• Image Handling - Automatically download and attach images from source pages to imported posts
• Duplicate Prevention - Built-in mechanisms to avoid importing duplicate content across multiple imports
• Custom Post Types - Import content as standard posts, pages, or custom post types
• Taxonomy Assignment - Automatically assign categories, tags, and other taxonomies to imported content
• Scheduling Control - Set up cron-based scheduling for automated content imports at custom intervals

Best For

This plugin is designed for content aggregators, news curators, and website owners who want to automatically collect and republish content from multiple external sources. It serves users managing content-heavy sites who need to populate their WordPress blog with curated content from industry sources, competitor monitoring, or syndicated feeds without manual copying and pasting.