Live News Mode is one of the most-used features of AI Content Generator: point it at an RSS feed, let it summarise the day’s stories into original editorial. It works cleanly with mainstream feeds — but some publishers (major newspapers, aggregators, forum front-pages) either block WordPress’ wp_remote_get(), gate the feed behind Cloudflare, or serve JavaScript-rendered pages instead of raw RSS. When the feed can’t be reached, the campaign stalls with an empty queue and an error in the log.
This tutorial covers a two-hour setup that solves the problem permanently: a Cloudflare Worker that acts as a friendly RSS proxy between your feed and AI Content Generator. Free tier, no server to run, ~40 lines of code, works with any feed WordPress can’t reach directly.
Why the direct fetch fails
Three common reasons a feed URL that works in your browser returns empty inside WordPress:
- User-Agent block — publisher blocks any User-Agent containing “WordPress” or PHP’s default fetch UA.
- Cloudflare bot challenge — the feed sits behind a Cloudflare-protected page that returns a JS challenge instead of XML.
- Country / IP filtering — the feed is limited to certain regions and your WordPress host isn’t in one.
All three can be bypassed by fetching from a Cloudflare Worker, which runs at 300+ edge locations, has a browser-like User-Agent by default, and can transform the response to remove anything that trips up WP’s parser.
Step 1 — Create the Cloudflare Worker
Free Cloudflare account, dash.cloudflare.com → Workers & Pages → Create Worker → give it a name (rss-proxy) → Deploy → Edit Code. Paste this:
export default {
async fetch(request) {
const url = new URL(request.url);
const target = url.searchParams.get('u');
if (!target) return new Response('missing u parameter', { status: 400 });
const upstream = await fetch(target, {
headers: {
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) ' +
'AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126',
'Accept': 'application/rss+xml, application/xml, text/xml, */*'
}
});
let body = await upstream.text();
// strip BOM and stray XML declaration issues
body = body.replace(/^/, '').trim();
return new Response(body, {
status: upstream.status,
headers: {
'Content-Type': 'application/rss+xml; charset=utf-8',
'Cache-Control': 'public, max-age=300',
'Access-Control-Allow-Origin': '*'
}
});
}
};
Save and Deploy. Cloudflare gives you a URL like https://rss-proxy.YOUR-SUBDOMAIN.workers.dev.
Step 2 — Test the proxy in a browser
Take any feed URL that WordPress can’t reach, URL-encode it, append as ?u= parameter:
https://rss-proxy.YOUR-SUBDOMAIN.workers.dev/?u=https%3A%2F%2Fexample.com%2Ffeed
Open in a browser. You should see the raw RSS XML. If you see a Cloudflare error page, the Worker is running but the target is blocking the Worker’s IPs too — you’ll need a different feed or a rotating proxy chain (out of scope).
Step 3 — Configure Live News Mode with the proxied URL
In your AI Content Generator campaign, open Live News Mode. In the RSS feed URL field, paste the proxied URL (the full Worker URL with ?u= parameter, URL-encoded target). Save the campaign.
Trigger a test generation. If the feed was previously empty, it now populates with recent stories. The Article log confirms which feed URL was consulted.
Step 4 — Add caching (optional but recommended)
The Cache-Control: public, max-age=300 header tells Cloudflare’s edge cache to hold the response 5 minutes. Reduces load on the upstream publisher and cuts Worker CPU time. If the campaign runs hourly, the cache is warm for every fetch after the first.
Verify caching works: fetch the proxied URL twice within 5 minutes. The second should be nearly instant and the response header should include cf-cache-status: HIT.
Step 5 — Multiple feeds through one Worker
One Worker handles unlimited feeds. Each Live News Mode configuration in AI Content Generator uses the same Worker with a different ?u= parameter. Manage 10 different campaigns → 10 different URLs, all proxied through the one Worker.
Rate limits and cost
Cloudflare Workers free tier gives 100,000 requests/day. A typical AI Content Generator setup with 3 campaigns polling every 30 minutes uses ~144 requests/day — well within the free tier. If you exceed it, upgrade to the $5/month tier for 10 million requests/day.
Legal considerations
Fetching an RSS feed is technical access to publicly-offered content — legally uncontroversial in most jurisdictions. What you do with the content matters more: AI Content Generator’s Live News Mode summarises rather than republishing, and cites the original source. Keep the source citation enabled and the summary well under fair-use thresholds (short quotes, not full articles).
If you’re proxying a feed the publisher specifically requested you not fetch, respect that. The Worker approach solves technical failures, not legal disagreements.
Troubleshooting
- 403 from upstream — publisher blocking the Worker’s IP range. Try a different feed.
- HTML instead of XML — feed URL you passed is a webpage, not a real feed. Find the actual
<link rel="alternate" type="application/rss+xml">URL. - Empty queue in AI Content Generator — the Worker responded with 200 but empty body. Test the Worker URL manually to see what it returns.
- Feed works then breaks — publisher noticed and started blocking. Add a small delay + retry to your Worker code.
You’re done
Cloudflare Worker RSS proxy → AI Content Generator Live News Mode → reliable feed fetching regardless of source publisher’s WordPress block. Two hours of setup, near-zero recurring cost, and the pipeline runs unattended for months.