RSS Tutorial

Scrape a Website Feed

Summary

Websites generally do not create RSS feeds automatically. There are various ways to create RSS feeds from websites. Some programs read (scrape) websites and create RSS feeds.

Programs are available that scrape a website to create a feed
The Programs in general do not do well trying to decide what is important or "What's New"
The Programs are controversial because they can be used improperly.

Detail

Programs are available that scrape a website to create a feed

Programs use the technique known as "scraping" or "screen scraping", which is reading a website, webpage screen, or HTML and creating the RSS Feed.

Programs read the website and generally create the RSS Feed as follows:

Webpage = RSS Feed Item
Webpage Title = RSS Item title
Webpage Description meta tag = RSS Item description
Webpage URL = RSS Item link.

Programs do not do well trying to decide what is important or "What's New"

There are two typical characteristics of a website RSS Feed.

It is a headline or an announcement that brings people back to the website.
It is a "What's New?" feature for the website.

Each of those characteristics means that it is probably going to take manual intervention to create effective feeds.

If you are using the RSS Feed to announce a new article, you want to decide how much of the article is enough to entice someone to click and read the entire article.
If you are using the RSS Feed to introduce a new product, you want to decide what should be in the RSS Feed to entice people to come to the website, review the product, and potentially buy the product.
If you are using the RSS Feed to announce a meeting or event, you want to include enough to bring people to the website and have them review the details of the event or meeting.

The scrape programs are a brute force approach that do not make intelligent decisions.

The Programs are controversial because they can be used improperly.

The scrape programs are controversial because they can be used to read the website of someone else and steal the content.

I do not recommend any program, product, or system that reads the website or webpage to try to figure out what should be on the RSS Feed. They just do not do a good job.

I certainly do not recommend these sorts of programs to steal content.