First step: Discover a data source based on the home page of the news site
const { discoverPostSource, getPostList } = require("post-feed-reader")
// Looks for metadata pointing to the Wordpress REST API or Atom/RSS Feeds
const source = await discoverPostSource('https://www.nytimes.com');
Second step: Fetch the post list data
// Retrieves the posts from the given source
const list = await getPostList(source);
This object can be iterated as needed
list.posts.map(post => post.title)
Other types of source can also be used
const techCrunchSource = await discoverPostSource('https://techcrunch.com');
const techCrunchList = await getPostList(techCrunchSource, {
fillTextContents: true, // Automatically converts HTML contents into plain text
wordpress: {
fetchBlogInfo: true, // Ensures blog title, description and favicons are available for WordPress sources
},
});
Sources that support it can be paginated
const nextPage = await getPostList(techCrunchList.pagination.next);
no comments
sign in to comment