Puppeteer GitHub Actions Scraper

How It Works

A Node.js script (scrape.js) uses Puppeteer to open quotes.toscrape.com.
The script scrapes the first 10 quotes.
The data is saved to a file: public/data.json.
A GitHub Actions workflow (.github/workflows/scrape.yml) runs this script on a schedule.
The workflow will push the new data.json file to the repository if there are changes.
This page fetches the data from data.json and displays it below.

Configuration

You can change the schedule by editing the cron value inside .github/workflows/scrape.yml. Find the schedule event to adjust the timing.

You can also configure which script to run. In the same file, find the step named Run scraper and change the command in the run property, for example: run: node your-other-script.js.

How to Run Locally

Clone this repository.

Install dependencies: npm install

Run the scrape script: node scrape.js

The headless option in scrape.js determines whether the browser is visible (false) or hidden (true). Change headless: true to false if you want to see the scraping process in the browser window.

When running locally, the script will automatically try several common browser paths (Chrome/Edge). If no browser is found, make sure the paths in the localCandidates array in scrape.js match the actual browser location on your computer. You can copy and paste the path from Windows Explorer; the script will automatically normalize the format.

How to copy the browser path in Windows: Search for your browser on the desktop or in the search bar, right-click and choose "Open file location". If it still shows a shortcut, right-click the shortcut again and select "Open file location" until you find the actual chrome.exe or msedge.exe file. Then right-click that file and choose "Copy as path". Paste this path into the localCandidates array in scrape.js.

Open the index.html file in your browser.

A Guide to Scraping with Puppeteer & GitHub Actions

How It Works

Configuration

Scraped Results: Top 10 Quotes