Scrapy Web Crawling API

Creating a Web Scraping application using Scrapy spiders to crawl and ScrapyRT to provide an API access.

Scrapyrt will provide API access to kick off the scrapy crawl on a specified URL and provide a json output once the required information is scraped.

Technologies Used:

  • Python
  • Scrapy

View Source