Best Seller Udemy Online Course – Learn Anywhere, Anytime, from Experts.
It is your opportunity to earn more income through learning Scrapy. Why? When it comes to scraping sophisticated websites, you need a robust crawling framework – crawling is currently necessary for many web scraping and data extraction jobs. That is why Scrapy a frequently required web scraping framework for both freelancing and in-house jobs.
This Scrapy course covers the following topics:
- What Scrapy is, the differences between Scrapy and other Python-based web scraping libraries such as BeautifulSoup, LXML, Requests, and Selenium, and when it is better to use Scrapy.
- Creating a Scrapy project and and then build a basic Spider to scrape data from a website.
- Exploring XPath commands and how to use it with Scrapy to extract data.
- Building a more advanced Scrapy spider to iterate multiple pages of a website and scrape data from each page.
- Scrapy Architecture: the overall layout of a Scrapy project; what each field represents and how you can use them in your spider code.
- Web Scraping best practices to avoid getting banned by the websites you are scraping.
- Deploying a Scrapy web crawler to the Scrapy Cloud platform easily. Scrapy Cloud is a platform from ScrapingHub to run, automate, and manage your web crawlers in the cloud, without the need to set up your own servers.
- Using Scrapy for web scraping authenticated (logged in) user sessions, i.e. on websites that require logging in with a username and password before displaying data.
- Creating an advanced web crawler with Scrapy. We will cover using Scrapy CrawlSpider which is the most commonly used spider for crawling regular websites, as it provides a convenient mechanism for following links by defining a set of rules. We will also use Link Extractor object which defines how links will be extracted from each crawled page; it allows us to grab all the links on a page, no matter how many of them there are.
- Functions that Scrapy offers after the spider is done with web scraping, and how to edit and use Scrapy parameters.
- Writing the output to CSV, Excel, JSON, and XML files.
- Downloading images from a website.
- Using pipelines and add more power to Scrapy.
- Storing the data extracted by Scrapy into MySQL and MongoDB databases.
- Several practical projects.
- All source code files are downloadable.
- Captions of all course videos in English, Portuguese, and Spanish.
- Q&A section inside the course for all your questions.