14.1. Scrape all the Cottage Inn Pizza locations¶
Let’s say that you want to make a list of all the Cottage Inn Pizza locations. When you go to their website, it turns out that there are a lot of locations.
If only you could write a little Python to easily collect them all…
It turns out that you can! Run the code below to see what it collects.
Before you keep reading...
Making great stuff takes time and $$. If you appreciate the book you are reading now and want to keep quality materials free for other students please consider a donation to Runestone Academy. We ask that you consider a $10 donation, but if you can give more thats great, if $10 is too much for your budget we would be happy with whatever you can afford as a show of support.
This code probably seems a bit complicated. In this ebook, we will break down web scraping into a few common “plans”. This example is made up of three plans. Click on each of them to learn more.
Plan 2: Get a soup from a URL# Load libraries for web scraping from bs4 import BeautifulSoup import requests # Get a soup from a URL url = 'https://web.archive.org/web/20200427175705/https://cottageinn.com/pick-a-location/' r = requests.get(url) soup = BeautifulSoup(r.content, 'html.parser')
Plan 5: Get info from all tags of a certain type# Get all tags of a certain type from the soup
tags = soup.find_all('h3')
# Collect info from the tags
collect_info = []
for tag in tags:
# Get info from tag
info = tag.text
collect_info.append(info)
Plan 9: Print the info# Print the info print(collect_info)