14.10. Code writing activity part 2
On this page, you will complete a second activity to write code that:
Scrapes all the comments on the Rate My Professor page for Prof. Ericson and Prof. Oney and prints them
Here is the link to Prof. Ericson's Rate My Professor page.
Here is the link to Prof. Oney's Rate My Professor page.
You can see that both the pages have the same layout.
The comments all have the same tag name, which is 'div' tag with class='Comments__StyledComments-dzzyvm-0 dvnRbr'. Here’s what it looks like when you inspect Prof. Ericson’s page:
Choose which of the following plans you will use, and put them in the correct order.
# Load libraries for web scraping
from bs4 import BeautifulSoup
import requests
# Get a soup from multiple URLs
base_url = ________________________________
endings = ________________________________
for ending in endings:
url = base_url + ending
r = requests.get(url)
soup = BeautifulSoup(r.content, 'html.parser')
---
# Load libraries for web scraping
from bs4 import BeautifulSoup
import requests
# Get a soup from a URL
url = _________________________
r = requests.get(url)
soup = BeautifulSoup(r.content, 'html.parser')#paired
---
# Get all tags of a certain type from the soup
tags = soup.find_all(___________)
# Collect info from the tags
collect_info = []
for tag in tags:
info = tag.____________
collect_info.append(info)
---
# Get first tag of a certain type from the soup
tag = soup.find(___________)
# Get info from the tag
info = tag.____________#paired
---
# Get first tag of a certain type from the soup
first_tag = soup.find(___________)
# Get all tags of a certain type from the first tag
tags = first_tag.find_all(____________)
# Collect info from the tags
collect_info = []
for tag in tags:
info = tag.____________
collect_info.append(info)#paired
---
# Print the info
print(____________)
---
# Load library for json files
import json
# Put info into file
f = open(____________, 'w')
json.dump(____________, f)
f.close()#paired
In solving the preceding problem I invested:
- 1. Very, very low mental effort
- 2. Very low mental effort
- 3. Low mental effort
- 4. Rather low mental effort
- 5. Neither low nor high mental effort
- 6. Rather high mental effort
- 7. High mental effort
- 8. Very high mental effort
- 9. Very, very high mental effort
You have attempted
of
activities on this page
