9th Session- Integrative Lab: Webscraping

For this assignment, you will again web scrape videos from two YouTube channels. For troubleshooting with webscraper.io, please see footnote below.** UCCSS_Lab_Webscraping_2.pdf
You will be assigned two channels to scrape. In contrast to our previous exercise, you will NOT scrape the featured videos of the specified news channel, but the search result of the name of the news channel in combination with your name:

1) Scrape the search results when you search for the [name of media outlet] + [your first & last name (your given & family name)]. For example, for the media outlet "The Young Turks", Prof. Hilbert would search:

Take a screenshot of the search result (Note: if you don't get any or very few search results, take a screenshot to show this and try again with only your first or last name, or a variant of your names).

2) Scrape both all the videos that appear as your search result + the recommended videos inside each of the 'search result' videos. For rough guidance, see our previous tutorial: UCCSS_LAB_webscraping  (you can, but you do not need to scroll down on the search results: it is enough to scrape the videos that first appear from the search).

3) Do step (1) and step (2) this for both indicated channels (see task below)

4) Organize both channels in one single csv file, ready for upload in the SNA software Gephi. For guidance, see our previous tutorial: UCCSS_LAB_SNA: Data Wrangling (12min)

5) Upload the single .csv file here. The file must be named with your channels and full name, for example >TheYoungTurks_DemocracyNow_MartinHilbert.csv<.

6) Then, click on "Assignment detail" / "Attempt" and in "Add comment" upload both screenshots (see step 1). If you cannot find it, go to "Grades" then click on this assignment:  

** Footnote: We are working in a real-time commercial environment, not in a practice sandbox. Google is constantly updating the YouTube interface and its Chrome browser, so you might run into some differences from the tutorial UCCSS_Lab_Webscraping2.pdf, especially when it comes to recommended videos. Students report several workarounds in case of problems:

• Try working with https://www.firefox.com as your browser, or even switching computers.
• Try uninstalling and installing the webscraper extension.
• When selecting recommended videos, after clicking on the first video, click on a few more videos after the first one (could be the 6th or 7th), and then again back to the first one. Some report that scrolling and loading recommended videos and going even further down helps.
• Feel free to use another webscraping tool. Several students report that Browserflow.app is more intuitive (here a tutorial, here a YouTube Flow, and, of course, feel free to consult LLMs like ChatGPT, Claude, and Gemini to make it work for you!)  
• Never hesitate to reach out on Piazza: please be very concrete and add screenshots.