Download Files From a URL Using Python скачать в хорошем качестве

Download Files From a URL Using Python

In this lesson we are going to do a simple web scraping- downloading files from a given URL using python. The code I used in this session is this- -*- coding: utf-8 -*- """ Created on Mon Jul 18 11:48:18 2022 @author: karin """ import requests from requests.auth import HTTPBasicAuth from bs4 import BeautifulSoup def get_url_paths(url, ext=''): response = requests.get(url) response = requests.get(url, auth=HTTPBasicAuth('username', 'password')) if response.ok: response_text = response.text else: return response.raise_for_status() soup = BeautifulSoup(response_text, 'html.parser') parent = [url + node.get('href') for node in soup.find_all('a') if node.get('href').endswith(ext)] return parent def main(): url = 'https://www.ncei.noaa.gov/data/total-... ext = '.nc' url = 'http://wwlln.net/hostdata//' ext = '.loc' result = get_url_paths(url, ext) for file in result: f_name = file[-19:-13] f_name = file[-12:-4] r = requests.get(file) r = requests.get(file, auth=HTTPBasicAuth('username', 'password')) with open(f'C:/Users/karin/Desktop/Work/PikFix/WebScraping/Folder1/{f_name}.nc', 'wb') as f: f.write(r.content) if _name_ == '__main__': main()