Skip to content

download: Add config to set local servers where cache tarballs #270

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rgonzalezfluendo wants to merge 3 commits into
base: 1.0
Choose a base branch
from
Open

download: Add config to set local servers where cache tarballs #270

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
869dec6
download: Add config to set local servers where cache tarballs
rgonzalezfluendo Apr 1, 2020
1d52107
Revert "download: Add config to set local servers where cache tarballs"
rgonzalezfluendo Apr 1, 2020
57c1431
download: check mirror first and then upstream
rgonzalezfluendo Apr 1, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions cerbero/utils/shell.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -436,11 +436,15 @@ def download(url, destination=None, check_cert=True, overwrite=False, logfile=No
@param check_cert: whether to check certificates or not
@type check_cert: bool
@param overwrite: whether to overwrite the destination or not
@type check_cert: bool
@type overwrite: bool
@param logfile: path to the file to log instead of stdout
@type logfile: str
@param mirrors: list of mirrors to use as fallback
@type logfile: list
@param user: HTTP or FTP server user
@type user: str
@param password: HTTP or FTP server password
@type password: str
'''
if not overwrite and os.path.exists(destination):
if logfile is None:
Expand All @@ -451,12 +455,13 @@ def download(url, destination=None, check_cert=True, overwrite=False, logfile=No
os.makedirs(os.path.dirname(destination))
log("Downloading {}".format(url), logfile)

urls = [url]
if mirrors is not None:
filename = os.path.basename(url)
# Add a traling '/' the url so that urljoin joins correctly urls
# in case users provided it without the trailing '/'
urls += [urllib.parse.urljoin(u + '/', filename) for u in mirrors]
urls = [urllib.parse.urljoin(u + '/', filename) for u in mirrors] + [url]
else:
urls = [url]

# wget shipped with msys fails with an SSL error on github URLs
# https://githubengineering.com/crypto-removal-notice/
Expand Down