soimort on develop
[common] new parameters for get… [tiktok] fix extraction for alt… (compare)
soimort on develop
Fixed a typo (compare)
you-get --debug -c bl.txt --format=hdflv2 https://www.bilibili.com/bangumi/play/ep401367
[DEBUG] get_content: https://www.bilibili.com/bangumi/play/ep401367
you-get: This bangumi currently has 7 videos. (use --playlist to download all videos.)
[DEBUG] get_content: https://api.bilibili.com/pgc/player/web/playurl?avid=588418696&cid=349418858&qn=0&type=&otype=json&ep_id=401367&fnver=0&fnval=16
[DEBUG] get_content: https://api.bilibili.com/pgc/player/web/playurl?avid=588418696&cid=349418858&qn=120&type=&otype=json&ep_id=401367&fnver=0&fnval=8
[DEBUG] get_content: https://api.bilibili.com/pgc/player/web/playurl?avid=588418696&cid=349418858&qn=80&type=&otype=json&ep_id=401367&fnver=0&fnval=8
[DEBUG] get_content: https://api.bilibili.com/pgc/player/web/playurl?avid=588418696&cid=349418858&qn=64&type=&otype=json&ep_id=401367&fnver=0&fnval=8
[DEBUG] get_content: https://api.bilibili.com/pgc/player/web/playurl?avid=588418696&cid=349418858&qn=32&type=&otype=json&ep_id=401367&fnver=0&fnval=8
[DEBUG] get_content: https://api.bilibili.com/pgc/player/web/playurl?avid=588418696&cid=349418858&qn=16&type=&otype=json&ep_id=401367&fnver=0&fnval=8
[DEBUG] get_content: https://api.bilibili.com/pgc/player/web/playurl?avid=588418696&cid=349418858&qn=120&type=&otype=json&ep_id=401367&fnver=0&fnval=16
[DEBUG] get_content: https://api.bilibili.com/pgc/player/web/playurl?avid=588418696&cid=349418858&qn=80&type=&otype=json&ep_id=401367&fnver=0&fnval=16
[DEBUG] get_content: https://api.bilibili.com/pgc/player/web/playurl?avid=588418696&cid=349418858&qn=64&type=&otype=json&ep_id=401367&fnver=0&fnval=16
[DEBUG] get_content: https://api.bilibili.com/pgc/player/web/playurl?avid=588418696&cid=349418858&qn=32&type=&otype=json&ep_id=401367&fnver=0&fnval=16
[DEBUG] get_content: https://api.bilibili.com/pgc/player/web/playurl?avid=588418696&cid=349418858&qn=16&type=&otype=json&ep_id=401367&fnver=0&fnval=16
[DEBUG] get_content: http://comment.bilibili.com/349418858.xml
site: Bilibili
title: 镇魂街 第二季:重制版第7话 以血洗血
stream:
- format: hdflv2
container: flv
quality: 高清 1080P+
size: 783.8 MiB (821847203 bytes)
# download-with: you-get --format=hdflv2 [URL]
Downloading 镇魂街 第二季:重制版第7话 以血洗血.flv ...
0.0% ( 0.0/783.8MB) ├────────────────────────────────────────┤[1/1] you-get: version 0.4.1527, a tiny downloader that scrapes the web.
you-get: Namespace(URL=['https://www.bilibili.com/bangumi/play/ep401367'], auto_rename=False, cookies='bl.txt', debug=True, extractor_proxy=None, first=None, force=False, format='hdflv2', help=False, http_proxy=None, info=False, input_file=None, insecure=False, itag=None, json=False, last=None, no_caption=False, no_merge=False, no_proxy=False, output_dir='.', output_filename=None, password=None, player=None, playlist=False, size=None, skip_existing_file_size_check=False, socks_proxy=None, stream=None, timeout=600, url=False, version=False)
Traceback (most recent call last):
File "C:\Users\Allen\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 1350, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "C:\Users\Allen\AppData\Local\Programs\Python\Python38\lib\http\client.py", line 1240, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Users\Allen\AppData\Local\Programs\Python\Python38\lib\http\client.py", line 1286, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Users\Allen\AppData\Local\Programs\Python\Python38\lib\http\client.py", line 1235, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Users\Allen\AppData\Local\Programs\Python\Python38\lib\http\client.py", line 1006, in _send_output
self.send(msg)
File "C:\Users\Allen\AppData\Local\Programs\Python\Python38\lib\http\client.py", line 946, in send
self.connect()
File "C:\Users\Allen\AppData\Local\Programs\Python\Python38\lib\http\client.py", line 1409, in connect
self.sock = self._context.wrap_socket(self.sock,
Downloading ADA Lecture 1.1: Introduction (19-09-12).mp4 ...
100% ( 63.7/ 63.7MB) ├███████████████████████████████████████████████████████████████████████████████████████████████┤[1/1] 2 MB/s
- title: ADA Lecture 2: Divide and Conquer - Part 1
size: 428.5 MiB (449283621 bytes)
url: https://youtu.be/ywJJvBwha4s
Downloading ADA Lecture 2: Divide and Conquer - Part 1.mp4 ...
27.7% (118.8/428.5MB) ├██████████████████████████─────────────────────────────────────────────────────────────────────┤[1/1] 75 kB/s
you-get https://www.bilibili.com/video/BV1nQ4y127v5 -d
[DEBUG] get_content: https://www.bilibili.com/video/BV1nQ4y127v5
[DEBUG] get_content: https://www.bilibili.com/video/BV1nQ4y127v5
[DEBUG] get_content: https://interface.bilibili.com/v2/playurl?appkey=iVGUTjsxvpLeuDCf&cid=386731535&otype=json&qn=112&quality=112&type=&sign=4b37737a57503187200d424a6df906fa
[DEBUG] get_content: https://interface.bilibili.com/v2/playurl?appkey=iVGUTjsxvpLeuDCf&cid=386731535&otype=json&qn=80&quality=80&type=&sign=b1e7d04a5f1db8b0596b477c1824def8
[DEBUG] get_content: https://interface.bilibili.com/v2/playurl?appkey=iVGUTjsxvpLeuDCf&cid=386731535&otype=json&qn=64&quality=64&type=&sign=3a9e9e9e8eb74e88d7a38e927807bfcd
[DEBUG] get_content: https://interface.bilibili.com/v2/playurl?appkey=iVGUTjsxvpLeuDCf&cid=386731535&otype=json&qn=32&quality=32&type=&sign=5db7edab80ea5ebea30cc86dc30acf5d
[DEBUG] get_content: https://api.bilibili.com/x/player/playurl?avid=717288641&cid=386731535&qn=16&type=&otype=json&fnver=0&fnval=16&fourk=1
[DEBUG] get_content: https://interface.bilibili.com/v2/playurl?appkey=iVGUTjsxvpLeuDCf&cid=386731535&otype=json&qn=16&quality=16&type=&sign=5853c176690811ef49cd2829d84738f0
[DEBUG] get_content: http://comment.bilibili.com/386731535.xml
site: Bilibili
title: 【看理想 | 当代艺术】中国最好的艺术家之一,理解自己的作品吗?
stream:
- format: dash-flv
container: mp4
quality: 高清 1080P
size: 37.4 MiB (39169220 bytes)
# download-with: you-get --format=dash-flv [URL]
Downloading 【看理想 - 当代艺术】中国最好的艺术家之一,理解自己的作品吗?.mp4 ...
100% ( 37.4/ 37.4MB) ├████████████████████████████████████████┤[1/2] 199 MB/syou-get: version 0.4.1545, a tiny downloader that scrapes the web.
you-get: Namespace(URL=['https://www.bilibili.com/video/BV1nQ4y127v5'], auto_rename=False, cookies=None, debug=True, extractor_proxy=None, first=None, force=False, format=None, help=False, http_proxy=None, info=False, input_file=None, insecure=False, itag=None, json=False, last=None, no_caption=False, no_merge=False, no_proxy=False, output_dir='.', output_filename=None, password=None, player=None, playlist=False, size=None, skip_existing_file_size_check=False, socks_proxy=None, stream=None, timeout=600, url=False, version=False)
Traceback (most recent call last):
File "C:\Users\Allen\AppData\Local\Programs\Python\Python38\lib\urllib\request.py", line 1350, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "C:\Users\Allen\AppData\Local\Programs\Python\Python38\lib\http\client.py", line 1240, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Users\Allen\AppData\Local\Programs\Python\Python38\lib\http\client.py", line 1286, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Users\Allen\AppData\Local\Programs\Python\Python38\lib\http\client.py", line 1235, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Users\Allen\AppData\Local\Programs\Python\Python38\lib\http\client.py", line 1006, in _send_output
self.send(msg)
File "C:\Users\Allen\AppData\Local\Programs\Python\Python38\lib\http\client.py", line 946, in send
self.connect()
File "C:\Users\Allen\AppData\Local\Programs\Python\Python38\lib\http\client.py", line 1409, in connect
self.sock = self._context.wrap_socket(self.sock,
File "C:\Users\Allen\AppData\Local\Programs\Python\Python38\lib\ssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
File "C:\Users\Allen\AppData\Local\Programs\Python\Python38\lib\ssl.py", line 1040, in _create
self.do_handshake()
File "C:\Users\Allen\AppData\Local\Programs\Python\Python38\lib\ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Allen\AppData\Local\Programs\Python\Py
发现报issue有点麻烦了,所以直接post到这里。
问题:B站使用cookies下载视频可能会失败
版本:0.4.1545
命令:
you-get --cookies <your-firefox-profile-path>\cookies.sqlite -i https://www.bilibili.com/video/BV1744y1h7CM
traceback:
Traceback (most recent call last):
File "e:\python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "e:\python39\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "E:\Python39\Scripts\you-get.exe\__main__.py", line 7, in <module>
File "e:\python39\lib\site-packages\you_get\__main__.py", line 92, in main
main(**kwargs)
File "e:\python39\lib\site-packages\you_get\common.py", line 1834, in main
script_main(any_download, any_download_playlist, **kwargs)
File "e:\python39\lib\site-packages\you_get\common.py", line 1716, in script_main
download_main(
File "e:\python39\lib\site-packages\you_get\common.py", line 1348, in download_main
download(url, **kwargs)
File "e:\python39\lib\site-packages\you_get\common.py", line 1825, in any_download
m.download(url, **kwargs)
File "e:\python39\lib\site-packages\you_get\extractor.py", line 48, in download_by_url
self.prepare(**kwargs)
File "e:\python39\lib\site-packages\you_get\extractors\bilibili.py", line 192, in prepar
e
initial_state = json.loads(initial_state_text)
File "e:\python39\lib\json\__init__.py", line 339, in loads
raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not NoneType
分析:初步分析发现是在get_content
函数中, 无脑将所有cookies都发送出去了,B站那边检测到后就不返回内容。
修复:打算给get_content
添加一个cookie_filter
参数,只提交相关的cookies。
代码示例:
def bilibili_cookie(cookie):
retrun 'bilibili' in cookie.domain
def get_content(url, headers={}, decoded=True, cookie_filter=None):
# 省略无关代码
if cookies:
cookie_strings = []
for cookie in list(cookies):
if cookie_filter and not cookie_filter(cookie):
continue
cookie_strings.append(cookie.name + '=' + cookie.value)
cookie_headers = {'Cookie': '; '.join(cookie_strings)}
req.headers.update(cookie_headers)
如果觉得没问题的话我就提交PR修复。
C:\Users\Administrator>you-get -o E/Videos https://www.bilibili.com/video/BV1aR4y147gK?from=search&seid=11754669618879750134&spm_id_from=333.337.0.0
site: Bilibili
title: 《孤泳者》“职业不是攀登高峰 赖得越久才越值得歌颂”
stream:
- format: dash-flv
container: mp4
quality: 高清 1080P
size: 28.9 MiB (30300167 bytes)
# download-with: you-get --format=dash-flv [URL]
Downloading 《孤泳者》“职业不是攀登高峰 赖得越久才越值得歌颂”.mp4 ...
0.0% ( 0.0/ 28.9MB) ├────────────────────────────────────────┤[1/2] you-get: [error] oops, something went wrong.
you-get: don't panic, c'est la vie. please try the following steps:
you-get: (1) Rule out any network problem.
you-get: (2) Make sure you-get is up-to-date.
you-get: (3) Check if the issue is already known, on
you-get: https://github.com/soimort/you-get/wiki/Known-Bugs
you-get: https://github.com/soimort/you-get/issues
you-get: (4) Run the command with '--debug' option,
you-get: and report this issue with the full output.
'seid' 不是内部或外部命令,也不是可运行的程序
或批处理文件。
'spm_id_from' 不是内部或外部命令,也不是可运行的程序
或批处理文件。