【2020-09-04】day04：通过bilibili外部接口获取视频资源

个人知识碎片的体系化理解与整理，缓慢更新，理解不到位与错漏之处望见谅与指正。

day04摘要思想：主要涉及hutool发送http请求，alibaba fastjson处理json数据，LinkedHashMap的使用，文件流的创建与保存等后台常见应用场景；需要调用b站的对外接口获取实际数据

开发环境：Intellij IDEA 2017.2.6

1.相关工具类Maven引入

hutool发送http请求工具类：

<dependency> <groupId>cn.hutool</groupId> <artifactId>hutool-all</artifactId> <version>5.4.1</version> </dependency>

alibaba fastjson处理json工具类：

<dependency> <groupId>com.alibaba</groupId> <artifactId>fastjson</artifactId> <version>1.2.70</version> </dependency>

所有的maven引入可直接在https://mvnrepository.com/artifact进行搜索获取当前最新版本；

 

2.基础请求地址

需要先获取该av号/bv号所指向视频资源的实际cid，再根据其cid获取实际下载地址：

//获取cid地址 private static final String cidUrl = "https://api.bilibili.com/x/web-interface/view"; //获取下载地址 private static final String downloadUrl = "https://api.bilibili.com/x/player/playurl"; //下载请求头 private static final String refererUrl = "https://www.bilibili.com/video/$";

 

3.获取视频实际分P的cid

av号请求地址示例：https://api.bilibili.com/x/web-interface/view?aid=380160

bv号请求地址示例：https://api.bilibili.com/x/web-interface/view?bvid=BV1Tx411P7fo

实际返回json过长，此处仅截取部分核心数据：

{ "title": "【合集】Kanon 雪之少女【BDRip】【澄空】【1080P】", "pages": [{ "cid": 2933938, "page": 1, "from": "vupload", "part": "白銀の序曲(オーバーチュア)～overture～ （银白色的序曲～overture～）", "duration": 1453, "vid": "", "weblink": "", "dimension": { "width": 0, "height": 0, "rotate": 0 } }] }

获取cid代码如下：

//获取cid与视频名称，根据pageSize从第一P开始获取，根据aid或bvid重载该方法 private LinkedHashMap<String,JSONObject> getCid(Long aid , int pageSize) { if(pageSize < 1) { throw new FrameworkException("分P数量至少为1！"); } String responseBody = HttpRequest.get(cidUrl + "?aid=" + aid.toString()).timeout(20000).execute().body(); LinkedHashMap<String,JSONObject> resultMap = new LinkedHashMap<>(); JSONObject responseJson = JSONObject.parseObject(responseBody); JSONObject responseData = responseJson.getJSONObject("data"); JSONObject titleObject = new JSONObject(); titleObject.put("title",responseData.getString("title")); resultMap.put("0",titleObject); JSONArray pageData = responseData.getJSONArray("pages"); if(pageData.size() >= pageSize) { for(int i = 0; i < pageSize; i++) { resultMap.put(pageData.getJSONObject(i).getString("page"),pageData.getJSONObject(i)); } } else { throw new FrameworkException("所请求分P数量超过实际数量！"); } return resultMap; } private LinkedHashMap<String,JSONObject> getCid(String bvid , int pageSize) { if(pageSize < 1) { throw new FrameworkException("分P数量至少为1！"); } String responseBody = HttpRequest.get(cidUrl + "?bvid=" + bvid).timeout(20000).execute().body(); LinkedHashMap<String,JSONObject> resultMap = new LinkedHashMap<>(); JSONObject responseJson = JSONObject.parseObject(responseBody); JSONObject responseData = responseJson.getJSONObject("data"); JSONObject titleObject = new JSONObject(); titleObject.put("title",responseData.getString("title")); resultMap.put("0",titleObject); JSONArray pageData = responseData.getJSONArray("pages"); if(pageData.size() >= pageSize) { for(int i = 0; i < pageSize; i++) { resultMap.put(pageData.getJSONObject(i).getString("page"),pageData.getJSONObject(i)); } } else { throw new FrameworkException("所请求分P数量超过实际数量！"); } return resultMap; }

 

4.获取下载地址

av号请求地址示例：https://api.bilibili.com/x/player/playurl?avid=380160&cid=2933938&otype=json

bv号请求地址示例：https://api.bilibili.com/x/player/playurl?cid=2401526&otype=json&bvid=BV1Tx411P7fo

实际返回json：

{ "code": 0, "message": "0", "ttl": 1, "data": { "from": "local", "result": "suee", "message": "", "quality": 32, "format": "flv480", "timelength": 1427200, "accept_format": "flv,flv720,flv480,flv360", "accept_description": [ "高清 1080P", "高清 720P", "清晰 480P", "流畅 360P" ], "accept_quality": [ 80, 64, 32, 16 ], "video_codecid": 7, "seek_param": "start", "seek_type": "offset", "durl": [{ "order": 1, "length": 1427200, "size": 165066794, "ahead": "", "vhead": "", "url": "http://upos-sz-mirrorhw.bilivideo.com/upgcxcode/26/15/2401526/2401526_da3-1-32.flv?e=ig8euxZM2rNcNbN37zUVhoMgnwuBhwdEto8g5X10ugNcXBlqNxHxNEVE5XREto8KqJZHUa6m5J0SqE85tZvEuENvNo8g2ENvNo8i8o859r1qXg8xNEVE5XREto8GuFGv2U7SuxI72X6fTr859r1qXg8gNEVE5XREto8z5JZC2X2gkX5L5F1eTX1jkXlsTXHeux_f2o859IB_&uipk=5&nbs=1&deadline=1599213874&gen=playurl&os=hwbv&oi=1032858202&trid=a8616fed86c74f04b57f82c78237a983u&platform=pc&upsig=c3ae99bb212cb13cfbdb28a6362f3501&uparams=e,uipk,nbs,deadline,gen,os,oi,trid,platform&mid=0&orderid=0,3&agrr=1&logo=80000000", "backup_url": [ "http://upos-sz-mirrorhw.bilivideo.com/upgcxcode/26/15/2401526/2401526_da3-1-32.flv?e=ig8euxZM2rNcNbN37zUVhoMgnwuBhwdEto8g5X10ugNcXBlqNxHxNEVE5XREto8KqJZHUa6m5J0SqE85tZvEuENvNo8g2ENvNo8i8o859r1qXg8xNEVE5XREto8GuFGv2U7SuxI72X6fTr859r1qXg8gNEVE5XREto8z5JZC2X2gkX5L5F1eTX1jkXlsTXHeux_f2o859IB_&uipk=5&nbs=1&deadline=1599213874&gen=playurl&os=hwbv&oi=1032858202&trid=a8616fed86c74f04b57f82c78237a983u&platform=pc&upsig=c3ae99bb212cb13cfbdb28a6362f3501&uparams=e,uipk,nbs,deadline,gen,os,oi,trid,platform&mid=0&orderid=1,3&agrr=1&logo=40000000", "http://upos-sz-mirrorhwb.bilivideo.com/upgcxcode/26/15/2401526/2401526_da3-1-32.flv?e=ig8euxZM2rNcNbN37zUVhoMgnwuBhwdEto8g5X10ugNcXBlqNxHxNEVE5XREto8KqJZHUa6m5J0SqE85tZvEuENvNo8g2ENvNo8i8o859r1qXg8xNEVE5XREto8GuFGv2U7SuxI72X6fTr859r1qXg8gNEVE5XREto8z5JZC2X2gkX5L5F1eTX1jkXlsTXHeux_f2o859IB_&uipk=5&nbs=1&deadline=1599213874&gen=playurl&os=hwbbv&oi=1032858202&trid=a8616fed86c74f04b57f82c78237a983u&platform=pc&upsig=147d8ba29455ad58f529abe187087a7f&uparams=e,uipk,nbs,deadline,gen,os,oi,trid,platform&mid=0&orderid=2,3&agrr=1&logo=40000000" ] }] } }

获取下载地址代码如下：

//获取视频下载地址，根据aid或bvid重载该方法 private LinkedHashMap<String,JSONObject> getUrl(Long avid , int pageSize) { LinkedHashMap<String,JSONObject> cidMap = getCid(avid , pageSize); for(Integer i = 1; i < cidMap.size(); i++) { String cid = cidMap.get(i.toString()).getString("cid"); String responseBody = HttpRequest.get(downloadUrl + "?avid=" +avid.toString() + "&cid=" + cid + "&otype=json") .timeout(20000).execute().body(); JSONObject responseJson = JSONObject.parseObject(responseBody); JSONObject responseData = responseJson.getJSONObject("data"); JSONArray durlArray = responseData.getJSONArray("durl"); JSONObject durlObject = (JSONObject)durlArray.get(0); String url = durlObject.getString("url"); cidMap.get(i.toString()).put("url",url); } log.info(cidMap.toString()); return cidMap; } private LinkedHashMap<String,JSONObject> getUrl(String bvid , int pageSize) { LinkedHashMap<String,JSONObject> cidMap = getCid(bvid , pageSize); for(Integer i = 1; i < cidMap.size(); i++) { String cid = cidMap.get(i.toString()).getString("cid"); String responseBody = HttpRequest.get(downloadUrl + "?bvid=" + bvid + "&cid=" + cid + "&otype=json") .timeout(20000).execute().body(); JSONObject responseJson = JSONObject.parseObject(responseBody); JSONObject responseData = responseJson.getJSONObject("data"); JSONArray durlArray = responseData.getJSONArray("durl"); JSONObject durlObject = (JSONObject)durlArray.get(0); String url = durlObject.getString("url"); cidMap.get(i.toString()).put("url",url); } log.info(cidMap.toString()); return cidMap; }

 

5.进行批量下载

代码如下，注意所有的下载请求均需加入Referer请求头

//流处理 private byte[] readInputStream(InputStream inputStream) throws IOException { byte[] buffer = new byte[1024]; int len = 0; ByteArrayOutputStream bos = new ByteArrayOutputStream(); while((len = inputStream.read(buffer)) != -1) { bos.write(buffer, 0, len); } bos.close(); return bos.toByteArray(); } //进行批量下载，默认flv文件，清晰度480P private void downloadByUrlMap(LinkedHashMap<String,JSONObject> infoMap, String savePath, Object id) { try { String title = infoMap.get("0").getString("title"); for(Integer i = 1; i < infoMap.size(); i++) { URL connURL = new URL(infoMap.get(i.toString()).getString("url")); HttpURLConnection connection = (HttpURLConnection)connURL.openConnection(); connection.setConnectTimeout(3*1000); connection.setRequestProperty("Referer",refererUrl + id.toString()); InputStream inputStream = connection.getInputStream(); byte[] getData = readInputStream(inputStream); File saveDir = new File(savePath + title + "\\"); if(!saveDir.exists()){ saveDir.mkdir(); } File file = new File(saveDir + File.separator + infoMap.get(i.toString()).getString("part") + ".flv"); FileOutputStream fos = new FileOutputStream(file); fos.write(getData); if(fos!=null){ fos.close(); } if(inputStream!=null){ inputStream.close(); } } log.info("提示：" + title + "————总" +(infoMap.size()-1)+ "P已下载成功..."); } catch (Exception e) { e.printStackTrace(); } }

 

6.测试

av号与bv号实际参数可直接从b站获取；下载P数与保存路径自行指定

//测试入口 public static void main(String args[]) { BilibiliUtils bilibiliUtils = new BilibiliUtils(); Long testAvid = 380160L; String testBvid = "BV1Tx411P7fo"; LinkedHashMap<String,JSONObject> testAvInfoMap = bilibiliUtils.getUrl(testAvid,1); LinkedHashMap<String,JSONObject> testBvInfoMap = bilibiliUtils.getUrl(testBvid,1); bilibiliUtils.downloadByUrlMap(testAvInfoMap,"F:\\",testAvid); bilibiliUtils.downloadByUrlMap(testBvInfoMap,"F:\\",testBvid); }

实际控制台输出与文件保存：

16:17:33.103 [main] INFO com.dream.summer.util.BilibiliUtils - {0={"title":"【合集】Kanon 雪之少女【BDRip】【澄空】【1080P】"}, 1={"duration":1453,"vid":"","part":"白銀の序曲(オーバーチュア)～overture～ （银白色的序曲～overture～）","weblink":"","from":"vupload","page":1,"dimension":{"rotate":0,"width":0,"height":0},"url":"http://upos-sz-mirrorhw.bilivideo.com/upgcxcode/38/39/2933938/2933938_da3-1-32.flv?e=ig8euxZM2rNcNbN37zUVhoMgnwuBhwdEto8g5X10ugNcXBlqNxHxNEVE5XREto8KqJZHUa6m5J0SqE85tZvEuENvNo8g2ENvNo8i8o859r1qXg8xNEVE5XREto8GuFGv2U7SuxI72X6fTr859r1qXg8gNEVE5XREto8z5JZC2X2gkX5L5F1eTX1jkXlsTXHeux_f2o859IB_&uipk=5&nbs=1&deadline=1599214646&gen=playurl&os=hwbv&oi=1902277611&trid=6272fc057ad9445382252c9057b41b3bu&platform=pc&upsig=bed90ed44adb10d81c2c86fd1ebcf9df&uparams=e,uipk,nbs,deadline,gen,os,oi,trid,platform&mid=0&orderid=0,3&agrr=1&logo=80000000","cid":2933938}} 16:17:33.509 [main] INFO com.dream.summer.util.BilibiliUtils - {0={"title":"【BD1080P】SOLA【雪飘】"}, 1={"duration":1427,"vid":"","part":"天蓝色的伞","weblink":"","from":"vupload","page":1,"dimension":{"rotate":0,"width":0,"height":0},"url":"http://upos-sz-mirrorhw.bilivideo.com/upgcxcode/26/15/2401526/2401526_da3-1-32.flv?e=ig8euxZM2rNcNbN37zUVhoMgnwuBhwdEto8g5X10ugNcXBlqNxHxNEVE5XREto8KqJZHUa6m5J0SqE85tZvEuENvNo8g2ENvNo8i8o859r1qXg8xNEVE5XREto8GuFGv2U7SuxI72X6fTr859r1qXg8gNEVE5XREto8z5JZC2X2gkX5L5F1eTX1jkXlsTXHeux_f2o859IB_&uipk=5&nbs=1&deadline=1599214646&gen=playurl&os=hwbv&oi=1902277611&trid=613baba6c5d44711b7adfbcf012868c8u&platform=pc&upsig=9e7dbf7a2efd6465d042681cb1995bbe&uparams=e,uipk,nbs,deadline,gen,os,oi,trid,platform&mid=0&orderid=0,3&agrr=1&logo=80000000","cid":2401526}} 16:27:23.985 [main] INFO com.dream.summer.util.BilibiliUtils - 提示：【合集】Kanon 雪之少女【BDRip】【澄空】【1080P】————总1P已下载成功... 16:37:01.757 [main] INFO com.dream.summer.util.BilibiliUtils - 提示：【BD1080P】SOLA【雪飘】————总1P已下载成功... 进程已结束,退出代码0

待改进：使用HttpURLConnection下载文件时暂未显示实时进度；视频画质可在获取下载地址时通过qn、fnval等参数调整

未完待续...