select * from xxx where id > 0 limit 1000
然后取最后一个 ID 为 1022
select * from xxx where id > 1022 limit 1000
想开几个线程都可以。

@cr4fun

你的需求很简单 我之前写过一个一键转换文档中英文的插件。
自己维护了一个中文文档-》英文文档的 map
自动更新原理就是把数据放到 github，每次启动浏览器都从 github 请求一些文件。

源码就几行。
https://github.com/DiamondYuan/switch-document/blob/master/background.js

docker-compose 了解一下

和我的想法一样。
我的 OCR 软件 OCR 会发一条推送。推送点击会去 GitHubPage。
然后把结果写入到 URL 链接里。

最终的链接是这样的。


https://diamondyuan.github.io/ocr/?ocr_result=%25E5%25AF%25B9%25E4%25BA%258E%25E4%25B9%25A0%25E6%2583%25AF%25E5%259C%25A8+HotSpot%25E8%2599%259A%25E6%258B%259F%25E6%259C%25BA%25E4%25B8%258A%25E5%25BC%2580%25E5%258F%2591%25E3%2580%2581%25E9%2583%25A8%25E7%25BD%25B2%25E7%25A8%258B%25E5%25BA%258F%25E7%259A%2584%25E5%25BC%2580%25E5%258F%2591%25E8%2580%2585%25E6%259D%25A5%25E8%25AF%25B4%252C%25E5%25BE%2588%25E5%25A4%259A%25E4%25BA%25BA%25E9%2583%25BD%25E6%259B%25B4%25E6%2584%25BF%25E6%2584%258F%25E6%258A%258A%25E6%2596%25B9%25E6%25B3%2595%25E5%258C%25BA+%250A%25E7%25A7%25B0%25E4%25B8%25BA%25E2%2580%259C%25E6%25B0%25B8%25E4%25B9%2585%25E4%25BB%25A3%25E2%2580%259D%2528+Permanent+Generation%2529%252C%25E6%259C%25AC%25E8%25B4%25A8%25E4%25B8%258A%25E4%25B8%25A4%25E8%2580%2585%25E5%25B9%25B6%25E4%25B8%258D%25E7%25AD%2589%25E4%25BB%25B7%252C%25E4%25BB%2585%25E4%25BB%2585%25E6%2598%25AF%25E5%259B%25A0%25E4%25B8%25BA+HotSpot%25E8%2599%259A%25E6%258B%259F%25E6%259C%25BA+%250A%25E7%259A%2584%25E8%25AE%25BE%25E8%25AE%25A1%25E5%259B%25A2%25E9%2598%259F%25E9%2580%2589%25E6%258B%25A9%25E6%258A%258AGC%25E5%2588%2586%25E4%25BB%25A3%25E6%2594%25B6%25E9%259B%2586%25E6%2589%25A9%25E5%25B1%2595%25E8%2587%25B3%25E6%2596%25B9%25E6%25B3%2595%25E5%258C%25BA%252C%25E6%2588%2596%25E8%2580%2585%25E8%25AF%25B4%25E4%25BD%25BF%25E7%2594%25A8%25E6%25B0%25B8%25E4%25B9%2585%25E4%25BB%25A3%25E6%259D%25A5%25E5%25AE%259E%25E7%258E%25B0%25E6%2596%25B9%25E6%25B3%2595%25E5%258C%25BA%25E8%2580%258C%25E5%25B7%25B2%252C%25E8%25BF%2599%25E6%25A0%25B7+%250AHotSpot%25E7%259A%2584%25E5%259E%2583%25E5%259C%25BE%25E6%2594%25B6%25E9%259B%2586%25E5%2599%25A8%25E5%258F%25AF%25E4%25BB%25A5%25E5%2583%258F%25E7%25AE%25A1%25E7%2590%2586Java%25E5%25A0%2586%25E4%25B8%2580%25E6%25A0%25B7%25E7%25AE%25A1%25E7%2590%2586%25E8%25BF%2599%25E9%2583%25A8%25E5%2588%2586%25E5%2586%2585%25E5%25AD%2598%252C%25E8%2583%25BD%25E5%25A4%259F%25E7%259C%2581%25E5%258E%25BB%25E4%25B8%2593%25E9%2597%25A8%25E4%25B8%25BA%25E6%2596%25B9%25E6%25B3%2595%25E5%258C%25BA%25E7%25BC%2596%25E5%2586%2599+%250A%25E5%2586%2585%25E5%25AD%2598%25E7%25AE%25A1%25E7%2590%2586%25E4%25BB%25A3%25E7%25A0%2581%25E7%259A%2584%25E5%25B7%25A5%25E4%25BD%259C%25E3%2580%2582%25E5%25AF%25B9%25E4%25BA%258E%25E5%2585%25B6%25E4%25BB%2596%25E8%2599%259A%25E6%258B%259F%25E6%259C%25BA%2528%25E5%25A6%2582+BEA+JRockit%25E3%2580%2581IBMJ9%25E7%25AD%2589%2529%25E6%259D%25A5%25E8%25AF%25B4%25E6%2598%25AF%25E4%25B8%258D%25E5%25AD%2598%25E5%259C%25A8%25E6%25B0%25B8%25E4%25B9%2585%25E4%25BB%25A3+%250A%25E7%259A%2584%25E6%25A6%2582%25E5%25BF%25B5%25E7%259A%2584%25E3%2580%2582%25E5%258E%259F%25E5%2588%2599%25E4%25B8%258A%252C%25E5%25A6%2582%25E4%25BD%2595%25E5%25AE%259E%25E7%258E%25B0%25E6%2596%25B9%25E6%25B3%2595%25E5%258C%25BA%25E5%25B1%259E%25E4%25BA%258E%25E8%2599%259A%25E6%258B%259F%25E6%259C%25BA%25E5%25AE%259E%25E7%258E%25B0%25E7%25BB%2586%25E8%258A%2582%252C%25E4%25B8%258D%25E5%258F%2597%25E8%2599%259A%25E6%258B%259F%25E6%259C%25BA%25E8%25A7%2584%25E8%258C%2583%25E7%25BA%25A6%25E6%259D%259F%252C%25E4%25BD%2586%25E4%25BD%25BF%25E7%2594%25A8%25E6%25B0%25B8+%250A%25E4%25B9%2585%25E4%25BB%25A3%25E6%259D%25A5%25E5%25AE%259E%25E7%258E%25B0%25E6%2596%25B9%25E6%25B3%2595%25E5%258C%25BA%252C%25E7%258E%25B0%25E5%259C%25A8%25E7%259C%258B%25E6%259D%25A5%25E5%25B9%25B6%25E4%25B8%258D%25E6%2598%25AF%25E4%25B8%2580%25E4%25B8%25AA%25E5%25A5%25BD%25E4%25B8%25BB%25E6%2584%258F%252C%25E5%259B%25A0%25E4%25B8%25BA%25E8%25BF%2599%25E6%25A0%25B7%25E6%259B%25B4%25E5%25AE%25B9%25E6%2598%2593%25E9%2581%2587%25E5%2588%25B0%25E5%2586%2585%25E5%25AD%2598%25E6%25BA%25A2%25E5%2587%25BA%25E9%2597%25AE%25E9%25A2%2598%2528%25E6%25B0%25B8%25E4%25B9%2585+%250A%25E4%25BB%25A3%25E6%259C%2589-+XX%253A+Max+Perm+Size%25E7%259A%2584%25E4%25B8%258A%25E9%2599%2590%252CJ9%25E5%2592%258C+JRockit%25E5%258F%25AA%25E8%25A6%2581%25E6%25B2%25A1%25E6%259C%2589%25E8%25A7%25A6%25E7%25A2%25B0%25E5%2588%25B0%25E8%25BF%259B%25E7%25A8%258B%25E5%258F%25AF%25E7%2594%25A8%25E5%2586%2585%25E5%25AD%2598%25E7%259A%2584%25E4%25B8%258A%25E9%2599%2590%252C%25E4%25BE%258B%25E5%25A6%258232+%250A%25E4%25BD%258D%25E7%25B3%25BB%25E7%25BB%259F%25E4%25B8%25AD%25E7%259A%25844GB%252C%25E5%25B0%25B1%25E4%25B8%258D%25E4%25BC%259A%25E5%2587%25BA%25E7%258E%25B0%25E9%2597%25AE%25E9%25A2%2598%2529%252C%25E8%2580%258C%25E4%25B8%2594%25E6%259C%2589%25E6%259E%2581%25E5%25B0%2591%25E6%2595%25B0%25E6%2596%25B9%25E6%25B3%2595%2528%25E4%25BE%258B%25E5%25A6%2582+String.+intern%2529%25E4%25BC%259A%25E5%259B%25A0%25E8%25BF%2599%25E4%25B8%25AA%25E5%258E%259F+%250A%25E5%259B%25A0%25E5%25AF%25BC%25E8%2587%25B4%25E4%25B8%258D%25E5%2590%258C%25E8%2599%259A%25E6%258B%259F%25E6%259C%25BA%25E4%25B8%258B%25E6%259C%2589%25E4%25B8%258D%25E5%2590%258C%25E7%259A%2584%25E8%25A1%25A8%25E7%258E%25B0%25E3%2580%2582%25E5%259B%25A0%25E6%25AD%25A4%252C%25E5%25AF%25B9%25E4%25BA%258E+Hotspot%25E8%2599%259A%25E6%258B%259F%25E6%259C%25BA%252C%25E6%25A0%25B9%25E6%258D%25AE%25E5%25AE%2598%25E6%2596%25B9%25E5%258F%2591%25E5%25B8%2583%25E7%259A%2584%25E8%25B7%25AF%25E7%25BA%25BF%25E5%259B%25BE%25E4%25BF%25A1+%250A%25E6%2581%25AF%252C%25E7%258E%25B0%25E5%259C%25A8%25E4%25B9%259F%25E6%259C%2589%25E6%2594%25BE%25E5%25BC%2583%25E6%25B0%25B8%25E4%25B9%2585%25E4%25BB%25A3%25E5%25B9%25B6%25E9%2580%2590%25E6%25AD%25A5%25E6%2594%25B9%25E4%25B8%25BA%25E9%2587%2587%25E7%2594%25A8+Native+Memory%25E6%259D%25A5%25E5%25AE%259E%25E7%258E%25B0%25E6%2596%25B9%25E6%25B3%2595%25E5%258C%25BA%25E7%259A%2584%25E8%25A7%2584%25E5%2588%2592%25E4%25BA%2586e%252C%25E5%259C%25A8%25E7%259B%25AE%25E5%2589%258D%25E5%25B7%25B2+%250A%25E7%25BB%258F%25E5%258F%2591%25E5%25B8%2583%25E7%259A%2584JDK17%25E7%259A%2584+HotSpot%25E4%25B8%25AD%252C%25E5%25B7%25B2%25E7%25BB%258F%25E6%258A%258A%25E5%258E%259F%25E6%259C%25AC%25E6%2594%25BE%25E5%259C%25A8%25E6%25B0%25B8%25E4%25B9%2585%25E4%25BB%25A3%25E7%259A%2584%25E5%25AD%2597%25E7%25AC%25A6%25E4%25B8%25B2%25E5%25B8%25B8%25E9%2587%258F%25E6%25B1%25A0%25E7%25A7%25BB%25E5%2587%25BA%25E3%2580%2582

任何一个支持 js 的网站，只要访问了就有可能中。包括微信小程序。

浏览器就可以做到了。

很重要的多部构建没有提到。可以极大的缩小镜像体积。

@WuMingyu 分布式就是多个爬虫共享进度 任务 不会重复。

感觉爬虫难点在于 IP 反爬 分布式调度。楼主这方面有轮子推荐吗。

@SingeeKing 服务号认证要 300

看到塞尔达了

@teleme 七牛的证书需要手动配置的。

ctrl u

各种背单词软件。
抓包爬虫，啥都可以干。今天刚刚写完一个单词软件的爬虫。


翻译可以爬柯林斯。但是要注意版权问题，自用可以，商业化应该不行。

@hahamy 什么网站

阿里云

文件名是 项目名称+uuid+版本 hash 理论上来说绝对不可能重复。

部署就是把 index.html 复制到 nginx 容器里部署。

我是写了一个 webpack 七牛插件，webpack build 快完成的时候，把文件列表传到后端服务获取上传 token。然后前端再根据 token 上传到七牛。构建完成后就已经把全部 css 和 js 发布成功了。

如果有公司账户 你只需要写一个前端页面提供然后 web view 就可以了。