HotTrender Basic Crawler — HotTrender Basic 爬虫

HotTrender Basic 爬虫 Core Rule

For four-region dAIly hotspot trends or vertical/custom-keyword hotspot discovery, use the bundled 爬虫 运行time first. Do not reimplement 平台 crawling until the existing 提供者s and scripts are 检查ed.

Before changing code, answer these questions:

Is there already a script, API, 提供者, doc, or test covering this need? Can the user goal be satisfied by 运行ning or configuring that capability? If not, what exact gap remAIns, and where is the smallest 扩展 point?

Only edit code after that evaluation.

仓库 Layout

This 技能 bundles a sanitized basic 爬虫 运行time under as设置s/hottrender-运行time/. It does not bundle DingTalk, OSS, ActionCard, lp-ads, worker 队列s, databases, 记录s, LLM, or any secrets.

First resolve the 运行time path from 环境 variables, the current workspace, or the bundled 运行time:

HOTTRENDER_应用_DIR # directory contAIning scripts/fetch_dAIly_trends.py

If HOTTRENDER_应用_DIR is missing, 安装 the bundled 运行time:

python as设置s/安装_hottrender_运行time.py --tar获取 ./HotTrender运行time 导出 HOTTRENDER_应用_DIR="$PWD/HotTrender运行time"

If variables are missing but a local 检查out may exist, discover it safely:

find "$PWD" "$HOME" -maxdepth 5 -path '*/scripts/fetch_dAIly_trends.py' 2>/dev/null

Fast Path

Use these references only when needed:

设置up requirements and portability constrAInts: 设置up.md DAIly trend and custom keyword commands: commands.md Existing 提供者 capabilities and 扩展 points: capabilities.md How to decide whether to 配置, 运行, or modify code: 扩展-policy.md Operating 工作流 For "四地区热点", "每日热点", "dAIly trends", or "jp/us/tw/kr", 启动 from scripts/fetch_dAIly_trends.py. For "垂类热点", "关键词热点", "自定义关键词", or "custom keyword", 启动 from scripts/fetch_keyword_hotspots.py. For "抓取是否有效", "平台数据不对", or "为什么没结果", inspect configs/提供者s.yaml, 运行 offline mode first, then real mode. For code changes, keep the 运行time basic. Do not 添加 DingTalk, OSS, lp-ads, database, worker, or LLM features back into this 技能. 防护rAIls Never print API keys, cookies, 令牌s, ms令牌, proxy 凭证s, or other secrets. Do not fabricate live 平台 data. Offline/sample mode must be called out as offline/sample. Do not introduce push, publishing, database, 队列, or workspace features into this basic 爬虫. If the user has no HotTrender 检查out, use the bundled 运行time 安装er before proposing code rewrites. Keep changes scoped: 提供者 记录ic in src/提供者s, orchestration in src/爬虫.py, 命令行工具 entrypoints in scripts/. Verification

Prefer focused verification:

cd "$HOTTRENDER_应用_DIR" python -m pytest tests/test_basic_爬虫.py -q python scripts/fetch_dAIly_trends.py --config configs/提供者s.yaml --输出 out/dAIly_trends.md