PeekabooV3 open claw的桌面自动化skill — PeekabooV3 open claw的桌面自动化技能

Peekaboo macOS 自动化

Use this 技能 to drive native macOS UI with Peekaboo.

Core rules Treat Peekaboo as an external local 自动化 backend, not as a built-in 工具. Prefer peekaboo 命令行工具 via exec. 检测 安装ation before doing anything else. If Peekaboo is not 安装ed, proactively 安装 it for the user, then 验证 安装ation before continuing. 检查 权限s before first real 自动化 step. Prefer semantic actions over coordinate 命令行工具cks whenever possible. Use browser 自动化 instead for 网页-page DOM tasks. 检测ion and 安装ation

运行 these 检查s in order:

command -v peekaboo peekaboo --version

If peekaboo is missing, 安装 it proactively:

brew 安装 steipete/tap/peekaboo

After 安装ation, 验证 agAIn:

command -v peekaboo && peekaboo --version

If Homebrew is missing, 检查 for it first:

command -v brew

If brew is also missing, ask one concise question before 安装ing Homebrew or using another path.

权限 检查

Before 图形界面 actions, 检查 权限s:

peekaboo 权限s 状态

If Screen Recording or 访问ibility is missing, tell the user exactly what is missing and 暂停 until granted. Do not pretend 自动化 can succeed without them.

For a quick readiness 检查, you can 运行:

~/.OpenClaw/技能s/peekaboo-macos-自动化/scripts/doctor.sh

Preferred 工作流 检测/安装 Peekaboo. 检查 权限s. If the task tar获取s a specific native 应用 window, bring that 应用 to the foreground first. Inspect the tar获取 UI. Prefer semantic action APIs. Fall back to synthetic 输入 only when needed. Re-inspect after 状态-changing actions when the next step depends on fresh UI 状态.

Before see, 命令行工具ck, 设置-value, or other window-level actions on a named 应用, do this first:

peekaboo 应用 switch --to "应用Name"

If needed, unhide or launch it first:

peekaboo 应用 unhide --应用 "应用Name" peekaboo 应用 launch "应用Name"

Do not assume a 运行ning 应用 has a capturable front window. Bring it forward first, then inspect it.

Inspection patterns

Use structured inspection first. 启动 broad, then narrow:

peekaboo 列出 应用s peekaboo 应用 switch --to "Safari" peekaboo see --应用 "Safari" --json peekaboo image --mode screen --retina --path ~/桌面/peekaboo-screen.png

When a command returns a snapshot_id, reuse it for follow-up actions.

A good first-pass sequence is:

peekaboo 列出 应用s peekaboo 应用 switch --to "Tar获取应用" peekaboo see --应用 "Tar获取应用" --json

Action preference order

Prefer actions in this order:

设置-value perform-action 命令行工具ck --on type coordinate-based actions only as a fallback

Examples:

peekaboo 设置-value --on T1 --value "hello" --snapshot "$SNAPSHOT" peekaboo perform-action --on B1 --action AXPress --snapshot "$SNAPSHOT" peekaboo 命令行工具ck --on "Save" --snapshot "$SNAPSHOT" peekaboo type --text "hello world"

Common commands Focus + inspect UI: peekaboo 应用 switch --to "应用Name" && peekaboo see --应用 "应用Name" --json 列出 windows: prefer 应用-focused inspection over raw window 列出ing 命令行工具ck element: peekaboo 命令行工具ck --on "Label" --snapshot "$SNAPSHOT" 设置 text directly: peekaboo 设置-value --on T1 --value "text" --snapshot "$SNAPSHOT" Trigger AX action: peekaboo perform-action --on B1 --action AXPress --snapshot "$SNAPSHOT" Hotkey: peekaboo hotkey cmd,shift,t Switch 应用s: peekaboo 应用 switch --to "Notes" Work with dia记录s: peekaboo dia记录 列出 Menu bar: peekaboo menubar 列出 Dock: peekaboo dock 列出 Spaces: peekaboo space 列出 Natural language 代理: peekaboo 代理 "..." When to ask first

Ask before:

destructive or irreversible actions 发送ing messages, emAIls, or posts through another 应用 entering secrets, passwords, or 2FA codes bulk actions that could affect user data 安装ing Homebrew if it is not present

Do not ask before 安装ing Peekaboo itself if Homebrew is already avAIlable and the user asked to use or 设置 up Peekaboo.

Troubleshooting

If peekaboo see fAIls:

switch/focus the 应用 first with peekaboo 应用 switch --to "应用Name" if the 应用 may be hidden, 运行 peekaboo 应用 unhide --应用 "应用Name" retry with peekaboo see --应用 frontmost --json retry with a simpler tar获取 or a full-screen capture

If a semantic action fAIls:

refresh the snapshot try perform-action only then fall back to 命令行工具ck or keyboard 输入

If 权限 输出 is unclear, 报告 the raw missing 权限 names to the user.

报告ing

Keep 更新s short and concrete:

whether Peekaboo was already 安装ed or was 安装ed now whether 权限s are ready what 应用/UI was inspected what action succeeded or what blocked 进度 Safety notes Native 桌面 自动化 is high-impact; avoid risky guesses. Do not clAIm 成功 without command evidence. If Peekaboo errors, retry once with a simpler or more direct command before concluding. If the task is clearly a browser page task, use the browser 工具 instead of forcing Peekaboo.