Help Swarmbot AI write an image-finding script: give it a useful button crop

To help a Swarmbot script locate a phone button, provide more than “tap Confirm.” Identify the page, help the Script Assistant save a distinctive crop from the target device as a script image, then check the match before any tap.

A Swarmbot image-finding script can use a small picture of a button or icon to locate its counterpart on the target device's current screen. This is useful for an Android phone or supported Jinpeng cloud phone when the label is unreliable but the visual target is fairly stable. In Swarmbot's script development view, open and save the script project, choose the authorized device and target page, and ask the Script Assistant to prepare an image resource before writing a chain of taps.

Describe the target, not just the tap

“Click Confirm” leaves too much to guess. A more useful request is:

On this app's confirmation page, find the blue Confirm button at the lower right. First check the selected device and page. Capture the button with a little distinctive surrounding detail from the device's original screenshot and save it as an image in this script project. Test the match without tapping. If other buttons look similar, tell me what else you need to distinguish them.

The page, nearby visual clues, and the instruction not to tap yet all matter. If the screen shows an account, private messages or customer information, use a safe test page or provide only a crop without that material. Do not paste a whole private screen into a public question.

Make a crop the script can use

A Swarmbot script project keeps its image resources under images/. Its region-screenshot capability can save a crop from the target device into the current project. Ask the AI to confirm the project has been saved and the right device is selected, then use the region-capture entry described in the current local documentation. You can also select part of a screenshot in the development view and save it to the resource panel. For actual image matching, however, check that the saved image retains the target device's original pixel size and orientation. A crop cut from a scaled device-wall preview is not automatically a suitable template.

Keep the crop focused, but not so tight that it contains only a flat patch of color. Include the button text, icon outline or a little distinctive edge. Avoid clocks, notification badges, avatars and rotating ads. If two buttons look alike, give the AI the page position, adjacent text or a restricted search area as well. OpenCV's template-matching explanation describes looking for a patch similar to a smaller image inside a larger one. A visual match alone does not establish that it is the right button for your task.

Check the match before adding an action

Ask the AI to consult Swarmbot's current script-function documentation for JPFindImage and reference the image just saved. On one authorized device, first find without tapping. Does it match on the intended page? Is the reported location actually on the right button? Does it avoid a match on a page where that button should not exist? A miss or resource error should be reported as such, not replaced with a guessed coordinate. Swarmbot's image search does not tap by itself or automatically rescale a template across screen resolutions; recheck the crop after a change in device orientation, UI scale or app design.

Only after the target checks out should the AI connect the search, a conditional decision and the next action, then verify that the screen reached the intended state. Start with one device and a small reversible flow rather than a fleet. For the broader workflow, see our introduction to AI-assisted Swarmbot scripting. Check the public Swarmbot release information for the current version.

参考来源