Swarmbot script keeps finding the wrong identical button? Give AI a smaller search area
When a screen repeats the same icon, tell Swarmbot AI which screen, nearby label, and card contain the target. Check the image match before allowing a click.
A Swarmbot script can use an image to find a button on a phone screen. When the same icon appears on several cards, however, the image alone may lead it to the wrong one. This guide is for users who already have an image resource in a Swarmbot script project and need to distinguish lookalike targets. Start on the intended screen, tell Swarmbot AI which card and nearby label identify the right button, and ask it to check the match before adding any click.
Describe the target's context
Imagine an order list with a “View” icon on both a Pending card and a Completed card. “Find View and tap it” leaves the choice ambiguous. A better request is:
I am on the order list on this device connected to Swarmbot. I want the View icon on the Pending card, not the identical icon on the Completed card below it. Use the image saved in this script project. Confirm the screen and the Pending label, then limit the image search to that card. First show me where it matched without tapping. If the Pending card is missing or the area still contains several plausible matches, stop and explain what you need from me.
That gives AI three useful clues: the screen, the neighboring label, and the part of the screen to search. The current Swarmbot JPFindImage script function accepts a search area; ask AI to use the function documentation in the client rather than scanning the whole screen. Image matching identifies visually similar regions, not the business meaning of a Pending order. The OpenCV template matching guide explains that underlying distinction.
Verify the match before allowing a tap
Ask AI to separate finding the icon from tapping it. On one authorized device, check that the returned position lands inside the intended card. On a screen without the Pending card, the script should report no suitable target rather than silently search the entire screen. Only then add the tap and a check that the next screen is the one you expected.
If cards can change order, do not hard-code “always in the top half.” Ask AI to locate the current card from its label first, then search within that card. Recheck the screen after a sign-in prompt, popup, or content change. Share only a cropped, redacted view when account or customer information is visible.
Need a better image resource first? Read how to prepare a button crop for a Swarmbot image-finding script. That guide helps define what the icon looks like; this one helps choose the right instance of that icon.