replied to a while ago:
I'm not saying you should, but when I'm designing things and want the agent to be able to use its vision facilities - I get it to write Playwright tests and sometimes even chunk the screenshots to isolate specific areas.
Last job I was building a kiosk for a client and got my agent to make standalone scripts to get desired output so here's an example you could feed your agent for inspiration if you want:
replied to:Thanks! Has been fixed now (I think).