Invisible Text on Android Screens: A New Threat to AI Assistants
Researchers find vulnerabilities in AI agents controlling Android phones, with risks extending to PCs.
Key points
- Researchers showed 7 attacks on 5 open-source Android AI agent frameworks in October 2023.
- An Android app can hide text on screens that AI agents read and follow.
- One attack can jump from a phone to a PC, executing commands remotely.
A new research study has uncovered serious vulnerabilities in AI agents, which are pieces of software that can read your screen and interact with it like a human would, on Android phones. This study, first reported by ThreatVectr, reveals how such agents, used in five open-source frameworks like AppAgent and AppAgentX, can be hijacked by invisible text planted by malicious apps.
What did the researchers discover?
Researchers demonstrated that by leveraging common Android permissions, a rogue app can display text invisible to humans but visible to AI agents. The AI, unable to distinguish between genuine user commands and these hidden instructions, can be tricked into performing tasks it should not, like taking screenshots or sending data back to a controlling PC. The failure lies in the AI's inability to differentiate between real and malicious text inputs.
Should users be worried?
For now, there's no immediate threat to typical users. The frameworks tested are research projects, not consumer apps you might download from the Google Play Store. However, as AI assistants that can see and act for you are developed by big tech companies like Google and Apple, these vulnerabilities could become more relevant. AI agents in commercial products might inherit similar weaknesses since they work by reading screen pixels.
How does the attack work?
The attack starts with an app asking for two common Android permissions: one to draw over other apps (think chat bubbles) and another to write files to shared storage. These are standard requests, often seen in legitimate apps. The malicious app then writes invisible text on the screen, which the AI agent reads and acts upon. In the most severe attack, the AI can be made to run commands on a connected PC, creating a significant security breach.
| Attack Steps | Description |
|---|---|
| Step 1 | App requests draw and write permissions. |
| Step 2 | Invisible text is placed on the screen. |
| Step 3 | AI agent reads and executes hidden commands. |
| Step 4 | Commands can be relayed to a controlling PC. |
What can users do?
While the technology progresses, users should scrutinize apps requesting permissions to draw over other apps or access shared storage. Free utilities, wallpapers, and keyboards asking for these permissions should be treated with caution. Until screen-reading AI agents can safely handle untrusted inputs, being selective with app permissions is your best defense.
Common questions
Can my phone be hacked by invisible text today?
Currently, the risk is low since the vulnerable frameworks are not consumer-facing. However, staying vigilant about app permissions is wise.
Should I avoid all apps asking for draw permissions?
Not necessarily, but only grant these permissions to apps from reputable developers and when the function is essential for the app's operation.
