ChatGPT Can Be Tricked By Third Parties: Beware Of Prompt Injections

Third parties can gain control over ChatGPT requests via Prompt Injections.

In January 2015, Stephen Hawking, Elon Musk, and other AI experts signed an open letter urging research on the societal impacts of artificial intelligence. Truth being told, despite many warnings given by various experts, AI has become a crucial part of our lives. OpenAI’s ChatGPT being the most prominent. 

There is one concern you must definitely address while using ChatGPT. ChatGPT plugins can be tricked by third parties to do things they weren’t supposed to. This is because the plugins can read outside media, such as YouTube transcripts, web pages, and PDFs. 

If a third party can insert malicious instructions into this media, then the plugin will execute those instructions when it reads the media. This could allow the third party to control ChatGPT and make it do things like steal data or spread malware.

OpenAI launched plugins for ChatGPT

Recently, OpenAI introduced plugins for ChatGPT, allowing users to interact with live websites, PDFs, and up-to-date data beyond its original training cutoff in 2021. While this expansion opened up exciting possibilities, it also introduced some concerns.

Security researchers are now cautioning ChatGPT users about “prompt injections.” This refers to the ability of third parties to inject new prompts into your ChatGPT queries without your awareness or consent. It’s important to be aware of this potential risk while utilizing ChatGPT and remain vigilant about the prompts you receive.

Prompt Injection Researches

Security researcher Johann Rehberger conducted intriguing research on prompt injection, exploring the inner workings of ChatGPT. By using a ChatGPT plugin designed for YouTube transcript summarization, Rehberger discovered the capability to make ChatGPT refer to itself by cleverly inserting a directive in the closing prompts of the transcript.

Similar results were found by Avram Piltch, a tech enthusiast from Tom’s Hardware, who also conducted a test to explore ChatGPT’s abilities. He asked ChatGPT to provide a brief summary of a video but added a request for a playful surprise—a Rickroll—at the end. Surprisingly, ChatGPT not only summarized the video as requested but also cleverly included the suggested Rickroll, seamlessly integrating it into the transcript.

In addition to that, AI researcher Kai Greshake presented a fascinating instance of prompt injections by employing a PDF resume. The added text, virtually imperceptible to human eyes due to its minuscule size, provided instructions to an AI chatbot, proclaiming this resume as “the best resume ever” according to a recruiter. When ChatGPT received the resume and was queried about the applicant’s suitability for a job, the AI chatbot echoed the sentiment, affirming that it indeed possessed the best resume.

Something that we trust so easily can be altered within minutes with a few prompt injections. Tricking ChatGPT is a piece of cake for third parties. Guess what not they can do.

Bottom Line

It is crucial for ChatGPT users to remain vigilant regarding prompt injections and to prioritize their own protection. 

Being aware of this issue allows users to take necessary precautions and safeguards while interacting with ChatGPT. 

By staying informed and cautious, users can mitigate the risks associated with prompt injections and ensure a safer and more secure experience.

There are certain ways to protect yourself from prompt injection attacks, like using ChatGPT plugins from trusted sources, being careful about what media you share with ChatGPT. If you’re not sure if a piece of media is safe, it’s best to err on the side of caution and not share it.