I’ve been through a couple of booms in my career, the most notable would have to be the dot-com boom. This was a paradigm shift in how things worked and what businesses did. We went from a world where the internet was interesting to people in academia to a world where everyone had to have a web presence.
Web developers appeared in the world and have gradually specialized and split into subgroups, frontend, backend, devops, site reliability engineers, etc… (the list of job titles is endless)
Everything settled down for a while, and then the app store revolution appeared. Now it wasn’t enough just to have a website - you had to have a mobile app as well!
A new type of developer appeared - mobile app developers, with sub species of iOS and Android specialists.
We’re now faced with a third boom. OpenAI released ChatGPT and suddenly AI went mainstream. ChatGPT gained 1 million users in just 5 days and is probably approaching 200 million users now.
There are no official numbers on how many users are actually paying for ChatGPT+ but surveys have suggested that up to 10% of users would be willing to pay for it. If we look at revenue announcements from OpenAI, we can see that they predict ChatGPT will generate \$200 million in revenue by the end of 2023. Given the monthly cost of \$20 for a ChatGPT+ subscription, we can work out that there are probably around 800,000 paying subscribers.
OpenAI dropped another bombshell in March and announced ChatGPT plugins.
We’ll soon be faced with another question to add to the “Do I need a website? - Yes”, “Do I need an App? - Maybe”.
We’ll now have to ask: “Do I need a ChatGPT plugin?”
At the same time, you’ll probably be asking yourself: “For goodness sake, we’ve already got a frontend web team, a backend web team, an iOS team, and an Android team… Now we’re going to have to hire a plugin team! When does it end?”
Well, it may not be as bad as it first appears.
Exactly What Are ChatGPT Plugins?
One of the big limitations with ChatGPT is that its knowledge cuts off after September 2021. The other limitation is that it can’t interact with anything outside of its chat. ChatGPT plugins enable it to access and provide up-to-date information, perform computations and execute actions on third-party services, and interact with a wide range of external applications. This enhances the AI’s ability to provide current, relevant, and actionable responses to user queries.
Creating a ChatGPT plugin might sound complex, but the reality is quite the opposite. Unlike the complexities of website and app development, building a plugin for ChatGPT doesn’t require a team of highly trained specialists. If your team of developers knows how to make an API, they already have what they need to create a plugin.
One of the biggest barriers to entry at present is simply the fact that developers need to join a waitlist for access and access is being rolled out extremely slowly.
How Do Plugins Work
The magic of plugins lies in their seamless integration with ChatGPT. The language model does the heavy lifting, interpreting user requests and mapping them onto your plugin API.
As part of your plugin, you provide some simple text describing what your plugin is and how to use it. You also provide an OpenAPI document (these used to be called Swagger files) - your development team will be very familiar with these as they will often be using them when they integrate APIs from third parties.
This is all ChatGPT needs to build calls to your API. The fact it can do this is both overwhelming and underwhelming at the same time. Overwhelming in that it can take extremely vague requests from the user (e.g. “I’m going to bed”) and translate them into concrete API calls (e.g. get the list of lights, select the appropriate light for the bedroom, switch it on). It’s underwhelming because everyone has seen how good ChatGPT is at generating and understanding code. It’s actually quite a fun exercise to get ChatGPT to generate all the required code for a plugin - there are even plugins now that will generate new plugins!
To Create or Not to Create a Plugin
While a plugin’s utility is undeniable, the challenge lies in defining a unique, valuable purpose for it. The real problem is not, “How do we build a plugin?” but, “What purpose will our plugin serve and who will use it?”
Plugin Security
How do we keep our plugin secure. Here are some easy recommendations:
- Build an API specifically for your plugin - don’t just point ChatGPT at your API and let it run wild!
- Lock the IP address range that can use the plugin to ChatGPT’s address range.
- Alternatively, implement service authentication so you know it’s ChatGPT talking to you.
- Add rate limiting and other anti-abuse safeguards to your API. ChatGPT will send you a conversation id along with an ephemeral user identifier in its requests so you can easily block and control access.
If you want to provide data that is specific to users on your system, then you can support OAuth login.
If you do have users logging in and performing actions on their data, then the usual security guidelines apply. The API should only be able to do things that the logged-in user is authorized to do.
There are a lot more complex questions that need to be thought about - what’s happening with your data - it’s being fed back into ChatGPT - which means it will be sent to servers that you have no control over. There are implications here around data security, GDPR, etc… it’s potentially a minefield.
Managing Plugin Behavior
One of the challenges developers will face is making ChatGPT behave in the way they want it to. If you’ve used ChatGPT, then I’m sure you will have experienced it not quite doing what you wanted it to, or interpreting your wishes in a different way from what you intended.
Guiding ChatGPT on how to use your plugin is more of an art than a science - you’ll have heard the term “prompt engineering” - sounds complicated, but it’s really just a case of experimenting to find the right way of expressing things to get the model to do what you want.
The “description for the model” in the plugin’s manifest file along with documentation in the OpenAPI specification file provide the main guidance. Treat ChatGPT like a person trying to manually operate your API. Your plugin should be coded defensively and provide good error messages when things go wrong.
For example, if you have parameters that can only take a limited set of values (e.g. country codes), then make sure these are validated - if ChatGPT sends an incorrect value, then it’s very helpful to send back an error with a list of valid values. ChatGPT will do its usual “I apologize….” and then try again with hopefully a correct value.
Another very useful thing to know is that in the responses to ChatGPT you can include an extra field EXTRA_INFORMATION_TO_ASSISTANT this can be used to help guide ChatGPT on what to do with the response and how to proceed.
For displaying data, Markdown is your friend. You can send back Markdown to show images, tables, and formatted text. It’s often helpful to include a hint in the EXTRA_INFORMATION_TO_ASSISTANT telling ChatGPT to use the markdown and show it to the user.
What you should be doing now
There’s no denying the potential of plugins in shaping the future of AI. Whether you’re a large company or an independent developer, it’s worth investing some time to experiment with plugin development to see what’s possible.
If you’re responsible for IT or directing development, then make sure you have someone tasked with keeping on top of what’s happening. At some point, you’re going to be asked about plugins and it’s always good to be prepared!
It’s clear that plugins offer a valuable opportunity to extend the capabilities of AI in a practical, user-friendly way. The question is, will you be ready when the time comes?
You can see an interesting plugin in action here that shows how good ChatGPT is at interpreting vague instructions:
You can find lots of other demos on the web showcasing the built in plugins and third party plugins. It’s going to be a wild ride.
