Using ChatGPT (and other Generative AI) as a development tool — tread carefully

Developers are particularly concerned. After all, a simple, straightforward request can get ChatGPT to generate perfectly usable code in seconds. But I’m not concerned about generative AI replacing developers anytime soon, if ever. I think there will always be a need for human developers who can solve complex problems. And even for simple tasks, no responsible company would outsource all development to ChatGPT. Along with other generative AI platforms, ChatGPT still suffers from a significant hallucination problem. Several publications have published articles generated entirely by generative AI only to discover, to their great embarrassment, that though these pieces read well, they were rife with errors and even contained facts that had been completely fabricated. 

The same thing can definitely happen with code. There’s no doubt that ChatGPT can generate code, but it may need important adjustments — and those adjustments require the attention of a skilled coder. So, for the foreseeable future, ChatGPT is not a threat to developers. But even with its limitations, if used properly, it can still be a very powerful tool that can make the coding process significantly more efficient in the hands of an experienced developer.

How we use ChatGPT

If you use a well-formed prompt, ChatGPT can enable a developer to get to a solution faster than without its help. Of course, we don’t use the code as-is — that would be reckless — but code generated by ChatGPT does provide a good framework on which to build. But don’t expect ChatGPT to help you build large, complex functionality. You can’t type, “Provide the underlying code to create a consumer hotel reservation application,” and expect to get anything useful. Likely, you won’t get anything at all. But for specific, discrete tasks that coders find themselves doing over and over? ChatGTP is perfect. Need to shape a suite of automated tests or make a data connector between two applications — ChatGPT is up to the task.

The more unique and complex your task is, the less likely you are to get a usable code framework from ChatGPT. And when you do get code, treat it as you would something you pulled off a forum or from Stack Overflow. After all, these kinds of forums are very likely the source on which ChatGPT is basing the code it generated. Make sure you understand exactly how the code works to make sure it does what you want it to do and not something else, and make sure it’s properly formatted without errors. 

ChatGPT can also help developers gain knowledge. New languages and frameworks appear almost daily. It’s impossible to keep up. Great with Python, but a novice with Ruby? Tell ChatGPT to provide examples of concepts you don’t quite grasp. Faced with a completely new language and you don’t know what a higher-order function looks like? ChatGTP is there to help.

Another great use case for ChatGPT is producing QA tests. Certainly, a lot of testing these days is automated, but typically a developer still needs to code up the tests. With a strong prompt that provides a comprehensive description of the test, ChatGPT can generate a great foundation to work from, and an experienced developer can generate even better iterations with further prompts that enrich the context. All that’s required is a few minutes of careful code review and a few adjustments to create a working test. It can save everyone a ton of time.  

Also, developers, I’m sad to say this but, ChatGPT can reduce the amount of time you have to spend writing documentation. Yes, I know this is your favorite task, and that you dream of writing documentation even on your days off, just for fun, but there’s no need. ChatGPT does a fine job of creating documentation for routine stuff like the outputs and inputs to expect from a connector. All kidding aside, you’ll need to review the documentation carefully, but even then, it will allow you to spend so much more time doing the tasks you actually enjoy doing -- solving complex problems for customers and clients. 

You can also use ChatGPT on an ongoing basis to automate tasks. Auto-GPT is an AI agent that performs functions on a loop. There are tons of uses for this, but if you’re a development shop, you could have it send you a weekly list of companies with at least 200 employees with a Wordpress-based web page that’s more than four years old — not a bad lead generation tool. 

ChatGPT best practices for developers

So, we’ve told you to be careful with ChatGPT, but what exactly does that mean in practice? Here’s the best practices we’ve adopted internally when we take advantage of this powerful new tool. 

• Make your requests detailed and specific: Include plenty of relevant context and don’t forget to tell ChatGPT the format in which you want your feedback. Otherwise, you might get a big block of useless text and a tiny snippet of code. Here’s an example of a request I might make: “Provide sample code that will connect a Postgres database hosted in Microsoft Azure using Ruby as my language. Include a connection string that replaces connection parameters with a curly brace.”

• Know what acceptable code and solid answers should look like: ChatGPT is a great source of knowledge, but it can all go awry in the hands of inexperienced programmers. If you don’t have the experience to see when something is off in the code, you simply can’t use ChatGPT effectively, because the risk of committing buggy code is simply too great.

• Don’t share anything you wouldn’t want to be published publicly: ChatGTP is still training, and it can use your prompts for that purpose. See this in the ChatGPT FAQ:

Who can view my conversations?

As part of our commitment to safe and responsible AI, we review conversations to improve our systems and to ensure the content complies with our policies and safety requirements.

Will you use my conversations for training?

Yes. Your conversations may be reviewed by our AI trainers to improve our systems.

So, unless you’re comfortable with your prompt turning up as a result in someone else’s query, don’t use it. The terms for ChatGPT API say they won’t use input for training, so using it instead of the Web interface may provide some protection. Nevertheless, be cautious. Better safe than sorry. 

Generative AI is going to transform how developers work, no doubt, and it’s impossible to predict what the profession will look like a decade from now. So, best to start learning how to take advantage of generative AI now. In the right hands, it’s a powerful tool that creates more time to focus on those tasks that require the attention of a skilled developer.