I like simple sayings that communicate complex ideas succinctly. As I experiment with ChatGPT, one mantra by Alistair Carruthers (French, Maule & Papamichail 2009) about problem-solving keeps coming to mind. It summarizes the 4 steps one should follow when solving problems.

1. Create questions
2. Question questions
3. Answer questions
4. Question answers

For a long time now, advances in data analytics have been accelerating capabilities to perform the third step, Answer questions. ChatGPT’s launch is clearly an evolutionary leap forward in this advancement, but it is only a prototype. Generative AI is likely to only improve at an exponential rate from here.

I find some solace in the mantra above because while “answers” only appears in two steps, “questions” exist in each one. It reminds me that as the Answer Questions task becomes more-automated, humans should respond by focusing more on how to ask the right questions at the proper time.

Ask Questions

ChatGPT responds to the questions asked. Asking the AI better questions yields better answers. This has always been true in decision-making processes. The first imperative is to ask the right questions. This is not easy, and it is often done only superficially in the urgency to find some answer quickly. As John Tukey (1962) said, however, “Far better an approximate answer to the right question, …. than an exact answer to the wrong question.” So, spending more time in this step is likely to be rewarding, up to a point.

Asking the right questions generally starts with a statement of what the problem seems to be. This may be a “mess statement” to start the process in the right ball park, or it may be a description of a symptom of what is suspected to be a larger problem, like a patient seeing a doctor about a high temperature. More questions should follow to explore the “problem space” and “dig deeper” Into potential root causes. There are many, well-developed techniques to do this from the operational research, project management, and creativity disciplines.

It is also important to look at the problem from the perspective of key stakeholders. Their stakes in the outcomes should be weighed by the decision-maker to guide and evaluate future answers. While this step may yield one overriding question, it is more likely to result in a network of interconnected questions. This is particularly true if the problem is more strategic than tactical.

Question Questions

Questions from the first step will likely require additional refinement to make them suitable for analysis. For instance, when designing customer surveys for market research, one needs to think carefully about how the answers will be perceived and interpreted by the respondents.

The questions should also be screened for unintended biases or loaded words that trigger adverse responses. All of this is also true for conversations with generative AI.

Answer Questions

Answering questions is where data analytics has been making rapid progress. As AI provides answers with unnerving speed and confidence, the danger is that the source of and reasoning behind those answers becomes a black box that people are unable to evaluate. This already happens with my students when using simple Excel formulas in spreadsheets. How much more likely are they to do this with the answers provided by ChatGPT and its competitors.

What is the motivation to learn to do things the long way yourself when the easier option is so available? The motivation should be a healthy fear of relying on uninvestigated answers to make decisions (or complete homework), but that fear often requires bad experiences later in life or a career to develop.

Question Answers

This brings us to the importance of the last phase, Questioning Answers. In order to make decisions from information, one should be able to verify its credibility and explain to others what it means. Particularly in this age of fake information, questioning the credibility of the sources and the robustness of analytical results is more important than ever.

ChatGPT’s lack of sourcing is currently a major weakness. One should also screen for errors, however unintended, that might have biased the results. Last mentioned here, but most importantly, one should screen the answers for consistency with one’s values and prior beliefs. There is nothing inherently ethical about AI.

Takeaway

ChatGPT is well-named. It is a conversation in which answers are given to the question asked. Instead of worrying that one’s skills in developing answers are being obsoleted by AI’s advances, we should focus on improving our ability to ask the right questions at the right time throughout the conversation.

Picture: www.shutterstock.com