AI Fails to Predict…The Future?

If you had “Russian uprising” and “Titanic-related disaster” on your 2023 bingo card, congratulations! You’re either a time traveler from the Edwardian Era or very, very prescient. But the news is full of stories that could never have happened 100 years ago. Crypto prices are inching back up, despite the fact that so many promoters are wearing ankle monitors (Sam Bankman-Fried), sitting in jail (Do Kwon), on the lam (Kyle Davies and Su Zhu), or facing subpoenas (pretty much everyone else). And artificial intelligence continues to dominate headlines, especially as more experts warn AI could someday pose an existential threat to human life. Most of that discussion centers on GPT-4, a “multimodal large language model” that resembles what might happen if Google’s autofill got bitten by a radioactive spider.

But AI is still far from infallible. Back in May, two New York lawyers used ChatGPT (an earlier model of GPT-4) to write a brief on behalf of a client suing Avianca. The brief cited more than half a dozen court decisions, including Martinez v. Delta Air Lines, Zicherman v. Korean Air Lines and Varghese v. China Southern Airlines. Unfortunately, none of them were real. ChatGPT had simply made them up. (Oops.) Last week, the judge in the case fined the lawyers $5,000. You already know it’s a bad idea to eat gas station sushi—now you have to worry about ChatGPT lawyers, too!

We’ve touched before on the role that artificial intelligence is likely to play with taxes here. It’s safe to assume that AI will affect how we calculate and pay them going forward in America, even if we don’t yet know how. You could certainly ask ChatGPT, but you probably shouldn’t trust the answer you get.

There are tasks that computers can do faster, cheaper, and more efficiently than people. Preparing tax returns probably strikes you as work that falls into that category. But broader accounting isn’t there, at least not yet. (In April, a group of accounting professors tested ChatGPT on a collection of 27,000 accounting questions. The chatbot scored 47.4%, far less than the student average of 76.7%.) And sophisticated tax planning remains outside of any large language model’s grasp, too.

Here’s why tax preparation and proactive planning are so different. Filing taxes involves compiling specific numbers from various sources like W-2s, 1099s, and K-1s, then adding more numbers from business and personal records, then putting those numbers in the right boxes on the right forms. That’s a fully observable, deterministic, static environment. It’s like chess—there’s an enormous quantity of possibilities and decisions, to be sure, but they’re all made within a predictably closed loop.

Proactive tax planning, on the other hand, means taking on the challenge of an open environment. It’s only partially observable because we don’t know what future tax laws or financial developments will appear. It’s stochastic because it’s random in nature. And it’s dynamic because future actions and circumstances will change the recommended best course of action. For example, deferring tax on 401(k) contributions may be a tax-smart option under today’s rules. But if rates go up tomorrow, deferring that tax may turn out to be an expensive mistake.

Today’s news stories sometimes deliver predictable results. You can assume, among other things, that the guy who organized that Russian rebellion will “fall” out a window someday. And they probably ought to pour their own tea from here on out. But don’t assume that AI can replace the value of a good tax planner, any more than it can replace the value of an honest lawyer doing his homework. So call us with your questions before you trust them to ChatGPT!