June 27, 2024
This Big Caveat Raises Red Flags for Open-Source Finance AIs
Dear Subscriber,
 |
| By Jurica Dujmovic |
As a developer and tech columnist, I've been fascinated by a growing role of artificial intelligence in the financial markets.
With the recent explosion of ChatGPT and its specialized GPTs — customized versions of ChatGPT that users can access through their subscription account — it seems like everyone's trying to harness AI to up their investing.
But how useful are new AI tools for an average investor, compared to traditional methods? And more importantly, how can we use them to inform our investment decisions?
To answer these questions, I decided to put one of ChatGPT's finance-focused GPTs to the test.
I subscribed to ChatGPT and asked a custom GPT named "Finance & Trading: Stock, Crypto, Forex Investing" a straightforward but crucial question: What are the top 10 S&P 500, Dow and Nasdaq stocks to sell right now?
While this brief exploration doesn’t answer all questions, it still offers valuable insights into how AI is shaping the world of finance … and whether it can truly add value to your investment strategies.
Here is the exact prompt I entered into the custom GPT: What are the top 10 S&P 500, Dow, and Nasdaq stocks to sell right now? I need one unified list.
And here is the response:
“Based on recent analyst recommendations and market performance, here are ten stocks from the S&P 500, Dow Jones and Nasdaq that are currently advised to be sold:
- GameStop (GME): Facing significant declines with a consensus price target much lower than its current value, indicating a potential downside of over 50% (MarketBeat).
- Harmony Gold Mining (HMY): Analysts are suggesting strong sell ratings due to consistent underperformance and a significant potential downside (MarketBeat).
- Etsy (ETSY): Struggling with declining gross merchandise sales and net income, making it a risky investment at its current high valuation (InvestorPlace).
- Netflix (NFLX): Although historically strong, Netflix faces challenging comparables moving forward and a high P/E ratio that might not justify its current price amidst slowing growth (InvestorPlace).
- Unity Software (U): Experiencing a steep decline in stock value and revenue, compounded by leadership changes and lack of profitability (InvestorPlace).
- Intel (INTC): Slated to face headwinds due to increased competition and declining market share in key segments (MarketBeat).
- 3M (MMM): Besieged by legal issues and declining sales in its core segments, leading to negative sentiment among analysts (MarketBeat).
- AT&T (T): Ongoing struggles with debt and underperformance relative to its peers make it a less attractive investment (MarketBeat).
- Carnival Corp (CCL): The cruise line faces significant challenges in recovering from the pandemic, with high debt levels and uncertain demand (MarketBeat).
-
Boeing (BA): Continues to grapple with production issues and delays, coupled with a high valuation that does not align with its current performance and outlook (MarketBeat) (NerdWallet: Finance smarter).
Listed stocks have been highlighted due to various issues such as declining revenues, high valuations, legal challenges and competitive pressures that could lead to further declines in stock prices.”
Now, let me be loud and clear: This information has been generated by an open-source AI program and not vetted. It should not be taken as a call to action or instruction in any way.
My goal is solely to demonstrate whether AI can be used as a tool for investors.
And my answer is … sort of. But not the way you’d likely hope.
See, what ChatGPT did here is precisely what all large language models do: It processed vast amounts of text data and complex statistical patterns to generate a human-like response.
That is to say, it came up with its response based on the raw data available and by examining how other sources online have answered similar questions.
Think of it like that kid in middle school who would copy someone else’s homework. He didn’t study for himself, so he would copy the facts of another person’s answer and dress it up in his own phrasing. Whether or not he was right would depend entirely on the data he was copying from.
ChatGPT, and all large language models are the same. They don't produce unique financial expertise. Nor do they create genuinely novel insights.
Without current market data or real-world expert analysis, they can't offer insight beyond a blend of superficially plausible recycled information and questionable projections.
What we're observing is the AI's effort to cobble together and reframe data it's gleaned from its training, not perform actual financial analysis.
And the devil is in those details. Here’s what my finance GPT answer got wrong.
- Questionable Interpretation: Look at ChatGPT’s cited sources. One in particular, Marketbeat, is used frequently. But if you click through to the slideshow, it states clearly that these aren’t sell recommendations. Rather the list is just of the “20 companies that have the lowest average analyst recommendations from Wall Street's equities research analysts over the last 12 months.”
- Out-of-Date Data: The latest data used from Marketbeat was from January of this year. I don’t know if you’ve noticed, but market conditions tend to shift as time goes on. And ChatGPT didn’t take that into account.
-
Incorrect Citations: To make matters worse, several of the citations don’t even line up. Look at the ones for Intel, 3M, AT&T and Carnival Corp. They’re not featured in the Marketbeat slideshow.
That means the AI is attributing facts it created based on the data it did have to the incorrect source. This is the same problem we’ve been hearing about AI programs for a while: When an exact answer cannot be found in the data, it’ll use its language processing to create answers … even if they’re not completely accurate.
While on the surface these AI models seem to be capable of great feats, they fall short when faced with specific and highly deterministic demands.
Forget about being superior. They struggle to maintain the necessary accuracy to make them merely comparable to their human counterparts.
In fact, without extensive further development, these models cannot achieve what is crucial for any deterministic software: consistently reproducible output.
So, when it comes to large language model AI programs like ChatGPT, I can say that they have their usefulness … but they cannot be considered a reasonable substitution for your own research, let alone an expert’s nuanced analysis.
What they can do is help you sort through the accessible data to give you a starting point for your own analysis.
But that doesn’t mean there’s no way forward for AI programs in this field. In fact, Weiss Ratings developed two specialized AI models. These aren’t large language models, however. They use machine learning techniques specifically designed for financial analysis.
And next week, I’ll get into how those models answered the very same question I asked ChatGPT.
Keep an eye out for that next Thursday.
Best,
Jurica Dujmovic
Tidak ada komentar:
Posting Komentar