Product idea: a LLM trained separately from mainline LLMs that anticipate market trends by analyzing how mainline LLMs will invest. As retail investors will probably use mainline AI for decisions going forward , one could get an edge.

"The AI-driven Market Hypothesis"

Please let me know where I should pick up my Nobel prize.

Stock analysts have a success rate of 47% or lower for directional predictions. That's worse than a coin flip. All AI has to do is product fair 50/50 results and it can beat analysts. But you can do it too for the price of a quarter.

We measure skill differentiation between frontier / last-gen LLMs across our environments, and one of our curated coding environments is a closed-system market simulator, containing only other agents and some system participants (a market maker and a liquidity provider via issuance / buybacks) whose behavior is fully defined for all of the agents.

This has the least measured skill differentiation of all of our environments, and not because forecasting/markets don't require skill or intelligence. Even the best models are so far from anticipating the behavior of the other agents and understanding the emergent effects that a 2025 model with a naive strategy can often outperform over the timeframes of the simulation simply because some other models in the simulation chose a similar self-reinforcing strategy. This likely happens to some degree in real markets.

You can watch these simulations here https://gertlabs.com/spectate?game=market

Aren't forecasters already using 'artificial intelligence' for decades in the form of non-llm machine learning models?

If 10,000 people guess 10,000 fair coin flips each one of them will get more guesses right than any of the others, one of them will get fewer guesses right than any of the others, and the gulf between the two is likely to be over 4 standard deviations wide. I'm certain that I, being an untutored schmuck from Pittsburgh and having thought of this almost immediately after reading about this contest, cannot be the first person to realize this is a potential problem for a forecasting contest. But I can't find anything they've done to mitigate that problem. Can anyone clue me in?

This has to be the least surprising development to date given ml is a universal function estimator

The test is when reflexivity kicks in and the prediction itself changes market behavior. LLMs usually melt there

At the risk of sounding extremely naieve i have a question for the Wall St / quant / HFT folks lurking here ... but how hard would it actually be to brute force the math/algos behind Medallion Fund (or something in that general class) or even some of the average quant funds

I know it’s not just the math but execution, infrastructure, risk management, data, colocation (if ur an HFT) etc ... but LLMs seem like a pretty powerful apparatus for running experiments that .. a few years ago would have required fairly deep multidisplinary skills across coding .. stats .. and math ..

So assuming you have decent intuition for ideas .. how difficult would it actually be to reverseengineer / rediscover some of the underlying stuff?

It will be interesting to see if this changes because presumably AI is using very predictable historical models, but it seems like the climate is shifting into something unseen that we won't have models for?

Interesting. This was one of the two areas the AI as Normal Technology folks specifically called out as a bet that AI will not outperform humans at.

> Concretely, we propose two such areas: forecasting and persuasion. We predict that AI will not be able to meaningfully outperform trained humans (particularly teams of humans and especially if augmented with simple automated tools) at forecasting geopolitical events (say elections). We make the same prediction for the task of persuading people to act against their own self-interest.

Curious to hear what their take is now.

https://www.normaltech.ai/p/ai-as-normal-technology

Since I can't read the article I believe they are referencing to https://www.metaculus.com/tournament/metaculus-cup-summer-20...

It seems to be that they were trying beat a guy named Dylan Mathews. And seems like the community beat him in making predictions for 58 questions about the future

The markets are a highly complex dynamic system. There are many instances of it exhibiting disastrous behavior, especially in response to changes and shocks.

AI trading and investment advice meaningfully changes the system and its dynamics. It seems highly probable that this will result in it failing in new ways.

Investing used to be a resource-weighted signal of human economic projections, where resources flow to better projections. Public markets had social value for their resource allocation and signalling/coordination benefits. Now? How could they avoid hallucination contagions?

So my plan to go from a developer to an economist is scrapped. What now?

So I guess the AI companies can stop with their plans to infest AI with ads and they'll instead fully fund themselves by using their AI to gamble on stocks and the prediction market right? Surely the chatbots will just print money!

Are the models going to skew their analysis to preserve AI companies as a form of self-preservation?

The best human forecasters working with artificial intelligence are going to do even better than either alone, the dichotomy is artificial.

That is too bad for The Economist. Exor N.V and Agnelli might replace some pundits at The Economist.

See also The AI Superforecasters Are Here https://www.astralcodexten.com/p/the-ai-superforecasters-are... and discussion https://news.ycombinator.com/item?id=48806296

Given the training data isn’t that more a win for the wisdom of the crowd?

Hasn't this been true for like 40 years

Anyone who believes this news story should create their LLM slop bot to trade on prediction markets like Polymarket and Kalshi. These acceletards provide a great influx of money to many human traders on these platforms.

Paywall

[flagged]

[dead]

[dead]

[dead]

[dead]

Archive Link: http://archive.today/IVreS