Frank replied – you DON’T find patterns that “don’t exist!”
Kind of correct Frank.
The problem is not that the pattern doesn’t exist in the data set examined, the problem is the probability that the pattern will persist into the future.
Goodhart’s Law (When a measure becomes a target it ceases to be a good measure) is an example of the problem in respect of complex systems. In complex systems, historical measures are not consistently good predictors of future outcomes.
Human minds are based in biased neural networks – that allows us to make useful predictions on relatively sparse information in many common contexts, and it also comes with a host of problems (like addiction to simplistic notions like True/False, right/wrong, etc rather than acknowledging the many ways in which reality is fundamentally unknowable and unpredictable, and having a somewhat less arrogant and more tolerant and respectful attitude to diversity generally – and this isn’t getting at you specifically – it is a general problem with the human condition – all levels – me as much as anyone else).
[followed by to Mark]
Even the idea of “correct theories” can only be probabilistic, due to many causes, Heisenberg uncertainty, maximal computational complexity, irrational numbers, etc.
The very idea that we can know anything about reality with 100% confidence has been disproven beyond any shadow of reasonable doubt.
Certainly, we can get close enough in some cases that we can effectively ignore the difference in practice, but that isn’t quite the same thing; and getting overconfident in new contexts can lead to all sorts of problems – like the idea that markets can solve problems involving fully automated systems (known to be disproved, but ignored by almost everyone because it is Heresy).