Statistics, Probability, Randomness, and Surprise: True and False Propositions in Investing

Statistics

Method: It is a mathematical method and scientific field aimed at collecting, organizing, analyzing, and interpreting data. It involves gathering large amounts of data and then organizing and summarizing that data.

Purpose: Through data analysis, it provides objective, quantified information to help us better understand and explain phenomena, and to make reasonable decisions and inferences.

Result: Descriptive statistics are used to describe and summarize data, such as calculating the mean, variance, and percentages. Inferential statistics draw conclusions from samples to make inferences and predictions, such as conducting hypothesis testing and building confidence intervals.

Statistics has two possible outcomes:

First: After analyzing all the data we can collect from the past, we find there is only one single outcome — that is "true" (but it can be falsified in the future; before it is falsified, we can only take it as true). For example, "all humans need oxygen" is a "true" proposition, but if new evidence is discovered showing that someone can survive without oxygen, the earlier proposition can be falsified.

The basic idea of the "presumption of innocence" is that in the eyes of the law, the defendant is regarded as innocent until proven guilty. This means the "presumption of innocence" is "true." The prosecutor bears the responsibility of providing sufficient evidence to prove the defendant's guilt; it is not the defendant who must prove his own innocence. You cannot prove something that does not exist.

So why the presumption of innocence? Because "innocent" is the proposition held true, and the prosecutor must falsify it by presenting evidence of guilt — proving guilt, that is, proving "innocent" to be false. If it were a presumption of guilt, then proving innocence would be impossible; you cannot prove something that does not exist, and no matter how much evidence of innocence you present, you cannot prove your own innocence.

Take, for example, the old belief in Europe that "all swans are white." This is a proposition. Unless you find a swan that is not white — which would prove the proposition "all swans are white" wrong, "false" — this is exactly how a true proposition can be falsified. But if no one in the world had ever seen a black swan, and someone said "swans are not only white," then no matter how many white swans you find (because at that time no one had ever seen a black swan), you could not prove "swans are not only white" to be "true" — and that is a false proposition. So before the black swan was discovered, "all swans are white" was a true proposition, and "swans are not only white" was a false proposition.

So in the stock market, when we say "as long as you invest in the U.S. broad market index for more than 20 years, you will not lose money," this is a true proposition, because it can be falsified. Before anyone proves that investing in the U.S. broad market for more than 20 years resulted in a loss, this proposition remains true. You can go and look for anyone who invested in the U.S. broad market index for more than 20 years and still lost money — this can be falsified. So "as long as you invest in the U.S. broad market index for more than 20 years, you will not lose money" is a true proposition, and so far it is "true."

But if someone says "that was true in the past, but investing in the U.S. broad market index for 20 years in the future will not necessarily make money," this is not a true proposition, because you cannot verify a proposition about something that does not exist. It is a false proposition about the future — a false proposition that cannot be falsified.

So when discussing things with one another, you must think logically. With a person who has no logical thinking, no matter how much you explain, you will never make it clear, because many people have no concept of logic. When they argue, they assume that because a claim cannot be refuted, it must be true — when in fact it is a false proposition. So during a discussion, you must pay very close attention to whether what the other person says is a logically valid true proposition. If someone can only argue using false propositions, you can only say he does not understand logic — just stay away from such a person.

Second: In statistics, there is not always only one single "true" outcome. When many possibilities can occur, you get a probability distribution. That is, after analyzing all the data we can collect from the past, if we find that there are many possible outcomes distributed across a range — that is a probability distribution.

Probability distribution: every event that occurs has a probability of occurrence under specified conditions.

Purpose: probability is the distribution of past occurrences, used to infer the likely probability of long-term sampling in the future. The longer the time and the larger the sample size, the closer it will come to the probability distribution found in past statistics.

So when we say that investing in a U.S. broad market index fund for more than 25 years gives you an annualized return of 7.94%–17.24%, that is a probability distribution. In other words, if you take any 25-year sample from past statistics, your annualized return falls between 7.94% and 17.24%. This is the probability distribution of returns, and we use this statistical probability distribution as the basis for long-term investing.

But many people invest for only three or five years and expect this kind of return, unwilling to accept any loss. This makes it impossible to truly achieve the return of the 25-year probability distribution. It is a mismatch in setup. If you invest for five years, you must take the probability distribution of five-year statistics as your expected value. The probability distribution for a five-year investment lies between -5.58% and +28.76%, so a loss over a five-year investment cannot be judged against the expected value of the 20-year or 25-year probability distribution.

Randomness: When the sample size is insufficient (too short a time period is also insufficient sampling), events are random. The events and outcomes that occur cannot be determined; they happen randomly, and there is no regularity in the order of occurrence that can be predicted. The higher-probability outcome does not necessarily happen first, nor does the higher-probability outcome necessarily keep happening, nor does it necessarily fall within the expected range of the probability distribution (a probability distribution has a range and interval; if an outcome is not within this range and interval (too little sampling), you cannot obtain the expected result. The probability distribution derived from inferential statistics has a tested and established confidence-interval range). When sampling is insufficient, the higher-probability outcome does not necessarily occur. The lowest-probability event might happen on the very first try — that is a random event outside the established confidence-interval range. Random events cannot be predicted.

Surprise: When you make a definite prediction about a random event, and the event that actually occurs falls outside that definite prediction, it is called a surprise. For example, if you say it will definitely rain tomorrow, and it does not rain, that is a surprise. Short-term stock market ups and downs are random; if you predict the market will definitely fall tomorrow, and instead it rises, that is a surprise. If you say there is a 35% chance of rain tomorrow, then we are back to the probability event above (it may or may not happen, but you cannot predict it with certainty).

Statistics is a trajectory of past history, obtained by analyzing all the data over a very long period in the past. A single outcome is a true proposition; otherwise it is a probability distribution.

Statistics uses a relatively long historical period and a relatively large amount of data to arrive at a relatively certain answer. For example, "as long as you invest in a U.S. index fund for 25 years, you will not lose money" is confirmed by a hundred years of past data. For example, the U.S. market rose 27,000-fold over 200 years — that is a statistical result. (If, based on this, we say the U.S. stock market index should rise more than 2,000-fold over the next hundred years, that too is a true proposition based on a statistical result, and it can be falsified over the coming hundred years.) A proposition made on the basis of statistical results can be proven (or falsified) over time. A proposition based on statistical results is true until it is falsified!

Probability uses what is known now, or what one personally believes, to predict future possibilities. For example, "there is a 35% chance of rain tomorrow" is a probability prediction! Probability is usually just the likelihood of an event distribution; it does not necessarily happen, and it can be falsified in a short time. But to say with certainty that the market will rise tomorrow, or to say with certainty that a little bird's next move will be to fly up, or that its next move will be to hop two steps forward (a bird's movements and the short-term ups and downs of the stock market are both random and cannot be predicted), and so on — these are all overconfident guesses, not worth believing. Making a definite prediction based on the probability-distribution results of statistical data and then placing a bet (gambling) on it is overconfidence; just listen and let it go, do not believe it, and above all do not use your resources to place a bet on it.

Probability is a mathematical concept used to describe the likelihood of an event occurring. It is a number between 0 and 1, used to express how likely a certain event is to occur (it does not guarantee what will happen the next time). And it cannot determine which specific outcome will occur — it is merely a probability distribution derived after statistical analysis.

It should be noted that probability describes the likelihood of a random event; it does not guarantee that a specific event will or will not occur. Probability is only used to assess how likely an event is to occur. And it cannot determine which specific outcome will occur.

Randomness means an unpredictable outcome. Whether it will rain tomorrow is actually close to random; even though the weather forecast says there is a 70% (probability) chance of rain, it is not truly certain whether it will actually rain. If it is predicted that there is a 99% chance of rain and instead it does not rain, or the day turns out bright and sunny, that is a fat-tail event (a low-probability event occurring at random) — a surprise happening.

In life, you must never make predictions or assertions about random events and then act on them with firm conviction. That is gambling, and it is bound to bring disaster — it should be avoided. As Mark Twain's famous line goes: what troubles people is not the things they know nothing about, but the things they are utterly convinced of.

Randomness (Random) means that the outcome of an event or phenomenon cannot be accurately predicted (especially under conditions outside the statistical framework; usually the sample must be large enough to approach the statistical probability distribution, otherwise it is all just random, and predicting the event will produce the surprise of a wrong prediction). Randomness has the characteristic of uncertainty. In a random situation, every possible outcome has a chance of occurring, and it cannot be determined which specific outcome will occur.

It should be noted that a probability distribution follows a regular pattern, but the process itself is an unpredictable random phenomenon. A probability distribution is a determinate result. Randomness is an uncertainty of outcome, because randomness is the result of insufficient sampling.

Randomness means you cannot determine how or when something will happen, so you cannot manage risk, because you cannot foresee what will happen or how it will happen. When you have no idea what will happen or when it will happen, you simply cannot avoid the risk, and risk management is simply impossible. If someone claims he can control risk by predicting a random event, that is probably overconfidence. Facing random risk, the only thing you can do is stay away from it (do not go near unlucky things, do not gamble; keep away from bad luck, and opportunity will come on its own). If someone is very certain about how a random event will unfold, that is usually human overconfidence. For example: the bird will hop down within five seconds; or wait a moment (within one second) and the bird will hop three steps forward; or "I will not be struck by lightning today"; or the market will rise in the next hour, the next candlestick will fall, it will rise tomorrow, it will fall next month — statements like these are all overconfidence! Even with a probability distribution to rely on, you cannot guarantee against the surprise of insufficient sampling; what you should fear is the surprise happening (a fat-tail event).

Take, for example, a bag with a hundred balls: only one is white, and the other 99 are red. If you draw one ball at random, the probability of drawing the white ball is a random 1/100. But if you are certain that in a single draw the ball you pull out must be red — that is overconfidence. Just like a speculator who says the market will definitely rise tomorrow! Or the stock will definitely fall! And so on — that too is overconfidence. Even saying there is a 70% chance the market will rise tomorrow is still random; it cannot be fully trusted, and you certainly must not use your money to bet on a random event because of it. Usually, the surprise arrives sooner than the good luck.

A surprise is human overconfidence: being certain that a random event will definitely happen, and then the outcome not matching expectations — that becomes a surprise. In life, the point is not to focus on good things happening, and above all not to say "certain" about random events, but rather to avoid the surprises born of overconfidence — and good luck will come on its own.