Understanding SP500 Gen2 Persistent Current Trend (PCT) probability model

Background

The Gen2 probability model we maintain for PRO subscribers for the SP500 provides for short, medium, long-term and macro-term probabilities of market troughs/peaks.

It does this by examining current up/down trends across the 4 time-horizons mentioned and compares them to a 30-year historical record of said trends with regards to both duration of trend and gains/losses achieved for the trends. By comparing the duration and gain/loss of the current up/down trend with the historical record, we can impute two probabilities, namely:

1. The probability that the current trend will endure longer and
2. The probability that the current trend will gain/lose more.

To assess an appropriate probability of a trend reversal for the current trend we find ourselves in, we merely create and average of the two above probabilities. This provides us with a likelihood assessment of a trend-reversal taking place when considering the long-term historical record of past trends with regards to both duration and gains/losses of said trends.

To categorize historical trends into short, medium, long-term and macro buckets we need an algorithm for trend classification across these multiple time-horizons. The algorithm used by the Gen2 probability model considers intraday highs and lows on the daily SP500 candles to determine true tough-to-peak gains or peak-to-trough losses of trends and works simply as follows, which is an alternate spin on the popular “Zig-Zag” technical indicator:

1. If we are in a down-trend and there is a higher low within x-days ahead than yesterday’s low, then change to an up-trend.
2. If we are in an up-trend and there is a lower high within x-days ahead than yesterday’s high, then change to a down-trend.

The values of x used for our model for our various trend time-horizons were determined through optimization research  based on what appeared to work best in general in representing the targeted trends for the nearly three decade historical period, and are currently as follows:

1. Short-term trends : 7 days (market sessions) for 211 entries for the historical database
2. Medium-term trends : 17 days for 89 entries for the database
3. Long-term trends : 34 sessions for 45 database entries
4. Macro-term trends : 72 sessions for 24 database entries.

The charts below depict the gains/losses (intensity) and durations for the short and medium-term trends respectively as of Wednesday, 24th January 2024. We note that if we view the current trend we are on as a medium-term up-trend, it has gained 19.49% versus a 11.2% historical average, which is one standard deviation above the mean. It is not shown on the chart, but this transcribes into a 22% probability the trend will continue, or conversely, a 78% probability the trend will reverse based on the gains. We also note that if we view the current trend we are on as a medium-term up-trend, it has lasted 59 sessions versus a 35-session historical average, which is 0.7 standard deviation above the mean. It is not shown on the chart, but this transcribes into a 26% probability the trend will continue, or conversely, a 74% probability the trend will reverse based on the duration.

The useful thing with these ACTUALS charts is if a trend has continued for some time, such as the deemed medium-term trend in the lower two charts above, we can draw a horizontal orange line showing the current gains/losses or duration so we can visually see how often in the past, and when these levels we are currently witnessing have been exceeded. These visual cues sometime are easier to digest and provide more dimensions (when and how many times) when assessing likelihood of reversal than a single dimension probability percentage derived from nonlinear math equations.

Below is the same chart depicting the deemed long-term trends. These translate into probabilities of a trend reversal of  53% according to the current gains and 47% according to the current duration.

Finally, here are the deemed macro-term trends:

Introducing PCT

The selection of x in trend-determination along short, medium and long-term horizons, apart from our optimization research, is fairly arbitrary. It’s never perfect and is designed to capture intended trends for most of the time. But for a short-term up-trend using x=7 that suddenly comes to an end on a given day, what if using x=8 does NOT reverse the trend on the given day and the market pushes a new high after another 8 days? Is x=8 a better value to use than x=7? We can keep on with this argument until x gets sufficiently long enough to eliminate such choices but by then x will be too large to be useful as a “short term” trend latcher. In fact, it is perfectly reasonable that for the prior up-trend, x=7 was best in terms of timeliness but for this current up-trend, x=8 will be better for accuracy.

Let us examine a very recent example of this. On 29th December 2023 we were in an up-trend, and the high for the previous day, the 28th December, was 4,793.3. The high for the 7-day look-ahead was only 4,790.8, below the  high of the 28th, so the algorithm marked 29th December as a reversal day, the first day of a new short-term down-trend. On 08 Jan 2024 the low for the 7-day look-ahead (4,714.8 on 17 Jan 2024) was higher than the low of the previous day (4,682.1 on 05 Jan), and thus we had another reversal day and a new short-term up-trend ensued:

On the surface, this looks entirely reasonable and appropriately captures a definition of short-term trends. We see that on the downtrend reversal day of 29 Dec, the probability of a short-term top on the previous day, the up-trend peak, was sitting at 91.4% an entirely appropriate warning of the impending short-term top.

Persistent Current Trend (PCT) attempts to avoid a reversal day by expanding the look-ahead value x by the minimum amount to preserve the current trend. It is a tacit admission from the algorithm that “Hey, this might not actually be a short-term reversal, and since x=7 is a generalized one-size-fits-all parameter, we can provide traders a more comprehensive review of the short-term risks by ignoring the reversal, but we’re going to have to find the smallest, larger value of x that will allow us to do this.” In this particular instance, we were fortunate to only have to increase x to 8 to maintain the current up-trend on 29th December, since the witnessed high 8 days into the future from the 29th was 4,798.5 on 11 Jan – higher than the 4,793.3 witnessed on 28th December:

So we are now giving the observer an alternate potential short-term reality – that the current short-term up-trend is ongoing. Note how for x=8, the probability of a market peak on 28th December was 89% as opposed to 91.4% witnessed with the x=7 trend algorithm. Also the PCT model is now flagging a probability of a market top on 24th January as 95% as opposed to the 41% from the x=7 trend algorithm which is now on the assessment that we are in a new, young short-term uptrend.

The chart sets we provide subscribers will include the PCT algorithm assumptions in addition to the standard assumptions, and they are available in the PCT tab in PRO>SP500 Charts . Both the probabilities of the duration and gains/losses are provided in the first two PCT charts, whilst the actual durations and gains/losses are provided in the bottom two PCT charts. Numbers in brackets represent readings for the prior day so we can assess how much the probabilities change from day-to-day:

Using the above example, we look at the top chart and take the PCT rally duration top-probability of 96% and average it with the PCT rally gains top-probability of 95% to derive a two-dimensional average market top probability of 95.5%

At this juncture the PCT algorithm is providing the investor/trader the opportunity to assess multiple potential short-term realities. Looking at the examples we provided above, in hindsight, it appears as if x=8 is indeed providing a more realistic and likely interpretation of the current short-term trend environment than x=7. We do not however use this as a reason to go and modify x to 8 permanently, since traders make heavy use of the short-term algorithm for market-timing and trend-following purposes, and we want the smallest value for x as possible that provides the most accurate generalized long-term results in order to preserve timeliness.

In this particular example shown, x only had to be increased from 7 to 8 to preserve the short-term uptrend, but you should note that on many occasions, x has to increase by quite large numbers to retain the persistent current trend assumption. In many cases x will jump into the mid to high teens to preserve a current short-term trend which means the PCT trend is no longer a viable short-term alternate reality assumption. In fact, large jumps triggered in PCT’s x-variable merely cement that the current x=7 short-term trend is most likely the actual, true short-term trend reality. At that stage, PCT is no longer suited to alternate reality assumptions on the short-term horizon but starts providing alternate realities for the medium-term trends. Thus, once PCT is in the teens it starts looking to reversals in the medium-term trends (x=17) to take its cues for finding alternate realities – this time on the medium-term timeframe.

Pre-empting the look-ahead with ST-ALGO

We provide a specialized market timing chart  in the ST-ALGO tab in PRO>SP500 Charts that allows short-term trend followers and market-timers to pre-empt x=7 short-term trends without having to wait for the full 7-days look-ahead to pass. Not only is it around 80% accurate in forecasting what the full look-ahead algorithm will provide as trends, but it does it with a 2-3 day lag as opposed to 7-day lag. Additionally, it also allows us to assess the risks of a future short-term x=7 trend reversal together with target levels for the SP500 that are most likely to trigger said reversals.

ST-ALGO is the process whereby we try make determinations of SP500 short-term trend reversals (directional changes) within the 7-day look-ahead process which the probability model uses for determining non-revisable short-term trends which get finalized, frozen and catalogued into the short-term trend historical database for use in determining current and future short-term trend reversal probabilities used in all the other Gen2 charts. The advantage is we dont have to wait for the full 7 market sessions to transpire to determine what the trend is after it changes.

The new ST-ALGO provides over 80-90% confidence of early trend reversal recognition within the estimation zone. This means trend estimations are made within the estimation zone that have only 20% to 10% probability of eventually turning out wrong and being overwritten by the trend detection algorithm using the full 7-day look-ahead.

The lag this algorithm provides to the actual finalized locked-in non-revisable turning (pivot) points is non-deterministic but ranges from 2 to 4 days, averaging about 3 days. This is still far more desirable than waiting the full 7 days for non-revisable trend lock, which is obviously useless from a market timing perspective. ST-ALGO as before, still determines turning points by the provision of a daily changing high/low STOP that must be breached in order to trigger the trend reversal, but this time it is not some one-size fits all 2.6% number but rather a dynamic number that
1. Takes into account recent support/resistance points,
2. That is till close (small) enough to make timely recognition of turning points and
3. Is still further (larger) enough to have statistically-derived high confidence levels of being accurate (not overwritten by the final full-lookahead algorithm)

The new ST-ALGO chart now also incorporates at least ONE intraday chart update and has been revised as follows with additional on-chart cues, timestamp and more descriptive summary text with citing of confidence levels associated with the chosen stop:

You will see from the above chart that it has already made a trend reversal estimation two days after the 05 Aug intraday lows (a best-case example) and made an accurate downtrend estimation for the previous downtrend within 3 days of the last SP500 all-time high (ATH). It did not make an incorrect uptrend estimation in the 25 July to 01 Aug bull-trap since this rally failed to make highs that set new resistance levels chosen by ST-ALGO.

If the SP500 should print an intraday low below the provided stop, the confidence level of 79.5% means that the historical database of non-revisable short-term uptrends catalogued by the Gen2 algorithm has shown short-term uptrends exceeding a high-low drawdown of 3.21% has only occurred 20.5% of the time, or 42 times in 212 instances in 25 years since 1999.

You should also note that the trend-estimation latch-freeze has been halved more recently from 2 to 1 days. Besides this, the latch is just an extra precaution for the wary timer and you may elect to act on trend changes on latch day, since statistical confidence levels provided ignore any latching holds. It is just that the research team has noted on many occasions that the latch-freeze gets you in at a better price even though it is an extra one-day lag. At this stage we are still debating dropping the latch condition in its entirety to get better timeliness, but for now the algorithm on the chart display will be observing the latch freeze. You may elect to front-run the algorithm therefore by ignoring the latch-freeze.