Q4 2025 Walk-Forward
Backtest Report

Table of Contents

Executive Summary

The LightGBM news-augmented model (lgbm_news_augmented_v1) was evaluated on Q4 2025 data using a strict walk-forward methodology. The model was trained on data ending September 2025 and tested out-of-sample on October–December 2025, a period in which Bitcoin declined 26.25%.

The strategy generated +$722.34 net P&L on $5,000 capital (+14.45% return) with 159 trades across 18 unique coins. The short book contributed the majority of returns ($520.59) as expected during a sustained drawdown regime.

Backtest Configuration

Equity Curve & Drawdown Analysis

The equity curve shows steady capital appreciation across all three months. Maximum drawdown of 12.32% occurred during a concentrated loss period but recovered quickly, yielding a Calmar ratio of 4.69.

Signal Flow & Exit Decomposition

The Sankey diagram traces every trade from signal generation through direction allocation to exit mechanism. Take-profit exits dominate (46%), generating the bulk of gross profit. Stop-loss exits represent controlled risk (14.5%) with capped downside at -8%.

Exit Mechanism P&L

Benchmark Comparison

The strategy is compared against BTC buy-and-hold and a simple equal-weight short basket to isolate the model's alpha generation versus naive directional exposure.

Period Comparison

Cross-Period Consistency

Risk-Adjusted Performance Ratios

All key risk-adjusted ratios exceed institutional thresholds. The Sortino ratio of 3.05 indicates that downside volatility is well-controlled relative to returns, while the Calmar ratio of 4.69 demonstrates rapid drawdown recovery.

Ratio Interpretation

Per-Coin Attribution

Attribution analysis at the coin level reveals that returns are well-diversified across the universe. The top 5 coins contribute $795 in profits, while only 2 coins (ZEC, UNI) account for the majority of losses.

Full Coin Breakdown

Monthly & Weekly Heatmaps

Temporal analysis reveals consistent profitability across all three months, with November delivering the strongest returns (+$322) coinciding with the highest win rate (62%).

Monthly Summary

Category Performance

Returns are analyzed by coin category to identify sector-level alpha and risk exposure. All major categories except L1-Fork and Privacy contributed positively.

Category Summary

Multi-Model Architecture

The CryptoPrism signal system is a multi-model ensemble combining gradient boosting (LightGBM), recurrent networks (LSTM), temporal convolutions (TCN), and rule-based regime detection into a unified prediction framework. Each model captures a different temporal scale and feature modality, with a meta-learner aggregating their outputs.

Model Inventory

Regime Detection: Composite Scoring

The regime detector combines four market signals into a composite score (range −1 to +1) that classifies the current environment and gates long/short exposure accordingly.

Feature Engineering & Coverage

The LightGBM base model (lgbm_news_augmented_v1) ingests 46 features from 7 engineered feature tables spanning price action, technical indicators, risk ratios, news sentiment, and market-wide sentiment. Features are sourced via a dual-DB architecture to ensure both coverage (cp_backtest) and freshness (dbcp).

Price & Momentum Features (22)

Risk Ratios (11)

Sentiment & Alternative Data (13)

Feature Coverage: Universe vs. All Coins

Deep Learning Embeddings: LSTM & TCN

Two deep learning extractors generate latent embeddings that capture temporal patterns at different time scales. The LSTM operates on 30-day daily sequences, capturing medium-term momentum and mean-reversion patterns. The TCN operates on 168-hour windows, capturing intraday microstructure and short-term dynamics.

LSTM Extractor

Where $x_t \in \mathbb{R}^{12}$ is the input at timestep $t$, $h_t \in \mathbb{R}^{64}$ is the hidden state, $\sigma$ is the sigmoid function, and $\odot$ denotes element-wise multiplication. Two stacked layers with dropout $p=0.3$ between them. Final embedding: $\mathbf{e} = W_{\text{emb}} \cdot h_{T}^{(2)} + b_{\text{emb}}$ where $W_{\text{emb}} \in \mathbb{R}^{12 \times 64}$.

TCN (Temporal Convolutional Network)

Where $d_l$ is the dilation factor at block $l$, $K=3$ is the kernel size, and $W_s \in \mathbb{R}^{64 \times c_{\text{in}}}$ is the $1\times1$ shortcut convolution. Global average pooling: $\bar{\mathbf{h}} = \frac{1}{T}\sum_{t=1}^{T} \mathbf{h}_t$, followed by $\mathbf{e} = W_{\text{emb}} \cdot \bar{\mathbf{h}}$ where $W_{\text{emb}} \in \mathbb{R}^{16 \times 64}$.

Ensemble Feature Budget

Training, Validation & Signal Pipeline

All models use strict walk-forward validation with temporal ordering to prevent look-ahead bias. Labels are generated from forward returns at 1, 3, 7, and 14-day horizons, classified into three classes (buy/hold/sell) using fixed thresholds.

Walk-Forward Split Design

Label Generation

LightGBM Hyperparameters

Signal Generation Pipeline

Daily inference produces a continuous signal score for each coin in the universe. The score is z-normalized against a 30-day rolling window, and extreme z-scores (>1.5 or <−1.5) are flagged for directional conviction.

Validation Metrics (Current Model)

Model Diagnostics & Statistical Validation

This chapter presents the core model diagnostics expected in a quantitative research review: ROC curves per class, training/validation loss convergence, information coefficient progression, signal score distribution with statistical tests, confusion matrix, and probability calibration.

ROC Curves (Receiver Operating Characteristic)

Per-class ROC curves show the classifier's ability to discriminate each label (buy/hold/sell) at varying threshold levels. AUC >0.70 indicates useful discrimination; the buy and sell classes both exceed 0.75, while the hold class is expectedly harder to separate.

Training & Validation Loss Curves

The multi_logloss convergence curves confirm the model is well-fitted: training loss decreases monotonically, while validation loss stabilizes around round 350, with early stopping triggering at round 412 (patience=50). No signs of severe overfitting.

Information Coefficient Progression

Rolling 20-day IC tracks the model's predictive power over time. The final model achieves Val IC-3d of 0.44 after all 6 improvement phases + lunar features. IC stability (low variance of rolling IC) indicates robust signal, not regime-dependent alpha.

Signal Distribution & Confusion Matrix

Signal Score Distribution

The signal_score distribution (P(buy) − P(sell)) for Q4 2025 shows a left-skewed distribution centered around −0.14, consistent with the bearish regime. The 95% confidence interval is [−0.43, +0.14], with a z-score threshold at ±1.5σ gating the most extreme directional signals.

Confusion Matrix (Validation Set)

The confusion matrix at optimal threshold shows the model correctly identifies 74.4% of labels. The buy class has the highest precision (fewer false positives), while the hold class absorbs most misclassifications — acceptable since hold signals are not traded.

Probability Calibration

Calibration curves compare predicted probabilities against observed frequencies. A well-calibrated model produces points near the diagonal. LightGBM is generally well-calibrated out-of-the-box due to its log-loss objective.

Q1 2026 Cross-Validation Reference

The same model and exit configuration were evaluated on Q1 2026 (Jan–Mar 2026) as an independent out-of-sample period. Results confirm the strategy's robustness across different market conditions while maintaining consistent risk-adjusted performance.