← back_to_portfolio
CASE STUDY // BACKTEST ENGINE

SENTIMENT BACKTEST ENGINE

Vectorized long/short crypto backtesting driven by NLP sentiment signals

3 Strategies
vectorbt Powered
Long + Short Signals
Ratio Validated Entries

> the_problem

Most backtesting frameworks are long-only. They assume you buy low and sell high — full stop. But crypto is different: sentiment swings hard, and short-side alpha is real during bearish regimes. The problem was that no off-the-shelf tool could simultaneously:

Integrate NLP sentiment signals with financial ratio validation — and run both long and short strategies in the same vectorized engine without lookahead bias.

Raw sentiment signals are noisy. A bearish tweet score doesn't mean the coin is a short — you need ratio context (beta, omega) to confirm conviction. Off-the-shelf tools weren't built for this layered signal logic. Building on vectorbt meant extending it, not forking it.

> my_approach

The design principle was separation of concerns. Strategy files emit signals — they don't know about execution, risk, or portfolio accounting. The engine handles everything downstream.

  • Signal Extension Extended vectorbt to support 4-tuple signals: (entry_long, exit_long, entry_short, exit_short). The engine auto-detects whether a strategy returns a 2-tuple or 4-tuple and routes accordingly — no breaking changes to existing strategies.
  • Ratio Filter Layer Every sentiment signal passes through a validation gate against FE_RATIOS data (beta/omega ratios). Only signals that meet ratio thresholds proceed to entry. Forward-fill is applied at pipeline start — never inside the signal loop.
  • Strategy Separation Strategies live in a strategies/ folder and are discovered automatically via CLI. Adding a new strategy requires zero changes to the engine — just drop a class in the folder.
  • MA Baseline Strategies MATimeframes (8h/24h dual crossover) and MAOnlyFast strategies give a non-sentiment baseline to compare against — critical for knowing if sentiment actually adds alpha.

> architecture

┌─────────────────────────────────────────────────────────┐
│                  DATA SOURCES                           │
│   cp_ai (NLP sentiment)    cp_backtest_h (OHLCV)        │
└──────────────┬──────────────────────┬───────────────────┘
               │                      │
               ▼                      ▼
┌──────────────────────┐   ┌──────────────────────────┐
│   FE_RATIOS Table    │   │    OHLCV Data Loader     │
│  beta / omega ratios │   │  (forward-fill aligned)  │
└──────────┬───────────┘   └────────────┬─────────────┘
           │                            │
           ▼                            │
┌──────────────────────┐               │
│  Ratio Filter Gate   │◄──────────────┘
│  (validates signals) │
└──────────┬───────────┘
           │
           ▼
┌──────────────────────────────────────────┐
│         STRATEGY SIGNAL GENERATION        │
│  SentimentLong / SentimentShort /        │
│  SentimentLongShort / MA strategies      │
│  (auto-discovered from strategies/)      │
└──────────────────┬───────────────────────┘
                   │  2-tuple or 4-tuple signals
                   ▼
┌──────────────────────────────────────────┐
│         EXTENDED VECTORBT ENGINE          │
│  Portfolio simulation, position sizing,  │
│  mutual exclusion enforcement            │
└──────────────────┬───────────────────────┘
                   │
                   ▼
┌──────────────────────────────────────────┐
│         PERFORMANCE ANALYTICS             │
│  Sharpe · Profit Factor · Win Rate       │
│  Max Drawdown · Period Comparison        │
└──────────────────┬───────────────────────┘
                   │
                   ▼
┌──────────────────────────────────────────┐
│         POSTGRESQL RESULTS               │
│  cp_backtest_h — results persisted       │
│  + 4 analysis utility scripts            │
└──────────────────────────────────────────┘

> hard_challenges

01 // Extending vectorbt without breaking 2-tuple strategies vectorbt's portfolio simulation API expects a specific signal shape. Adding short-side support required wrapping the portfolio call to detect signal dimensionality at runtime. If a strategy returns a 2-tuple, the engine routes to the standard long-only path. 4-tuple goes through the extended long/short path. Both paths share the same analytics output — callers see no difference.
02 // Lookahead bias in ratio forward-fill Forward-filling ratio data sounds simple until you realise that Pandas ffill() on a misaligned index can silently peek at future values if the join isn't carefully ordered. The fix: forward-fill is computed once on the full ratio time series before any signal generation begins — never inline during backtesting loops. The filled series is then sliced to the backtest window, ensuring no future data can bleed into past signals.
03 // Mutual exclusion in SentimentLongShort The combined strategy must never simultaneously emit a long entry and a short entry for the same coin on the same bar. This is a correctness constraint, not just a risk constraint. Solved by computing long signals and short signals independently, then applying a mutual exclusion mask: if both fire on the same timestep, both are suppressed and flagged for review.
04 // Data alignment across two databases cp_ai stores sentiment indicators and cp_backtest_h stores OHLCV — different schemas, potentially different timezone conventions. Alignment required explicit UTC normalisation on both sides before any merge, plus assertion checks that the merged DataFrame has no unexpected NaN spikes that would indicate a missed join row.

> results

3 Live Strategies Long, Short, Combined — all production-deployed
4 Analysis Tools Trade extraction, breakdown, period compare, portfolio metrics
Zero Lookahead Bias Forward-fill architecture guarantees clean signals
CLI Strategy Discovery --list-strategies, --list-ranges, coin/strategy filters

> lessons_learned

vectorbt's portfolio simulation is genuinely fast — backtesting across hundreds of coins and thousands of bars runs in seconds rather than minutes. But the API is dense. The learning curve is steep, and the documentation assumes you already know what a portfolio accessor is. Worth it.

Ratio validation cuts false positives dramatically. Without the beta/omega gate, raw sentiment signals produce a lot of noise entries — especially during sideways markets where sentiment swings but price doesn't follow. The filter is the difference between a signal and a trade.

Period comparison analysis (bull vs bear market regimes) is as important as overall Sharpe. A strategy with a mediocre Sharpe might have exceptional bear-market short alpha that the aggregate number hides. The period comparison script was added specifically to surface this.

Auto-discovery of strategy files from a folder was a small engineering decision that paid large dividends — it made experimentation frictionless and kept the engine/strategy boundary clean.