Beyond classic multi-factor models, research and practice have accumulated a large set of candidate factors. This chapter consolidates former chapters 18, 21, 19, 22, and 20 into one: anomaly mining and testing pitfalls first, then deeper fundamentals, alternative data, macro factors, and high-frequency factors.
Anomaly mining seeks return patterns that, in theory, should not persist.
“Should not persist” means that if markets are efficient, public information should already be in prices. In practice, strategies that keep outperforming still appear—these are anomalies. The opportunity set looks large, but so do false signals and overfitting risk.
In academia, an anomaly is a statistically significant return-forecasting pattern that mainstream pricing models cannot explain. Classic size, value, and momentum effects are anomalies.
Anomalies usually fall into two classes:
The key question: is the pattern real?
A common story: the backtest looks excellent, live trading fails. Often the “anomaly” was noise, not signal.
Data-mining bias (overfitting / data snooping) means that among many candidate factors, some will look significant by chance alone.
Financial data have very low signal-to-noise: random return variation dominates. With enough trials, spurious “significant” patterns are inevitable.
Extreme example: backtest 1,000 pure-noise series and 47 factors show Sharpe above 1.5—pure coincidence.
A common mistake: screen and backtest on the same full sample. A flashy 30% annualized return may be look-ahead. Screening and validation must use different samples.
Common sources of data-mining bias:
More trials raise the false-positive rate, so correction is required—that is the point of multiple hypothesis testing.
If you independently test N factors at significance level α (e.g., 0.05), the probability of at least one false positive is:
At N=20 the probability already exceeds 64%—more than a 60% chance of finding a spurious anomaly when testing 20 factors.
Hence the need for correction. Two common methods:
Bonferroni divides the significance threshold by the number of tests.
Example: test 100 factors with a raw p-value cutoff of 0.05; after correction it is 0.0005. Only factors with p < 0.0005 count as significant.
The method is strict and can be too conservative. Bonferroni is workable for small N (e.g., 10–20); beyond ~50 it rarely finds anything. It also assumes independent tests, which financial factors usually violate due to high correlation.
FDR controls the expected share of false rejections among rejected hypotheses—allowing some false positives if that share stays controlled.
The most common procedure is Benjamini–Hochberg:
Example: 100 factors, q=0.1. The 5th-smallest p is 0.008 vs threshold (5/100)×0.1=0.005—fail. The 4th-smallest is 0.003 vs (4/100)×0.1=0.004—pass. Reject the first 4; treat those four as significant.
For initial screens, prefer FDR (q=0.2): let some false positives through and cull them in out-of-sample tests. Bonferroni is so strict it can discard real factors. The goal is not absolute truth but patterns that are likely to be profitable.
A practical workflow:
| Step | Action | Correction | Notes |
|---|---|---|---|
| 1 | Initial screen (100–500 candidates) | FDR (q=0.2) | Cast a wide net; do not over-filter |
| 2 | In-sample validation | Bonferroni | Strict gate on survivors |
| 3 | Out-of-sample test | None; judge live performance | Real results beat p-values |
| 4 | Live monitoring | Ongoing | Anomalies decay or disappear |
Anomaly mining is not about finding more factors—it is about finding real ones. Data-mining controls and multiple-testing correction filter surface-significant noise. One carefully corrected factor beats a hundred untested ones.
Do not over-rely on p-values—they are statistical tools, not absolute standards. Treating p < 0.05 as sufficient can be costly. Real anomalies must survive out-of-sample tests, economic logic, and time.
Anomaly mining mixes opportunity and risk; the core task is avoiding false signals in the data.
Sections 18.2–18.3 covered data-mining bias and Bonferroni/FDR. A further point: even without explicitly mining hundreds of factors, repeatedly trying sort details can manufacture significance.
Common degrees of freedom include:
Multiple-testing correction therefore applies not only to the “factor zoo” but also to specification search on the same factor. Robust reporting includes:
Link to Chapter 6: Sections 6.5–6.6 give templates for one-/two-way sorts and Fama–MacBeth; the discipline here constrains how freely those template parameters are searched.
Fundamental factors assess how solid a company’s economics are. Many quants start with prices and volumes as “purer.” Long practice shows fundamentals are often the ballast of the portfolio.
This section covers four repeatedly validated themes: financial quality, earnings growth, analyst expectations, and classic post-earnings announcement drift. Each has pitfalls and useful regimes.
Financial quality asks whether reported profits are real cash.
A common pattern: the income statement shows large profits while operating cash flow is buried in receivables—paper wealth.
Core metric set: Accruals ratio: (net income − operating cash flow) / total assets. Higher means worse earnings quality. Gross-margin stability: std of gross margin over the past 5 years—more stable is better (moat). Asset turnover: revenue / total assets—efficiency of asset use.
In practice these three are typically combined into Z-scores as follows:
# Pseudocode example
def financial_quality_score(df):
# 1. Accruals ratio (reverse: higher is worse)
accrual = (df['net_income'] - df['operating_cf']) / df['total_assets']
accrual_z = -1 * (accrual - accrual.mean) / accrual.std
# 2. Gross-margin stability (reverse: more vol is worse)
gross_margin_std = df.groupby('stock_id')['gross_margin'].rolling(20).std
stability_z = -1 * (gross_margin_std - gross_margin_std.mean) / gross_margin_std.std
# 3. Asset turnover (positive)
turnover = df['revenue'] / df['total_assets']
turnover_z = (turnover - turnover.mean) / turnover.std
# Equal-weight composite
quality_score = (accrual_z + stability_z + turnover_z) / 3
return quality_score
Caveat: in 2018, using accruals alone surfaced many cash-rich but growth-poor utilities. Adding gross-margin stability and asset turnover improved results. Single metrics overfit; prefer composites.
Earnings-growth factors are widely used, but year-over-year growth alone is often too crude.
Example: earnings from 1 to 2 imply 100% growth—whether it persists is another question.
Examine growth along three dimensions:
| Dimension | Metric | Notes |
|---|---|---|
| Growth speed | 3-year EPS CAGR | Smooths short-term noise; long-run trend |
| Growth stability | 5-year ROE coefficient of variation | Lower is better; more predictable growth |
| Growth source | Revenue growth vs net-income growth | Profit growth far above revenue growth → watch for accounting |
Note: growth factors tend to work better in bulls and face Davis double kills in bears. Pair with valuation—high growth plus cheap price is the stronger mix.
Analyst expectations capture how bullish the Street is. What matters is not the level of expectations but the expectation gap.
Prices already embed known forecasts; only the surprise moves prices.
Common metrics:
Practice note: in backtests, estimate-revision ratios are more effective in A-shares than in the US—possibly a higher retail share makes research more influential. The factor works best around earnings seasons and fades otherwise.
Post-earnings announcement drift (PEAD) is a classic anomaly: after good earnings, prices keep rising for a while; after bad earnings, they keep falling.
Markets react with a lag. Investors need time to digest news, or face trading costs and limited attention, so they cannot rebalance instantly.
A typical PEAD construction:
Prefer buying at the open on the second trading day after the announcement, not the announcement day. Gap opens make chasing costly; waiting a day for cooler sentiment often helps.
Each of the four factors can produce alpha alone; together they work better. A common framework: screen with financial quality and earnings growth, gauge sentiment with analyst expectations, and time entry with PEAD.
This framework has held up reasonably in A-shares and Hong Kong equities. There is no perfect factor—what matters is the economic logic and how each factor behaves across regimes.
Closing note: factor investing is not one trick forever. Market structure evolves and factors decay. Continuous learning and iteration are how strategies survive.
On alternative data, the industry once argued even about whether volume was a valid factor. Today takeout-order data is used to forecast consumer stocks—adoption has been rapid.
The core value of alternative data is information asymmetry. While most watch filings and charts, knowing a factory parking lot is empty early can itself be alpha. More data is not always better; fit to the question matters.
Sentiment factors turn news, social media, and stock-forum posts into numbers—usually polarity and attention.
Core logic: positive sentiment → buy; negative → sell; attention spikes → higher volatility.
Implementation example: a project processing millions of Weibo posts daily roughly follows:
Watch lexicon staleness. In 2019 a strategy still used an old dictionary that tagged “blow-up” slang as purely negative, while forums often used it as dark humor. Switching to dynamic word vectors improved results.
News sentiment resembles social sentiment but focuses on events—earnings, management changes, recalls—with pulse-like price impact.
Common processing ideas:
Do not rely on headlines alone. In one backtest, headline-only sentiment had Sharpe ~0.3; adding body text lifted it to ~0.8—clickbait titles distort, bodies are closer to true tone.
Typical code flow:
# Pseudocode: news sentiment factor
def news_sentiment_factor(news_df):
# news_df: date, ticker, title, content
# 1. Sentiment scores
news_df['sentiment'] = model.predict(news_df['content'])
# 2. Time decay
news_df['weight'] = np.exp(-0.1 * (today - news_df['date']).days)
# 3. Aggregate to daily
factor = news_df.groupby(['date', 'stock']) \
.apply(lambda x: (x['sentiment'] * x['weight']).sum / x['weight'].sum)
return factor
Satellite data once felt exotic; commercial costs have fallen sharply. Early skepticism faded after successfully forecasting a retailer’s quarterly sales.
Logic: parking-lot cars → foot traffic → revenue. In practice, watch:
Pitfall: forecasting ag names without seasonal adjustment—summer greenery inflates “crop” area. Year-over-year changes stabilize the signal.
Processing usually has three steps:
Example satellite series for a logistics company:
| Date | Parking-lot cars | Roof reflective area | Factor |
|---|---|---|---|
| 2024-01-05 | 342 | 1200 m² | 0.78 |
| 2024-01-06 | 287 | 980 m² | 0.62 |
| 2024-01-07 | 401 | 1350 m² | 0.91 |
Cars and reflective area move together—active warehouse operations. Three straight down days warrant caution.
Supply-chain data is a major research focus: a firm’s risk often lives in its suppliers and customers.
If a phone maker’s chip supplier fails, the phone maker’s stock will eventually suffer. Traditional factors wait for filings; supply-chain data can warn weeks earlier.
Common sources:
Factor construction often uses network analysis:
# Supply-chain network factor
def supply_chain_factor(company, network):
# First- and second-tier suppliers
suppliers = network.get_suppliers(company, depth=2)
# Average supplier risk score
risk_score = np.mean([s.risk for s in suppliers])
# High supplier risk → negative company factor
return -risk_score
Key insight: second-tier suppliers often matter more than first-tier. First-tier problems get switched quickly; second-tier issues take time to transmit—that lag is alpha.
Backtests suggest supply-chain factors work better in manufacturing and tech (~8% annualized excess, IR ~1.2) and almost not at all in banks, which lack a traditional supply chain.
Alternative data is not magic. Spending millions on satellite feeds can still underperform simple momentum. Understand the business hypothesis—what claim does the data test, and in which regimes?—before building.
Do not start too broad. Pick one familiar industry and one data type; run the full pipeline. For consumer stocks, begin with supply-chain data; expand once it is stable.
Many quants first find macro factors too abstract and prefer high-frequency price/volume signals, viewing macro as low-frequency and noisy for trading.
Adding a simple rate factor to a multi-factor model once lifted backtests by more than ten points—after that, macro got serious attention.
A macro factor system turns cycle views into quantifiable signals. Focus on four pillars: growth, inflation, rates, and credit spreads.
Growth is the anchor of the macro system—it tracks expansion versus contraction.
Common quant inputs:
Direct GDP growth as a factor is often too late—markets have already reacted. Composites of PMI and electricity generation work better.
Tip: growth usually correlates positively with equities and negatively with bonds. Industry betas differ—cyclicals (steel, metals) are far more sensitive than defensives (pharma, food).
Inflation is a double-edged sword: mild inflation supports growth; hyperinflation destroys value.
Common inflation metrics:
Why it matters: rising inflation can trigger hikes that compress equity valuations and erode real bond returns—material for both stocks and bonds.
YoY CPI alone is often too smooth to catch turns. MoM annualized rates plus core CPI raise sensitivity.
Caveat: seasonal noise (e.g., food around Chinese New Year) can distort the signal. Seasonally adjust before use.
Rates are the price of money—among the most active and sensitive macro factors.
Common rate factors:
Rate effects are broad: higher rates hurt bond prices and equity valuations and pressure property; banks can benefit via wider NIMs.
In multi-factor models, rates and growth can be highly correlated and destabilize estimates—orthogonalize to separate them.
Credit spreads are the risk thermometer of the macro system—the market’s price of credit risk.
Credit spread = corporate bond yield − government bond yield
Wider spreads signal pessimism and lower risk appetite; tighter spreads signal optimism and willingness to take risk.
Spreads matter most for HY credit, convertibles, and small-caps—assets with more credit risk and higher sensitivity.
Practice tip: treat credit spreads as a sentiment proxy. Rapid widening is a risk-off signal; sustained tightening can justify more risk.
The four factors are not independent; they transmit into each other. The map below sketches the logic:
Macro factor system map Growth · Inflation · Rates · Credit spreads Demand pull; central-bank hikes; financing costs Transmission and interactions Growth ↑ → inflation ↑ → rates ↑ → spreads ↑ (higher corporate funding costs) Growth ↓ → inflation ↓ → rates ↓ → spreads ↑ (higher default risk) Rates ↑ → growth ↓ (weaker investment/consumption) → inflation ↓ Equities · Bonds · Commodities · FX Note: solid arrows = direct effects; dashed = indirect Factors interact and jointly price asset classes
After the theory, how do macro factors enter a multi-factor model?
Common construction steps:
Example: a composite growth factor:
# Composite growth factor (pseudocode)
def build_growth_factor:
# 1. Raw data
pmi = get_data('PMI')
industrial = get_data('工业增加值')
electricity = get_data('发电量')
# 2. Standardize
pmi_z = (pmi - pmi.mean) / pmi.std
industrial_z = (industrial - industrial.mean) / industrial.std
electricity_z = (electricity - electricity.mean) / electricity.std
# 3. Equal-weight composite
growth_factor = (pmi_z + industrial_z + electricity_z) / 3
# 4. Lag one period (align to trading time)
growth_factor = growth_factor.shift(1)
return growth_factor
Practice tip: macro factors work better in low-frequency strategies (weekly/monthly rebalance). For high-frequency trading they add little—macros do not update daily.
Alongside strengths, know the limits. Common issues:
In 2015, a historically calibrated macro model failed completely in the crash—heavy policy intervention took markets off normal regimes.
Recommendation: combine macro with price/volume, fundamental, and sentiment factors for complementarity; do not over-rely on macro alone.
That closes the macro system. Growth, inflation, rates, and credit spreads are the pillars. Mastering them shifts focus from single names to the whole—an essential investment perspective.
A common view in quant practice: low-frequency factors are about logic; high-frequency factors are about data. Daily factors like PE, PB, and momentum/reversal are well known; competition turns on clean logic and backtest discipline. HF factors compete on microstructure understanding and the ability to process massive tick data.
This section covers HF factors: data features, microstructure, order-flow imbalance and downsampling, and common pitfalls.
HF data mainly means trade-by-trade and order-by-order records. A-shares can generate tens of millions of ticks per day—storage and compute are non-trivial.
Core features of HF data:
Core view: HF data are not for raw prediction; they are for extracting microstructure features. Feature quality determines HF factor efficacy.
After obtaining raw ticks, the first step is usually cleaning—not factor math: drop obvious errors (price 0, absurd sizes). Tedious but essential. Poor cleaning can make order-flow factors look great in backtests and diverge live—often because auction-session special records slipped in.
Microstructure factors use the order book and trades to reflect participant behavior and supply–demand.
Common microstructure factors:
Buy–sell pressure example:
buy_volume = sum(bid size levels 1–5)
sell_volume = sum(ask size levels 1–5)
pressure = (buy_volume - sell_volume) / (buy_volume + sell_volume)
The metric ranges from −1 to 1. Positive means buy pressure; negative means sell pressure. It often works in intraday reversal—extreme pressure tends to mean-revert shortly after.
Tip: microstructure factors usually need intraday standardization. Book depth varies by time and name. Z-score with rolling mean/std over the past N minutes.
Order flow imbalance (OFI) is finer-grained than simple pressure because it uses the aggressive side of each trade.
Classify aggressive buy vs sell by trade price:
OFI calculation:
# Order flow imbalance
def order_flow_imbalance(trades):
buy_volume = 0
sell_volume = 0
for trade in trades:
if trade['side'] == 'buy':
buy_volume += trade['volume']
else:
sell_volume += trade['volume']
imbalance = (buy_volume - sell_volume) / (buy_volume + sell_volume)
return imbalance
Caveat: OFI is highly sensitive to data quality. Wrong side flags reverse the signal. Add validation—reclassify side from trade price vs the book.
Note: vendors differ on side conventions (aggressive buy/sell vs buyer-/seller-initiated). Know the rule or the factor logic will be systematically biased.
HF signals can be strong, but tick-level trading is often too costly—hundreds of trades a day can erase most alpha in fees.
Hence downsample: convert HF signals to daily or hourly features.
Common downsampling methods:
Example: minute OFI down to daily:
# Downsample minute factors to daily
def daily_factor(minute_imbalance):
# Method 1: simple average
avg_imbalance = minute_imbalance.mean
# Method 2: weighted average (higher weight near close)
weights = np.linspace(0.5, 1.5, len(minute_imbalance))
weighted_avg = np.average(minute_imbalance, weights=weights)
# Method 3: extreme signal
extreme_signal = 1 if minute_imbalance.max > 0.8 else 0
return avg_imbalance, weighted_avg, extreme_signal
Rule of thumb: do not rely on one downsampling method. Compute several versions and combine—e.g., average OFI plus an extreme flag is often more stable than either alone.
Why? Different methods capture different information. Averages reflect the overall trend; extremes capture tail events. Together you see both the forest and the trees.
From HF data features through microstructure and OFI to downsampling into usable lower-frequency signals—each step has its own pitfalls and techniques.
Emphatically: HF factors are not a panacea, but they are a valuable complement in a multi-factor system. The next chapter discusses combining HF factors with traditional low-frequency factors into a full framework.