Contents

中文EN

Theory: Factor Models & Styles

1. Single-Factor ModelsCode2. Multi-Factor Models3. Classic Style Factors4. Expanding the Factor Universe

Practice: Factor Production & Management

5. Factor Construction and Neutralization6. Factor Testing FrameworkCode7. Portfolio Optimization and Attribution8. Factor Timing and Crowding

Extension: Macro Quant & Asset Allocation

9. Macro Data and Factors10. Macro Regime Identification11. Asset AllocationCode12. A Decade of Macro Quant NotesCode

1. Single-Factor Models

We first cover a few short sections on core factor-investing concepts (originally Chapter 1’s introduction), then move to CAPM and an A-share single-factor replication.

These are personal notes and code I keep while learning quant research—frameworks from single- to multi-factor models, implementation patterns, and interim reflections. The aim is not a definitive answer key, but a honest record of understanding, mistakes, and reviews. If they help others on the same path, good; feedback is welcome. Contact: adingminjie@163.com

Intro.1 What Is a Factor?

A factor, put simply, is a shared characteristic that helps explain stock returns. Why do some stocks outperform? They may be cheap, highly profitable, or riding strong market sentiment. Those selection logics are factors.

Examples:

• Value: buy cheap stocks
• Size: buy small-cap stocks
• Profitability: buy stocks with strong, stable earnings
• Momentum: buy recent winners
• Quality: buy stocks with durable earning power
• Low volatility: buy low-vol stocks

Each factor corresponds to a risk premium: bear a risk, earn a compensation. That is the core logic. Factors are not invented out of thin air—they need an economic rationale. A factor without logic is noise, however pretty the backtest.

Intro.2 Origins and Evolution of Factor Investing

1960s: Sharpe’s CAPM. The prevailing view was that stock returns depend only on market risk—bear market risk, earn the market return.

1970s–80s: Fama and French challenged that view. Small-caps and low book-to-market stocks tended to outperform over long horizons. In 1993 they proposed the three-factor model: market, size, and value.

1990s–today: the factor zoo kept growing:

• Momentum (Jegadeesh & Titman, 1993)
• Quality (Novy-Marx, 2013)
• Low volatility (Ang et al., 2006)
• Dividend, growth, sentiment, and more...

Mainstream factor libraries list hundreds of candidates; perhaps a dozen or so are reliably useful. Focus first on 5–10 classics, internalize their economics, then expand.

Factor investing is now core material in university finance curricula. At Peking University’s Guanghua School of Management, the sophomore course Securities Investment already covers the full factor-investing arc; at Tsinghua PBC School of Finance, PhD sequences include Empirical Asset Pricing and Theoretical Asset Pricing.

Intro.3 Core Logic of Factor Investing

The case for factor investing in three lines:

1. Returns are decomposable: any stock’s return can be written as factor exposures times factor returns
2. Risk is manageable: controlling exposures lets you manage portfolio risk with precision
3. Premia are somewhat predictable: factor returns are more stable over long horizons; short-run noise does not erase the long-run direction

Theory holding does not make implementation easy. Markets change and factor efficacy drifts—that is the central practical challenge.

In practice, three principles usually follow:

1. Diversify: avoid concentrated single-factor bets; multi-factor mixes are more robust.
2. Control risk: keep exposures within a tolerable band so one factor cannot sink the book.
3. Adjust dynamically: track factor life cycles and refresh allocations periodically.

Also guard against overfitting. Strong in-sample performance does not imply live stability. Prefer out-of-sample tests, rolling validation, and simpler structures.

In factor research, CAPM is the unavoidable baseline—and one of the first models finance undergraduates meet systematically.

Beginners often find it too simple: one $\beta$ linking asset returns to market risk seems unable to capture market heterogeneity. With more research and practice, CAPM’s value becomes clearer: not to explain everything, but to supply the minimal theory of risk pricing. That sparse structure is the methodological base for later multi-factor extensions and tests.

1.1 Introduction to CAPM

CAPM was introduced by Sharpe (1964) and refined by Lintner (1965) and Mossin (1966). The formula:

$$ E(R_i) = R_f + \beta_i \bigl(E(R_m) - R_f\bigr) $$

In words: expected stock return = risk-free rate + $\beta$ × market risk premium.

$\beta$ (beta) is sensitivity to the market. At $\beta=1$, a 1% market move maps to 1%; at $\beta=1.5$, it maps to 1.5%.

CAPM prices only systematic (market) risk. Idiosyncratic risk can be diversified away, so the market does not pay extra for it.

High-$\beta$ stocks usually show more upside elasticity in bull markets and deeper drawdowns in selloffs. Empirically, that highlights CAPM’s limit: with only the market factor, it cannot fully capture return differences driven by style, industry, and fundamentals.

1.2 Single-Factor Regression Analysis

How to estimate $\beta$? The direct route is linear regression. For individual stocks, use a rolling window—commonly 252 trading days (about one year). Shorter windows are noisy; longer ones react slowly.

import numpy as np
import pandas as pd
import statsmodels.api as sm

def calculate_beta(stock_returns, market_returns, window=252):
    """
    Rolling beta estimation.
    stock_returns: stock daily return series
    market_returns: market daily return series
    window: rolling window length
    """
    betas = pd.Series(index=stock_returns.index, dtype=float)

    for i in range(window, len(stock_returns)):
        y = stock_returns.iloc[i - window:i]
        x = market_returns.iloc[i - window:i]
        x = sm.add_constant(x)

        model = sm.OLS(y, x).fit
        betas.iloc[i] = model.params.iloc[1]  # slope is beta

    return betas

def calculate_beta_newey_west(stock_returns, market_returns, window=36, nw_lags=3, return_stats=True):
    """
    Rolling beta with Newey-West (HAC) robust standard errors.
    stock_returns: stock return series (same frequency as market_returns recommended)
    market_returns: market return series
    window: rolling window length (monthly data often uses 24/36/60)
    nw_lags: Newey-West lag order
    return_stats: True returns alpha/beta/t/p; False returns beta series only
    """
    df = pd.concat([stock_returns, market_returns], axis=1)
    df.columns = ["ret_i", "ret_m"]
    df = df.dropna.copy

    if nw_lags is None:
        nw_lags = max(1, int(window ** 0.25))

    result = pd.DataFrame(
        index=df.index,
        columns=["alpha", "beta", "beta_se_nw", "beta_t_nw", "beta_p_nw"],
        dtype=float,
    )

    for i in range(window, len(df) + 1):
        win = df.iloc[i - window:i]
        y = win["ret_i"]
        X = sm.add_constant(win["ret_m"])

        model = sm.OLS(y, X).fit(cov_type="HAC", cov_kwds={"maxlags": nw_lags})
        t = win.index[-1]

        result.loc[t, "alpha"] = model.params.get("const", np.nan)
        result.loc[t, "beta"] = model.params.get("ret_m", np.nan)
        result.loc[t, "beta_se_nw"] = model.bse.get("ret_m", np.nan)
        result.loc[t, "beta_t_nw"] = model.tvalues.get("ret_m", np.nan)
        result.loc[t, "beta_p_nw"] = model.pvalues.get("ret_m", np.nan)

    if return_stats:
        return result
    return result["beta"]

# Monthly data example:
# beta_df = calculate_beta_newey_west(stock_returns, market_returns, window=36, nw_lags=3, return_stats=True)
# beta_series = calculate_beta_newey_west(stock_returns, market_returns, window=36, nw_lags=3, return_stats=False)

A basic econometrics note: include an intercept. Without it, the fit is forced through the origin and beta is biased.

On returns, note that $\log(1+r)\approx r$. Simple or log returns both work for $\beta$, but stock, market, and risk-free series must use the same return definition.

1.3 Factor Returns and Factor Exposures

Factor return and factor exposure are easy to confuse.

Factor exposure: the loading / open risk on that factor. Example: beta = 1.2 is exposure to the market factor.

Factor return: what the factor itself earned. If the market rose 3% this month, the market factor return is 3%.

In short: stock return = factor exposure × factor return + residual.

1.4 Limits of Single-Factor Models

CAPM cannot fully explain stock moves. A classic case is the 2017 A-share “white horse” rally: Kweichow Moutai rose over 100% while the market index gained only about 6%. Under CAPM that would look like a huge $\alpha$. That excess need not be mispricing—it more often reflects exposures to other systematic factors such as value and quality.

That is why pricing models kept expanding—from Fama–French three- to five-factor models and then to higher-dimensional frameworks. The path is CAPM plus successively recovering omitted risk sources.

So CAPM is not “wrong”; its boundary is clear. It captures the basic market-risk compensation but not the full return structure. The single-factor model’s lasting value is as the analytical starting point for factor pricing; multi-factor systems build stronger explanation and application on top of it.

1.5 A-Share Data and CAPM Estimation

This section prepares A-share monthly data and runs CAPM regressions (pooled + stock-level), providing a data-definition benchmark for the rolling-beta work in Section 2.6.

Load Data

import pandas as pd
from urllib.parse import urlparse, parse_qs, urlencode, urlunparse
from IPython.display import display

# 1) Dropbox share links (original share-page URLs)
dropbox_urls = {
    "market_return": "https://www.dropbox.com/scl/fi/wd4zxa3safkrjyt4xjy3y/AShare_Market_Return_Monthly.xlsx?rlkey=glixs89u4unkwu3j0w478rdmh&st=38c5588m&dl=0",
    "riskfree_rate": "https://www.dropbox.com/scl/fi/hfl350d4t9fy3mw6dd9cp/AShare_Riskfree_Rate_Monthly.xlsx?rlkey=jrcfsu9vk856kdoigodfr03o7&st=ex1vxhiy&dl=0",
    "stock_return": "https://www.dropbox.com/scl/fi/roaossdn05bvxyr4b8n1i/AShare_Stocks_Return_Monthly.xlsx?rlkey=tw72id38dgyxd7qfso090hjgg&st=27ej4fqj&dl=0",
}

# 2) Convert share links to direct-download URLs (dl=1)
def to_direct_download(url: str) -> str:
    u = urlparse(url)
    q = parse_qs(u.query)
    q["dl"] = ["1"]  # force file download instead of web preview
    new_query = urlencode(q, doseq=True)
    return urlunparse((u.scheme, u.netloc, u.path, u.params, new_query, u.fragment))

direct_urls = {k: to_direct_download(v) for k, v in dropbox_urls.items}

# 3) Load Excel files into dict dfs; subsequent cells read from dfs
dfs = {}
for name, url in direct_urls.items:
    dfs[name] = pd.read_excel(url)

# 4) Quick check: shape and head of each table
for name, df in dfs.items:
    print(f"\n{name} shape = {df.shape}")
    display(df.head)

d:\pythonprojects\venv\Lib\site-packages\openpyxl\styles\stylesheet.py:237: UserWarning: Workbook contains no default style, apply openpyxl's default
  warn("Workbook contains no default style, apply openpyxl's default")
d:\pythonprojects\venv\Lib\site-packages\openpyxl\styles\stylesheet.py:237: UserWarning: Workbook contains no default style, apply openpyxl's default
  warn("Workbook contains no default style, apply openpyxl's default")
d:\pythonprojects\venv\Lib\site-packages\openpyxl\styles\stylesheet.py:237: UserWarning: Workbook contains no default style, apply openpyxl's default
  warn("Workbook contains no default style, apply openpyxl's default")

market_return shape = (8119, 4)

	Markettype	Trdmnt	Cmretwdos	Cmretwdtl
0	市场类型	交易月份	考虑现金红利再投资的综合月市场回报率(流通市值加权平均法)	考虑现金红利再投资的综合月市场回报率(总市值加权平均法)
1	没有单位	没有单位	没有单位	没有单位
2	5	1990-12	NaN	NaN
3	15	1990-12	NaN	NaN
4	21	1990-12	NaN	NaN

riskfree_rate shape = (13016, 3)

	Nrr1	Clsdt	Nrrmtdt
0	无风险利率基准	统计日期	月度化无风险利率(%)
1	没有单位	没有单位	没有单位
2	NRI01	1990-04-15	0.8035
3	NRI01	1990-08-21	0.693
4	NRI01	1990-12-19	0.693

stock_return shape = (942618, 8)

	Stkcd	Trdmnt	Clsdt	Mclsprc	Msmvosd	Msmvttl	Mretwd	Markettype
0	证券代码	交易月份	月收盘日期	月收盘价	月个股流通市值	月个股总市值	考虑现金红利再投资的月个股回报率	市场类型
1	没有单位	没有单位	没有单位	元/股	千元	千元	没有单位	没有单位
2	000001	1991-04	30	43.68	1157520	2118487.47	NaN	4
3	000001	1991-05	31	38.34	1016010	1859496.56	-0.122253	4
4	000001	1991-06	28	33.99	900735	1648520.81	-0.113459	4

CAPM Tests (China Market, Markettype = 53)

The code below does four things:

1. Auto-identify key fields in the three tables (stock/monthly returns, market return, risk-free rate)
2. Filter to market type 53 (Shanghai/Shenzhen A-shares + ChiNext + STAR)
3. Build stock and market excess returns
4. Report CAPM results (full sample + cross-section of stock-level fits)

import numpy as np
import statsmodels.api as sm

# Three raw tables (from dfs in the previous cell)
stock_raw = dfs["stock_return"].copy
market_raw = dfs["market_return"].copy
rf_raw = dfs["riskfree_rate"].copy

# Manual column mapping (CSMAR field names)
# Note: fixed names are more stable and reproducible than auto-detection.
STOCK_CODE_COL = "Stkcd"
STOCK_MONTH_COL = "Trdmnt"
STOCK_RET_COL = "Mretwd"

MARKET_TYPE_COL = "Markettype"
MARKET_MONTH_COL = "Trdmnt"
MARKET_RET_COL = "Cmretwdos"  # dividends reinvested + float-cap weighted

RF_MONTH_COL = "Clsdt"
RF_COL = "Nrrmtdt"  # monthly risk-free rate (%)

def to_month_str(x):
    """Normalize date columns to YYYY-MM strings for monthly merges."""
    return pd.to_datetime(x, errors="coerce").dt.to_period("M").astype("string")

def to_return_decimal(s):
    """
    Convert return columns to decimal units when needed:
    - If the median is clearly > 0.5, treat as percent (e.g. 3.2) and divide by 100
    - Otherwise treat as already decimal (e.g. 0.032)
    """
    s = pd.to_numeric(s, errors="coerce")
    s_clean = s.dropna
    if len(s_clean) == 0:
        return s
    if s_clean.abs.median > 0.5:
        return s / 100.0
    return s

# Check required columns exist to avoid silent failures later
required_cols = {
    "stock": [STOCK_CODE_COL, STOCK_MONTH_COL, STOCK_RET_COL],
    "market": [MARKET_TYPE_COL, MARKET_MONTH_COL, MARKET_RET_COL],
    "rf": [RF_MONTH_COL, RF_COL],
}

missing_cols = {
    "stock": [c for c in required_cols["stock"] if c not in stock_raw.columns],
    "market": [c for c in required_cols["market"] if c not in market_raw.columns],
    "rf": [c for c in required_cols["rf"] if c not in rf_raw.columns],
}

if any(missing_cols[k] for k in missing_cols):
    raise ValueError(f"Column names do not match; please check: {missing_cols}")

print("Column mapping is fixed; auto-detection disabled.")
print("stock:", required_cols["stock"])
print("market:", required_cols["market"])
print("rf:", required_cols["rf"])

列名映射已固定，不再自动识别。
stock: ['Stkcd', 'Trdmnt', 'Mretwd']
market: ['Markettype', 'Trdmnt', 'Cmretwdos']
rf: ['Clsdt', 'Nrrmtdt']

# 1) Stock data: select columns -> standard names -> unify time -> decimal returns
START_MONTH = "2000-01" # keep sample from 2000 onward
stock_df = stock_raw[[STOCK_CODE_COL, STOCK_MONTH_COL, STOCK_RET_COL]].copy
stock_df.columns = ["stock_code", "month", "ret_i"]
stock_df["month"] = to_month_str(stock_df["month"])
stock_df["ret_i"] = to_return_decimal(stock_df["ret_i"])

stock_df = stock_df.dropna(subset=["stock_code", "month", "ret_i"])

stock_df["stock_code"] = stock_df["stock_code"].astype(str).str.strip
stock_df = stock_df[stock_df["stock_code"].str.match(r"^\d{6}$", na=False)]
stock_df = stock_df[stock_df["month"] >= START_MONTH].copy

market_df = market_raw[[MARKET_TYPE_COL, MARKET_MONTH_COL, MARKET_RET_COL]].copy
market_df.columns = ["market_type", "month", "ret_m"]
market_df["month"] = to_month_str(market_df["month"])
market_df["ret_m"] = to_return_decimal(market_df["ret_m"])

market_df["market_type_num"] = pd.to_numeric(market_df["market_type"], errors="coerce")

market_df = market_df[(market_df["market_type_num"] == 53) | (market_df["market_type"].astype(str).str.strip == "53")].copy
market_df = market_df[["month", "ret_m"]].dropna.drop_duplicates(subset=["month"])
market_df = market_df[market_df["month"] >= START_MONTH].copy

rf_df = rf_raw[[RF_MONTH_COL, RF_COL]].copy
rf_df.columns = ["month", "rf"]
rf_df["month"] = to_month_str(rf_df["month"])

rf_df["rf"] = pd.to_numeric(rf_df["rf"], errors="coerce") / 100.0
rf_df = rf_df.dropna.drop_duplicates(subset=["month"])
rf_df = rf_df[rf_df["month"] >= START_MONTH].copy

print(f"Sample window: {START_MONTH} onward")
print("Sample size:")
print("stock_df:", stock_df.shape)
print("market_df (type=53):", market_df.shape)
print("rf_df:", rf_df.shape)
display(stock_df.head)
display(market_df.head)
display(rf_df.head)
print("\nrf describe (decimal units; e.g. 0.006092 means 0.6092%):")
display(rf_df["rf"].describe)

<ipython-input-4-379ecc8467f1>:24: UserWarning: Could not infer format, so each element will be parsed individually, falling back to `dateutil`. To ensure parsing is consistent and as-expected, please specify a format.
  return pd.to_datetime(x, errors="coerce").dt.to_period("M").astype("string")

样本窗口: 2000-01 及以后
样本规模:
stock_df: (890537, 3)
market_df (type=53): (319, 2)
rf_df: (320, 2)

<ipython-input-4-379ecc8467f1>:24: UserWarning: Could not infer format, so each element will be parsed individually, falling back to `dateutil`. To ensure parsing is consistent and as-expected, please specify a format.
  return pd.to_datetime(x, errors="coerce").dt.to_period("M").astype("string")
<ipython-input-4-379ecc8467f1>:24: UserWarning: Could not infer format, so each element will be parsed individually, falling back to `dateutil`. To ensure parsing is consistent and as-expected, please specify a format.
  return pd.to_datetime(x, errors="coerce").dt.to_period("M").astype("string")

	stock_code	month	ret_i
107	000001	2000-01	0.061891
108	000001	2000-02	-0.011333
109	000001	2000-03	0.002729
110	000001	2000-04	0.037017
111	000001	2000-05	-0.055118

	month	ret_m
2066	2000-01	0.160838
2085	2000-02	0.122024
2104	2000-03	0.055926
2123	2000-04	0.013014
2142	2000-05	0.027691

	month	rf
3304	2000-01	0.001856
3335	2000-02	0.001856
3364	2000-03	0.001856
3395	2000-04	0.001856
3425	2000-05	0.001856

rf描述统计（应为小数口径，例如0.006092表示0.6092%）：

count    320.000000
mean       0.001752
std        0.000612
min        0.000788
25%        0.001241
50%        0.001635
75%        0.002060
max        0.003386
Name: rf, dtype: float64

# 3) Merge three tables on month and build excess returns
capm_df = (
    stock_df
    .merge(market_df, on="month", how="inner")
    .merge(rf_df, on="month", how="inner")
)
capm_df = capm_df.dropna(subset=["ret_i", "ret_m", "rf", "stock_code", "month"]).copy
capm_df["excess_i"] = capm_df["ret_i"] - capm_df["rf"]
capm_df["excess_m"] = capm_df["ret_m"] - capm_df["rf"]

print("CAPM merged sample:", capm_df.shape)
print("n_stocks:", capm_df["stock_code"].nunique, "n_months:", capm_df["month"].nunique)
display(capm_df.head)

# 4) Pooled CAPM: single regression on the stacked sample
X_pool = sm.add_constant(capm_df[["excess_m"]])
y_pool = capm_df["excess_i"]
pool_model = sm.OLS(y_pool, X_pool, missing="drop").fit(cov_type="HAC", cov_kwds={"maxlags": 3})

print("\nPooled CAPM (HAC, maxlags=3):")
print(pool_model.summary)

# 5) Stock-level CAPM: estimate alpha/beta per stock
rows = []
min_obs = 24  # at least 24 months to avoid unstable short samples

for code, g in capm_df.groupby("stock_code"):
    g = g.dropna(subset=["excess_i", "excess_m"])

    if len(g) < min_obs:
        continue

    X = sm.add_constant(g[["excess_m"]])
    y = g["excess_i"]
    m = sm.OLS(y, X).fit(cov_type="HAC", cov_kwds={"maxlags": 3})

    rows.append({
        "stock_code": code,
        "n_obs": len(g),
        "alpha": m.params.get("const", np.nan),
        "beta": m.params.get("excess_m", np.nan),
        "alpha_t": m.tvalues.get("const", np.nan),
        "beta_t": m.tvalues.get("excess_m", np.nan),
        "alpha_p": m.pvalues.get("const", np.nan),
        "beta_p": m.pvalues.get("excess_m", np.nan),
        "r2": m.rsquared,
    })

capm_by_stock = pd.DataFrame(rows)
print("\nStock-level results:", capm_by_stock.shape)
display(capm_by_stock.head)

# 6) Summary metrics: quick read on CAPM fit in sample
if len(capm_by_stock) > 0:
    summary_df = pd.DataFrame({
        "metric": ["alpha_mean", "beta_mean", "r2_mean", "alpha_sig_5pct_ratio"],
        "value": [
            capm_by_stock["alpha"].mean,
            capm_by_stock["beta"].mean,
            capm_by_stock["r2"].mean,
            (capm_by_stock["alpha_p"] < 0.05).mean,
        ],
    })
    print("\nSummary stats:")
    display(summary_df)
else:
    print("No stocks meet min_obs=24; check the date range or field mapping.")

CAPM合并后样本: (890537, 7)
股票数: 5977 月份数: 319

	stock_code	month	ret_i	ret_m	rf	excess_i	excess_m
0	000001	2000-01	0.061891	0.160838	0.001856	0.060035	0.158982
1	000001	2000-02	-0.011333	0.122024	0.001856	-0.013189	0.120168
2	000001	2000-03	0.002729	0.055926	0.001856	0.000873	0.054070
3	000001	2000-04	0.037017	0.013014	0.001856	0.035161	0.011158
4	000001	2000-05	-0.055118	0.027691	0.001856	-0.056974	0.025835

Pooled CAPM (HAC, maxlags=3):
                            OLS Regression Results                            
==============================================================================
Dep. Variable:               excess_i   R-squared:                       0.207
Model:                            OLS   Adj. R-squared:                  0.207
Method:                 Least Squares   F-statistic:                 1.622e+05
Date:                Sat, 08 Aug 2026   Prob (F-statistic):               0.00
Time:                        21:00:30   Log-Likelihood:             4.6529e+05
No. Observations:              890537   AIC:                        -9.306e+05
Df Residuals:                  890535   BIC:                        -9.306e+05
Df Model:                           1                                         
Covariance Type:                  HAC                                         
==============================================================================
                 coef    std err          z      P>|z|      [0.025      0.975]
------------------------------------------------------------------------------
const          0.0045      0.000     30.965      0.000       0.004       0.005
excess_m       1.1083      0.003    402.792      0.000       1.103       1.114
==============================================================================
Omnibus:                  1713749.296   Durbin-Watson:                   2.060
Prob(Omnibus):                  0.000   Jarque-Bera (JB):      47072717237.924
Skew:                          14.038   Prob(JB):                         0.00
Kurtosis:                    1128.976   Cond. No.                         15.1
==============================================================================

Notes:
[1] Standard Errors are heteroscedasticity and autocorrelation robust (HAC) using 3 lags and without small sample correction

个股回归结果: (5729, 9)

	stock_code	n_obs	alpha	beta	alpha_t	beta_t	alpha_p	beta_p	r2
0	000001	317	0.001696	1.000171	0.435371	17.537020	0.663293	7.474557e-69	0.509825
1	000002	313	0.005495	0.872107	0.983000	8.214996	0.325607	2.121721e-16	0.254732
2	000004	309	-0.001135	1.055256	-0.141812	10.194937	0.887228	2.088800e-24	0.224673
3	000005	275	-0.001002	1.338899	-0.108143	7.944917	0.913882	1.943202e-15	0.216694
4	000006	314	0.004957	1.240507	0.904831	13.150238	0.365555	1.696403e-39	0.407076

汇总统计:

	metric	value
0	alpha_mean	0.004637
1	beta_mean	1.163326
2	r2_mean	0.237669
3	alpha_sig_5pct_ratio	0.035259

Reading CAPM Results (Pooled + Stock Cross-Section)

A brief reading of Pooled CAPM (HAC, maxlags=3) and the stock-level regressions:

1. The market factor is strongly significant

• The excess_m coefficient is about 1.1083, with z=402.792 and p<0.001.
• Stock excess returns co-move tightly with market excess returns.
• A coefficient above 1 means that, on average in this sample, stocks load more than one-for-one on market moves.

2. The intercept (alpha) is positive and significant

• const=0.0045 (monthly decimal ≈ 0.45% per month).
• After controlling for the market, average excess return remains significantly positive.
• That usually signals that single-factor CAPM is incomplete—omitted factors (size, value, quality, momentum, etc.) may matter.

3. Explanatory power is limited but expected

• R-squared=0.207: CAPM explains about 20.7% of stock excess-return variation.
• The remaining ~79.3% is other risk factors, idiosyncrasy, and noise.
• Consistent with CAPM as a baseline with bounded explanatory power.

4. Strong statistical significance ≠ economic significance

• Sample size N=890,537 and a very large F-statistic.
• In huge samples, significance is unsurprising; focus on coefficient size, tradability, and robustness.

5. Residuals are heavy-tailed

• Skew=14.038, Kurtosis=1128.976, and a huge significant JB.
• Residuals are far from normal—extremes and fat tails are common in equity returns.
• HAC (Newey–West) standard errors are appropriate to mitigate heteroskedasticity and autocorrelation.

6. Durbin–Watson near 2

• DW=2.060 suggests little first-order residual autocorrelation.

7. Broad stock-level coverage

• Stock-level results shape (5729, 9): alpha/beta/t/p/R2 for 5,729 names.
• Next checks: mean/median alpha and significance share; beta stability; whether industry/style exposures create spurious alpha.

One-line takeaway: CAPM holds directionally in this sample (beta is significant) but is incomplete (modest R² and significant alpha)—empirical motivation to move from single- to multi-factor models.

1.6 Rolling-Window Beta Estimation (Full A-Share Replication)

This section replicates the standard market-beta pipeline in empirical asset pricing on A-share data: single-stock CAPM time-series regressions, then closed-form rolling-window estimates for the full market, comparing monthly and daily frequencies.

Field mapping (names aligned with common academic code):

Field here	Meaning	Section 2.5 counterpart
permno	Stock code	stock_code
date	Month-end / month-start date	month
ret_excess	Stock excess return	excess_i
mkt_excess	Market excess return	excess_m

The CAPM time-series regression is

$$ r_{i,t}-r_{f,t}=\alpha_i+\beta_i(r_{m,t}-r_{f,t})+\varepsilon_{i,t}. $$

Prefer local CSMAR exports under 202608_02_传统量化/Data (same field definitions as the Section 2.5 Dropbox files). If Section 2.5 already produced capm_df, the monthly panel can reuse that merge logic.

1.6.1 Environment and Dependencies

Implemented with pandas / numpy / statsmodels / matplotlib. The I/O shape of roll_capm_estimation matches common Python replications (permno, date, coefficient, estimate, t_statistic).

import warnings
from pathlib import Path

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from statsmodels.regression.linear_model import OLS
from statsmodels.tools.tools import add_constant
from IPython.display import display

warnings.filterwarnings("ignore", category=UserWarning)

# CJK-capable fonts (fallback by local availability; same as earlier backtest plots)
from matplotlib import font_manager

_cn_candidates = ["Microsoft YaHei", "SimHei", "PingFang SC", "Noto Sans CJK SC", "Arial Unicode MS"]
_available = {f.name for f in font_manager.fontManager.ttflist}
for _name in _cn_candidates:
    if _name in _available:
        plt.rcParams["font.sans-serif"] = [_name]
        break
plt.rcParams["axes.unicode_minus"] = False

DATA_DIR = Path(r"D:/A_Topics/202607_03_TidyFinanceAShare/202608_02_传统量化/Data")
CACHE_DIR = DATA_DIR / "_cache_beta"
CACHE_DIR.mkdir(parents=True, exist_ok=True)

print("DATA_DIR =", DATA_DIR)
print("exists =", DATA_DIR.exists)

DATA_DIR = D:\A_Topics\202607_03_TidyFinanceAShare\202608_02_传统量化\Data
exists = True

1.6.2 Load and Clean the Monthly Panel

Same steps as Section 2.5:

1. Load stock monthly returns, market monthly returns, and the risk-free rate;
2. Keep market type 53 (SSE/SZSE A-shares + ChiNext + STAR; float-cap weighted with dividends reinvested);
3. Convert to decimal returns and build excess returns.

Then map columns to permno / date / ret_excess / mkt_excess so the rolling estimator can be applied directly.

def read_csmar_excel(path: Path) -> pd.DataFrame:
    """Read CSMAR-style Excel: row 1 = field names; next two rows often Chinese labels/units."""
    df = pd.read_excel(path, header=0)
    # Always skip two annotation rows (Chinese name/units); do not filter by numeric first column (RF uses NRI01)
    if len(df) >= 2:
        df = df.iloc[2:].copy
    return df.reset_index(drop=True)

def to_month_period(x) -> pd.Series:
    return pd.to_datetime(x, errors="coerce").dt.to_period("M")

def to_return_decimal(s: pd.Series) -> pd.Series:
    s = pd.to_numeric(s, errors="coerce")
    clean = s.dropna
    if len(clean) == 0:
        return s
    if clean.abs.median > 0.5:
        return s / 100.0
    return s

def load_ashare_monthly(data_dir: Path, start_month: str = "2000-01") -> pd.DataFrame:
    stock_raw = read_csmar_excel(data_dir / "AShare_Stocks_Return_Monthly.xlsx")
    market_raw = read_csmar_excel(data_dir / "AShare_Market_Return_Monthly.xlsx")
    rf_raw = read_csmar_excel(data_dir / "AShare_Riskfree_Rate_Monthly.xlsx")

    stock = stock_raw[["Stkcd", "Trdmnt", "Mretwd", "Msmvosd"]].copy
    stock.columns = ["permno", "month", "ret", "mktcap"]
    stock["permno"] = stock["permno"].astype(str).str.zfill(6)
    stock["month"] = to_month_period(stock["month"])
    stock["ret"] = to_return_decimal(stock["ret"])
    stock["mktcap"] = pd.to_numeric(stock["mktcap"], errors="coerce")

    market = market_raw[["Markettype", "Trdmnt", "Cmretwdos"]].copy
    market.columns = ["market_type", "month", "ret_m"]
    market["market_type"] = pd.to_numeric(market["market_type"], errors="coerce")
    market = market[market["market_type"] == 53].copy
    market["month"] = to_month_period(market["month"])
    market["ret_m"] = to_return_decimal(market["ret_m"])
    market = market[["month", "ret_m"]].drop_duplicates("month")

    rf = rf_raw.copy
    if "Nrr1" in rf.columns:
        rf = rf[rf["Nrr1"].astype(str).str.upper.str.contains("NRI01", na=False)]
    rf = rf[["Clsdt", "Nrrmtdt"]].copy
    rf.columns = ["month", "rf"]
    rf["month"] = to_month_period(rf["month"])
    rf["rf"] = pd.to_numeric(rf["rf"], errors="coerce") / 100.0  # Nrrmtdt is in percent
    rf = rf.dropna.drop_duplicates("month")

    out = (
        stock.merge(market, on="month", how="inner")
        .merge(rf, on="month", how="inner")
        .dropna(subset=["permno", "month", "ret", "ret_m", "rf"])
    )
    out = out[out["month"] >= pd.Period(start_month, freq="M")].copy
    out["date"] = out["month"].dt.to_timestamp(how="start")
    out["ret_excess"] = out["ret"] - out["rf"]
    out["mkt_excess"] = out["ret_m"] - out["rf"]
    # Lagged market cap for later size-group descriptions
    out = out.sort_values(["permno", "date"])
    out["mktcap_lag"] = out.groupby("permno")["mktcap"].shift(1)
    return out.reset_index(drop=True)

cache_monthly = CACHE_DIR / "crsp_monthly_ashare.parquet"
if cache_monthly.exists:
    crsp_monthly = pd.read_parquet(cache_monthly)
    print("loaded cache:", cache_monthly)
else:
    crsp_monthly = load_ashare_monthly(DATA_DIR, start_month="2000-01")
    crsp_monthly.to_parquet(cache_monthly, index=False)
    print("saved cache:", cache_monthly)

print(crsp_monthly.shape)
print("stocks:", crsp_monthly["permno"].nunique, "months:", crsp_monthly["date"].nunique)
display(crsp_monthly.head)

loaded cache: D:\A_Topics\202607_03_TidyFinanceAShare\202608_02_传统量化\Data\_cache_beta\crsp_monthly_ashare.parquet
(890537, 10)
stocks: 5977 months: 319

	permno	month	ret	mktcap	ret_m	rf	date	ret_excess	mkt_excess	mktcap_lag
0	000001	2000-01	0.061891	19843822.88	0.160838	0.001856	2000-01-01	0.060035	0.158982	NaN
1	000001	2000-02	-0.011333	19618933.36	0.122024	0.001856	2000-02-01	-0.013189	0.120168	19843822.88
2	000001	2000-03	0.002729	19672478.48	0.055926	0.001856	2000-03-01	0.000873	0.054070	19618933.36
3	000001	2000-04	0.037017	20400692.17	0.013014	0.001856	2000-04-01	0.035161	0.011158	19672478.48
4	000001	2000-05	-0.055118	19276244.57	0.027691	0.001856	2000-05-01	-0.056974	0.025835	20400692.17

1.6.3 Single-Stock CAPM Example

Run a full-sample time-series CAPM on one representative name to fix the regression and coefficient interpretation. Here 600519 (Kweichow Moutai) plays the role of the Apple example in the original text.

example_permno = "600519"

model_fit = OLS(
    crsp_monthly.loc[crsp_monthly["permno"] == example_permno, "ret_excess"],
    add_constant(
        crsp_monthly.loc[crsp_monthly["permno"] == example_permno, "mkt_excess"]
    ),
    missing="drop",
).fit

coefficients = pd.DataFrame(
    {
        "Estimate": model_fit.params,
        "Std. Error": model_fit.bse,
        "t value": model_fit.tvalues,
        "Pr(>|t|)": model_fit.pvalues,
    }
)
coefficients.index = ["Intercept", "mkt_excess"]
display(coefficients)

print(
    "Kweichow Moutai full-sample beta ≈ {:.3f}; alpha ≈ {:.4f}/month".format(
        model_fit.params["mkt_excess"], model_fit.params["const"]
    )
)

	Estimate	Std. Error	t value	Pr(>|t|)
Intercept	0.018621	0.005104	3.648361	3.118157e-04
mkt_excess	0.633779	0.067782	9.350247	2.229768e-18

贵州茅台全样本 beta ≈ 0.634；alpha ≈ 0.0186/月

1.6.4 Closed-Form Rolling-Window Estimation

Fitting CAPM stock-by-stock and month-by-month for the whole market is slow. For a single-regressor model, rolling OLS has a closed form. Let $x$ be market excess return and $y$ stock excess return. Within each window,

$$ \hat\beta_i=\frac{S_{xy}}{S_{xx}},\qquad \hat\alpha_i=\bar y-\hat\beta_i\bar x, $$

where

$$ S_{xy}=\sum_t x_t y_t-\frac{1}{n}\sum_t x_t\sum_t y_t,\qquad S_{xx}=\sum_t x_t^2-\frac{1}{n}\left(\sum_t x_t\right)^2. $$

Rolling sums of $\sum x,\sum y,\sum x^2,\sum y^2,\sum xy$ therefore deliver vectorized betas and $t$-stats for all stocks. Default window: 60 months, with at least 48 valid observations (a common setting).

def roll_capm_estimation(
    data: pd.DataFrame,
    look_back: int = 60,
    min_obs: int = 48,
) -> pd.DataFrame:
    """Closed-form rolling CAPM estimation (API aligned with common roll_capm_estimation).

    Parameters
    ----------
    data : must contain permno, date, ret_excess, mkt_excess
    look_back : rolling window length (periods after aggregating by date; usually months).
    min_obs : minimum raw observations in the window. For monthly data ≈ minimum months;
        after truncating daily dates to months, n is total trading days in the window
        (e.g. 200). Do not pass this as pandas rolling min_periods (daily look_back=12
        with min_obs=200 would then set min_periods > window).
    """
    df = data[["permno", "date", "ret_excess", "mkt_excess"]].dropna.copy
    df["date"] = pd.to_datetime(df["date"], errors="coerce")
    df = df.dropna(subset=["date"])
    df["xx"] = df["mkt_excess"] ** 2
    df["yy"] = df["ret_excess"] ** 2
    df["xy"] = df["mkt_excess"] * df["ret_excess"]

    cumulants = (
        df.groupby(["permno", "date"], as_index=False)
        .agg(
            n=("mkt_excess", "size"),
            sum_x=("mkt_excess", "sum"),
            sum_y=("ret_excess", "sum"),
            sum_xx=("xx", "sum"),
            sum_yy=("yy", "sum"),
            sum_xy=("xy", "sum"),
        )
        .sort_values(["permno", "date"])
    )

    pieces = []
    for permno, g in cumulants.groupby("permno", sort=False):
        g = g.sort_values("date").copy
        # rolling min_periods = minimum months/periods; separate from min_obs (raw obs count)
        rolled = g[["n", "sum_x", "sum_y", "sum_xx", "sum_yy", "sum_xy"]].rolling(
            window=look_back, min_periods=1
        ).sum
        est = g[["permno", "date"]].copy
        est = est.join(rolled)
        est = est[est["n"] >= min_obs].copy
        if est.empty:
            continue

        est["s_xx"] = est["sum_xx"] - est["sum_x"] ** 2 / est["n"]
        est["s_xy"] = est["sum_xy"] - est["sum_x"] * est["sum_y"] / est["n"]
        est["s_yy"] = est["sum_yy"] - est["sum_y"] ** 2 / est["n"]
        # avoid divide-by-zero when market returns have no variation in the window
        est = est[est["s_xx"].abs > 1e-18].copy
        if est.empty:
            continue
        est["beta"] = est["s_xy"] / est["s_xx"]
        est["alpha"] = (est["sum_y"] - est["beta"] * est["sum_x"]) / est["n"]
        est["sigma2"] = (est["s_yy"] - est["beta"] * est["s_xy"]) / (est["n"] - 2)
        est.loc[est["sigma2"] <= 0, "sigma2"] = np.nan
        est["t_alpha"] = est["alpha"] / np.sqrt(
            est["sigma2"]
            * (1.0 / est["n"] + (est["sum_x"] / est["n"]) ** 2 / est["s_xx"])
        )
        est["t_beta"] = est["beta"] / np.sqrt(est["sigma2"] / est["s_xx"])
        pieces.append(est)

    if not pieces:
        return pd.DataFrame(
            columns=["permno", "date", "coefficient", "estimate", "t_statistic"]
        )

    estimates = pd.concat(pieces, ignore_index=True)
    alpha = estimates.assign(
        coefficient="alpha", estimate=estimates["alpha"], t_statistic=estimates["t_alpha"]
    )[["permno", "date", "coefficient", "estimate", "t_statistic"]]
    beta = estimates.assign(
        coefficient="mkt_excess",
        estimate=estimates["beta"],
        t_statistic=estimates["t_beta"],
    )[["permno", "date", "coefficient", "estimate", "t_statistic"]]
    out = pd.concat([alpha, beta], ignore_index=True)
    return out.sort_values(["permno", "date", "coefficient"]).reset_index(drop=True)

print("roll_capm_estimation ready")

roll_capm_estimation ready

1.6.5 Rolling Betas for Example Stocks

Validate on a few well-known names: Kweichow Moutai, CATL, Ping An, and Wuliangye. Window = 60 months.

examples = pd.DataFrame(
    {
        "permno": ["600519", "300750", "601318", "000858"],
        "company": ["Kweichow Moutai", "CATL", "Ping An", "Wuliangye"],
    }
)

capm_examples = roll_capm_estimation(
    crsp_monthly[crsp_monthly["permno"].isin(examples["permno"])],
    look_back=60,
    min_obs=48,
)
display(capm_examples.head(10))
print("rows:", len(capm_examples))

	permno	date	coefficient	estimate	t_statistic
0	000858	2003-12-01	alpha	0.003234	0.328274
1	000858	2003-12-01	mkt_excess	0.609631	3.699961
2	000858	2004-01-01	alpha	0.003228	0.334763
3	000858	2004-01-01	mkt_excess	0.609517	3.798020
4	000858	2004-02-01	alpha	0.005334	0.550281
5	000858	2004-02-01	mkt_excess	0.650284	4.045008
6	000858	2004-03-01	alpha	0.005596	0.588700
7	000858	2004-03-01	mkt_excess	0.652414	4.108064
8	000858	2004-04-01	alpha	0.006662	0.710796
9	000858	2004-04-01	mkt_excess	0.623492	4.052087

rows: 1504

Figure 1: Monthly Rolling Betas for Example Stocks

Keep only slope estimates with coefficient == "mkt_excess" and plot the time series. Betas vary over time and are positive in most periods.

beta_examples = (
    capm_examples[capm_examples["coefficient"] == "mkt_excess"]
    .merge(examples, on="permno", how="left")
)

fig, ax = plt.subplots(figsize=(10, 4.5))
for company, g in beta_examples.groupby("company"):
    ax.plot(g["date"], g["estimate"], label=company, linewidth=1.5)
ax.set_title("Monthly Rolling Betas for Example Stocks (5-Year Window)")
ax.set_xlabel("")
ax.set_ylabel("beta")
ax.legend(frameon=False, ncol=2)
ax.grid(True, alpha=0.3)
plt.tight_layout
plt.show

1.6.6 Full-Sample Monthly Rolling Estimation

Estimate once on the full A-share monthly panel. With the closed form this usually finishes in tens of seconds (machine-dependent), without explicit per-stock OLS loops.

cache_capm_m = CACHE_DIR / "capm_monthly_ashare.parquet"
if cache_capm_m.exists:
    capm_monthly = pd.read_parquet(cache_capm_m)
    print("loaded cache:", cache_capm_m)
else:
    capm_monthly = roll_capm_estimation(crsp_monthly, look_back=60, min_obs=48)
    capm_monthly.to_parquet(cache_capm_m, index=False)
    print("saved cache:", cache_capm_m)

print(capm_monthly.shape)
display(capm_monthly.head)

beta_monthly = (
    capm_monthly[capm_monthly["coefficient"] == "mkt_excess"]
    [["permno", "date", "estimate"]]
    .rename(columns={"estimate": "beta"})
    .assign(return_type="monthly")
)
print("beta_monthly:", beta_monthly.shape)

loaded cache: D:\A_Topics\202607_03_TidyFinanceAShare\202608_02_传统量化\Data\_cache_beta\capm_monthly_ashare.parquet
(1246512, 5)

	permno	date	coefficient	estimate	t_statistic
0	000001	2003-12-01	alpha	-0.008803	-0.999836
1	000001	2003-12-01	mkt_excess	0.945177	6.418647
2	000001	2004-01-01	alpha	-0.008221	-0.951793
3	000001	2004-01-01	mkt_excess	0.956954	6.657053
4	000001	2004-02-01	alpha	-0.007131	-0.835241

beta_monthly: (623256, 4)

1.6.7 Beta Distribution by Size Quintile

The original text used industry boxplots. The local monthly file has no industry codes, so we use lagged float-cap quintiles for the same cross-sectional description: average each stock’s beta in-sample, then boxplot by average market cap.

Expect small-cap groups to differ from large-caps in beta dispersion or level; persistently negative market betas are rare.

size_panel = (
    beta_monthly.merge(
        crsp_monthly[["permno", "date", "mktcap_lag"]],
        on=["permno", "date"],
        how="left",
    )
    .dropna(subset=["beta", "mktcap_lag"])
)

firm_beta = (
    size_panel.groupby("permno", as_index=False)
    .agg(beta=("beta", "mean"), mktcap_lag=("mktcap_lag", "mean"))
)
firm_beta["size_group"] = pd.qcut(
    firm_beta["mktcap_lag"],
    5,
    labels=["Q1 Small", "Q2", "Q3", "Q4", "Q5 Large"],
)

fig, ax = plt.subplots(figsize=(8, 4.5))
order = ["Q1 Small", "Q2", "Q3", "Q4", "Q5 Large"]
data_box = [firm_beta.loc[firm_beta["size_group"] == k, "beta"].values for k in order]
ax.boxplot(data_box, labels=order, showfliers=False)
ax.axhline(1.0, color="gray", ls="--", lw=1)
ax.set_title("Distribution of Average Stock Betas by Size Quintile")
ax.set_ylabel("Average beta")
ax.grid(True, axis="y", alpha=0.3)
plt.tight_layout
plt.show

print(firm_beta.groupby("size_group", observed=True)["beta"].describe[["count", "mean", "50%"]])

             count      mean       50%
size_group                            
Q1 小市值      1026.0  1.083252  1.065115
Q2          1025.0  1.136182  1.118821
Q3          1025.0  1.156031  1.135611
Q4          1025.0  1.192340  1.162915
Q5 大市值      1026.0  1.151092  1.129414

<ipython-input-14-8c79d19563c4>:23: MatplotlibDeprecationWarning: The 'labels' parameter of boxplot() has been renamed 'tick_labels' since Matplotlib 3.9; support for the old name will be dropped in 3.11.
  ax.boxplot(data_box, labels=order, showfliers=False)

1.6.8 Cross-Sectional Beta Quantiles over Time

Each month, compute the 10%–90% quantiles of market-wide betas to track co-movement and dispersion in risk exposure.

quantiles = np.arange(0.1, 1.0, 0.1)
rows = []
for dt, g in beta_monthly.groupby("date"):
    qs = g["beta"].quantile(quantiles)
    for q, v in qs.items:
        rows.append({"date": dt, "quantile": int(round(q * 100)), "beta": v})
beta_quantiles = pd.DataFrame(rows)

fig, ax = plt.subplots(figsize=(10, 4.5))
for q, g in beta_quantiles.groupby("quantile"):
    ax.plot(g["date"], g["beta"], label=str(q), linewidth=1.2)
ax.set_title("Cross-Sectional Quantiles of Monthly Betas (10%–90%)")
ax.set_xlabel("")
ax.set_ylabel("beta")
ax.legend(title="Quantile", ncol=5, frameon=False, fontsize=8)
ax.grid(True, alpha=0.3)
plt.tight_layout
plt.show

1.6.9 Estimating Beta with Daily Returns

Daily inputs come from local files—no synthetic market series or monthly RF spread to daily:

• Stock daily returns: AShare_Stocks_Return_Daily_1.xlsx … _7.xlsx (Dretwd)
• Market daily returns: AShare_Market_Return_Daily.xlsx (Cdretwdos for Markettype=53, float-cap weighted with dividends reinvested)
• Daily risk-free rate: AShare_Riskfree_Rate_Daily.xlsx (Nrrdaydt for NRI01, percent → decimal)

Because the stock daily sample is relatively short from about 2021-08, use a 12-month window with min_obs=200. With a longer history, restore look_back / min_obs to 60 / 1000. Truncate trading dates to month-start, then call the same roll_capm_estimation.

def load_ashare_daily_panel(data_dir: Path) -> pd.DataFrame:
    """Load ready-made daily stock/market returns and RF; build excess-return panel."""
    files = sorted(data_dir.glob("AShare_Stocks_Return_Daily_*.xlsx"))
    if not files:
        raise FileNotFoundError("AShare_Stocks_Return_Daily_*.xlsx not found")

    parts = []
    for f in files:
        raw = read_csmar_excel(f)
        keep = raw[["Stkcd", "Trddt", "Dretwd", "Trdsta"]].copy
        keep.columns = ["permno", "date", "ret", "trdsta"]
        parts.append(keep)
        print("read", f.name, "rows", len(keep))

    daily = pd.concat(parts, ignore_index=True)
    daily["permno"] = daily["permno"].astype(str).str.zfill(6)
    daily["date"] = pd.to_datetime(daily["date"], errors="coerce")
    daily["ret"] = pd.to_numeric(daily["ret"], errors="coerce")
    daily["trdsta"] = pd.to_numeric(daily["trdsta"], errors="coerce")
    daily = daily.dropna(subset=["permno", "date", "ret"])
    # Trade status 1 = normal trading (common CSMAR convention)
    daily = daily[(daily["trdsta"].isna) | (daily["trdsta"] == 1)].copy

    # Ready-made daily market return: type 53, float-cap weighted with dividends reinvested
    market_raw = read_csmar_excel(data_dir / "AShare_Market_Return_Daily.xlsx")
    market = market_raw[["Markettype", "Trddt", "Cdretwdos"]].copy
    market.columns = ["market_type", "date", "ret_m"]
    market["market_type"] = pd.to_numeric(market["market_type"], errors="coerce")
    market = market[market["market_type"] == 53].copy
    market["date"] = pd.to_datetime(market["date"], errors="coerce")
    market["ret_m"] = pd.to_numeric(market["ret_m"], errors="coerce")
    market = market[["date", "ret_m"]].dropna.drop_duplicates("date")

    # Ready-made daily risk-free rate (percent -> decimal)
    rf_raw = read_csmar_excel(data_dir / "AShare_Riskfree_Rate_Daily.xlsx")
    rf = rf_raw.copy
    if "Nrr1" in rf.columns:
        rf = rf[rf["Nrr1"].astype(str).str.upper.str.contains("NRI01", na=False)]
    rf = rf[["Clsdt", "Nrrdaydt"]].copy
    rf.columns = ["date", "rf_daily"]
    rf["date"] = pd.to_datetime(rf["date"], errors="coerce")
    rf["rf_daily"] = pd.to_numeric(rf["rf_daily"], errors="coerce") / 100.0
    rf = rf.dropna.drop_duplicates("date")

    out = (
        daily.merge(market, on="date", how="inner")
        .merge(rf, on="date", how="inner")
        .dropna(subset=["ret", "ret_m", "rf_daily"])
    )
    out["ret_excess"] = out["ret"] - out["rf_daily"]
    out["mkt_excess"] = out["ret_m"] - out["rf_daily"]
    return out[["permno", "date", "ret_excess", "mkt_excess"]].reset_index(drop=True)

cache_daily = CACHE_DIR / "crsp_daily_ashare.parquet"
# Data definition now uses ready-made daily market/RF; drop stale caches
for _stale in [
    cache_daily,
    CACHE_DIR / "capm_daily_ashare.parquet",
]:
    if _stale.exists:
        _stale.unlink
        print("removed old cache:", _stale)

crsp_daily = load_ashare_daily_panel(DATA_DIR)
crsp_daily.to_parquet(cache_daily, index=False)
print("saved cache:", cache_daily)

print(crsp_daily.shape)
print(
    "daily range:",
    crsp_daily["date"].min.date,
    "->",
    crsp_daily["date"].max.date,
    "stocks:",
    crsp_daily["permno"].nunique,
)
display(crsp_daily.head)

removed old cache: D:\A_Topics\202607_03_TidyFinanceAShare\202608_02_传统量化\Data\_cache_beta\crsp_daily_ashare.parquet
removed old cache: D:\A_Topics\202607_03_TidyFinanceAShare\202608_02_传统量化\Data\_cache_beta\capm_daily_ashare.parquet
read AShare_Stocks_Return_Daily_1.xlsx rows 1000000
read AShare_Stocks_Return_Daily_2.xlsx rows 1000000
read AShare_Stocks_Return_Daily_3.xlsx rows 1000000
read AShare_Stocks_Return_Daily_4.xlsx rows 1000000
read AShare_Stocks_Return_Daily_5.xlsx rows 1000000
read AShare_Stocks_Return_Daily_6.xlsx rows 1000000
read AShare_Stocks_Return_Daily_7.xlsx rows 370309
saved cache: D:\A_Topics\202607_03_TidyFinanceAShare\202608_02_传统量化\Data\_cache_beta\crsp_daily_ashare.parquet
(6167307, 4)
daily range: 2021-08-05 -> 2026-08-04 stocks: 5782

	permno	date	ret_excess	mkt_excess
0	000001	2021-08-05	-0.007340	-0.005102
1	000001	2021-08-06	0.010140	-0.001528
2	000001	2021-08-09	0.067148	0.009096
3	000001	2021-08-10	0.035111	0.011574
4	000001	2021-08-11	0.004014	0.001276

# Truncate daily dates to month-start so the rolling window stays month-aligned.
# Note: look_back=12 means 12 months; min_obs=200 means at least 200 trading days in-window,
# filtered on aggregated n inside roll_capm_estimation—not pandas rolling min_periods.
crsp_daily_month = crsp_daily.copy
crsp_daily_month["date"] = pd.to_datetime(
    crsp_daily_month["date"], errors="coerce"
).dt.to_period("M").dt.to_timestamp
crsp_daily_month = crsp_daily_month.dropna(subset=["date", "ret_excess", "mkt_excess"])

cache_capm_d = CACHE_DIR / "capm_daily_ashare.parquet"
# Recompute automatically if a prior bad parameter run left an empty cache
need_recompute = True
if cache_capm_d.exists:
    capm_daily = pd.read_parquet(cache_capm_d)
    if len(capm_daily) > 0:
        need_recompute = False
        print("loaded cache:", cache_capm_d)
    else:
        print("empty cache detected, recomputing...")

if need_recompute:
    capm_daily = roll_capm_estimation(
        crsp_daily_month,
        look_back=12,   # local daily history is short; use 12 months
        min_obs=200,    # minimum trading days inside the window
    )
    capm_daily.to_parquet(cache_capm_d, index=False)
    print("saved cache:", cache_capm_d)

print(capm_daily.shape)
display(capm_daily.head)

beta_daily = (
    capm_daily[capm_daily["coefficient"] == "mkt_excess"]
    [["permno", "date", "estimate"]]
    .rename(columns={"estimate": "beta"})
    .assign(return_type="daily")
)
beta = pd.concat([beta_monthly, beta_daily], ignore_index=True)
print("beta stacked:", beta.shape, beta["return_type"].value_counts.to_dict)

saved cache: D:\A_Topics\202607_03_TidyFinanceAShare\202608_02_传统量化\Data\_cache_beta\capm_daily_ashare.parquet
(511308, 5)

	permno	date	coefficient	estimate	t_statistic
0	000001	2022-06-01	alpha	-0.000465	-0.358986
1	000001	2022-06-01	mkt_excess	0.966083	8.728630
2	000001	2022-07-01	alpha	-0.000928	-0.765064
3	000001	2022-07-01	mkt_excess	0.953419	8.980394
4	000001	2022-08-01	alpha	-0.000816	-0.701891

beta stacked: (878910, 4) {'monthly': 623256, 'daily': 255654}

Figure 4: Daily vs Monthly Betas for Example Stocks

Daily estimates are usually smoother (more effective observations) but should track monthly trends. Given the later daily start, compare mainly on the overlapping window.

# For a fair comparison, re-estimate monthly with a 12-month window on the overlap (Figure 4 only)
capm_examples_m12 = roll_capm_estimation(
    crsp_monthly[crsp_monthly["permno"].isin(examples["permno"])],
    look_back=12,
    min_obs=10,
)
beta_m12 = (
    capm_examples_m12[capm_examples_m12["coefficient"] == "mkt_excess"]
    [["permno", "date", "estimate"]]
    .rename(columns={"estimate": "beta"})
    .assign(return_type="monthly")
)
beta_d12 = beta_daily[beta_daily["permno"].isin(examples["permno"])].copy
beta_comparison = pd.concat([beta_m12, beta_d12], ignore_index=True).merge(
    examples, on="permno", how="inner"
)

companies = examples["company"].tolist
fig, axes = plt.subplots(len(companies), 1, figsize=(10, 8), sharex=True)
if len(companies) == 1:
    axes = [axes]
for ax, company in zip(axes, companies):
    g = beta_comparison[beta_comparison["company"] == company]
    for rt, gg in g.groupby("return_type"):
        ax.plot(gg["date"], gg["beta"], label=rt, linewidth=1.4)
    ax.set_ylabel("beta")
    ax.set_title(company)
    ax.grid(True, alpha=0.3)
    ax.legend(frameon=False, loc="best")
fig.suptitle("Daily vs Monthly Beta (Both 12-Month Windows)", y=1.01)
plt.tight_layout
plt.show

1.6.10 Coverage and Summary Statistics

Two sanity checks:

1. How many stocks get a beta each month (coverage);
2. Distribution summaries for daily vs monthly betas.

Large coverage gaps usually point to alignment or window-parameter problems.

# Coverage: monthly stock-month universe as denominator
universe = crsp_monthly[["permno", "date"]].drop_duplicates
coverage_rows = []
for rt, b in [("monthly", beta_monthly), ("daily", beta_daily)]:
    tmp = universe.merge(
        b[["permno", "date", "beta"]],
        on=["permno", "date"],
        how="left",
    )
    cov = tmp.groupby("date")["beta"].apply(lambda s: s.notna.mean).reset_index
    cov.columns = ["date", "share"]
    cov["return_type"] = rt
    coverage_rows.append(cov)
beta_coverage = pd.concat(coverage_rows, ignore_index=True)

fig, ax = plt.subplots(figsize=(10, 4))
for rt, g in beta_coverage.groupby("return_type"):
    ax.plot(g["date"], g["share"], label=rt, linewidth=1.5)
ax.set_ylim(0, 1)
ax.set_title("Month-End Share of Securities with a Beta Estimate")
ax.set_ylabel("share")
ax.legend(frameon=False)
ax.grid(True, alpha=0.3)
plt.tight_layout
plt.show

summary = (
    beta.groupby("return_type")["beta"]
    .agg(
        count="count",
        mean="mean",
        std="std",
        min="min",
        q05=lambda s: s.quantile(0.05),
        q50="median",
        q95=lambda s: s.quantile(0.95),
        max="max",
    )
    .round(3)
)
display(summary)

# Persist for later chapters (portfolio sorts, etc.)
out_path = DATA_DIR / "beta_ashare.parquet"
beta.to_parquet(out_path, index=False)
print("wrote", out_path)

	count	mean	std	min	q05	q50	q95	max
return_type
daily	255654	1.228	0.451	-18.801	0.564	1.192	1.992	37.881
monthly	623256	1.118	0.377	-7.364	0.585	1.099	1.707	12.785

wrote D:\A_Topics\202607_03_TidyFinanceAShare\202608_02_传统量化\Data\beta_ashare.parquet

1.6.11 Key Takeaways

1. Market beta is the slope from regressing stock excess returns on market excess returns; rolling windows capture time-varying risk exposure.
2. Closed-form rolling estimation turns “millions of regressions” into rolling sums—what makes a full-sample run feasible.
3. Long monthly windows suit medium-/long-horizon exposure; daily estimates are often smoother when samples are rich, but are more sensitive to history length and microstructure noise.
4. For A-shares, watch market-portfolio definition (monthly uses type 53 here), RF frequency conversion, and ST/suspension filters (daily uses Trdsta).
5. The output beta_ashare.parquet feeds later one-way sorts and low-beta anomaly tests (see Chapter 6 on layered backtests and univariate sorts).