diff --git a/README.md b/README.md index f8bddbad..9e7adff2 100644 --- a/README.md +++ b/README.md @@ -122,7 +122,6 @@ print("forecast:", forecast) | Causation | `nns_causation`, `causal_matrix` | | Regression and classification | `nns_reg`, `nns_m_reg`, `nns_stack`, `nns_boost` | | Forecasting | `nns_seas`, `nns_arma`, `nns_arma_optim`, `nns_var` | -| Nowcast panels | `nns_nowcast_panel`, `CsvNowcastProvider` | | Distribution tools | `nns_cdf`, `nns_anova`, `nns_norm` | | Stochastic dominance | `fsd`, `ssd`, `tsd`, `nns_sd_cluster`, `sd_efficient_set` | | Stochastic superiority and simulation | `nns_ss`, `nns_mc`, `nns_meboot` | @@ -157,7 +156,6 @@ Runnable examples live in [`docs/examples`](docs/examples): | Regression | [`regression.py`](docs/examples/regression.py) | | Classification | [`classification.py`](docs/examples/classification.py) | | Forecasting | [`forecasting.py`](docs/examples/forecasting.py) | -| Nowcast panel | [`nowcast_panel.py`](docs/examples/nowcast_panel.py) | Run one example: @@ -179,7 +177,6 @@ Notebook workflows are also available under [`docs/examples/notebooks`](docs/exa - [Behavior conventions and intentional divergences](docs/conventions.md) - [Benchmarks](docs/benchmarks.md) - [Examples](docs/examples/README.md) -- [Nowcast design](docs/specs_nowcast.md) ## Development diff --git a/docs/api_status.md b/docs/api_status.md index aefd9eef..6f316217 100644 --- a/docs/api_status.md +++ b/docs/api_status.md @@ -54,8 +54,6 @@ invariant, and property coverage. | Boost: `nns_boost` | partial | medium | Deterministic and stochastic structures are implemented; one high-feature threshold path remains guarded to match installed-R failure behavior. | | Seasonality: `nns_seas` | implemented | high | Non-plotting installed-R path is implemented and cached defensively. | | ARMA and VAR: `nns_arma`, `nns_arma_optim`, `nns_var` | partial | medium | Numeric forecasting and supported VAR dimension-reduction paths are implemented on focused fixtures. Explicit numeric multi-lag ARMA uses actual-lag weighting instead of installed R's position-based weighting quirk. VAR's multivariate stack stage matches R's effective time-series holdout sizing; the remaining macro-like VAR strict xfail is inherited from ARMA optimizer period selection. Stochastic interval streams are structural/statistical parity only. | -| Nowcast panel: `nns_nowcast_panel` | implemented | medium | Python-native deterministic monthly panel helper backed by `nns_var`. R NNS 13.0 does not export `NNS.nowcast`, so this is no longer an R-export parity target. | -| Providers: `CsvNowcastProvider` | implemented | medium | Produces explicit local/offline payloads for `nns_nowcast_panel`. | | Bootstrap/Monte Carlo: `nns_meboot`, `nns_mc` | implemented | medium | Deterministic diagnostics are parity-tested; exact stochastic replicate parity with R is not expected. | | Stochastic dominance/superiority: `fsd`, `ssd`, `tsd`, `.uni` wrappers, `nns_ss`, `nns_sd_cluster`, `sd_efficient_set` | implemented | medium | Public structures and deterministic paths are covered. SD uses exact pure-NumPy prefix-pair kernels plus a degree-1 discrete order-statistic matrix path; R's C++ core remains faster on full finance fixtures. Stochastic intervals use NNS Python RNG. | | ANOVA: `nns_anova` | implemented | high | Binary, multi-group, pairwise, and degenerate `NaN` conventions are covered. | @@ -75,10 +73,6 @@ invariant, and property coverage. ## Intentional Design Boundaries - No hidden network fetching happens by default. -- NNS Python does not export `nns_nowcast`; R NNS 13.0 does not export `NNS.nowcast`. -- Nowcast providers are payload builders for `nns_nowcast_panel`, not implicit - public forecast wrappers. -- `CsvNowcastProvider` is local/offline. - Library code does not auto-load `.env` files. - External data clients and dataframe libraries are not dependencies. - NNS Python uses explicit Python errors for some cases where R silently truncates, @@ -94,23 +88,6 @@ invariant, and property coverage. degree-1 discrete calls. Optional compiled SD backends remain deferred until benchmark evidence justifies the added packaging and maintenance cost. -## Provider Boundary - -Nowcast provider support is explicit. Providers return payloads; callers pass -the payload to `nns_nowcast_panel`: - -```python -from nns import nns_nowcast_panel -from nns.providers import CsvNowcastProvider - -provider = CsvNowcastProvider("monthly_panel.csv") -payload = provider.fetch((), "2000-01-03") -result = nns_nowcast_panel(payload["series"], h=2, tau=12, dates=payload["dates"]) -``` - -NNS Python does not ship a default Yahoo, FRED, or other live-data workflow hidden -behind a public nowcast wrapper. - ## Intentional Divergences And Caveats The detailed behavior notes live in `docs/conventions.md`. Release-relevant diff --git a/docs/conventions.md b/docs/conventions.md index 5833a0e4..614ff772 100644 --- a/docs/conventions.md +++ b/docs/conventions.md @@ -404,20 +404,6 @@ logic, and R-style relevance extraction. The function returns `multivariate` and `relevant_variables` in the same shape/naming pattern expected by `NNS.VAR`. -`nns_nowcast_panel` is the deterministic nowcast core for user-supplied monthly -numeric panels. It accepts array-like panels or ordered mappings of column names -to numeric series, delegates numeric forecasting to `nns_var`, and returns VAR -fields plus `dates` and `metadata` dictionaries. Date labels are metadata rather -than array indices. Without dates, forecast rows are labeled `t+1`, `t+2`, ... -With dates, inputs are normalized to `YYYY-MM`, must be sorted and unique, and -forecast labels advance monthly. R NNS 13.0 does not export `NNS.nowcast`, so NNS Python -does not export a public `nns_nowcast` wrapper. `CsvNowcastProvider` remains an -explicit payload builder whose `fetch(series, start_date)` method returns -`{"series": ..., "dates": ..., "metadata": ...}` for callers to pass to -`nns_nowcast_panel`. `CsvNowcastProvider` is offline and local-file only. -Library code does not read `.env` files. NNS Python does not ship an implicit -FRED/Yahoo provider. - ## Meboot `nns_meboot` maps to R's `NNS.meboot` maximum-entropy bootstrap algorithm and diff --git a/docs/examples/README.md b/docs/examples/README.md index 35162fc5..a5fa4854 100644 --- a/docs/examples/README.md +++ b/docs/examples/README.md @@ -25,7 +25,6 @@ you want short Python call patterns that are kept in sync with NNS Python. | Regression | [regression.py](regression.py) | `nns_reg`, fitted values, point estimates, regression output shape | `NNSvignette_Clustering_and_Regression.Rmd` | | Classification | [classification.py](classification.py) | `nns_reg(..., type="class")`, numeric class-code predictions | `NNSvignette_Classification.Rmd` | | Forecasting | [forecasting.py](forecasting.py) | `nns_arma`, `nns_arma_optim`, `nns_var` | `NNSvignette_Forecasting.Rmd` | -| Nowcast panel | [nowcast_panel.py](nowcast_panel.py) | deterministic user-supplied panel, date metadata, VAR-backed forecast output | `NNS.VAR` nowcast/frequency-alignment material | ## Notebooks @@ -33,7 +32,6 @@ you want short Python call patterns that are kept in sync with NNS Python. |---|---| | Partial-moment risk workflow | [01_partial_moments_risk_workflow.ipynb](notebooks/01_partial_moments_risk_workflow.ipynb) | | Regression, classification, factors | [02_regression_classification_workflow.ipynb](notebooks/02_regression_classification_workflow.ipynb) | -| Forecasting and local nowcast panel | [03_forecasting_nowcast_workflow.ipynb](notebooks/03_forecasting_nowcast_workflow.ipynb) | | Distribution, dominance, simulation | [04_distribution_dominance_simulation_workflow.ipynb](notebooks/04_distribution_dominance_simulation_workflow.ipynb) | | Boston Housing regression parity example | [05_boston_housing_regression_workflow.ipynb](notebooks/05_boston_housing_regression_workflow.ipynb) | diff --git a/docs/examples/notebooks/03_forecasting_nowcast_workflow.ipynb b/docs/examples/notebooks/03_forecasting_nowcast_workflow.ipynb deleted file mode 100644 index 54e5595b..00000000 --- a/docs/examples/notebooks/03_forecasting_nowcast_workflow.ipynb +++ /dev/null @@ -1,273 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Forecasting and Nowcast Workflow\n" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [], - "source": [ - "from pathlib import Path\n", - "from tempfile import TemporaryDirectory\n", - "\n", - "import numpy as np\n", - "\n", - "from nns import nns_arma, nns_arma_optim, nns_nowcast_panel, nns_seas, nns_var\n", - "from nns.providers import CsvNowcastProvider\n", - "\n", - "np.set_printoptions(precision=4, suppress=True)\n", - "rng = np.random.default_rng(21)\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Monthly series\n" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "panel shape: (48, 3)\n", - "last observed rows:\n", - "[[166.973 116.1881 66.6082]\n", - " [169.8529 121.9687 70.8091]\n", - " [177.4491 124.202 74.7791]]\n" - ] - } - ], - "source": [ - "t = np.arange(1, 49, dtype=np.float64)\n", - "dates = [f\"2020-{month:02d}\" for month in range(1, 13)] + [f\"2021-{month:02d}\" for month in range(1, 13)] + [f\"2022-{month:02d}\" for month in range(1, 13)] + [f\"2023-{month:02d}\" for month in range(1, 13)]\n", - "revenue = 120.0 + 1.2 * t + 9.0 * np.sin(2.0 * np.pi * t / 12.0) + rng.normal(0.0, 1.5, t.size)\n", - "orders = 85.0 + 0.7 * t + 6.0 * np.sin(2.0 * np.pi * (t + 2.0) / 12.0) + rng.normal(0.0, 1.2, t.size)\n", - "activity = 50.0 + 0.4 * t + 5.0 * np.cos(2.0 * np.pi * t / 6.0) + rng.normal(0.0, 1.0, t.size)\n", - "panel = np.column_stack((revenue, orders, activity))\n", - "panel_with_missing = panel.copy()\n", - "panel_with_missing[10, 1] = np.nan\n", - "panel_with_missing[27, 2] = np.nan\n", - "\n", - "print(\"panel shape:\", panel_with_missing.shape)\n", - "print(\"last observed rows:\")\n", - "print(panel_with_missing[-3:])\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Seasonality and ARMA\n" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "candidate periods: [ 3 6 9 12 15 18]\n", - "best period: 3\n", - "ARMA lin forecast: [183.51 186.32 191.99 189.21]\n", - "ARMA both forecast: [184.24 185.83 190.68 188.25]\n", - "optimized periods/method: [12] lin\n", - "optimized forecast: [183.22 186.03 191.7 188.92]\n" - ] - } - ], - "source": [ - "seas = nns_seas(revenue, modulo=[3, 6, 12], mod_only=True)\n", - "arma_lin = nns_arma(revenue, h=4, seasonal_factor=12, method=\"lin\")\n", - "arma_both = nns_arma(revenue, h=4, seasonal_factor=12, method=\"both\")\n", - "optim = nns_arma_optim(\n", - " revenue,\n", - " h=4,\n", - " seasonal_factor=[6, 12],\n", - " lin_only=True,\n", - " print_trace=False,\n", - ")\n", - "\n", - "print(\"candidate periods:\", seas[\"periods\"])\n", - "print(\"best period:\", seas[\"best.period\"])\n", - "print(\"ARMA lin forecast:\", np.round(arma_lin, 2))\n", - "print(\"ARMA both forecast:\", np.round(arma_both, 2))\n", - "print(\"optimized periods/method:\", optim[\"periods\"], optim[\"method\"])\n", - "print(\"optimized forecast:\", np.round(optim[\"results\"], 2))\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## VAR forecast\n" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "interpolated missing rows:\n", - "[[129.5761 96.1406 55.516 ]\n", - " [163.7713 104.8407 60.3538]]\n", - "univariate forecast:\n", - "[[183.22 118.2 72.9 ]\n", - " [186.03 119.87 67.23]\n", - " [191.7 119.54 64.63]]\n", - "multivariate forecast:\n", - "[[173.02 113.83 65.31]\n", - " [174.92 114.22 65.66]\n", - " [177.45 124.2 74.78]]\n", - "ensemble forecast:\n", - "[[175.06 114.7 66.83]\n", - " [177.14 115.35 65.97]\n", - " [180.3 123.27 72.75]]\n", - "relevant variables:\n", - "[['x2_tau_0' 'x1_tau_0' 'x1_tau_0']\n", - " ['x3_tau_0' 'x3_tau_0' 'x2_tau_0']\n", - " ['x1_tau_1' 'x1_tau_1' 'x1_tau_1']\n", - " ['x2_tau_2' 'x2_tau_2' 'x2_tau_2']\n", - " ['x3_tau_3' 'x3_tau_3' 'x3_tau_3']]\n" - ] - } - ], - "source": [ - "var = nns_var(\n", - " panel_with_missing,\n", - " h=3,\n", - " tau=[1, 2, 3],\n", - " dim_red_method=\"cor\",\n", - " naive_weights=False,\n", - " status=False,\n", - ")\n", - "print(\"interpolated missing rows:\")\n", - "print(var[\"interpolated_and_extrapolated\"][[10, 27]])\n", - "print(\"univariate forecast:\")\n", - "print(np.round(var[\"univariate\"], 2))\n", - "print(\"multivariate forecast:\")\n", - "print(np.round(var[\"multivariate\"], 2))\n", - "print(\"ensemble forecast:\")\n", - "print(np.round(var[\"ensemble\"], 2))\n", - "print(\"relevant variables:\")\n", - "print(var[\"relevant_variables\"])\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": "## Local nowcast panel\nR NNS 13.0 does not export `NNS.nowcast`; NNS Python keeps the local panel workflow.\n" - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "names: ['revenue', 'orders', 'activity']\n", - "forecast dates: ['2024-01', '2024-02']\n", - "ensemble forecast:\n", - "[[173.35 120.88 73.69]\n", - " [173.47 123.53 71.78]]\n", - "metadata: {'source': 'user_panel', 'freq': 'monthly', 'tau': 12, 'dim_red_method': 'cor', 'naive_weights': False}\n" - ] - } - ], - "source": [ - "monthly_payload = {\n", - " \"revenue\": revenue[-24:].copy(),\n", - " \"orders\": orders[-24:].copy(),\n", - " \"activity\": activity[-24:].copy(),\n", - "}\n", - "nowcast = nns_nowcast_panel(\n", - " monthly_payload,\n", - " h=2,\n", - " tau=12,\n", - " dates=dates[-24:],\n", - " dim_red_method=\"cor\",\n", - " naive_weights=False,\n", - ")\n", - "print(\"names:\", nowcast[\"names\"])\n", - "print(\"forecast dates:\", nowcast[\"dates\"][\"forecast\"])\n", - "print(\"ensemble forecast:\")\n", - "print(np.round(nowcast[\"ensemble\"], 2))\n", - "print(\"metadata:\", nowcast[\"metadata\"])\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## CSV provider\n" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "provider metadata: {'provider': 'csv', 'path': '', 'date_column': 'date', 'series_columns': ['revenue', 'orders', 'activity']}\n", - "provider dates: ['2022-07', '2022-08', '2022-09'] ... ['2023-10', '2023-11', '2023-12']\n", - "one-step CSV nowcast: [[172.2 122.28 68.33]]\n" - ] - } - ], - "source": [ - "with TemporaryDirectory() as tmp:\n", - " path = Path(tmp) / \"monthly_panel.csv\"\n", - " rows = [\"date,revenue,orders,activity\"]\n", - " for date, row in zip(dates[-18:], panel[-18:], strict=True):\n", - " rows.append(f\"{date},{row[0]:.6f},{row[1]:.6f},{row[2]:.6f}\")\n", - " path.write_text(\"\\n\".join(rows) + \"\\n\", encoding=\"utf-8\")\n", - "\n", - " provider = CsvNowcastProvider(path)\n", - " payload = provider.fetch((), dates[-18])\n", - " csv_result = nns_nowcast_panel(payload[\"series\"], h=1, tau=6, dates=payload[\"dates\"])\n", - "\n", - "metadata = dict(payload[\"metadata\"])\n", - "metadata[\"path\"] = \"\"\n", - "print(\"provider metadata:\", metadata)\n", - "print(\"provider dates:\", payload[\"dates\"][:3], \"...\", payload[\"dates\"][-3:])\n", - "print(\"one-step CSV nowcast:\", np.round(csv_result[\"ensemble\"], 2))\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "name": "python", - "pygments_lexer": "ipython3" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} \ No newline at end of file diff --git a/docs/examples/nowcast_panel.py b/docs/examples/nowcast_panel.py deleted file mode 100644 index d33b53be..00000000 --- a/docs/examples/nowcast_panel.py +++ /dev/null @@ -1,69 +0,0 @@ -from __future__ import annotations - -from collections import OrderedDict -from tempfile import NamedTemporaryFile - -import numpy as np - -from nns import nns_nowcast_panel -from nns.providers import CsvNowcastProvider - - -def main() -> None: - t = np.arange(1, 25, dtype=np.float64) - panel = OrderedDict( - ( - ("employment", 100.0 + 0.3 * t + np.sin(t / 4.0)), - ("inflation", 3.0 + 0.05 * np.cos(t / 3.0)), - ("production", 80.0 + 0.5 * t + np.cos(t / 5.0)), - ) - ) - dates = [f"2024-{month:02d}" for month in range(1, 13)] + [ - f"2025-{month:02d}" for month in range(1, 13) - ] - - result = nns_nowcast_panel(panel, h=2, tau=2, dates=dates) - matrix_result = nns_nowcast_panel( - np.column_stack(tuple(panel.values())), - h=1, - tau=[1, 2, 2], - names=list(panel), - naive_weights=True, - ) - - with NamedTemporaryFile("w", suffix=".csv", delete=True) as handle: - handle.write("date,employment,inflation,production\n") - for row, month in enumerate(dates): - handle.write( - f"{month},{panel['employment'][row]}," - f"{panel['inflation'][row]},{panel['production'][row]}\n" - ) - handle.flush() - payload = CsvNowcastProvider(handle.name).fetch((), "2024-01") - provider_result = nns_nowcast_panel( - payload["series"], - h=1, - naive_weights=True, - tau=12, - dates=payload["dates"], - ) - - assert result["names"] == list(panel) - assert result["ensemble"].shape == (2, 3) - assert result["dates"]["forecast"] == ["2026-01", "2026-02"] - assert matrix_result["names"] == list(panel) - assert matrix_result["dates"]["forecast"] == ["t+1"] - assert provider_result["ensemble"].shape == (1, 3) - - print("series:", result["names"]) - print("forecast dates:", result["dates"]["forecast"]) - print("ensemble forecast:") - print(result["ensemble"]) - print("matrix-input next-step forecast:") - print(matrix_result["ensemble"]) - print("csv-provider forecast:") - print(provider_result["ensemble"]) - - -if __name__ == "__main__": - main() diff --git a/pyproject.toml b/pyproject.toml index 27ab04c3..4c0f0a07 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "ovvo-nns" -version = "1.0.1" +version = "1.0.2" description = "Python port of nonlinear nonparametric statistics from R NNS" readme = "README.md" requires-python = ">=3.11" diff --git a/src/nns/__init__.py b/src/nns/__init__.py index 4d561bb5..711284f1 100644 --- a/src/nns/__init__.py +++ b/src/nns/__init__.py @@ -4,7 +4,7 @@ from nns.pm_matrix import pm_matrix as pm_matrix -__version__ = "1.0.1" +__version__ = "1.0.2" _EXPORTS = { "FactorDesign": ("nns.regression", "FactorDesign"), @@ -48,7 +48,6 @@ "nns_mc": ("nns.mc", "nns_mc"), "nns_meboot": ("nns.meboot", "nns_meboot"), "nns_norm": ("nns.norm", "nns_norm"), - "nns_nowcast_panel": ("nns.nowcast", "nns_nowcast_panel"), "nns_part": ("nns.part", "nns_part"), "nns_reg": ("nns.regression", "nns_reg"), "nns_rescale": ("nns.central_tendencies", "nns_rescale"), diff --git a/src/nns/nowcast.py b/src/nns/nowcast.py deleted file mode 100644 index b584feb6..00000000 --- a/src/nns/nowcast.py +++ /dev/null @@ -1,193 +0,0 @@ -from __future__ import annotations - -from collections.abc import Mapping, Sequence -from datetime import date, datetime - -import numpy as np -from numpy.typing import NDArray - -from nns.var import nns_var - -_DEFAULT_NOWCAST_SERIES = ( - "PAYEMS", - "JTSJOL", - "CPIAUCSL", - "DGORDER", - "RSAFS", - "UNRATE", - "HOUST", - "INDPRO", - "DSPIC96", - "BOPTEXP", - "BOPTIMP", - "TTLCONS", - "IR", - "CPILFESL", - "PCEPILFE", - "PCEPI", - "PERMIT", - "TCU", - "BUSINV", - "ULCNFB", - "IQ", - "GACDISA066MSFRBNY", - "GACDFSA066MSFRBPHI", - "PCEC96", - "GDPC1", - "ICSA", - "DGS10", - "T10Y2Y", - "WALCL", - "PALLFNFINDEXM", - "FEDFUNDS", - "PPIACO", - "CIVPART", - "M2NS", - "ADPMNUSNERNSA", -) - - -def nns_nowcast_panel( - panel: object, - *, - h: int = 0, - tau: int | list[int] | list[list[int]] = 12, - dim_red_method: str = "cor", - naive_weights: bool = False, - dates: Sequence[object] | None = None, - names: Sequence[str] | None = None, -) -> dict[str, object]: - """Deterministic nowcast core for user-supplied monthly panels.""" - if h < 0: - raise ValueError("h must be non-negative.") - - matrix, panel_names = _panel_matrix_and_names(panel, names) - observed_dates, forecast_dates = _normalize_nowcast_dates(dates, matrix.shape[0], h) - - result = nns_var( - matrix, - h, - tau=tau, - dim_red_method=dim_red_method, - naive_weights=naive_weights, - ) - output: dict[str, object] = dict(result) - output["names"] = panel_names - if "relevant_variables" in output: - output["relevant_variables"] = _rename_relevant_variables( - output["relevant_variables"], - panel_names, - ) - output["dates"] = { - "observed": observed_dates, - "forecast": forecast_dates, - "interpolated_and_extrapolated": observed_dates, - } - output["metadata"] = { - "source": "user_panel", - "freq": "monthly", - "tau": tau, - "dim_red_method": dim_red_method, - "naive_weights": naive_weights, - } - return output - - -def _panel_matrix_and_names( - panel: object, - names: Sequence[str] | None, -) -> tuple[NDArray[np.float64], list[str]]: - if isinstance(panel, Mapping): - if names is not None: - raise ValueError("names cannot be provided when panel is a mapping.") - panel_names = [str(key) for key in panel] - columns = [np.asarray(values, dtype=np.float64).reshape(-1) for values in panel.values()] - if not columns: - raise ValueError("panel must contain at least one column.") - row_count = columns[0].size - if any(column.size != row_count for column in columns): - raise ValueError("mapping panel columns must have equal lengths.") - matrix = np.column_stack(columns) - else: - matrix = np.asarray(panel, dtype=np.float64) - if matrix.ndim != 2: - raise ValueError("panel must be a 2-D numeric matrix or an ordered mapping of columns.") - panel_names = [f"x{i + 1}" for i in range(matrix.shape[1])] - - if matrix.ndim != 2: - raise ValueError("panel must be a 2-D numeric matrix.") - if matrix.shape[0] == 0 or matrix.shape[1] == 0: - raise ValueError("panel must be non-empty.") - - if names is not None: - if len(names) != matrix.shape[1]: - raise ValueError("names length must match panel column count.") - panel_names = [str(name) for name in names] - - return matrix.astype(np.float64, copy=False), panel_names - - -def _normalize_nowcast_dates( - dates: Sequence[object] | None, - row_count: int, - h: int, -) -> tuple[list[str] | None, list[str]]: - if dates is None: - return None, [f"t+{step}" for step in range(1, h + 1)] - if len(dates) != row_count: - raise ValueError("dates length must match panel row count.") - - observed = [_normalize_month_label(value) for value in dates] - if len(set(observed)) != len(observed): - raise ValueError("dates must not contain duplicate months.") - if observed != sorted(observed): - raise ValueError("dates must be sorted in ascending monthly order.") - return observed, _forecast_month_labels(observed[-1], h) - - -def _normalize_month_label(value: object) -> str: - if isinstance(value, np.datetime64): - return str(value.astype("datetime64[M]")) - if isinstance(value, datetime | date): - return f"{value.year:04d}-{value.month:02d}" - text = str(value) - try: - parsed = datetime.fromisoformat(text) - return f"{parsed.year:04d}-{parsed.month:02d}" - except ValueError: - pass - try: - parsed_month = np.datetime64(text, "M") - except ValueError as exc: - raise ValueError("dates must be parseable as monthly date labels.") from exc - return str(parsed_month) - - -def _forecast_month_labels(last_observed: str, h: int) -> list[str]: - year_text, month_text = last_observed.split("-") - year = int(year_text) - month = int(month_text) - labels: list[str] = [] - for _ in range(h): - month += 1 - if month > 12: - year += 1 - month = 1 - labels.append(f"{year:04d}-{month:02d}") - return labels - - -def _rename_relevant_variables(values: object, names: Sequence[str]) -> object: - mapping = {f"x{i + 1}": name for i, name in enumerate(names)} - array = np.asarray(values, dtype=object).copy() - for index, item in np.ndenumerate(array): - if item is None: - continue - text = str(item) - for old, new in mapping.items(): - if text == old: - text = new - elif text.startswith(f"{old}_tau_"): - text = f"{new}{text[len(old) :]}" - array[index] = text - return array diff --git a/src/nns/providers/__init__.py b/src/nns/providers/__init__.py deleted file mode 100644 index 1f519dc0..00000000 --- a/src/nns/providers/__init__.py +++ /dev/null @@ -1,5 +0,0 @@ -from __future__ import annotations - -from nns.providers.nowcast import CsvNowcastProvider - -__all__ = ["CsvNowcastProvider"] diff --git a/src/nns/providers/nowcast.py b/src/nns/providers/nowcast.py deleted file mode 100644 index aafb556c..00000000 --- a/src/nns/providers/nowcast.py +++ /dev/null @@ -1,121 +0,0 @@ -from __future__ import annotations - -import csv -from collections import OrderedDict -from collections.abc import Mapping, Sequence -from pathlib import Path -from typing import Any, cast - -from nns.nowcast import _normalize_month_label - - -class CsvNowcastProvider: - """Local CSV provider for deterministic nowcast panels.""" - - def __init__( - self, - path: str | Path, - *, - date_column: str = "date", - series_columns: Sequence[str] | None = None, - ) -> None: - self.path = Path(path) - self.date_column = date_column - self.series_columns = ( - None if series_columns is None else [str(name) for name in series_columns] - ) - - def fetch(self, series: Sequence[str], start_date: str) -> dict[str, object]: - del series - rows, fieldnames = self._read_rows() - selected_columns = self._selected_columns(fieldnames) - dates, values = self._parse_rows(rows, selected_columns, start_date) - return { - "dates": dates, - "series": values, - "metadata": { - "provider": "csv", - "path": str(self.path), - "date_column": self.date_column, - "series_columns": selected_columns, - }, - } - - def _read_rows(self) -> tuple[list[Mapping[str, str]], list[str]]: - if not self.path.exists(): - raise FileNotFoundError(f"CSV nowcast provider file does not exist: {self.path}") - with self.path.open(newline="", encoding="utf-8") as handle: - reader = csv.DictReader(handle) - if reader.fieldnames is None: - raise ValueError("CSV nowcast provider file is empty.") - fieldnames = [str(name) for name in reader.fieldnames] - rows = cast(list[Mapping[str, str]], list(reader)) - if not rows: - raise ValueError("CSV nowcast provider file has no data rows.") - if self.date_column not in fieldnames: - raise ValueError(f"CSV nowcast provider missing date column: {self.date_column}") - return rows, fieldnames - - def _selected_columns(self, fieldnames: Sequence[str]) -> list[str]: - if self.series_columns is None: - selected = [name for name in fieldnames if name != self.date_column] - else: - selected = list(self.series_columns) - missing = [name for name in selected if name not in fieldnames] - if missing: - raise ValueError( - f"CSV nowcast provider missing selected series column: {missing[0]}" - ) - if not selected: - raise ValueError("CSV nowcast provider requires at least one usable series column.") - return selected - - def _parse_rows( - self, - rows: Sequence[Mapping[str, str]], - selected_columns: Sequence[str], - start_date: str, - ) -> tuple[list[str], OrderedDict[str, list[float | None]]]: - start_month = _normalize_month_label(start_date) - dates: list[str] = [] - values: OrderedDict[str, list[float | None]] = OrderedDict( - (name, []) for name in selected_columns - ) - for row_number, row in enumerate(rows, start=2): - raw_date = row.get(self.date_column) - if raw_date is None: - raise ValueError(f"CSV nowcast provider row {row_number} is missing a date value.") - month = _normalize_month_label(raw_date) - if month < start_month: - continue - dates.append(month) - for column in selected_columns: - values[column].append(_parse_optional_float(row.get(column), column, row_number)) - - if not dates: - raise ValueError("CSV nowcast provider has no rows on or after start_date.") - if len(set(dates)) != len(dates): - raise ValueError("CSV nowcast provider dates must not contain duplicate months.") - if dates != sorted(dates): - raise ValueError( - "CSV nowcast provider dates must be sorted in ascending monthly order." - ) - lengths = {len(column_values) for column_values in values.values()} - if lengths != {len(dates)}: - raise ValueError("CSV nowcast provider series columns must have equal lengths.") - return dates, values - - -def _parse_optional_float(value: Any, column: str, row_number: int) -> float | None: - if value is None: - return None - text = str(value).strip() - if text == "" or text.lower() in {"na", "nan", "null", "none"}: - return None - try: - return float(text) - except ValueError as exc: - raise ValueError( - f"CSV nowcast provider column {column!r} row {row_number} " - f"contains a nonnumeric value: {value!r}" - ) from exc diff --git a/tests/invariants/test_export_surface.py b/tests/invariants/test_export_surface.py index 8c388266..0196e84b 100644 --- a/tests/invariants/test_export_surface.py +++ b/tests/invariants/test_export_surface.py @@ -1,11 +1,22 @@ from __future__ import annotations +import importlib + import pytest import nns +_REMOVED_NOWCAST_NAMES = ("nns_nowcast", "nns_nowcast_panel") + -def test_removed_r_nowcast_is_not_public() -> None: - assert "nns_nowcast" not in nns.__all__ +@pytest.mark.parametrize("name", _REMOVED_NOWCAST_NAMES) +def test_nowcast_names_are_not_public(name: str) -> None: + assert name not in nns.__all__ with pytest.raises(AttributeError): - nns.__getattr__("nns_nowcast") + nns.__getattr__(name) + + +def test_nowcast_modules_are_gone() -> None: + for module in ("nns.nowcast", "nns.providers", "nns.providers.nowcast"): + with pytest.raises(ModuleNotFoundError): + importlib.import_module(module) diff --git a/tests/invariants/test_nowcast.py b/tests/invariants/test_nowcast.py deleted file mode 100644 index bce2619e..00000000 --- a/tests/invariants/test_nowcast.py +++ /dev/null @@ -1,357 +0,0 @@ -from __future__ import annotations - -from collections import OrderedDict -from collections.abc import Mapping, Sequence -from typing import Any, cast - -import numpy as np -import pytest - -from nns import nns_nowcast_panel, nns_var -from nns.providers import CsvNowcastProvider - - -def _panel() -> np.ndarray: - idx = np.arange(1, 40, dtype=np.float64) - return np.column_stack( - ( - np.sin(idx / 3.0) + 2.0, - np.cos(idx / 5.0) + 3.0, - ) - ) - - -def test_nns_nowcast_panel_array_h0_matches_var_core() -> None: - panel = _panel() - - actual = nns_nowcast_panel(panel, h=0, tau=2) - expected = nns_var(panel, h=0, tau=2) - - assert set(actual) == { - "interpolated_and_extrapolated", - "names", - "dates", - "metadata", - } - np.testing.assert_allclose( - actual["interpolated_and_extrapolated"], - expected["interpolated_and_extrapolated"], - ) - assert actual["names"] == ["x1", "x2"] - assert actual["dates"] == { - "observed": None, - "forecast": [], - "interpolated_and_extrapolated": None, - } - assert actual["metadata"] == { - "source": "user_panel", - "freq": "monthly", - "tau": 2, - "dim_red_method": "cor", - "naive_weights": False, - } - - -def test_nns_nowcast_panel_array_h3_matches_var_core() -> None: - panel = _panel() - - actual = nns_nowcast_panel(panel, h=3, tau=2, dim_red_method="NNS.dep") - expected = nns_var(panel, h=3, tau=2, dim_red_method="NNS.dep", naive_weights=False) - - assert set(actual) == { - "interpolated_and_extrapolated", - "relevant_variables", - "univariate", - "multivariate", - "ensemble", - "names", - "dates", - "metadata", - } - for key in ("interpolated_and_extrapolated", "univariate", "multivariate", "ensemble"): - np.testing.assert_allclose(actual[key], expected[key]) - assert np.array_equal(actual["relevant_variables"], expected["relevant_variables"]) - assert actual["names"] == ["x1", "x2"] - assert actual["dates"]["observed"] is None - assert actual["dates"]["forecast"] == ["t+1", "t+2", "t+3"] - assert actual["dates"]["interpolated_and_extrapolated"] is None - - -def test_nns_nowcast_panel_mapping_preserves_column_order_and_names() -> None: - panel = OrderedDict( - ( - ("PAYEMS", [1.0, 2.0, 3.0, 4.0, 5.0]), - ("GDPC1", [2.0, 3.0, 4.0, 5.0, 6.0]), - ) - ) - - actual = nns_nowcast_panel(panel, h=0, tau=1) - - assert actual["names"] == ["PAYEMS", "GDPC1"] - np.testing.assert_allclose( - actual["interpolated_and_extrapolated"], - np.column_stack((panel["PAYEMS"], panel["GDPC1"])), - ) - - -def test_nns_nowcast_panel_rejects_mismatched_names() -> None: - with pytest.raises(ValueError, match="names length"): - nns_nowcast_panel(_panel(), h=0, names=["only_one"]) - - -def test_nns_nowcast_panel_normalizes_dates_and_forecast_months() -> None: - panel = _panel() - dates = ["2020-01-15", "2020-02", np.datetime64("2020-03-31")] - dates.extend(f"2020-{month:02d}" for month in range(4, 13)) - dates.extend(f"2021-{month:02d}" for month in range(1, 13)) - dates.extend(f"2022-{month:02d}" for month in range(1, 13)) - dates.extend(f"2023-{month:02d}" for month in range(1, 4)) - - actual = nns_nowcast_panel( - panel, - h=2, - tau=1, - dates=dates, - ) - - assert actual["dates"]["observed"][:3] == ["2020-01", "2020-02", "2020-03"] - assert actual["dates"]["forecast"] == ["2023-04", "2023-05"] - assert actual["dates"]["interpolated_and_extrapolated"] == actual["dates"]["observed"] - - -def test_nns_nowcast_panel_rejects_invalid_dates() -> None: - panel = np.array([[1.0, 2.0], [2.0, 3.0], [3.0, 4.0]], dtype=np.float64) - - with pytest.raises(ValueError, match="dates length"): - nns_nowcast_panel(panel, h=0, dates=["2020-01"]) - with pytest.raises(ValueError, match="duplicate"): - nns_nowcast_panel(panel, h=0, dates=["2020-01", "2020-01", "2020-02"]) - with pytest.raises(ValueError, match="sorted"): - nns_nowcast_panel(panel, h=0, dates=["2020-02", "2020-01", "2020-03"]) - - -def test_nns_nowcast_panel_missing_values_delegate_to_var() -> None: - panel = _panel() - panel[4, 0] = np.nan - panel[-1, 1] = np.nan - - actual = nns_nowcast_panel(panel, h=3, tau=2) - - assert np.all(np.isfinite(actual["interpolated_and_extrapolated"])) - assert np.all(np.isfinite(actual["univariate"])) - assert np.all(np.isfinite(actual["multivariate"])) - assert np.all(np.isfinite(actual["ensemble"])) - - -def _provider_payload() -> dict[str, Any]: - panel = _panel() - return { - "dates": [f"2020-{month:02d}" for month in range(1, 13)] - + [f"2021-{month:02d}" for month in range(1, 13)] - + [f"2022-{month:02d}" for month in range(1, 13)] - + [f"2023-{month:02d}" for month in range(1, 4)], - "series": OrderedDict( - ( - ("PAYEMS", panel[:, 0].tolist()), - ("UNRATE", panel[:, 1].tolist()), - ) - ), - "metadata": {"provider": "fixture"}, - } - - -def test_provider_payload_feeds_nowcast_panel_core() -> None: - payload = _provider_payload() - - actual = nns_nowcast_panel( - payload["series"], - h=2, - tau=12, - dates=payload["dates"], - naive_weights=False, - ) - expected = nns_nowcast_panel( - payload["series"], - h=2, - tau=12, - dates=payload["dates"], - naive_weights=False, - ) - - assert actual["names"] == ["PAYEMS", "UNRATE"] - assert actual["dates"]["forecast"] == ["2023-04", "2023-05"] - for key in ("interpolated_and_extrapolated", "univariate", "multivariate", "ensemble"): - np.testing.assert_allclose(actual[key], expected[key]) - assert np.array_equal(actual["relevant_variables"], expected["relevant_variables"]) - - -def test_csv_nowcast_provider_returns_payload(tmp_path: Any) -> None: - csv_path = tmp_path / "macro.csv" - csv_path.write_text( - "date,PAYEMS,UNRATE\n2020-01-15,1.0,4.0\n2020-02,2.0,5.0\n2020-03-31,3.0,6.0\n", - encoding="utf-8", - ) - - payload = CsvNowcastProvider(csv_path).fetch(("PAYEMS",), "2020-01-01") - - assert payload["dates"] == ["2020-01", "2020-02", "2020-03"] - series_payload = cast(Mapping[str, object], payload["series"]) - assert list(series_payload) == ["PAYEMS", "UNRATE"] - assert payload["series"] == OrderedDict( - ( - ("PAYEMS", [1.0, 2.0, 3.0]), - ("UNRATE", [4.0, 5.0, 6.0]), - ) - ) - assert payload["metadata"] == { - "provider": "csv", - "path": str(csv_path), - "date_column": "date", - "series_columns": ["PAYEMS", "UNRATE"], - } - - -def test_csv_provider_payload_matches_panel_core(tmp_path: Any) -> None: - csv_path = tmp_path / "macro.csv" - panel = _panel() - rows = ["date,PAYEMS,UNRATE"] - for index, month in enumerate( - [f"2020-{month:02d}" for month in range(1, 13)] - + [f"2021-{month:02d}" for month in range(1, 13)] - + [f"2022-{month:02d}" for month in range(1, 13)] - + [f"2023-{month:02d}" for month in range(1, 4)] - ): - rows.append(f"{month},{panel[index, 0]},{panel[index, 1]}") - csv_path.write_text("\n".join(rows), encoding="utf-8") - - payload = CsvNowcastProvider(csv_path).fetch((), "2000-01-03") - actual = nns_nowcast_panel( - payload["series"], - h=2, - tau=12, - dates=cast(Sequence[object], payload["dates"]), - ) - expected = nns_nowcast_panel( - OrderedDict( - ( - ("PAYEMS", panel[:, 0].tolist()), - ("UNRATE", panel[:, 1].tolist()), - ) - ), - h=2, - tau=12, - dates=[row.split(",", maxsplit=1)[0] for row in rows[1:]], - ) - - assert actual["names"] == ["PAYEMS", "UNRATE"] - assert actual["dates"]["forecast"] == ["2023-04", "2023-05"] - for key in ("interpolated_and_extrapolated", "univariate", "multivariate", "ensemble"): - np.testing.assert_allclose(actual[key], expected[key]) - assert np.array_equal(actual["relevant_variables"], expected["relevant_variables"]) - - -def test_csv_nowcast_provider_selects_and_orders_series_columns(tmp_path: Any) -> None: - csv_path = tmp_path / "macro.csv" - csv_path.write_text( - "date,PAYEMS,UNRATE,GDPC1\n2020-01,1.0,4.0,7.0\n2020-02,2.0,5.0,8.0\n", - encoding="utf-8", - ) - - payload = CsvNowcastProvider(csv_path, series_columns=["GDPC1", "PAYEMS"]).fetch((), "2020-01") - - series_payload = cast(Mapping[str, object], payload["series"]) - assert list(series_payload) == ["GDPC1", "PAYEMS"] - assert payload["series"] == OrderedDict((("GDPC1", [7.0, 8.0]), ("PAYEMS", [1.0, 2.0]))) - - -def test_csv_nowcast_provider_parses_missing_values(tmp_path: Any) -> None: - csv_path = tmp_path / "macro.csv" - csv_path.write_text( - "date,PAYEMS,UNRATE\n2020-01,1.0,\n2020-02,NA,5.0\n2020-03,nan,null\n", - encoding="utf-8", - ) - - payload = CsvNowcastProvider(csv_path).fetch((), "2020-01") - - assert payload["series"] == OrderedDict( - ( - ("PAYEMS", [1.0, None, None]), - ("UNRATE", [None, 5.0, None]), - ) - ) - - -def test_csv_nowcast_provider_filters_start_date(tmp_path: Any) -> None: - csv_path = tmp_path / "macro.csv" - csv_path.write_text( - "date,PAYEMS\n2020-01,1.0\n2020-02,2.0\n2020-03,3.0\n", - encoding="utf-8", - ) - - payload = CsvNowcastProvider(csv_path).fetch((), "2020-02-15") - - assert payload["dates"] == ["2020-02", "2020-03"] - assert payload["series"] == OrderedDict((("PAYEMS", [2.0, 3.0]),)) - - -def test_csv_nowcast_provider_rejects_bad_dates(tmp_path: Any) -> None: - duplicate_path = tmp_path / "duplicate.csv" - duplicate_path.write_text( - "date,PAYEMS\n2020-01,1.0\n2020-01,2.0\n", - encoding="utf-8", - ) - unsorted_path = tmp_path / "unsorted.csv" - unsorted_path.write_text( - "date,PAYEMS\n2020-02,2.0\n2020-01,1.0\n", - encoding="utf-8", - ) - - with pytest.raises(ValueError, match="duplicate"): - CsvNowcastProvider(duplicate_path).fetch((), "2020-01") - with pytest.raises(ValueError, match="sorted"): - CsvNowcastProvider(unsorted_path).fetch((), "2020-01") - - -def test_csv_nowcast_provider_rejects_missing_columns(tmp_path: Any) -> None: - csv_path = tmp_path / "macro.csv" - csv_path.write_text( - "month,PAYEMS\n2020-01,1.0\n", - encoding="utf-8", - ) - - with pytest.raises(ValueError, match="missing date column"): - CsvNowcastProvider(csv_path).fetch((), "2020-01") - with pytest.raises(ValueError, match="missing selected series column"): - CsvNowcastProvider(csv_path, date_column="month", series_columns=["UNRATE"]).fetch( - (), "2020-01" - ) - - -def test_csv_nowcast_provider_rejects_nonnumeric_values(tmp_path: Any) -> None: - csv_path = tmp_path / "macro.csv" - csv_path.write_text( - "date,PAYEMS\n2020-01,bad\n", - encoding="utf-8", - ) - - with pytest.raises(ValueError, match="nonnumeric"): - CsvNowcastProvider(csv_path).fetch((), "2020-01") - - -def test_csv_nowcast_provider_rejects_empty_or_no_series_csv(tmp_path: Any) -> None: - empty_path = tmp_path / "empty.csv" - empty_path.write_text("", encoding="utf-8") - header_only_path = tmp_path / "header_only.csv" - header_only_path.write_text("date,PAYEMS\n", encoding="utf-8") - no_series_path = tmp_path / "no_series.csv" - no_series_path.write_text( - "date\n2020-01\n", - encoding="utf-8", - ) - - with pytest.raises(ValueError, match="empty"): - CsvNowcastProvider(empty_path).fetch((), "2020-01") - with pytest.raises(ValueError, match="no data rows"): - CsvNowcastProvider(header_only_path).fetch((), "2020-01") - with pytest.raises(ValueError, match="at least one usable series"): - CsvNowcastProvider(no_series_path).fetch((), "2020-01")