Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "quantitative-data-analysis-rs"
version = "0.37.4"
version = "0.38.0"
edition = "2024"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
Expand All @@ -24,7 +24,7 @@ tokio = { version = "1.48.0", features = ["full"] }
async-trait = "0.1.89"
tryhard = "0.5.2"
reqwest = { version = "0.13.2", features = ["json"] }
query-string-builder = "0.7.1"
query-string-builder = "0.8.0"
rayon = "1.11.0"
deref-derive = "0.1.0"
dirs = "7.0.0"
Expand Down
40 changes: 40 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,46 @@ regression safety.

## Configuration

### J-Quants financial disclosures

The J-Quants statement repository retrieves every `/v2/fins/summary` page and
returns disclosure rows in ascending disclosure date and time (JST). Rows with
the same timestamp retain their source order; disclosure numbers do not establish
their chronological order. A failed page, invalid response, or malformed nonempty
field is an error, not a partial history. Rate limiting and retries remain the
caller's responsibility.

`FetchScoringData.financial_disclosures` contains the API's FY, quarterly, and
forecast-revision rows. Each entry retains its disclosure identifier and document
type, period and fiscal-year dates, and operating-profit actual and forecast
fields, including the `NC` variants. Amounts are in the units provided by
J-Quants (normally yen). Missing values are `null`; they do not mean unchanged or
withdrawn. `FOP` refers to the current fiscal year, `NxFOP` to the next fiscal
year. Compare both fiscal-year boundaries before associating forecasts.

The API's `OP` is preserved with its original period dates and
`operating_profit_period: "unspecified"`. Its interpretation as cumulative or
single-quarter is not guaranteed here; no single-quarter
differences are calculated. The `NC`-less fields may represent consolidated or
individual values depending on document type; neither accounting basis nor
consolidation status is inferred when the document does not establish it.
J-Quants does not provide the time at which a disclosure became available through
its API, and provider corrections can overwrite previously supplied values.
These records therefore do not guarantee point-in-time reconstruction.
J-Quants also does not guarantee snapshot consistency if data changes during
pagination.

`latest_statement` and `full_year_sales` remain FY projections, separate from the
full history. `get_row_statements` returns the full disclosure rows, whereas
`get_statements` remains an FY-only projection. Generated statements TSVs now
include a `disclosure_json` column;
`examples/make_statements.rs` writes every disclosure and stops on fetch failure.
Existing TSV data lacks the history and must be regenerated before using it with
`FetchScoringData`; it is rejected rather than silently producing incomplete
history. The JSON contract adds `financial_disclosures` and old JSON must be
regenerated. The example uses `assets/statements.tsv`, so regeneration and
deployment of that input must be handled separately.

### Stock price and CSV input contract

`Stock` requires `adj_open`, `adj_high`, `adj_low`, `adj_close`, and
Expand Down
23 changes: 9 additions & 14 deletions examples/make_statements.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
use anyhow::Context;
use bytes::Bytes;
use csv::WriterBuilder;
use quantitative_data_analysis_rs::domain::repositories::company::repository::Company;
Expand Down Expand Up @@ -38,38 +39,32 @@ async fn main() -> anyhow::Result<()> {
let started_at = Instant::now();
let total = companies.len();
let mut success_count = 0usize;
let mut failure_count = 0usize;

for (index, company) in companies.0.into_iter().enumerate() {
let code = company.code;
let query = get_statements::Query {
code: Some(code.as_str()),
};

match jquants_api.get_statements(&query).await {
Ok(statements) => {
for statement in statements.0 {
writer.serialize(statement_tsv::Structure::from(statement))?;
}
success_count += 1;
}
Err(error) => {
eprintln!("failed to fetch statements for code {}: {}", code, error);
failure_count += 1;
}
let statements = jquants_api
.get_row_statements(&query)
.await
.with_context(|| format!("failed to fetch statements for code {code}"))?;
for statement in statements {
writer.serialize(statement_tsv::Structure::try_from(statement)?)?;
}
success_count += 1;

if (index + 1) % PROGRESS_STEP == 0 || index + 1 == total {
let elapsed = started_at.elapsed();
let average_per_item = elapsed / (index as u32 + 1);
let remaining = total.saturating_sub(index + 1) as u32;
let eta = average_per_item * remaining;
eprintln!(
"progress: {}/{} success={} failed={} elapsed={:?} eta={:?}",
"progress: {}/{} success={} elapsed={:?} eta={:?}",
index + 1,
total,
success_count,
failure_count,
elapsed,
eta,
);
Expand Down
63 changes: 61 additions & 2 deletions src/domain/models/statement/model.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,53 @@
use chrono::NaiveDate;
use chrono::{NaiveDate, NaiveTime};
use deref_derive::{Deref, DerefMut};
use serde::Deserialize;
use serde::{Deserialize, Serialize};

/// Period interpretation of the API's actual operating-profit value.
#[derive(Serialize, Deserialize, Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
#[serde(rename_all = "snake_case")]
pub enum OperatingProfitPeriod {
/// The API source does not establish a general cumulative or single-quarter meaning.
Unspecified,
}

/// An unmodified J-Quants disclosure. Times are Japan Standard Time.
///
/// The API does not explicitly establish whether every operating profit value
/// is cumulative or for a single quarter. Period dates and source fields are
/// retained without deriving or classifying quarterly amounts.
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, PartialOrd)]
pub struct FinancialDisclosure {
pub code: String,
pub disclosed_date: NaiveDate,
pub disclosed_time: NaiveTime,
pub disclosure_number: String,
pub document_type: String,
pub current_period_type: Option<String>,
pub current_period_start: Option<NaiveDate>,
pub current_period_end: Option<NaiveDate>,
pub current_fiscal_year_start: Option<NaiveDate>,
pub current_fiscal_year_end: Option<NaiveDate>,
pub next_fiscal_year_start: Option<NaiveDate>,
pub next_fiscal_year_end: Option<NaiveDate>,
pub operating_profit: Option<f64>,
pub operating_profit_period: OperatingProfitPeriod,
pub forecast_operating_profit: Option<f64>,
pub next_forecast_operating_profit: Option<f64>,
pub forecast_operating_profit_2q: Option<f64>,
pub next_forecast_operating_profit_2q: Option<f64>,
pub non_consolidated_operating_profit: Option<f64>,
pub non_consolidated_forecast_operating_profit: Option<f64>,
pub non_consolidated_next_forecast_operating_profit: Option<f64>,
pub non_consolidated_forecast_operating_profit_2q: Option<f64>,
pub non_consolidated_next_forecast_operating_profit_2q: Option<f64>,
}

impl FinancialDisclosure {
/// J-Quants accepts a four-character code and returns its five-character form.
pub fn matches_code(&self, code: &str) -> bool {
self.code == code || (code.len() == 4 && self.code.strip_suffix('0') == Some(code))
}
}

#[derive(Deserialize, Debug, Clone, PartialEq, PartialOrd, Default)]
pub struct RowStatement {
Expand All @@ -16,6 +63,17 @@ pub struct RowStatement {
pub profit: Option<f64>,
pub equity: Option<f64>,
pub total_assets: Option<f64>,
pub disclosure: Option<FinancialDisclosure>,
}

impl RowStatement {
/// Retains the legacy FY projection for statement data without disclosure metadata.
pub fn is_usable_full_year(&self) -> bool {
self.disclosure.as_ref().is_none_or(|disclosure| {
disclosure.current_period_type.as_deref() == Some("FY")
&& (self.net_sales.is_some() || self.eps.is_some())
})
}
}

/// Represents a financial statement with various financial metrics.
Expand Down Expand Up @@ -93,6 +151,7 @@ impl TryFrom<RowStatement> for Statement {
profit,
equity,
total_assets,
disclosure: _,
} = value;

Ok(Self {
Expand Down
5 changes: 3 additions & 2 deletions src/domain/repositories/statement/repository.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,9 @@ pub trait Statement {
let row_statements = self.get_row_statements(query).await?;
let vec_statement = row_statements
.into_iter()
.filter_map(|row_statement| TryFrom::try_from(row_statement).ok())
.collect();
.filter(model::RowStatement::is_usable_full_year)
.map(TryFrom::try_from)
.collect::<Result<Vec<_>>>()?;
Ok(model::Statements(vec_statement))
}

Expand Down
108 changes: 100 additions & 8 deletions src/infrastructure/repositories/statement/dsv.rs
Original file line number Diff line number Diff line change
Expand Up @@ -35,31 +35,123 @@ where

#[cfg(test)]
mod tests {
use crate::domain::models::statement::model::OperatingProfitPeriod;
use crate::domain::repositories::statement::queries;
use crate::domain::repositories::statement::repository::Statement;
use crate::infrastructure::dsv::Dsv;
use crate::infrastructure::repositories::statement::structures::internal::tsv;
use bytes::Bytes;
use chrono::NaiveDate;
use serde_json::json;

const SAMPLE_CODE: &str = "72030";
const TSV: &[u8] = include_bytes!("../../../../assets/statements.tsv");

#[tokio::test]
async fn get_row_statement_returns_latest() -> anyhow::Result<()> {
let repository = Dsv::<tsv::Structure>::new(true, Bytes::from(TSV));
async fn get_row_statements_filters_code_and_preserves_disclosures() -> anyhow::Result<()> {
let mut writer = csv::WriterBuilder::new()
.delimiter(b'\t')
.from_writer(Vec::new());
for (code, date, kind, start, end, fiscal_end, profit) in [
(
SAMPLE_CODE,
"2026-05-08",
"FY",
"2025-04-01",
"2026-03-31",
"2026-03-31",
Some(100.0),
),
(
"13010",
"2026-05-11",
"FY",
"2025-04-01",
"2026-03-31",
"2026-03-31",
Some(50.0),
),
(
SAMPLE_CODE,
"2026-08-07",
"1Q",
"2026-04-01",
"2026-06-30",
"2027-03-31",
None,
),
] {
writer.serialize(tsv::Structure {
code: code.into(),
disclosed_date: NaiveDate::parse_from_str(date, "%Y-%m-%d")?,
current_fiscal_year_end_date: Some(NaiveDate::parse_from_str(
fiscal_end, "%Y-%m-%d",
)?),
opp: profit,
disclosure_json: Some(
json!({
"code": code,
"disclosed_date": date,
"disclosed_time": "15:00:00",
"disclosure_number": format!("{code}-{date}"),
"document_type": format!("{kind}FinancialStatements_Consolidated_JP"),
"current_period_type": kind,
"current_period_start": start,
"current_period_end": end,
"current_fiscal_year_start": start,
"current_fiscal_year_end": fiscal_end,
"operating_profit": profit,
"operating_profit_period": "unspecified"
})
.to_string(),
),
..Default::default()
})?;
}
let repository = Dsv::<tsv::Structure>::new(true, Bytes::from(writer.into_inner()?));
let query = queries::get_statements::Query {
code: Some(SAMPLE_CODE),
};
let actual = repository.get_row_statements(&query).await?;
assert_eq!(actual[0].code, SAMPLE_CODE);
assert_eq!(actual.len(), 2);
assert!(actual.iter().all(|row| row.code == SAMPLE_CODE));
assert_eq!(
actual[0].disclosed_date,
NaiveDate::from_ymd_opt(2022, 5, 11)
NaiveDate::from_ymd_opt(2026, 5, 8)
);
assert_eq!(actual[0].opp, Some(100.0));
let annual = actual[0].disclosure.as_ref().unwrap();
assert_eq!(annual.current_period_type.as_deref(), Some("FY"));
assert_eq!(annual.operating_profit, Some(100.0));
assert_eq!(
actual[1].disclosed_date,
NaiveDate::from_ymd_opt(2026, 8, 7)
);
let quarter = actual[1].disclosure.as_ref().unwrap();
assert_eq!(quarter.current_period_type.as_deref(), Some("1Q"));
assert_eq!(
quarter.current_period_end,
NaiveDate::from_ymd_opt(2026, 6, 30)
);
assert_eq!(quarter.operating_profit, None);
assert_eq!(
quarter.operating_profit_period,
OperatingProfitPeriod::Unspecified
);
assert_eq!(
repository
.get_row_statements(&queries::get_statements::Query { code: None })
.await?
.len(),
3
);
assert!(
repository
.get_row_statements(&queries::get_statements::Query {
code: Some("99990"),
})
.await?
.is_empty()
);
let actual = actual.last().unwrap();
assert_eq!(actual.code, SAMPLE_CODE);
assert_eq!(actual.disclosed_date, NaiveDate::from_ymd_opt(2026, 5, 8));
Ok(())
}
}
Loading
Loading