Skip to content

Implement SEC 13F XML parsing in the institutional data scraper #13

Description

@Abhilovescars

TLDR:
The SEC 13F scraper currently locates the filing XML but does not parse the holdings table. parse_13f_holdings() returns an empty DataFrame with a TODO. This blocks the repo’s institutional positioning data path. Please implement XML parsing for the entries and return a cleaned DataFrame with columns such as issuer, cusip, shares, market_value, and put_call.

Summary
The SEC 13F scraper currently retrieves filing metadata and locates the relevant XML document, but it does not parse the holdings table into a usable DataFrame.

In data/scrapers/sec_13f.py, parse_13f_holdings() currently returns an empty DataFrame with a TODO comment:

the filing index is fetched correctly,
the XML file is located correctly,
but the entries are not converted into structured holdings data.
Problem
This blocks the institutional positioning pipeline described in the repo’s data roadmap. The repo uses 13F filings as one of the public observables for the Level 1 / Level 2 mean-field institutional distribution, but the data scraper currently stops at a placeholder.

Expected behavior
parse_13f_holdings(cik, accession) should:

download the final filing XML from SEC EDGAR,
parse the entries,
extract the relevant columns such as:
issuer
cusip
shares
market_value
put_call
return a clean pandas DataFrame for downstream aggregation.
Why this matters
This is one of the key free-data sources identified for the institutional μ^(1) layer in DATA_REQUIREMENTS.md. Completing this parser would materially improve the repo’s ability to build the empirical distribution of institutional positions.

Scope
This issue is focused on:

XML parsing,
DataFrame normalization,
and validation against a known SEC filing sample.
Suggested implementation direction
Use Python’s xml.etree.ElementTree (or equivalent) to parse the nodes and map them to the expected columns described in the docstring.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions