Skip to content

Comments

URI resolver for external data fetching#50

Draft
vimmotions wants to merge 5 commits intomainfrom
vim/hyp-178-feature-uri-resolver-for-type-safe-external-data-fetching
Draft

URI resolver for external data fetching#50
vimmotions wants to merge 5 commits intomainfrom
vim/hyp-178-feature-uri-resolver-for-type-safe-external-data-fetching

Conversation

@vimmotions
Copy link

Extended Resolve Macro - URL Resolution Support

Overview

This PR extends the #[resolve] macro to support URL-based data fetching in addition to the existing Token metadata resolution via DAS API.

New Capability

The #[resolve] macro now supports fetching and extracting data from HTTP URLs:

// Token resolver (existing functionality)
#[resolve(address = "oreoU2P8bN6jkk3jbaiVxYnG1dCXcYxwhwyK9jSybcp")]
pub ore_metadata: Option<TokenMetadata>,

// URL resolver (new)
#[resolve(url = info.uri, extract = "image")]
pub resolved_image: Option<String>,

// URL resolver with HTTP method
#[resolve(url = api.endpoint, method = POST, extract = "data.result")]
pub result: Option<String>,

Resolver Type Detection

The macro automatically determines the resolver type based on parameters:

Parameters Resolver Type
url = ... URL Resolver (HTTP fetch + JSON extraction)
address = ... or from = ... Token Resolver (DAS API)

Parameters are mutually exclusive - specifying both url and address/from is a compile error.

URL Resolver Parameters

Parameter Required Description
url Yes Field path containing the URL to fetch (e.g., info.uri)
extract Yes JSON path to extract from response (e.g., "image", "data.nested.field")
method No HTTP method: GET (default) or POST

Example Usage

pub struct TokenInfo {
    #[from_instruction([Create::uri], strategy = SetOnce)]
    pub uri: Option<String>,

    // Fetch metadata JSON from uri and extract the "image" field
    #[resolve(url = info.uri, extract = "image")]
    pub resolved_image: Option<String>,
}

Changes

Files Modified

  • hyperstack-macros/src/parse/attributes.rs

    • Added url and method fields to ResolveAttributeArgs
    • Updated parser to handle dot-path syntax for URL field references
    • Added validation for mutually exclusive parameters
  • hyperstack-macros/src/stream_spec/entity.rs

    • Updated resolve branch to create ResolverType::Url when url parameter is present
  • hyperstack-macros/src/stream_spec/sections.rs

    • Same updates as entity.rs with section-name prefixing for unqualified paths
  • stacks/pumpfun/src/stack.rs

    • Added example usage of URL resolution

…raction paths

- Introduced `UrlResolverConfig` struct to define URL resolution parameters.
- Added `HttpMethod` enum for specifying GET and POST methods.
- Enhanced `ResolverType` to include URL resolvers.
- Implemented URL resolution logic in the code generation and runtime, allowing for batch processing and JSON path extraction from responses.
- Updated parsing logic to support new `url_resolve` attribute in macros.
- Updated the `hyperstack` macro documentation to reflect the new `#[resolve(...)]` attribute.
- Refactored parsing logic to support new parameters for URL resolution, including `url` and `method`.
- Removed the deprecated `url_resolve` attribute and integrated its functionality into the `resolve` attribute.
- Enhanced error handling for mutually exclusive parameters in the `resolve` attribute.
- Updated entity processing to accommodate the new resolver logic for URL and token sources.
@vercel
Copy link

vercel bot commented Feb 16, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hyperstack-docs Ready Ready Preview, Comment Feb 16, 2026 7:21pm

Request Review

@vimmotions vimmotions requested a review from adiman9 February 16, 2026 17:58
Copy link
Contributor

@adiman9 adiman9 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this is great. Follows the overall patterns and handles things well.

Few things i've noticed:

Highest prio is there are a few spots where we silently fail and don't retry. This happens in extract_json_path as well as failed http requests. The token resolver has some retry patterns you can reference.

I've also created HYP-180 as a future plan to consolidate some of the resolver stuff


#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, Hash, Default)]
#[serde(rename_all = "lowercase")]
pub enum HttpMethod {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've noticed these types are duplicated in the interpreter as well. My code from before is also duplicated. Not worth fixing now but worth making a note to consolidate those over time


url = Some(parts.join("."));
} else if ident_str == "method" {
method = Some(input.parse()?);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should probably fail for any mehtod that isn't get or post


// Process URL resolver requests
if !url_requests.is_empty() {
let url_client = hyperstack::runtime::hyperstack_interpreter::resolvers::UrlResolverClient::new();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably instantiate the client once at startup and attach it onto VmHandler

}
}
}
Err(err) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably requeue these so they get retried. Similar to the token resolver calling vm.restore_resolver_requests(unresolved)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dont want a minor network error to result in missing a value with SetOnce for example

}
};

if url.is_empty() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably requeue here as well: vm.restore_resolver_requests(vec![request])

}
}
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the updates in this block also apply to the other location that duplicates this logic which is on lines 1400+ below


// For URL resolvers with extract_path, the value is already extracted by
// resolve_with_extract, so source_path should be None to use the value directly
let source_path = match &spec.resolver {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The source_path = None branch for Url resolvers encodes knowledge that extraction happens inside UrlResolverClient rather than in apply_resolver_result. This couples the AST builder to a client implementation detail. The cleaner fix is to have UrlResolverClient return the raw JSON response and let apply_resolver_result do extraction via source_path — exactly how token resolvers work. Then this special case disappears and extraction is uniform across all resolver types

// Determine resolver type: URL resolver if url is present, otherwise Token resolver
let resolver = if let Some(url_path_raw) = resolve_attr.url.clone() {
// Qualify the url_path with section name if needed
let url_path = if url_path_raw.contains('.') {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

url_path is computed twice with identical logic in this block — once for UrlResolverConfig.url_path and again for from. The second derivation can just reuse the first: from: Some(url_path.clone()). As written, any change to the qualification logic needs to be applied in both places.

) -> HashMap<String, Value> {
let mut results = HashMap::new();

for (url, method, extract_path) in urls {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably should do this in parallel too

if let Some(next) = current.get(index) {
current = next;
} else {
return Ok(Value::Null);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be an Err maybe? Same below

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants