Conversation
…raction paths - Introduced `UrlResolverConfig` struct to define URL resolution parameters. - Added `HttpMethod` enum for specifying GET and POST methods. - Enhanced `ResolverType` to include URL resolvers. - Implemented URL resolution logic in the code generation and runtime, allowing for batch processing and JSON path extraction from responses. - Updated parsing logic to support new `url_resolve` attribute in macros.
- Updated the `hyperstack` macro documentation to reflect the new `#[resolve(...)]` attribute. - Refactored parsing logic to support new parameters for URL resolution, including `url` and `method`. - Removed the deprecated `url_resolve` attribute and integrated its functionality into the `resolve` attribute. - Enhanced error handling for mutually exclusive parameters in the `resolve` attribute. - Updated entity processing to accommodate the new resolver logic for URL and token sources.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
adiman9
left a comment
There was a problem hiding this comment.
Overall this is great. Follows the overall patterns and handles things well.
Few things i've noticed:
Highest prio is there are a few spots where we silently fail and don't retry. This happens in extract_json_path as well as failed http requests. The token resolver has some retry patterns you can reference.
I've also created HYP-180 as a future plan to consolidate some of the resolver stuff
|
|
||
| #[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, Hash, Default)] | ||
| #[serde(rename_all = "lowercase")] | ||
| pub enum HttpMethod { |
There was a problem hiding this comment.
I've noticed these types are duplicated in the interpreter as well. My code from before is also duplicated. Not worth fixing now but worth making a note to consolidate those over time
|
|
||
| url = Some(parts.join(".")); | ||
| } else if ident_str == "method" { | ||
| method = Some(input.parse()?); |
There was a problem hiding this comment.
we should probably fail for any mehtod that isn't get or post
|
|
||
| // Process URL resolver requests | ||
| if !url_requests.is_empty() { | ||
| let url_client = hyperstack::runtime::hyperstack_interpreter::resolvers::UrlResolverClient::new(); |
There was a problem hiding this comment.
We should probably instantiate the client once at startup and attach it onto VmHandler
| } | ||
| } | ||
| } | ||
| Err(err) => { |
There was a problem hiding this comment.
We should probably requeue these so they get retried. Similar to the token resolver calling vm.restore_resolver_requests(unresolved)
There was a problem hiding this comment.
Dont want a minor network error to result in missing a value with SetOnce for example
| } | ||
| }; | ||
|
|
||
| if url.is_empty() { |
There was a problem hiding this comment.
Should probably requeue here as well: vm.restore_resolver_requests(vec![request])
| } | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
All the updates in this block also apply to the other location that duplicates this logic which is on lines 1400+ below
|
|
||
| // For URL resolvers with extract_path, the value is already extracted by | ||
| // resolve_with_extract, so source_path should be None to use the value directly | ||
| let source_path = match &spec.resolver { |
There was a problem hiding this comment.
The source_path = None branch for Url resolvers encodes knowledge that extraction happens inside UrlResolverClient rather than in apply_resolver_result. This couples the AST builder to a client implementation detail. The cleaner fix is to have UrlResolverClient return the raw JSON response and let apply_resolver_result do extraction via source_path — exactly how token resolvers work. Then this special case disappears and extraction is uniform across all resolver types
| // Determine resolver type: URL resolver if url is present, otherwise Token resolver | ||
| let resolver = if let Some(url_path_raw) = resolve_attr.url.clone() { | ||
| // Qualify the url_path with section name if needed | ||
| let url_path = if url_path_raw.contains('.') { |
There was a problem hiding this comment.
url_path is computed twice with identical logic in this block — once for UrlResolverConfig.url_path and again for from. The second derivation can just reuse the first: from: Some(url_path.clone()). As written, any change to the qualification logic needs to be applied in both places.
| ) -> HashMap<String, Value> { | ||
| let mut results = HashMap::new(); | ||
|
|
||
| for (url, method, extract_path) in urls { |
There was a problem hiding this comment.
Probably should do this in parallel too
| if let Some(next) = current.get(index) { | ||
| current = next; | ||
| } else { | ||
| return Ok(Value::Null); |
There was a problem hiding this comment.
This should be an Err maybe? Same below
Extended Resolve Macro - URL Resolution Support
Overview
This PR extends the
#[resolve]macro to support URL-based data fetching in addition to the existing Token metadata resolution via DAS API.New Capability
The
#[resolve]macro now supports fetching and extracting data from HTTP URLs:Resolver Type Detection
The macro automatically determines the resolver type based on parameters:
url = ...address = ...orfrom = ...Parameters are mutually exclusive - specifying both
urlandaddress/fromis a compile error.URL Resolver Parameters
urlinfo.uri)extract"image","data.nested.field")methodGET(default) orPOSTExample Usage
Changes
Files Modified
hyperstack-macros/src/parse/attributes.rsurlandmethodfields toResolveAttributeArgshyperstack-macros/src/stream_spec/entity.rsResolverType::Urlwhenurlparameter is presenthyperstack-macros/src/stream_spec/sections.rsstacks/pumpfun/src/stack.rs