Extending the parser to implement a custom DSL-like language

[springcomp]

@mhutch you have been very helpful in the past when I previously asked about this:
#84

You have referred me to sample code that I took as inspiration.
I am indeed able to extend the parser and create new custom nodes.

However, going back to my first scenario, I seem to be unable to override the default XAttribute whose values are hardcoded to be strings.
You specifically highlighted in the past that you used a two-phase parsing of attributes, instead of going through the extension points from the parser here. I would be very interested to understand if it is at all possible with the current code or if I’m missing something obvious.

I think what I’m after is being able to create DOMs like there:

• XDocument
  • XElement
    • XAttribute
      • XAttributeValue (custom implementation possible)

Ideally, the XAttribute should have an XAttributeValue node that authors like myself could extend arbitrarily.

Another option, is like I’m trying in a proof-of-concept to add a generic XObject? Node { get; set; } property to XObject, giving it support for attaching any node. But that feels a bit kludgey.

Something like:

• XDocument
  • XElement
    • XAttribute (XAttribute.Node is provided as an attach point for custom nodes)

Please, can you provide some helpful guidance?

If there is anything that needs to be slightly changed in the current code, I’m happy to try and propose a pull request.

[KirillOsenkov]

Could you give an example of how you would like to extend the attribute specifically? It always helps to have concrete examples when designing something.

Also are you sure you want inheritance from XAttributeValue? What if it just had public object Payload {get; set;} property or something?

[springcomp]

@KirillOsenkov

I’m after syntax like this:

<policies>
  <inbound>
    <set-variable name="addition" value="@( 1 + 1 )" />
    <set-variable name="constant" value="@( "hello, world" )" />
    <set-variable name="indirect" value="@( context.Variables.GetValueOrDefault<string>("constant", "n/a") )" />
    <base />
  </inbound>
</policies>

Could you give an example of how you would like to extend the attribute specifically? It always helps to have concrete examples when designing something.

My first attempt had me
modifying the library to change XAttribute’s Value property from a string to a new XAttributeValue extensible node.

[A]re you sure you want inheritance from XAttributeValue? What if it just had public object Payload {get; set;} property or something?

That’s the thing. I’m not sure what is the best way to achieve this with the current code if it is at all possible.

That’s why my very short POC referred to previously needs a change in this library to somehow hold the new structure:

Core/Dom/XObject.cs

		protected XObject (TextSpan span) => Span = span;


		public XObject? Parent { get; internal protected set; }

+		// supports attaching arbitrary node
+		public XObject? Node { get; set; }

		public IEnumerable<XNode> Parents {
			get {
				var next = Parent as XNode;

This looks a lot like your suggestion to have public object Payload { get; set; } suggestion.

However, I would ideally think it best if we could extend the DOM itself. Like, currently an XAttribute is a leaf of the AST / DOM. Whereas I think we should be able to attach arbitrary child nodes to the structure.

Thank you very much for your feedback.
Really appreciated.