[BUG] Unclosed root tag when processing XML

[jmturner]

What seems to be the problem?

When uploading a GreenButton XML file, EMILE errors out with a "Unclosed root tag Line" error. The Line and Column of the error changes each time for the very same file, leading me to believe it may be an out-of-resources type of error. The GreenButton XML file in question is 22 MB. xmllint shows the file to be clean with no errors.

How can I recreate this problem?

1. Upload an XML file.
2. Refresh files to see if it has been processed
3. Clicking on the Processed tab shows the file parsing process has exited with the "Unclosed root tag" error.

What did you expect to happen?

The XML file should be properly processed.
Do you have pictures of the problem?

What kind of device and web browser are you using?
Ubuntu 24.04.3 amd64 with Firefox 143.0.4.

Is there anything else you can offer that might help?

[jmturner]

I tried ingesting the sample XML files from the Green Button site to rule out an issue with my provider's file. They all fail with the same error. https://green-button.github.io/samples/

[dangowans]

@jmturner You're probably right about the parser running out of resources. That was an ongoing issue during development. The parser, https://github.com/cityssm/node-green-button-parser, could digest the larger Green Button files, but the application had trouble.

Unfortunately, Green Button turned out to be too difficult to reliably implement and our staff responsible for energy reporting decided to go another way. Work on this project may start back up at some point, but right now, it's stalled. ☹️

[jmturner]

Thanks for the update. I don't think I can stop myself from helping to get this fixed. :) I have long awaited the delivery of real energy data from my provider and my mindset makes me determined to unlock it, I guess. I have a lot to get up to speed on before I'm productive but I will be here to help. :)

[dangowans]

I can see that, and I appreciate your dedication @jmturner .

One of our main issues with Green Button was the difficulty associated with subscribing for regular updates. For our instance, we wanted to subscribe to 100s of individual bills from two different utility providers. The subscription process is what derailed things.

If you're planning to download your data from your utility provider yourself, the subscription piece would be a non-issue.

The nice thing about the subscription though was the ability to more easily download data for shorter time ranges to keep from having memory issues.

I'm not sure what kind of time range your data file is, but is it possible to download a smaller one? Like if it's a year sized file, is it possible to download 12 month sized files?

[jmturner]

I will filter some elements out of my file and see if I can get it to load.

[jmturner]

So my original XML has three years worth of data - 225 MB - impressive! I pared it down to the last ~ 45 days which creates a <1MB file. Even the sub-megabyte file errors out around line 10407 with the Unclosed root tag, so it's clearly not an out-of-resources issue. I wonder now if a dependency somewhere (outside of the previously fixed csurf one) might be broken.