feat(#10903): sub contact attachment routing

[benkags]

Description

Every uploaded file is attached to the main doc regardless of the sibling (contact) or child doc it was uploaded in. For example, if you upload a photo inside a CHW (sibling) form during a health CHW Area creation, the photo is attached to the CHW Area doc.

This PR fixes that for contact forms only by

• Adding attachment - doc mapping logic
• Route file-widget and inline-binary attachments per-doc using the mapping, with per-doc processing
• Validations, filename sanitization & clean up

Closes #10903

Code review checklist

• UI/UX backwards compatible: Test it works for the new design (enabled by default). And test it works in the old design, enable can_view_old_navigation permission to see the old design. Test it has appropriate design for RTL languages.
• Readable: Concise, well named, follows the style guide
• Documented: Configuration and user documentation on cht-docs
• Tested: Unit and/or e2e where appropriate
• Internationalised: All user facing text
• Backwards compatible: Works with existing data and configuration or includes a migration. Any breaking changes documented in the release notes.
• AI disclosure: Please disclose use of AI per the guidelines.

License

The software is provided under AGPL-3.0. Contributions to this project are accepted under the same license.

[benkags]

This is ready for review.

[jkuester]

@benkags once again, I am so glad you are working through this! ✨ I spent hours this afternoon reading through and debugging around this code. I have not finished reading it all yet, but I ran out of time today and wanted to push the thoughts that I have so for. 👍

[benkags]

@jkuester please take another look

[jkuester]

Okay, I have been deep down the rabbit hole here. 😬 🐇

All of this is so complicated and interlocked that I ended up just creating benkags#10 with my suggested changes instead of trying to put everything in comments here. 😓 The main focus of my changes was getting all the attachments for the main report and sub-reports to be rendered as expected when viewing the report in the "Reports" tab.

I used file_uploads.xlsx to test with.

A couple things I learned in all my testing and debugging:

Current attachment naming strategies (before this PR):

1. New-style file type attachements (>=4.9.0):
   • Property value set to attachment name
   • Attachment name has user-file- prefix + property value
2. Old-style file type attachments (<4.9.0):
   • Property value set to attachment name
   • Attachment name has user-file/ prefex + path/to/property
3. All binary type attachments:
   • Property value set to ""
   • Attachment name has user-file/ prefex + path/to/property

#1 and #2 are what you got when using any of the image/audio/file/etc types in your xform and the user selects a file from the device to upload.

#3 should mainly be coming from the external Android app integration where you are getting base64 data loaded as the value for a field in the form with instance:: binary set to true. (Technically can also enter base64 text right in the form or load it as a default value.) This data does not go through the Enekto file manager.

Editing docs with attachments

Currently, (before this PR) you can edit docs with file attachments. The attachments with type=file are rendered properly in the form, but the type=binary attachments are not rendered. Depending on how we plan to load profile pictures into a contact form, this may or may not be something we need to worry about....

The rendering of the type=binary attachments is not fixed (even by my changes in the linked PR). I was up way too late last night trying to sort out a solution (see the first commit in my PR for what I came up with). I think there is something viable here that we could move forwards with in the future, but ultimately I felt it was out of scope for your PR here since the existing functionality remains unchanged.

---

The changes in my PR come without any updates to the tests 😓 or even any manual validation of the contact-attachment flow. I have been purely testing with reports. I am out of time for the week, though, and I figured I would post what I have and get your thoughts. 👍

[benkags]

Thanks @jkuester for going down the rabbit hole. I have been down there myself a couple of times with this PR 😃 . Thanks for PR benkags#10 - I agree, it a much better PR review feedback approach. This is what I picked from PR as net change. Let me know if I missed something

1. a bug fix that I'd missed: File-widget routing bug in xmlToDocs
2. updated xpath based attachment name logic; so that the sub docs have the relative file path
3. undoes the 'attachment-filename-in-the-field-value' for the binary types in favor of constructing the base64 string from the attachment itself for re-population when editing the report (the spirit of the first commit)

I think it really comes down to what approach to adopt between Approach A (the one I have here) and Approach B (in your PR) from a design perpective

• Approach A: standardize as much as possible with [type=file] approach by populating the type=binary field value with the attachment name. This saves us the extra logic of constructing base64 string from the attachment for edit. The report view is handled by reading the attachment directly and injecting it as an image like any other file object of type image. The cost to pay for here is to handle exiting reports before this PR by populating the binary field from the saved xml form model at display.
  • the reports edit / view works in the PR

• the report edit was broken before (but report view worked)

• Approach B: Does not save attachment name in the saved doc, reconstructs the attachment name from the xml at display and constructs the binary's base64 string to populate the type=binary field.

[jkuester]

I think you have pretty clearly understood my PR. 👍 Just a couple comments:

Approach A: standardize as much as possible with [type=file] approach by populating the type=binary field value with the attachment name.

I 100% was hoping this would work out as you have described. 😓 Unfortunately, it did not really seem to save us any complexity.

This saves us the extra logic of constructing base64 string from the attachment for edit. The report view is handled by reading the attachment directly and injecting it as an image like any other file object of type image.

The format-data-record service will still need to keep the extra logic to support any existing reports with the user-file/ format. And the logic in the reports-add component for loading the image attachments when editing a report will still not work for type=binary fields (because the form model knows that they are not type=file fields and so their handling logic is totally different).

On top of that, changing to store the attachment name as the property value for type=binary fields ends up creating new challenges in the edit workflow:

1. If we do not do anything and just leave the attachment name set as the value, it breaks the display-base64-image preview functionality so that a "invalid image" icon is shown in the form (instead of the current edit "functionality" where nothing is shown when editing a report for the display-base64-image fields.... 😬 (FTR, we can avoid the icon and actually display the image if we re-load the base64 data from the attachment back into the property value before rendering the form. However, as I found in my first commit, this is not super easy. We will probably need to support this eventually, though... 🤷 )
2. If/when we end up re-injecting the base64 image data for type=binary fields on edit, we immediately hit the challenge of how to avoid adding a duplicate attachment file each time the report is edited even if the type=binary field did not change. Using a random uuid-based attachment name is obviously a problem because we would always get different names. My first commit used the hash of the file contents which worked but added overhead/complexity. Another option would be to just use the path to the field (e.g. like the original type=binary logic was doing).

It ended up being similar complexity to just properly calculate the field path (even for the sub-docs). That was the point where I started trying to figure out what benefits I was actually getting from Approach A.

On the other hand, with Approach B., we end up with a minimal amount of code changed and all the type=binary logic still just follows what it was doing before (but now also works for sub-docs). We have not simplified anything, but we really have not added much more complexity either... 😅 🤷

---

@benkags if you have a strong preference for Approach A, I am not set against it (as long as we can find a maintainable way to sort out the above challenges). Otherwise, I do think Approach B (as drafted in my PR) should result in the functionality we need with the minimal amount of changes.

Also happy to jump on a call to discuss further if that would be helpful! 👍

[benkags]

Ready for review @jkuester