Add ZATCA specific conversion#71
Conversation
|
Fixed with: go get github.com/invopop/gobl@addon-sa |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #71 +/- ##
==========================================
+ Coverage 76.56% 77.63% +1.07%
==========================================
Files 26 28 +2
Lines 1933 2106 +173
==========================================
+ Hits 1480 1635 +155
- Misses 330 335 +5
- Partials 123 136 +13 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Tests are running OK locally but failing when pushing with error:
Copilot's explanation:The failure in job 74992376272 is caused by the step "Upload coverage reports to Codecov" using codecov/codecov-action@v4.0.1, resulting in an Error: write EPIPE. This error often indicates a broken pipe, typically due to issues communicating with the Codecov service or an incorrectly handled upload process. |
There was a problem hiding this comment.
Pull request overview
This PR adds Saudi Arabia ZATCA (Phase 2) support to the GOBL↔UBL converter/parser, along with cross-cutting updates to extension handling, multi-currency tax totals/exchange rates, attachments, and golden fixtures.
Changes:
- Introduce
ContextZATCAand ZATCA-specific mappings (address fields, delivery period, invoice headers/type code attributes, tax total behavior). - Update core conversion/parsing to use the new
tax.Extensionsmethod-based API and derive exchange rates from dualTaxTotalblocks. - Add UBL extension/signature struct scaffolding and refresh a large set of golden XML/JSON fixtures.
Reviewed changes
Copilot reviewed 46 out of 103 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| utils.go | Switch note parsing to tax.ExtensionsOf(...). |
| ubl.go | Register ext namespace; remove post-addon Validate(); replace ' in serialized XML. |
| totals.go | Add rounding to TaxTotal; ZATCA/BT-111 extra TaxTotal; derive exchange rates from dual TaxTotal; migrate extensions API usage. |
| signature.go | Add UBL document signature struct scaffolding (XAdES embedding). |
| extension.go | Add UBL extension wrapper structs and helpers. |
| payment.go | Make InstructionNote omitempty; ZATCA-specific use of InstructionNote; migrate extensions API checks. |
| payment_test.go | Update test to use new extensions constructor. |
| payment_parse.go | Update parsing to pass options/context and use tax.ExtensionsOf(...). |
| payment_parse_test.go | Update extension access to .Get(). |
| party.go | Add ZATCA address fields + mappings; context-aware party mapping; migrate extensions API usage. |
| party_parse.go | Context-aware party parsing; map ZATCA address fields; migrate extensions API usage. |
| party_parse_test.go | Update extension access to .Get(). |
| ordering.go | Context-aware ordering mapping (skip PEPPOL-specific requirements for ZATCA); add UUID to references. |
| ordering_parse.go | Migrate to .Set() for extensions; capture DocumentTypeCode into GOBL ext. |
| lines.go | Add line-level TaxTotal (ZATCA KSA-11); migrate extensions API usage. |
| lines_parse.go | Migrate extensions API usage in line parsing. |
| lines_parse_test.go | Update extension access to .Get(). |
| invoice.go | Add ZATCA header behavior (UUID, IssueTime, invoice type name attr); make InvoiceTypeCode an *IDType; ensure ext namespace on root; context-aware conversions. |
| invoice_test.go | Update expectations for InvoiceTypeCode pointer + extensions constructor. |
| invoice_parse.go | ZATCA parsing support (invoice type name attr, IssueTime); derive exchange rates from dual TaxTotal; context-aware payment/party parsing. |
| attachments.go | Export AddAttachments, add UUID support, make URL optional. |
| delivery.go | Add LatestDeliveryDate and map delivery period to date range (start/end). |
| delivery_parse.go | Parse ZATCA delivery period (Actual+Latest) into GOBL Period. |
| context.go | Add ContextZATCA and include it in reverse-lookup list. |
| common.go | Add EXT/SIG/SAC/SBC namespace constants; migrate extensions API check in getTypeCode; remove old extensions structs. |
| charges_parse.go | Migrate charges/discounts parsing to .Set() / tax.ExtensionsOf(...). |
| charges_parse_test.go | Update extension access to .Get(). |
| examples_test.go | Update Phive port; include ZATCA context in convert/parse example tests. |
| go.mod | Update GOBL version, add xmldsig/cloud deps, bump testify, refresh indirect deps. |
| test/data/parse/zatca/standard-usd-invoice.xml | New ZATCA parse fixture (USD doc currency + SAR tax currency). |
| test/data/parse/zatca/simplified-zero-rated.xml | New ZATCA parse fixture (simplified zero-rated scenario). |
| test/data/parse/zatca/out/standard-usd-invoice.json | Expected parsed GOBL output for ZATCA USD invoice. |
| test/data/parse/zatca/out/standard-invoice.json | Expected parsed GOBL output for ZATCA standard invoice. |
| test/data/parse/zatca/out/standard-debit-note.json | Expected parsed GOBL output for ZATCA debit note. |
| test/data/parse/zatca/out/standard-credit-note.json | Expected parsed GOBL output for ZATCA credit note. |
| test/data/parse/zatca/out/simplified-zero-rated.json | Expected parsed GOBL output for ZATCA simplified zero-rated. |
| test/data/parse/zatca/out/simplified-invoice.json | Expected parsed GOBL output for ZATCA simplified invoice. |
| test/data/parse/zatca/out/simplified-debit-note.json | Expected parsed GOBL output for ZATCA simplified debit note. |
| test/data/parse/zatca/out/simplified-credit-note.json | Expected parsed GOBL output for ZATCA simplified credit note. |
| test/data/parse/peppol/out/Allowance-example.json | Golden update for parsed exchange rates + identity ext. |
| test/data/parse/france-cius/out/b2b-reg.json | Golden update for identity ext structure. |
| test/data/parse/en16931/out/ubl-example5.json | Golden update for parsed exchange rates. |
| test/data/parse/en16931/out/custom-namespace-prefixes.json | Golden update for address num mapping. |
| test/data/parse/en16931/out/credit-note1.json | Golden update for address num mapping. |
| test/data/convert/zatca/standard-usd-invoice.json | New ZATCA convert input fixture (USD doc currency + SAR tax currency). |
| test/data/convert/zatca/standard-invoice.json | New ZATCA convert input fixture (standard invoice). |
| test/data/convert/zatca/standard-debit-note.json | New ZATCA convert input fixture (debit note). |
| test/data/convert/zatca/standard-credit-note.json | New ZATCA convert input fixture (credit note). |
| test/data/convert/zatca/simplified-zero-rated.json | New ZATCA convert input fixture (simplified zero-rated). |
| test/data/convert/zatca/simplified-invoice.json | New ZATCA convert input fixture (simplified invoice). |
| test/data/convert/zatca/simplified-debit-note.json | New ZATCA convert input fixture (simplified debit note). |
| test/data/convert/zatca/simplified-credit-note.json | New ZATCA convert input fixture (simplified credit note). |
| test/data/convert/zatca/out/standard-usd-invoice.xml | New expected UBL output for ZATCA USD invoice. |
| test/data/convert/zatca/out/standard-invoice.xml | New expected UBL output for ZATCA standard invoice. |
| test/data/convert/zatca/out/standard-credit-note.xml | New expected UBL output for ZATCA credit note (invoice-rooted per ZATCA). |
| test/data/convert/zatca/out/simplified-invoice.xml | New expected UBL output for ZATCA simplified invoice. |
| test/data/convert/zatca/out/simplified-debit-note.xml | New expected UBL output for ZATCA simplified debit note. |
| test/data/convert/zatca/out/simplified-credit-note.xml | New expected UBL output for ZATCA simplified credit note. |
| test/data/convert/xrechnung/out/invoice-xr-minimal.xml | Golden update to include ext namespace. |
| test/data/convert/xrechnung/out/invoice-due-date-with-notes.xml | Golden update to include tax currency + dual tax totals. |
| test/data/convert/xrechnung/out/credit-note-xr.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/out/peppol-reverse-charge.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/out/peppol-1.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/out/peppol-1-advance.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/out/invoice-with-prepayment.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/out/invoice-with-item-codes.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/out/invoice-with-document-discount.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/out/invoice-with-delivery.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/out/invoice-with-contract-ref.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/out/invoice-with-charges.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/out/invoice-prices-include-vat.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/out/invoice-partially-paid.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/out/invoice-multi-line.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/out/invoice-minimal.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/out/invoice-intra-comunity.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/out/invoice-cross-border-b2b.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/out/invoice-corrective.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/out/invoice-complete.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/out/credit-note.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/out/credit-note-peppol.xml | Golden update to include ext namespace. |
| test/data/convert/peppol/invoice-prices-include-vat.json | Golden update (line sums/totals + totals block). |
| test/data/convert/peppol-self-billed/out/self-billed-invoice.xml | Golden update to include ext namespace. |
| test/data/convert/peppol-self-billed/out/credit-note-self-billed.xml | Golden update to include ext namespace. |
| test/data/convert/france-extended/out/invoice-fr-extended.xml | Golden update to include ext namespace. |
| test/data/convert/france-extended/out/invoice-fr-extended-detailed.xml | Golden update to include ext namespace. |
| test/data/convert/france-cius/out/invoice-fr-cius.xml | Golden update (apostrophe entity fix + ext namespace). |
| test/data/convert/france-cius/out/credit-note-fr.xml | Golden update (apostrophe entity fix + ext namespace). |
| test/data/convert/en16931/out/invoice-proforma.xml | Golden update to include ext namespace. |
| test/data/convert/en16931/out/invoice-multi-vat-rates.xml | Golden update to include ext namespace. |
| test/data/convert/en16931/out/invoice-minimal.xml | Golden update to include ext namespace. |
| test/data/convert/en16931/out/invoice-export-zero-rated.xml | Golden update to include ext namespace. |
| test/data/convert/en16931/out/invoice-complete.xml | Golden update to include ext namespace. |
| test/data/convert/en16931/out/invoice-attachments.xml | Golden update to include ext namespace. |
| test/data/convert/en16931/out/credit-note-simple.xml | Golden update to include ext namespace. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
alvarolivie
left a comment
alvarolivie
left a comment
There was a problem hiding this comment.
Looks great, I've mentioned a few things. To make this cleaner you might want to create a PR to update GOBL so you remove all those changes that come from updating GOBL and keep this clean.
| // Some countries don't differentiate between Invoice and notes | ||
| // treating all the same. This helper returns the invoice type | ||
| // based on XML name instead of gobl's invoice type key |
There was a problem hiding this comment.
Quick doubt on this. Are we not already doing this somehow on parse?
There was a problem hiding this comment.
Yes, on parse we do infer the invoice type, but from the UNTDID document type code. Without this method, the snippet below would treat invoices as a credit notes triggering an error in ZATCA's systems.
Lines 63 to 67 in c7582f7
| return env, nil | ||
| } | ||
|
|
||
| func (ui *Invoice) goblInvoice(o *options) (*bill.Invoice, error) { |
There was a problem hiding this comment.
split into subfunctions, gocyclo shows when we start making a function too complicated
| if typeCode == nil { | ||
| return bill.InvoiceTypeOther | ||
| } | ||
| if ctx.Is(ContextZATCA) && typeCode.Name != nil { |
There was a problem hiding this comment.
Cant this be handled by the scenario with the new changes in GOBL?
| if val, ok := InvoiceTagMap[typeCode]; ok { | ||
| return val | ||
| func tagCodeParse(typeCode *IDType, ctx Context) []cbc.Key { | ||
| var tags []cbc.Key |
There was a problem hiding this comment.
Same as before, can't we get this information from the scenarios.
There was a problem hiding this comment.
Not sure how. This function populates the gobl scenarios by setting the types and tags that the ZATCA addon expects
| } | ||
|
|
||
| // Zatca specific KSA-11 | ||
| if context.Is(ContextZATCA) && l.Total != nil && len(l.Taxes) > 0 && l.Taxes[0].Percent != nil { |
There was a problem hiding this comment.
I would add a comment on why this is necessary/what is it doing
There was a problem hiding this comment.
Added a more specific comment. Basically, as per BT-KSA-11: ZATCA requires that standard invoices have a TaxTotal line amount.
| if ui.OrderReference == nil { | ||
| ui.OrderReference = &OrderReference{} | ||
| // BT-13: Ensure at least one of BuyerReference or OrderReference is set: PEPPOL-EN16931-R003 | ||
| // Optional to ZATCA |
There was a problem hiding this comment.
instead of doing !zatca, I would set this for the Peppol contexts
There was a problem hiding this comment.
How do you plan to handle this during parse?
There was a problem hiding this comment.
Its handled automatically by ordering_parse:
Lines 73 to 78 in c7582f7
There was a problem hiding this comment.
Just checked this with ZATCA's report and clearance endpoints and it doesn't flag it as an error so we can get rid off the if statement completely.
I initially added it because its marked as optional in ZATCA's specs
| if id.Ext != nil { | ||
| if s := id.Ext[iso.ExtKeySchemeID].String(); s != "" { | ||
| idType.SchemeID = &s | ||
| if s := id.Ext.Get(iso.ExtKeySchemeID).String(); s != "" { |
There was a problem hiding this comment.
Do they not have and ISO scheme? I think NO has something like this as well
There was a problem hiding this comment.
No, these are their only supported identites:
const (
IdentityTypeTIN cbc.Code = "TIN"
IdentityTypeCRN cbc.Code = "CRN"
IdentityTypeMom cbc.Code = "MOM"
IdentityTypeMLS cbc.Code = "MLS"
IdentityType700 cbc.Code = "700"
IdentityTypeSAG cbc.Code = "SAG"
IdentityTypeNational cbc.Code = "NAT"
IdentityTypeGcc cbc.Code = "GCC"
IdentityTypeIqa cbc.Code = "IQA"
IdentityTypePassport cbc.Code = "PAS"
IdentityTypeOTH cbc.Code = "OTH"
)| formattedDate := formatDate(*inv.Payment.Terms.DueDates[0].Date) | ||
| ui.PaymentMeans[0].PaymentDueDate = &formattedDate | ||
| } | ||
| if inv.Preceding != nil && ctx.Is(ContextZATCA) { |
There was a problem hiding this comment.
Not clear why we need this. I would try to explain when calling ctx.is why this guard is needed
There was a problem hiding this comment.
Should have a comment. This is the business rules that forces this.
- BR-KSA-17 Debit and credit note (invoice type code (BT-3) is equal to 383 or 381) must contain the reason (KSA-10) for this invoice type issuing.
|
|
||
| // Go's xml.Marshal encodes single quotes as ', | ||
| // this is a quick fix | ||
| b = bytes.ReplaceAll(b, []byte("'"), []byte("'")) |
There was a problem hiding this comment.
did you double check if this still works with the rest of the code
ZATCA support
ContextZATCAwith ZATCA customization/profile IDs andzatca.V1addon.BuildingNumber,PlotIdentification,CitySubdivisionNamefor ZATCA address requirements.LatestDeliveryDatesupport for ZATCA supply date ranges.extension.goandsignature.gowith UBL extension and document signature types for XAdES embedding.ext,sig,sac,sbc.Cross-cutting changes
tax.ExtensionsAPI migration: map-style access (ext[key]) replaced with.Get(),.Set(),tax.ExtensionsOf()across all files.InvoiceTypeCodechanged fromstringto*IDTypeto support thenameattribute needed by ZATCA.TaxTotalblocks.AddAttachmentsmade public, added UUID field, URL is now optional.Validate()inensureAddons— onlyCalculate()is called.'replaced with literal'in serialized output.Test coverage
simplified-invoice,simplified-credit-note,simplified-debit-note,simplified-zero-rated,standard-invoice,standard-credit-note,standard-debit-note,standard-usd-invoiceDependencies
Requires the
sa-addonbranch ofgobl(pointed to viago.modreplace directive).