GISF activity not visible in D-Portal

[siwhitehouse]

Brief Description
GISF is a new reporting organisation, using IATI Publisher. They published their first activity on Monday 20th April (https://iatipublisher-prod.s3.amazonaws.com/xml/mergedActivityXml/gisf-activities.xml). It is available in IATI tools such as the IATI Datastore, IATI Tables and the Dashboard.

It doesn't have any Validation errors. GISF don't appear in the Database logs in d-Portal or in https://notshi.github.io/cleanerr/ as far as I can see.

d-Preview processes the activity

D-Portal appears to have identified the file on the 22nd April, but browsing to the organisation page shows 0 activities and a search for the activity is unsuccessful.

Severity
Medium

Issue Location
https://d-portal.iatistandard.org/ctrack.html#view=search&reporting_ref=GB-COH-14937701

[siwhitehouse]

This activity is now visible in d-Portal, see https://d-portal.iatistandard.org/ctrack.html?&reporting_ref=GB-COH-14937701#view=act&aid=GB-COH-14937701-GISF-FCDO-0001

[IsabelBirds]

I have the same issue with another organisation - their org file is being pulled into d-portal from IATI Publisher, but not their activities. They have also reported other organisations having the same issue, I've asked for details.

The activities have been live for nearly a month, and I cannot see any reason for them not to be in d-portal. They're in all other IATI tooling.

• https://dashboard.iatistandard.org/publishers/fb/#p_files
• https://d-portal.iatistandard.org/ctrack.html?reporting_ref=SE-ON-802004-7737#view=main

@xriss @notshi Please could you take a look into this? We're seeing an uptick in issues cc @stevieflow

[xriss]

This one seems fine now?

The last one was a hard to spot data error, followed by me accidentally breaking data while trying to work out what was happening.

Looks like this mistake was bigger than I thought, ended up with more xml files with uppercase letters (slugname) when I was trying to work out why the file was not importing.

So most problems are probably related to this snapshot of old data.

Have changed the import step so this should not happen again after tonight, will take a look and make sure tomorrow.

[leilafarsani]

Hi @xriss - Thanks for looking into this.

I’m seeing the same behaviour for another organisation: Svalorna Latinamerika (SE-ON-802003-3083).
All the activities are not showing (0 activities), even though:

• activities have been published for ~2+ weeks
• no validation errors
• data is present in Datastore and IATI Tables
• activities render correctly in d-portal preview

Given your note about the import/slugname issue and old snapshots, this may be another dataset affected by that ingestion problem.

Links:

D-portal Org: https://d-portal.iatistandard.org/ctrack.html?publisher=SE-ON-802003-3083&nocache=1#view=main
Dashboard: https://dashboard.iatistandard.org/publishers/svalornala/

IATI Account: https://account.iatistandard.org/en/data/organisation/36e5fb06-2eb9-6fb6-72f4-69809f12bb3a
Activities: https://iatipublisher-prod.s3.amazonaws.com/xml/mergedActivityXml/svalornala-activities.xml

[xriss]

Looks fixed now?

There are two separate problems, the first one was not a problem just an emdash, the second one is caused by me trying to fix/diagnose the first non-problem.

I was checking all sorts of things before I spotted the emdash.

One of the things I tried was manual download and import which mistakenly created duplicate files but with capital letters. This was the big mistake, I used the raw slugnames without forcing them to lowercase. This is the cause of the new problem, these files hung around and replaced newer data with the same lowercase name ( randomly ).

All the debug/logs/etc are case insensitive so it was only by checking the actual files in the case sensitive filesystem that I realised the cause of the second problem.

Initially I thought it was just the emdash files but it turned out there where some other random files that I must have also downloaded when checking imports.

So the first partial fix of the second problem was just a manual fix of some files.

I have now changed the import and just stopped letting any old files linger so it should not happen again in the same way.