Skip to content

Add detections-vehicles dataset to backend#83

Merged
olafveerman merged 7 commits into
developfrom
feature/add-detections-vehicle-dataset
Jan 13, 2021
Merged

Add detections-vehicles dataset to backend#83
olafveerman merged 7 commits into
developfrom
feature/add-detections-vehicle-dataset

Conversation

@leothomas

@leothomas leothomas commented Nov 17, 2020

Copy link
Copy Markdown
Contributor

What I did:

Added detections-vehicles dataset - addresses covid-dashboard issue #440

How I did it:

Extracted scene_ids for each feature in the two geojsons currently in S3 and built + uploaded a detections_scenes.csv file.

Created a new dataset metadata file called detections-vehicles.json which is very similar to detection-plane.json and detections-ship.json.

Added detections-vehicle to all datasets which has detections-ship and detection-plane as exclusive datasets

How you can test it:

Run docker-compose up --build

  • Hit http://localhost:8000/v1/datasets notice a new dataset for detections-vehicles is present (with {spotlightId} in url, since no spotlight has been specified)
  • Hit http://localhost:8000/v1/datasets/la notice that the detections-vehicles is still present
  • Hist http://localhost:8000/v1/datasets/ny notices that the detections-vehicles is no longer present

@olafveerman @drewbo would love some feedback on the detections-vehicles.json metadata files (I copy-pasted the description from detections-ships.json, not sure if the info field is missing anything)

@danielfdsilva tagging you because the format of the geojsons in s3 seems different from the other detection-* datasets. Let me know if you need anything in order to implement in the front end

@lillythomas fyi

@leothomas leothomas added enhancement New feature or request data labels Nov 17, 2020
@leothomas leothomas self-assigned this Nov 17, 2020
@olafveerman

Copy link
Copy Markdown
Contributor

Thanks @leothomas

  • detections-ships.json seems to check out. My only comment is that the interface shows detections in orange (not red), I can update the description for all ML detections
  • had a quick look at the GeoJSON, and didn't notice any structural difference that would trip up the frontend

Running this with 1.8.0 of the dashboard, I noticed a couple of issues. Unsure whether they should be solved on the API, or frontend, but raising them here:

  • background imagery is not available
  • there are vehicle detections for two dates. The domain in the API response is: "domain": [ "2020-01-09T00:00:00Z", "2020-07-08T00:00:00Z" ]. Instead of treating these as two discrete dates, the frontend interprets this as a range daily dates with the domain being the start and end date

danielfdsilva added a commit to NASA-IMPACT/covid-dashboard that referenced this pull request Nov 18, 2020
@danielfdsilva

Copy link
Copy Markdown
Contributor

Tested this locally and the data seems to show up correctly.
I encountered some errors, but I think it's because I don't have a planet key.

I did notice the domain error reported by @olafveerman, and this is a problem with the frontend. We were considering domains with 2 dates to be a range according to the time unit. This has been the case and worked until this moment. The very useful isPeriodic property of the api response was not being used.

Long story short, here's the PR with the fix: NASA-IMPACT/covid-dashboard#455

@leothomas

Copy link
Copy Markdown
Contributor Author

Fixed the issue with detections-vehicles missing background scenes (issue was due to the fact that detections-vehicles relies on Skysat imagery, which is not available through the Planet tile endpoint (detections-plane and detections-ship rely on Planet imagery which is available through the Planet tile endpoint).

@olafveerman re: observations colour - I'm not seeing anywhere on the backend to specify the colours of the observations. If that's not fixable through the front end, I'm happy to update the info field of each of the detections-* metadata files to specify that the observations are show in orange (instead of red)

@olafveerman

Copy link
Copy Markdown
Contributor

This looks good! Holding off on merging for a couple of days, until the Story feature has been launched and stable.

@leothomas

Copy link
Copy Markdown
Contributor Author

@olafveerman This PR has been open for a while. Can we merge, now that the Story feature has been launched?

@olafveerman olafveerman merged commit 3c2d9f7 into develop Jan 13, 2021
@olafveerman olafveerman deleted the feature/add-detections-vehicle-dataset branch January 13, 2021 10:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants