REST API for SESAR Statistics

[lulin-song]

On 6/4/19 7:30 AM, Kerstin Lehnert wrote:

• How have sample types changes over the last 3-4 years (= number of samples registered by quarter grouped by object type)

• How many samples are registered by institution over the past 3 years?

• How many active users do we have by year (last 8 years if possible)?

• Number of batches processed by month over past 8 years

[lulin-song]

How many samples are registered by institution over the past 3 years?

• The following REST API is implemented.
  • /v1/igsns/count/institution?start_date=2011-01-01&end_date=2012-01-01

[lulin-song]

How many active users do we have by year (last 8 years if possible)?

Implemented the new user count by year/month REST APIs

• /v1/newusers/count/month
• /v1/newusers/count/year

Implemented the active user count by year/month REST APIs

• /v1/activeusers/count/month
• /v1/activeusers/count/year

[lulin-song]

number of samples registered by quarter grouped by object type

Implemented the following rest api. Get the sample count by sample type

• /v1/igsns/count/sampletype

[lulin-song]

• Number of batches processed by month over past 8 years

There is not enough uploaded batch information in the database. I will leave this for future implementation. See newly created #17

[ramdeensarah]

1. How many samples are registered by institution over the past 3 years?

• If provide a start date and no end date, search fails.
  http://grg-doc-dev.ldeo.columbia.edu:8082/v1/igsns/count/institution?start_date=2019-12-31

• If provide an end date in the future, search is successful
  http://grg-doc-dev.ldeo.columbia.edu:8082/v1/igsns/count/institution?end_date=2020-06-12&start_date=2019-12-31

• Search results for this year seem incorrect. Only lists Ronin and CNRS.
  http://grg-doc-dev.ldeo.columbia.edu:8082/v1/igsns/count/institution?end_date=2020-06-12&start_date=2019-12-31

• Search for samples registered within 2019 tested for hideprivate (set to 1 and null) successfully shows a change in number
  http://grg-doc-dev.ldeo.columbia.edu:8082/v1/igsns/count/institution?end_date=2019-12-12&start_date=2019-01-01

2. How many active/new users do we have by year (last 8 years if possible)?

• Implemented the new user count by year REST APIs
  Question about results This shows 90 new users for 2019, but the results for active users in 2019 was only 83 (see below).

• Implemented the new user count by month REST APIs
  Test appears successful. See note above.

• Implemented the active user count by year REST APIs
  Question about results I assume these are unique visitors, why are the numbers so high in some past years if this is on sesar-test?
  Results
  "2005": "7",
  "2006": "2",
  "2007": "10",
  "2008": "4",
  "2009": "7",
  "2010": "17",
  "2011": "27",
  "2012": "35",
  "2013": "45",
  "2014": "55",
  "2015": "87",
  "2016": "114",
  "2017": "112",
  "2018": "87",
  "2019": "83",
  "2020": "2"

• Implemented the active user count by month REST APIs
  Question about results -- This looks at the users on test-sesar. This seems low and there are months with gaps (was there no work on test-sesar during March and April?). Has Lulin logged into test-sesar this month? Should the count be at least 2?

Results
"2020-06": "1",
"2020-05": "1",
"2020-02": "1",
"2020-01": "2",

3. Implemented the following rest api. Get the sample count by sample type

• Execute with no end date and include unpublished metadata
  Test successful

• Execute with no end date and exclude unpublished metadata
  Test successful, some sample types show decrease in count.

• Execute with end date and include unpublished metadata
  Test successful. If put end date as a future date, test still successful.

• Execute with end date and exclude unpublished metadata
  Test successful, some sample types show decrease in count. If put end date as a future date, test still successful.

[lulin-song]

I updated the REST API and let the API connect to SESAR production database.

If provide a start date and no end date, search fails.

The end date is supposed to be required. I fixed the bug.

Implemented the new user count by year REST APIs
Question about results This shows 90 new users for 2019, but the results for active users in 2019 was only 83 (see below).

• New user count is from sesar_user table. If someone registered a sesar account, it will be counted.
• Active user count is from sample table. If a user registered a sample, the user will be counted as 'active user'. If a new user registered an account but the user did not register any sample. This user is not counted in 'active user'.

This is what it is coded now but we can discuss how 'active user' to be counted.

Implemented the active user count by month REST APIs
Question about results -- This looks at the users on test-sesar. This seems low and there are months with gaps (was there no work on test-sesar during March and April?). Has Lulin logged into test-sesar this month? Should the count be at least 2?

The test database has not been updated for a long time. I don't think I registered any sample in test database in March, April, 2020. I have switched the REST API to connect to production database, so the number reflects the real number now.