Evaluator feedback

[ltfschoen]

After an initial submission on 28th June 2024, I resubmitted the application on 16th July 2024.

Then we received the following feedback about our grant application on 22nd July 2024:

Currently the proposal feels too general and too broad. Would you consider resubmitting with a more limited scope of max two categories and to go deeper into the analyses/ research focus.
Web Technology Consulting: Can you provide more info about this organization? Any links would be helpful.

We resubmitted the application on 28th July 2024, where the changes between the initial submission and the revised submission are as shown here.

We received an email stating the following on 19th August 2024:

Thank you so much for applying to the inaugural JUST Grant. We have pored over your applications for weeks and have been so impressed by you.
Unfortunately, we haven’t selected you for the shortlist. We appreciate the effort you put into your application. If you have further questions about your application, I will do my best to get you a summary of the reviewer feedback. We’d also love to stay in touch as this is only the first grant we’re launching. We’d love for you to apply to future offerings.

We requested evaluator feedback was requested on 21th August 2024

We received evaluator feedback on 11th October 2024, as follows:

"The application failed to provide a clear and precise focus of study, which might prove to delay the research process beyond the grants programme scope of 12 months. Because the proposal lacks organisation and a clear story, the research goals are hard to assess based on the submission.

One of our goals is to support grantees in the publication of their research papers in academic magazines, complying with the Guidelines for Safeguarding Good Scientific Practice of the German Research Foundation (DFG): the publication of the research in your own publication by the end of the programme does not constitute a fulfilment of this requisite. Furthermore, the evaluation committee was unable to identify a connection with any academic institutions or research institutions in the European Union that could credit the work. Presenting your results in your own side-event associated with a Web3 or Open-Source conference is not enough fulfilment for this requirement.

The proposal outlines a clear and sensible methodology, but budget and timeline are not realistic. This was not corrected after re-submission and requires reworking for the plan to be accepted. Therefore, the proposal is too limited to have any long term impact."

[ltfschoen]

"the publication of the research in your own publication by the end of the programme does not constitute a fulfilment of this requisite"

"Presenting your results in your own side-event associated with a Web3 or Open-Source conference is not enough fulfilment for this requirement."

We only proposed publishing it ourselves as a "contingency" last resort incase our attempt to publicise the research paper in an major publication of an academic magazine decided to pull the plug on us. Hence why we stated or if that is not possible here, here, and here. We certainly wouldn't wish it upon ourselves to have to launch it ourselves unless JUST Open Source deemed it absolutely necessary, since we would have had to do that at our own cost, which would be immense.

If we had received that feedback during screening before it was forwarded to the evaluators then we would have simply removed mentioning that "contingency" plan.

It may have been better to have been less proactive by omitting the "Contingency" plan from the application and simply stated reactively in the "Assumptions" section that we assumed JUST Open Source would support us in finding an academic magazine major publication that would allow us to publish and market our research papers in by the end of the program that would comply with the Guidelines for Safeguarding Good Scientific Practice of the German Research Foundation (DFG), and that we assumed JUST Open Source would support us in finding an appropriate major academic event to present our research paper.

[ltfschoen]

"The application failed to provide a clear and precise focus of study"

We received initial screening feedback that recommended we limited the scope of our application to only two focus areas maximum, so we updated it to only focus on one focus topic: "Open-source communities", as shown here

Were the evaluators internally provided with the updated application that we submitted?

[ltfschoen]

Because the proposal lacks organisation

The proposal was "organised" in accordance with the application form that was published by JUST Open Source in June 2024 that we were required to use to "organise" our application. I have requested an updated version of the application form templates, and asked if there is an open-source sample of a compliant submission that applicants could refer to in order to avoid submitting applications that are not in alignment with the expectations of the evaluators at JUST Open Source.

[ltfschoen]

Because the proposal lacks organisation and a clear story, the research goals are hard to assess based on the submission.

Our "story" was primarily mentioned in the "Why are we researching", "What are we researching", and "Objectives" sections.

The "research goals" for assessment that related to the "story" were listed and described in detailed in the "Definition and Measurement of Success" section of our submission.

The stages were precisely mapped to our "Methodology", "Timeline", and "Budget"

[ltfschoen]

"Furthermore, the evaluation committee was unable to identify a connection with any academic institutions or research institutions in the European Union that could credit the work."

The grant application stated here that Sven who is a team member had a connection with the Technische Hochschule Augsburg in Germany, which is both an academic institution and research institution that is part of the European Union, since he is an alumni having graduated with a Master of Computer Science equivalent (Diplom Informatiker FH), Computer Science and Communication Design ("Multimedia") from there, and as such, they could have credited the work.

The grant application also stated here that Sven who is a team member was an research engineer and scientist at a research institution in Germany, which is part of the European Union, working on Framework Programmes (FP7), which was one of the Framework Programmes for Research and Technological Development, and is a funding programme created by the European Union/European Commission to support and foster research in the European Research Area, and as such, they could have credited the work.

[ltfschoen]

"but budget and timeline are not realistic. This was not corrected after re-submission and requires reworking for the plan to be accepted. Therefore, the proposal is too limited to have any long term impact."

The timeline of each grant application submission aligned with the timeline of the JUST Open Source grant period.
It was corrected after re-submission, since it stated here in our application that "The team will endeavour to try to complete the milestones sooner whereever possible, and shall notify the JUST Open Source Stiftung team as soon as possible in case of unforeseen circumstances arising that may cause delays.", to highlight that our proposal would be submitted as soon as possible to have long term impact.

The budget of both our original and revised grant application submissions all requested the maximum budget that was offered by JUST Open Source upon acceptance at the staged milestones that were specified in the "Budget" section.

The only reason why the budget was not realistic was because the actual cost to the applicants associated with the budget of each proposal submission was mostly going to be absorbed by the team members at their own cost, and they were willing to absorb that cost, since the maximum budget that was requested represented 4 weeks of full-time work by both of the two team members based on average OECD annual wages.

The feedback states that the "proposal is too limited", but the proposal was updated and resubmitted to focus on only the one focus topic "in-depth" rather than broadly covering multiple topics.