What to do now that simple-evals won't be updated?

[william-max-byte]

Hi,

I see that simple-evals won't be updated. That's understandable, but what should we do if we need to run evaluations on new tasks or datasets? Is there a recommended way to extend it or any community projects we can use?

[yduwkha]

Hey @william-max-byte,

I ran into the same issue with simple-evals no longer being updated. Found this community fork deep-analysis-research/simple-evals that works great! It keeps everything that was good about simple-evals and adds new tasks/datasets. You should give it a try.

[william-max-byte]

Thanks @yduwkha! Tested deep-analysis-research/simple-evals and it covers everything we need now that simple-evals isn't updating.

Since simple-evals isn't being updated, perhaps we could add a small note in the README about community forks like this? It would help others find active solutions.