Releases: braintrustdata/autoevals
Releases · braintrustdata/autoevals
autoevals Python v0.2.0
What's Changed
- Update README.md by davidatbraintrust in #1
- Setup: Include template *.yaml files in output by Kenny Wong (@wong-codaio) in #2
- Updates to Levenshtein + methods by Kenny Wong (@wong-codaio) in #3
- Make the package isomorphic by Andy Brenneke (@abrenneke) in #5
- Package uses cjs for commonjs, so .js needs to be module by Andy Brenneke (@abrenneke) in #6
- Add file extensions for true ESM by Andy Brenneke (@abrenneke) in #7
- Remove fancy regex from JS implementation by Ankur Goyal (@ankrgyl) in #8
- Update link to Autoevals docs by Kenny (@dashk) in #13
- Prettier: Fixes by Kenny (@dashk) in #11
- Github Action: Trigger pre-commit hooks by Kenny (@dashk) in #12
- Create JS workflow by Kenny (@dashk) in #15
- Create Python workflow by Kenny (@dashk) in #14
- Add numeric and json diff by Ankur Goyal (@ankrgyl) in #16
- Use function calling to parse responses by Ankur Goyal (@ankrgyl) in #18
- Add tracing support by Ankur Goyal (@ankrgyl) in #17
- Remove pydantic as a dependency by Ankur Goyal (@ankrgyl) in #19
- Enable azure openai engines by Edward Atkins (@ecatkins) in #20
- Move score tracing into sdk framework. by Manu Goyal (@manugoyal) in #22
- Update OpenAI version to v4 and add JS tracing by Ankur Goyal (@ankrgyl) in #21
- Disable threading checks and synchronize access with sync/async locks by Ankur Goyal (@ankrgyl) in #23
- Pin openai version to the one we use. by Manu Goyal (@manugoyal) in #24
- Update pydoc-markdown requirement by Manu Goyal (@manugoyal) in #25
- Improve mustache by Ankur Goyal (@ankrgyl) in #26
- Support multiple openai versions in python by Ankur Goyal (@ankrgyl) in #27
- Set the braintrust proxy as the api url to facilitate caching by Ankur Goyal (@ankrgyl) in #33
- Remove validity score by Manu Goyal (@manugoyal) in #34
- Reduce dependencies on SDK span logic. by Manu Goyal (@manugoyal) in #35
- Support python3.8 by Ankur Goyal (@ankrgyl) in #36
- Replace duplicated definitions with
@braintrust/core. by Manu Goyal (@manugoyal) in #37 - Update packaging script locations. by Manu Goyal (@manugoyal) in #38
- Embedding distance by Ankur Goyal (@ankrgyl) in #39
- Rename EmbeddingDistance to EmbeddingSimilarity by Ankur Goyal (@ankrgyl) in #41
- Update bundling to resemble sdk and proxy by Ankur Goyal (@ankrgyl) in #40
- Add Levenshtein by Ankur Goyal (@ankrgyl) in #42
- Add a manifest of all autoevals in typescript and fix export error by Ankur Goyal (@ankrgyl) in #43
- Add turbo.json by Ankur Goyal (@ankrgyl) in #45
- Fix binding issue when tracing inside autoevals. by Manu Goyal (@manugoyal) in #46
- Align with updates to JS tracing API. by Manu Goyal (@manugoyal) in #47
- Updates to track the fact that Scores can be null by Ankur Goyal (@ankrgyl) in #48
- Support OPENAI_BASE_URL in autoevals by Ankur Goyal (@ankrgyl) in #49
- Fix tracing bug. by Manu Goyal (@manugoyal) in #50
- Use openai wrapper by Ankur Goyal (@ankrgyl) in #52
- Add
Sqlscorer to python by Austin Moehle (@aphinx) in #53 - Merge
node-specific env initialization intoenv.ts. by Manu Goyal (@manugoyal) in #54 - README fix + use enums to improve classification accuracy by Ankur Goyal (@ankrgyl) in #55
- Fall back to BRAINTRUST_API_KEY if OPENAI_API_KEY is not set. by Manu Goyal (@manugoyal) in #57
- Don't swallow errors into scores. by Manu Goyal (@manugoyal) in #56
- Bump core version by Ankur Goyal (@ankrgyl) in #58
- fix: move tsx and @types/node to devDependencies by Travis Fischer (@transitive-bullshit) in #62
- Add list comparison metric by Ankur Goyal (@ankrgyl) in #59
- RAGAS retrieval metrics by Ankur Goyal (@ankrgyl) in #60
- Added Moderation as a scorer by Kenny (@dashk) in #63
- Add generation and end-to-end RAGAS metrics by Austin Moehle (@aphinx) in #64
- fix matter => manner typo by bardia-pourvakil in #66
- Add Valid JSON as a scorer by tara (@tara-nagar) in #67
- Update autoevals README to remove usage of
experiment.logby Austin Moehle (@aphinx) in #68 - Update openai package by Ankur Goyal (@ankrgyl) in #69
- Fix list contains DX by Ankur Goyal (@ankrgyl) in #71
- Some housekeeping updates by Ankur Goyal (@ankrgyl) in #72
- Add support for partial syntax by Ankur Goyal (@ankrgyl) in #73
- Readme fixes by Dan Lee (@danielericlee) in #75
- Add public evals by Ankur Goyal (@ankrgyl) in #74
- Update links for npm package by tara (@tara-nagar) in #76
- update pre-commit prettier to 3.3.2 by john (@j13huang) in #77
- Add ExactMatch scorer by Ankur Goyal (@ankrgyl) in #79
- Propagate args to embedding metrics by Ankur Goyal (@ankrgyl) in #83
- Change proxy URL by Ankur Goyal (@ankrgyl) in #85
- Fix rationale by Ankur Goyal (@ankrgyl) in #88
- Add scorer purpose by Ankur Goyal (@ankrgyl) in #87
- Propagate prompt template through manifest by Ankur Goyal (@ankrgyl) in #86
- Fix a few typos found via doc tests by Ankur Goyal (@ankrgyl) in #92
- Support AzureOpenAI service by Ben Perlmutter (@mongodben) in #91
- Add customizable embedding model to
AnswerRelevancymetric by Ben Perlmutter (@mongodben) in #94 - Ensure EmbeddingSimilarity scores do not exceed 1 by Dan Lee (@danielericlee) in #95
- Fix ExactMatch export by Ankur Goyal (@ankrgyl) in #97
- Fix embedding similarity crash by Austin Moehle (@aphinx) in #90
- Parse JSON in JSONDiff by Ankur Goyal (@ankrgyl) in #99
- feature: configurable python (OpenAI) client by Olmo Maldonado (@ibolmo) in #102
- Support non-strings in valid json by Ankur Goyal (@ankrgyl) in #103
- bump by Sachin Padmanabhan (@sachinpad) in #106
- update version by Olmo Maldonado (@ibolmo) in #107
- bump autoevals version by Sachin Padmanabhan (@sachinpad) in #108
- wrap openai moderations by Sachin Padmanabhan (@sachinpad) in #109
- escape objects to stringified in autoevals by Sachin Padmanabhan (@sachinpad) in #110
- Upgrade turborepo by Manu Goyal (@manugoyal) in #111
- Update autoevals manifest by Ankur Goyal (@ankrgyl) in #112
- Update docs to show usage with custom AI models through the AI proxy by Hurshal Patel (@choochootrain) in #113
- Improve readme for AI proxy examples based on feedback by Hurshal Patel (@choochootrain) in #114
- Pass through azure open ai params to cachedChatCompletion by Olmo Maldonado (@ibolmo) in #117
- bump version by Olmo Maldonado (@ibolmo) in #119
- Replace Levenshtein with polyeven by Zubin Pahuja (@zpahuja) in #122
- Bump the npm_and_yarn group across 1 directory with 2 updates by Dependabot (@dependabot)[bot] in #123
- Bump tsup from 8.3.5 to 8.4.0 in the npm_and_yarn group across 1 directory by Dependabot (@dependabot)[bot] in #124
- Bump tsup by Manu Goyal (@manugoyal) in #125
- Uniform improved
client=(direct & global) option for Python and JS by Olmo Maldonado (@ibolmo) in #121 - Revert "Unif...