HelpLogin
© 2022-2025 Spice AI, Inc.
SQL Query ReferenceDocsFAQSupport
PrivacyTerms
Status
Home
Datasets
Models
main
Edit on GitHub
Fork
/evals/scripts
battle_generator.py
Fix formatting/typing so pre-commit hooks pass (#1451) (Not an eval) **One-line summary**: Pre-commit hooks were failing. I identified the main cause, and then fixed all secondary pre-commit issues. I only changed the logic in one place, `oiaevalset.py`. I was having issues with type-hinting and identified that the old `typings` directory was causing the `from openai import OpenAI` import to register as an error. I decided to go through and fix all the issues that appeared in `pre-commit run --all-files`. NOTE: - I changed the logic in `oaievalset.py` by adding a `continue` statement if an `eval` or `eval.key` was missing. - As far as I can tell this should basically never happen, but is correct behavior. - Another option would be to assert that `eval` and `eval.key` are not `None` but forcing an error here doesn't match what I interpret as intended behavior. The manual work involved was mainly: 1. Deleting the `typings` directory, which was interfering with `openai` type-hints (such as `from openai import OpenAI`) 2. Fixing type issues in `oaievalset.py`. 3. Moving the `client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))` line below all the imports. 4. Breaking lines of length >767 into smaller chunks using line continuation. Thus this PR is broken into three parts: 1. Deleting `typings` (first commit) 2. Manually cleaning up issues (middle commits) 3. Applying autofixes from the pre-commit hooks (last commit)
a year ago
modelgraded_generator.py
Fix formatting/typing so pre-commit hooks pass (#1451) (Not an eval) **One-line summary**: Pre-commit hooks were failing. I identified the main cause, and then fixed all secondary pre-commit issues. I only changed the logic in one place, `oiaevalset.py`. I was having issues with type-hinting and identified that the old `typings` directory was causing the `from openai import OpenAI` import to register as an error. I decided to go through and fix all the issues that appeared in `pre-commit run --all-files`. NOTE: - I changed the logic in `oaievalset.py` by adding a `continue` statement if an `eval` or `eval.key` was missing. - As far as I can tell this should basically never happen, but is correct behavior. - Another option would be to assert that `eval` and `eval.key` are not `None` but forcing an error here doesn't match what I interpret as intended behavior. The manual work involved was mainly: 1. Deleting the `typings` directory, which was interfering with `openai` type-hints (such as `from openai import OpenAI`) 2. Fixing type issues in `oaievalset.py`. 3. Moving the `client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))` line below all the imports. 4. Breaking lines of length >767 into smaller chunks using line continuation. Thus this PR is broken into three parts: 1. Deleting `typings` (first commit) 2. Manually cleaning up issues (middle commits) 3. Applying autofixes from the pre-commit hooks (last commit)
a year ago
pattern_identification_generator.py
Fix formatting/typing so pre-commit hooks pass (#1451) (Not an eval) **One-line summary**: Pre-commit hooks were failing. I identified the main cause, and then fixed all secondary pre-commit issues. I only changed the logic in one place, `oiaevalset.py`. I was having issues with type-hinting and identified that the old `typings` directory was causing the `from openai import OpenAI` import to register as an error. I decided to go through and fix all the issues that appeared in `pre-commit run --all-files`. NOTE: - I changed the logic in `oaievalset.py` by adding a `continue` statement if an `eval` or `eval.key` was missing. - As far as I can tell this should basically never happen, but is correct behavior. - Another option would be to assert that `eval` and `eval.key` are not `None` but forcing an error here doesn't match what I interpret as intended behavior. The manual work involved was mainly: 1. Deleting the `typings` directory, which was interfering with `openai` type-hints (such as `from openai import OpenAI`) 2. Fixing type issues in `oaievalset.py`. 3. Moving the `client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))` line below all the imports. 4. Breaking lines of length >767 into smaller chunks using line continuation. Thus this PR is broken into three parts: 1. Deleting `typings` (first commit) 2. Manually cleaning up issues (middle commits) 3. Applying autofixes from the pre-commit hooks (last commit)
a year ago