Skip to content

Handle content_filter errors during document indexing #15

Description

@Allob

DescriptionRetirever indexing uses gpt-4o-mini to get image descriptions. Some documents may trigger content filter error which will look like that in the stream:

{'error': {'inner_error': {'code': 'ResponsibleAIPolicyViolation', 'content_filter_results': {'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': True, 'severity': 'high'}, 'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}}}, 'code': 'content_filter', 'message': "The response was filtered due to the prompt triggering Azure OpenAI's content management policy. Please modify your prompt and retry. To learn more about our content filtering policies please read our documentation: \r\nhttps://go.microsoft.com/fwlink/?linkid=2198766.", 'param': 'prompt', 'type': None}}

The main indicator is code=content_filter.

Currently, such error will be reported to the UI as the following message:

I'm sorry, but I can't process the documents because of the following errors:
<document>: Internal error

Dial RAG should be able to detect such errors in the stream and show some user-friendly message.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions