We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
在处理 SSE 数据推送时,由于 envoy 推送到 Wasm 插件的 response data chunk 可能并不是一个完整的 SSE 消息,所以插件里需要对数据进行缓存,凑够一个完整的 SSE 消息后再进行处理。
目前 ai-proxy 的这部分工作是由各个 provider 单独实现的。考虑到这部分功能很多 provider 都需要对接,可以改成由 ai-proxy 插件主体集中处理。这样 provider 就只需要关注消息本身,不再需要关注分段这些公共逻辑了。
将目前分散在各个 provider 实现中的 SSE 消息分段逻辑集中到 ai-proxy 本体。
相关代码:https://github.com/alibaba/higress/blob/main/plugins/wasm-go/extensions/ai-proxy/provider/qwen.go#L253
The text was updated successfully, but these errors were encountered:
cr7258
Successfully merging a pull request may close this issue.
Why do you need it?
在处理 SSE 数据推送时,由于 envoy 推送到 Wasm 插件的 response data chunk 可能并不是一个完整的 SSE 消息,所以插件里需要对数据进行缓存,凑够一个完整的 SSE 消息后再进行处理。
目前 ai-proxy 的这部分工作是由各个 provider 单独实现的。考虑到这部分功能很多 provider 都需要对接,可以改成由 ai-proxy 插件主体集中处理。这样 provider 就只需要关注消息本身,不再需要关注分段这些公共逻辑了。
How could it be?
将目前分散在各个 provider 实现中的 SSE 消息分段逻辑集中到 ai-proxy 本体。
Other related information
相关代码:https://github.com/alibaba/higress/blob/main/plugins/wasm-go/extensions/ai-proxy/provider/qwen.go#L253
The text was updated successfully, but these errors were encountered: