Skip to content

feat: async support for process functions #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

oseoin
Copy link
Collaborator

@oseoin oseoin commented Jun 9, 2025

Proposed changes

Adds support for running process_input and process_response as async. Have performed performance tests with both options and have not seen any meaningful difference between the two. pyright seems to be happy with the implementation.

Checklist

Before creating a PR, run through this checklist and mark each as complete.

  • I have written my commit messages in the Conventional Commits format.
  • I have read the CONTRIBUTING doc
  • I have added tests (when possible) that prove my fix is effective or that my feature works
  • I have checked that all unit tests pass after adding my changes
  • I have updated necessary documentation
  • I have rebased my branch onto main
  • I will ensure my PR is targeting the main branch and pulling from my branch from my own fork

@oseoin oseoin changed the title feat: async support for process_ feat: async support for process functions Jun 9, 2025
@oseoin oseoin added this to the 1.1.0 milestone Jun 9, 2025
@oseoin oseoin linked an issue Jun 9, 2025 that may be closed by this pull request
@oseoin oseoin marked this pull request as ready for review June 11, 2025 09:43
@oseoin oseoin requested a review from a team as a code owner June 11, 2025 09:43
@@ -225,6 +226,13 @@ def __init_subclass__(cls, **kwargs):
"The DEPRECATED 'process' method must not be implemented "
"alongside 'process_input' or 'process_response'."
)
if is_process_overridden and inspect.iscoroutinefunction(cls.process):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can suggest to unwrap decorators for a little bit more safety

Suggested change
if is_process_overridden and inspect.iscoroutinefunction(cls.process):
if is_process_overridden and inspect.iscoroutinefunction(inspect.unwrap(cls.process)):

return Result()

self.assertIsNotNone(AsyncImplementedProcessor())

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
async def test_async_message(self):
prompt = TEST_REQ_INPUT.model_dump_json()
metadata = {"key": "value"}
request = fake_multipart_request(
prompt=prompt,
metadata=metadata,
parameters={"modify": True, "annotate": True},
)
class AsyncInputProcessor(Processor):
def __init__(self):
super().__init__(
name="async-input-processor",
namespace="fake",
signature=INPUT_ONLY_SIGNATURE,
version="v1",
)
async def process_input(self, prompt, metadata, parameters, request) -> Result:
prompt.messages.append(Message(content="Test message"))
return Result(modified_prompt=prompt)
processor = AsyncInputProcessor()
response = await processor.handle_request(request)
self.assertStatusCodeEqual(response, HTTP_200_OK)
content = await self.buffer_response(response)
multipart = MultipartDecoderHelper(
content=content, content_type=response.headers["Content-Type"]
)
self.assertTrue(
multipart.has_prompt(),
"prompt should be in the response",
)
multipart_prompt = multipart.prompt
assert "Test message" in multipart_prompt.content

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature] Abstract process_input and process_response should be async
3 participants