Skip to content

Ability to Test Model Components in AGS #5728

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Tracked by #4006
victordibia opened this issue Feb 26, 2025 · 0 comments · Fixed by #5963
Closed
Tracked by #4006

Ability to Test Model Components in AGS #5728

victordibia opened this issue Feb 26, 2025 · 0 comments · Fixed by #5963
Assignees
Milestone

Comments

@victordibia
Copy link
Collaborator

victordibia commented Feb 26, 2025

We have a few valuable debugging capabilities in AGS today

  • Abilitity to test agent teams in teambuilder
  • Automatic validation of entire team

However, validation only gives us static checks for agent behavior (e.g a spec is valid .. valid provider, etc), it tells us nothing about actual functional behaviour e.g., an agent might have the right provider, but have missing required fields eg a model client, or a model client look good but its baseulr incorrect etc.

This could be useful in the Gallery where developers can create components , test them and then reliably reuse them.
This PR is meant to add the ability to run a component as a form of live test.

Currently will focus on the ability to test model clients. Others can be tested relatively easily (e.g., agents) by using the test teams already available.

How

  • Extend /validate endpoint to validate/test/component= component
  • For each coponent determine how we will test
    • agent - task.run("what is x")
    • model - model.create ("hi there")
    • tools - tool.run ...
@victordibia victordibia self-assigned this Feb 26, 2025
@victordibia victordibia added this to the 0.4.2-studio milestone Mar 1, 2025
@victordibia victordibia changed the title Ability to Test all Components in AGS Ability to Test Components in AGS Mar 16, 2025
@victordibia victordibia changed the title Ability to Test Components in AGS Ability to Test Model Components in AGS Mar 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant