Skip to content

[ML] Expose the Input Type option for the text_embedding task in the Inference API #117856

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
davidkyle opened this issue Dec 2, 2024 · 2 comments · Fixed by #122638
Closed
Assignees
Labels
>enhancement Feature:GenAI Features around GenAI :ml Machine learning Team:ML Meta label for the ML team

Comments

@davidkyle
Copy link
Member

Description

Many of the integrated inference services have an input type option for text embeddings which modifies the resulting embedding for a specific use. The inference API has an InputType class for internal use and depending on where the text embedding call is made the Inference API will pick either the search or ingest input type.

When ingesting documents with an ingest pipeline or into a semantic text field the ingest type is used, when used in a query the search type is used automatically, this way users don't have to worry about selecting the right type and will get consistent results.

Exposing the input_type option in the POST _inference API is a natural extension that gives users more control over their embeddings. This change should not alter the search or ingest behaviours.

@davidkyle davidkyle added :ml Machine learning >enhancement labels Dec 2, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

@elasticsearchmachine elasticsearchmachine added the Team:ML Meta label for the ML team label Dec 2, 2024
@ymao1
Copy link
Contributor

ymao1 commented Feb 10, 2025

@davidkyle So this is updating the Perform Inference API to accept

POST _inference/text_embedding/my-endpoint
{
  "input": "The sky above the port was the color of television tuned to a dead channel.",
  "input_type": "ingest"
}

while still accepting the previous format of

POST _inference/text_embedding/my-endpoint
{
  "input": "The sky above the port was the color of television tuned to a dead channel.",
  "task_settings": {
    "input_type": "ingest"
  }
}

Is that correct? Should we return a validation error if both are specified? Or always use the input type in the task settings?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>enhancement Feature:GenAI Features around GenAI :ml Machine learning Team:ML Meta label for the ML team
Projects
None yet
3 participants