Expose `input_type` option at root level for `text_embedding` task type in Perform Inference API #122638

ymao1 · 2025-02-14T18:45:17Z

Summary

Exposes the input_type parameter as a root level field in the Perform Inference API for text-embedding task types. Previously, this setting was available for certain services as a task setting. There are several services that do not accept an input_type specification or only a subset of values. Invalid inputs for these services will throw a validation exception.

Because we use the InferenceAction internally in several places, we want to ensure that calling perform inference with an input_type on the services that don't accept input_type parameters do not throw exceptions when used internally. Toward this end, this PR adds a InputType.INTERNAL_SEARCH and InputType.INTERNAL_INGEST that will bypass validation.

Verification

✅ - manually verified with credentials
🛑 - have not been able to manually verify

The following services already passed the input_type parameter:

✅ AlibabaCloud AI Search - docs
Only InputType.SEARCH and InputType.INGEST are supported so InputType.CLASSIFICATION and InputType.CLUSTERING will throw validation exceptions

✅ Cohere - docs
✅ Google Vertex AI - docs
✅ Jina AI - docs
✅ Voyage AI - docs
Only InputType.SEARCH and InputType.INGEST are supported so InputType.CLASSIFICATION and InputType.CLUSTERING will throw validation exceptions

The following services were updated to pass an input type parameter:

✅ Amazon Bedrock - docs

cohere provider updated to pass input_type to text embedding task.
amazontitan provider does not accept an input type parameter so updated to throw validation exception when specified.

✅ Azure OpenAI - docs
The docs say that input_type is an accepted string parameter but does not list the allowed values. Upon testing, it seems that this API accepts any arbitrary value for input_type without throwing any errors or changing the embedding results.

🛑 Azure AI Studio - docs

✅ Google AI Studio - docs
Updated to pass taskType parameter when modelId=embedding-001. Throws validation exception otherwise.

The following services do not support an input type parameter and have been updated to throw a validation exception if input_type is specified.

✅ Hugging Face - docs
✅ Mistral - docs
✅ OpenAI - docs
✅ IBM Watsonx - docs

…7856

davidkyle · 2025-03-05T09:59:03Z

server/src/main/java/org/elasticsearch/inference/InputType.java

@@ -19,7 +21,9 @@ public enum InputType {
    SEARCH,
    UNSPECIFIED,
    CLASSIFICATION,
-    CLUSTERING;
+    CLUSTERING,
+    INTERNAL_SEARCH,


…7856

ymao1 · 2025-03-11T13:23:25Z

run elasticsearch-ci/part-1

ymao1 · 2025-03-11T18:46:03Z

server/src/main/java/org/elasticsearch/inference/InputType.java

@@ -29,4 +35,20 @@ public String toString() {
    public static InputType fromString(String name) {
        return valueOf(name.trim().toUpperCase(Locale.ROOT));
    }
+
+    public static InputType fromRestString(String name) {
+        var inputType = InputType.fromString(name);


Check that the values passed from the REST API do not include the internal values

ymao1 · 2025-03-11T18:46:33Z

...plugin/core/src/main/java/org/elasticsearch/xpack/core/inference/action/InferenceAction.java

@@ -199,6 +199,14 @@ public ActionRequestValidationException validate() {
                }
            }

+            if (taskType.equals(TaskType.TEXT_EMBEDDING) == false


Throws validation exception if input_type is used with task types other than text-embedding or if task type is not specified

ymao1 · 2025-03-11T18:51:45Z

...g/elasticsearch/xpack/inference/services/cohere/embeddings/CohereEmbeddingsTaskSettings.java

@@ -88,30 +83,26 @@ public static CohereEmbeddingsTaskSettings fromMap(Map<String, Object> map) {
     * originalSettings.
     * @param originalSettings the settings stored as part of the inference entity configuration
     * @param requestTaskSettings the settings passed in within the task_settings field of the request
-     * @param requestInputType the input type passed in the request parameters
     * @return a constructed {@link CohereEmbeddingsTaskSettings}
     */
    public static CohereEmbeddingsTaskSettings of(


Previously, if a requestInputType was passed in (only used internally), it would take precedence over request task settings input type, which would take precedence over persisted task settings input type, and the task settings input type would be set to the "winner".

This caused some issues when trying to do this across services, some of which do not allow input type in task settings and some of which don't allow task settings at all.

For this reason, this PR refactored to set and validate the request input type at the service level, which is then passed directly to the service's request manager.

elasticsearchmachine · 2025-03-11T19:27:39Z

Hi @ymao1, I've created a changelog YAML for you.

elasticsearchmachine · 2025-03-11T23:16:40Z

Pinging @elastic/ml-core (Team:ML)

…7856

davidkyle

LGTM

...src/main/java/org/elasticsearch/xpack/inference/services/azureopenai/AzureOpenAiService.java

.../plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/ServiceUtils.java

…inference/services/azureopenai/AzureOpenAiService.java Co-authored-by: David Kyle <[email protected]>

…inference/services/ServiceUtils.java Co-authored-by: David Kyle <[email protected]>

ymao1 · 2025-03-17T18:33:51Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Questions ?

Please refer to the Backport tool documentation

…pe in Perform Inference API (elastic#122638) * wip * wip * [CI] Auto commit changes from spotless * Adding internal input types * [CI] Auto commit changes from spotless * Throwing validation exception for services that don't support input type * linting * hugging face * voyage ai * google ai studio * bedrock updates * Fixing tests * Fixing tests * Fixing tests * bedrock updates * elasticsearch * azure openai * [CI] Auto commit changes from spotless * Refactoring all the things * [CI] Auto commit changes from spotless * Everything compiles * spotless * external actions tests * external request tests * service tests * Fixing integration tests * Cleanup * Update docs/changelog/122638.yaml * Cleanup * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/azureopenai/AzureOpenAiService.java Co-authored-by: David Kyle <[email protected]> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/ServiceUtils.java Co-authored-by: David Kyle <[email protected]> * PR feedback --------- Co-authored-by: elasticsearchmachine <[email protected]> Co-authored-by: David Kyle <[email protected]> (cherry picked from commit 0b6a3cd) # Conflicts: # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/external/request/alibabacloudsearch/AlibabaCloudSearchEmbeddingsRequestEntity.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/external/request/alibabacloudsearch/AlibabaCloudSearchSparseRequestEntity.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/external/request/cohere/CohereEmbeddingsRequestEntity.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/external/request/voyageai/VoyageAIEmbeddingsRequestEntity.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/action/cohere/CohereEmbeddingsActionTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/action/googleaistudio/GoogleAiStudioEmbeddingsActionTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/action/huggingface/HuggingFaceActionCreatorTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/action/voyageai/VoyageAIActionCreatorTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/action/voyageai/VoyageAIEmbeddingsActionTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/request/alibabacloudsearch/AlibabaCloudSearchEmbeddingsRequestEntityTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/googleaistudio/GoogleAiStudioServiceTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/voyageai/VoyageAIServiceTests.java

…pe in Perform Inference API (#122638) (#125049) * wip * wip * [CI] Auto commit changes from spotless * Adding internal input types * [CI] Auto commit changes from spotless * Throwing validation exception for services that don't support input type * linting * hugging face * voyage ai * google ai studio * bedrock updates * Fixing tests * Fixing tests * Fixing tests * bedrock updates * elasticsearch * azure openai * [CI] Auto commit changes from spotless * Refactoring all the things * [CI] Auto commit changes from spotless * Everything compiles * spotless * external actions tests * external request tests * service tests * Fixing integration tests * Cleanup * Update docs/changelog/122638.yaml * Cleanup * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/azureopenai/AzureOpenAiService.java Co-authored-by: David Kyle <[email protected]> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/ServiceUtils.java Co-authored-by: David Kyle <[email protected]> * PR feedback --------- Co-authored-by: elasticsearchmachine <[email protected]> Co-authored-by: David Kyle <[email protected]> (cherry picked from commit 0b6a3cd) # Conflicts: # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/external/request/alibabacloudsearch/AlibabaCloudSearchEmbeddingsRequestEntity.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/external/request/alibabacloudsearch/AlibabaCloudSearchSparseRequestEntity.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/external/request/cohere/CohereEmbeddingsRequestEntity.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/external/request/voyageai/VoyageAIEmbeddingsRequestEntity.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/action/cohere/CohereEmbeddingsActionTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/action/googleaistudio/GoogleAiStudioEmbeddingsActionTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/action/huggingface/HuggingFaceActionCreatorTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/action/voyageai/VoyageAIActionCreatorTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/action/voyageai/VoyageAIEmbeddingsActionTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/external/request/alibabacloudsearch/AlibabaCloudSearchEmbeddingsRequestEntityTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/googleaistudio/GoogleAiStudioServiceTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/voyageai/VoyageAIServiceTests.java

…pe in Perform Inference API (elastic#122638) * wip * wip * [CI] Auto commit changes from spotless * Adding internal input types * [CI] Auto commit changes from spotless * Throwing validation exception for services that don't support input type * linting * hugging face * voyage ai * google ai studio * bedrock updates * Fixing tests * Fixing tests * Fixing tests * bedrock updates * elasticsearch * azure openai * [CI] Auto commit changes from spotless * Refactoring all the things * [CI] Auto commit changes from spotless * Everything compiles * spotless * external actions tests * external request tests * service tests * Fixing integration tests * Cleanup * Update docs/changelog/122638.yaml * Cleanup * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/azureopenai/AzureOpenAiService.java Co-authored-by: David Kyle <[email protected]> * Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/ServiceUtils.java Co-authored-by: David Kyle <[email protected]> * PR feedback --------- Co-authored-by: elasticsearchmachine <[email protected]> Co-authored-by: David Kyle <[email protected]>

wip

3b418ae

elasticsearchmachine added the v9.1.0 label Feb 14, 2025

ymao1 and others added 13 commits February 21, 2025 08:42

wip

5d020a7

Merge branch 'main' of github.com.:elastic/elasticsearch into es-117856

cceb308

Merge branch 'main' of github.com.:elastic/elasticsearch into es-117856

b515e84

[CI] Auto commit changes from spotless

625a5a2

Merge branch 'main' of github.com.:elastic/elasticsearch into es-117856

c064e26

Adding internal input types

f6dbf51

Merge branch 'es-117856' of github.com.:ymao1/elasticsearch into es-11…

6a25d08

…7856

[CI] Auto commit changes from spotless

b7c2481

Throwing validation exception for services that don't support input type

2d2b6db

Merge branch 'es-117856' of github.com.:ymao1/elasticsearch into es-11…

3dc38f7

…7856

linting

356d546

hugging face

a143af1

voyage ai

7f20d32

davidkyle reviewed Mar 5, 2025

View reviewed changes

ymao1 added 4 commits March 5, 2025 13:57

google ai studio

d6c2464

Merge branch 'main' of github.com.:elastic/elasticsearch into es-117856

f63e852

bedrock updates

f9962ce

Fixing tests

2905643

ymao1 changed the title ~~wip~~ Expose input_type option at root level for text_embedding task type in Perform Inference API Mar 6, 2025

ymao1 and others added 9 commits March 6, 2025 12:41

Fixing tests

f47538c

Fixing tests

5b4ee68

bedrock updates

be17339

elasticsearch

bba8ac5

azure openai

bf32efd

Merge

3dbeaff

[CI] Auto commit changes from spotless

c3c9cef

Refactoring all the things

cc96e9a

Merge branch 'es-117856' of github.com.:ymao1/elasticsearch into es-11…

fbc8791

…7856

Merge branch 'main' of github.com.:elastic/elasticsearch into es-117856

4ad3214

ymao1 added 2 commits March 11, 2025 15:25

Cleanup

9f744db

Merge branch 'main' of github.com.:elastic/elasticsearch into es-117856

4564680

ymao1 added >enhancement :ml Machine learning Team:ML Meta label for the ML team v8.19.0 labels Mar 11, 2025

ymao1 self-assigned this Mar 11, 2025

ymao1 commented Mar 11, 2025

View reviewed changes

Update docs/changelog/122638.yaml

aef7da8

ymao1 marked this pull request as ready for review March 11, 2025 23:16

ymao1 added 2 commits March 12, 2025 08:13

Merge

6cbd41e

Merge branch 'es-117856' of github.com.:ymao1/elasticsearch into es-11…

f364ad3

…7856

davidkyle approved these changes Mar 12, 2025

View reviewed changes

...src/main/java/org/elasticsearch/xpack/inference/services/azureopenai/AzureOpenAiService.java Outdated Show resolved Hide resolved

.../plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/ServiceUtils.java Outdated Show resolved Hide resolved

ymao1 and others added 6 commits March 17, 2025 09:15

Merging in main

113cf3d

Cleanup

cb8d337

Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/…

ef6074b

…inference/services/azureopenai/AzureOpenAiService.java Co-authored-by: David Kyle <[email protected]>

Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/…

61c8d7b

…inference/services/ServiceUtils.java Co-authored-by: David Kyle <[email protected]>

PR feedback

61091c3

Merge branch 'main' of github.com.:elastic/elasticsearch into es-117856

588c6fb

ymao1 merged commit 0b6a3cd into elastic:main Mar 17, 2025
17 checks passed

ymao1 mentioned this pull request Mar 17, 2025

[8.x] Expose input_type option at root level for text_embedding task type in Perform Inference API (#122638) #125049

Merged

lcawl mentioned this pull request Mar 19, 2025

Add missing inference API details elastic/elasticsearch-specification#3821

Closed

16 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Expose `input_type` option at root level for `text_embedding` task type in Perform Inference API #122638

Expose `input_type` option at root level for `text_embedding` task type in Perform Inference API #122638

ymao1 commented Feb 14, 2025 •

edited

Loading

davidkyle Mar 5, 2025

ymao1 commented Mar 11, 2025

ymao1 Mar 11, 2025

ymao1 Mar 11, 2025

ymao1 Mar 11, 2025

elasticsearchmachine commented Mar 11, 2025

elasticsearchmachine commented Mar 11, 2025

davidkyle left a comment

ymao1 commented Mar 17, 2025

Expose input_type option at root level for text_embedding task type in Perform Inference API #122638

Expose input_type option at root level for text_embedding task type in Perform Inference API #122638

Conversation

ymao1 commented Feb 14, 2025 • edited Loading

Summary

Verification

davidkyle Mar 5, 2025

Choose a reason for hiding this comment

ymao1 commented Mar 11, 2025

ymao1 Mar 11, 2025

Choose a reason for hiding this comment

ymao1 Mar 11, 2025

Choose a reason for hiding this comment

ymao1 Mar 11, 2025

Choose a reason for hiding this comment

elasticsearchmachine commented Mar 11, 2025

elasticsearchmachine commented Mar 11, 2025

davidkyle left a comment

Choose a reason for hiding this comment

ymao1 commented Mar 17, 2025

💚 All backports created successfully

Questions ?

Expose `input_type` option at root level for `text_embedding` task type in Perform Inference API #122638

Expose `input_type` option at root level for `text_embedding` task type in Perform Inference API #122638

ymao1 commented Feb 14, 2025 •

edited

Loading