Skip to content

Add elasticsearch enrich policy datasource #293

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Mar 23, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
## [Unreleased]
### Added
- New resource `elasticstack_elasticsearch_enrich_policy` to manage enrich policies ([#286](https://github.com./elastic/terraform-provider-elasticstack/pull/286)) ([Enrich API](https://www.elastic.co/guide/en/elasticsearch/reference/current/enrich-apis.html))
- New data source `elasticstack_elasticsearch_enrich_policy` to read enrich policies ([#293](https://github.com./elastic/terraform-provider-elasticstack/pull/293)) ([Enrich API](https://www.elastic.co/guide/en/elasticsearch/reference/current/enrich-apis.html))
- Add 'mapping_coerce' field to index resource ([#229](https://github.com./elastic/terraform-provider-elasticstack/pull/229))
- Add 'min_*' conditions to ILM rollover ([#250](https://github.com./elastic/terraform-provider-elasticstack/pull/250))
- Add support for Kibana connections ([#226](https://github.com./elastic/terraform-provider-elasticstack/pull/226))
Expand Down
84 changes: 84 additions & 0 deletions docs/data-sources/elasticsearch_enrich_policy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
---
subcategory: "Enrich"
page_title: "elasticstack_elasticsearch_enrich_policy Data Source - terraform-provider-elasticstack"
description: |-
Returns information about an enrich policy. See: https://www.elastic.co/guide/en/elasticsearch/reference/current/get-enrich-policy-api.html
---

# Data Source: elasticstack_elasticsearch_enrich_policy

Returns information about an enrich policy. See: https://www.elastic.co/guide/en/elasticsearch/reference/current/get-enrich-policy-api.html

## Example Usage

```terraform
provider "elasticstack" {
elasticsearch {}
}

resource "elasticstack_elasticsearch_index" "my_index" {
name = "my-index"

mappings = jsonencode({
properties = {
email = { type = "text" }
first_name = { type = "text" }
last_name = { type = "text" }
}
})
deletion_protection = false
}

resource "elasticstack_elasticsearch_enrich_policy" "policy1" {
name = "policy1"
policy_type = "match"
indices = [elasticstack_elasticsearch_index.my_index.name]
match_field = "email"
enrich_fields = ["first_name", "last_name"]
query = jsonencode({
bool = {
must = [{ term = { b = "A" } }]
must_not = [{ term = { a = "B" } }]
}
})
}

data "elasticstack_elasticsearch_enrich_policy" "policy" {
name = "policy1"
}

output "name" {
value = data.elasticstack_elasticsearch_enrich_policy.policy.name
}
output "match_field" {
value = data.elasticstack_elasticsearch_enrich_policy.policy.match_field
}
output "indices" {
value = data.elasticstack_elasticsearch_enrich_policy.policy.indices
}
output "policy_type" {
value = data.elasticstack_elasticsearch_enrich_policy.policy.policy_type
}
output "enrich_fields" {
value = data.elasticstack_elasticsearch_enrich_policy.policy.enrich_fields
}
output "query" {
value = jsondecode(data.elasticstack_elasticsearch_enrich_policy.policy.query)
}
```

<!-- schema generated by tfplugindocs -->
## Schema

### Required

- `name` (String) The name of the policy.

### Read-Only

- `enrich_fields` (Set of String) Fields to add to matching incoming documents. These fields must be present in the source indices.
- `id` (String) Internal identifier of the resource
- `indices` (Set of String) Array of one or more source indices used to create the enrich index.
- `match_field` (String) Field in source indices used to match incoming documents.
- `policy_type` (String) The type of enrich policy, can be one of geo_match, match, range.
- `query` (String) Query used to filter documents in the enrich index. The policy only uses documents matching this query to enrich incoming documents. Defaults to a match_all query.
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
provider "elasticstack" {
elasticsearch {}
}

resource "elasticstack_elasticsearch_index" "my_index" {
name = "my-index"

mappings = jsonencode({
properties = {
email = { type = "text" }
first_name = { type = "text" }
last_name = { type = "text" }
}
})
deletion_protection = false
}

resource "elasticstack_elasticsearch_enrich_policy" "policy1" {
name = "policy1"
policy_type = "match"
indices = [elasticstack_elasticsearch_index.my_index.name]
match_field = "email"
enrich_fields = ["first_name", "last_name"]
query = jsonencode({
bool = {
must = [{ term = { b = "A" } }]
must_not = [{ term = { a = "B" } }]
}
})
}

data "elasticstack_elasticsearch_enrich_policy" "policy" {
name = "policy1"
}

output "name" {
value = data.elasticstack_elasticsearch_enrich_policy.policy.name
}
output "match_field" {
value = data.elasticstack_elasticsearch_enrich_policy.policy.match_field
}
output "indices" {
value = data.elasticstack_elasticsearch_enrich_policy.policy.indices
}
output "policy_type" {
value = data.elasticstack_elasticsearch_enrich_policy.policy.policy_type
}
output "enrich_fields" {
value = data.elasticstack_elasticsearch_enrich_policy.policy.enrich_fields
}
output "query" {
value = jsondecode(data.elasticstack_elasticsearch_enrich_policy.policy.query)
}
76 changes: 76 additions & 0 deletions internal/elasticsearch/enrich/policy_data_source.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
package enrich

import (
"context"

"github.com./elastic/terraform-provider-elasticstack/internal/clients"
"github.com./hashicorp/terraform-plugin-sdk/v2/diag"
"github.com./hashicorp/terraform-plugin-sdk/v2/helper/schema"
)

func DataSourceEnrichPolicy() *schema.Resource {
policySchema := map[string]*schema.Schema{
"id": {
Description: "Internal identifier of the resource",
Type: schema.TypeString,
Computed: true,
},
"name": {
Description: "The name of the policy.",
Type: schema.TypeString,
Required: true,
},
"policy_type": {
Description: "The type of enrich policy, can be one of geo_match, match, range.",
Type: schema.TypeString,
Computed: true,
},
"indices": {
Description: "Array of one or more source indices used to create the enrich index.",
Type: schema.TypeSet,
Computed: true,
Elem: &schema.Schema{
Type: schema.TypeString,
},
},
"match_field": {
Description: "Field in source indices used to match incoming documents.",
Type: schema.TypeString,
Computed: true,
},
"enrich_fields": {
Description: "Fields to add to matching incoming documents. These fields must be present in the source indices.",
Type: schema.TypeSet,
Computed: true,
Elem: &schema.Schema{
Type: schema.TypeString,
},
},
"query": {
Description: "Query used to filter documents in the enrich index. The policy only uses documents matching this query to enrich incoming documents. Defaults to a match_all query.",
Type: schema.TypeString,
Computed: true,
},
}

return &schema.Resource{
Description: "Returns information about an enrich policy. See: https://www.elastic.co/guide/en/elasticsearch/reference/current/get-enrich-policy-api.html",
ReadContext: dataSourceEnrichPolicyRead,
Schema: policySchema,
}
}

func dataSourceEnrichPolicyRead(ctx context.Context, d *schema.ResourceData, meta interface{}) diag.Diagnostics {
client, diags := clients.NewApiClient(d, meta)
if diags.HasError() {
return diags
}

policyId := d.Get("name").(string)
id, diags := client.ID(ctx, policyId)
if diags.HasError() {
return diags
}
d.SetId(id.String())
return resourceEnrichPolicyRead(ctx, d, meta)
}
68 changes: 68 additions & 0 deletions internal/elasticsearch/enrich/policy_data_source_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
package enrich_test

import (
"fmt"
"testing"

"github.com./elastic/terraform-provider-elasticstack/internal/acctest"
sdkacctest "github.com./hashicorp/terraform-plugin-sdk/v2/helper/acctest"
"github.com./hashicorp/terraform-plugin-sdk/v2/helper/resource"
)

func TestAccDataSourceEnrichPolicy(t *testing.T) {
name := sdkacctest.RandStringFromCharSet(10, sdkacctest.CharSetAlphaNum)
resource.Test(t, resource.TestCase{
PreCheck: func() { acctest.PreCheck(t) },
ProtoV5ProviderFactories: acctest.Providers,
Steps: []resource.TestStep{
{
Config: testAccEnrichPolicyDataSource(name),
Check: resource.ComposeTestCheckFunc(
resource.TestCheckResourceAttr("data.elasticstack_elasticsearch_enrich_policy.test", "name", name),
resource.TestCheckResourceAttr("data.elasticstack_elasticsearch_enrich_policy.test", "policy_type", "match"),
resource.TestCheckResourceAttr("data.elasticstack_elasticsearch_enrich_policy.test", "match_field", "email"),
resource.TestCheckResourceAttr("data.elasticstack_elasticsearch_enrich_policy.test", "indices.0", name),
resource.TestCheckResourceAttr("data.elasticstack_elasticsearch_enrich_policy.test", "enrich_fields.0", "first_name"),
resource.TestCheckResourceAttr("data.elasticstack_elasticsearch_enrich_policy.test", "enrich_fields.1", "last_name"),
resource.TestCheckResourceAttr("data.elasticstack_elasticsearch_enrich_policy.test", "query", "{\"match_all\":{}}"),
),
},
},
})
}

func testAccEnrichPolicyDataSource(name string) string {
return fmt.Sprintf(`
provider "elasticstack" {
elasticsearch {}
}

resource "elasticstack_elasticsearch_index" "my_index" {
name = "%s"

mappings = jsonencode({
properties = {
email = { type = "text" }
first_name = { type = "text" }
last_name = { type = "text" }
}
})
deletion_protection = false
}

resource "elasticstack_elasticsearch_enrich_policy" "policy" {
name = "%s"
policy_type = "match"
indices = [elasticstack_elasticsearch_index.my_index.name]
match_field = "email"
enrich_fields = ["first_name", "last_name"]
query = <<-EOD
{"match_all": {}}
EOD
}

data "elasticstack_elasticsearch_enrich_policy" "test" {
name = elasticstack_elasticsearch_enrich_policy.policy.name
}
`, name, name)
}
1 change: 1 addition & 0 deletions provider/provider.go
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ func New(version string) *schema.Provider {
"elasticstack_elasticsearch_security_role_mapping": security.DataSourceRoleMapping(),
"elasticstack_elasticsearch_security_user": security.DataSourceUser(),
"elasticstack_elasticsearch_snapshot_repository": cluster.DataSourceSnapshotRespository(),
"elasticstack_elasticsearch_enrich_policy": enrich.DataSourceEnrichPolicy(),
},
ResourcesMap: map[string]*schema.Resource{
"elasticstack_elasticsearch_cluster_settings": cluster.ResourceSettings(),
Expand Down
16 changes: 16 additions & 0 deletions templates/data-sources/elasticsearch_enrich_policy.md.tmpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
subcategory: "Enrich"
page_title: "elasticstack_elasticsearch_enrich_policy Data Source - terraform-provider-elasticstack"
description: |-
Returns information about an enrich policy. See: https://www.elastic.co/guide/en/elasticsearch/reference/current/get-enrich-policy-api.html
---

# Data Source: elasticstack_elasticsearch_enrich_policy

Returns information about an enrich policy. See: https://www.elastic.co/guide/en/elasticsearch/reference/current/get-enrich-policy-api.html

## Example Usage

{{ tffile "examples/data-sources/elasticstack_elasticsearch_enrich_policy/data-source.tf" }}

{{ .SchemaMarkdown | trimspace }}