Skip to content

[CI] ProfileIntegTests testSuggestProfilesWithHint failing #117782

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
elasticsearchmachine opened this issue Nov 29, 2024 · 3 comments · Fixed by #117888
Closed

[CI] ProfileIntegTests testSuggestProfilesWithHint failing #117782

elasticsearchmachine opened this issue Nov 29, 2024 · 3 comments · Fixed by #117888
Assignees
@n1v0lg
Labels
low-risk An open issue or test failure that is a low risk to future releases :Security/Profile User Profiles & Personalisation :Security/Security Security issues without another label Team:Security Meta label for security team >test-failure Triaged test failures from CI

Comments

@elasticsearchmachine
Copy link
Collaborator

Build Scans:

Reproduction Line:

./gradlew ":x-pack:plugin:security:internalClusterTest" --tests "org.elasticsearch.xpack.security.profile.ProfileIntegTests.testSuggestProfilesWithHint" -Dtests.seed=87B674B0067A5850 -Dtests.locale=frr -Dtests.timezone=America/Swift_Current -Druntime.java=21 -Dtests.fips.enabled=true

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: 
Expected: <1>
     but: was <5>

Issue Reasons:

  • [main] 2 failures in test testSuggestProfilesWithHint (0.3% fail rate in 690 executions)
  • [main] 2 failures in pipeline elasticsearch-periodic (14.3% fail rate in 14 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.
@elasticsearchmachine elasticsearchmachine added :Security/Security Security issues without another label >test-failure Triaged test failures from CI Team:Security Meta label for security team needs:risk Requires assignment of a risk label (low, medium, blocker) labels Nov 29, 2024
@elasticsearchmachine
Copy link
Collaborator Author

Pinging @elastic/es-security (Team:Security)

@n1v0lg
Copy link
Contributor

n1v0lg commented Dec 3, 2024

Wow this actually reproduces 🎉

@n1v0lg n1v0lg self-assigned this Dec 3, 2024
@n1v0lg
Copy link
Contributor

n1v0lg commented Dec 3, 2024

This is a test setup failure where sometimes, the full name of the user is a substring of the other username (e.g., SER) -- when that's passed into suggest with the * space, we get a match on all profiles.

I'll follow this up with a fix since it's straight-forward. Labelling this low-risk.

@n1v0lg n1v0lg added low-risk An open issue or test failure that is a low risk to future releases :Security/Profile User Profiles & Personalisation and removed needs:risk Requires assignment of a risk label (low, medium, blocker) labels Dec 3, 2024
@n1v0lg n1v0lg mentioned this issue Dec 3, 2024
Merged
n1v0lg added a commit to n1v0lg/elasticsearch that referenced this issue Dec 5, 2024
@n1v0lg
Fix ProfileIntegTests (elastic#117888)
07ef385 
The test setup for `ProfileIntegTests` is flawed, where the full name of
a user can be a substring of other profile names (e.g., `SER` is a
substring of `User <random-string>-space1`) -- when that's passed into
suggest call with the `*` space, we get a match on all profiles, instead
of only the one profile expected in the test, since we are matching on
e.g. `SER*`. This PR restricts the setup to avoid the wildcard profile
for that particular test.

Closes: elastic#117782
n1v0lg added a commit to n1v0lg/elasticsearch that referenced this issue Dec 5, 2024
@n1v0lg
Fix ProfileIntegTests (elastic#117888)
88ace1a 
The test setup for `ProfileIntegTests` is flawed, where the full name of
a user can be a substring of other profile names (e.g., `SER` is a
substring of `User <random-string>-space1`) -- when that's passed into
suggest call with the `*` space, we get a match on all profiles, instead
of only the one profile expected in the test, since we are matching on
e.g. `SER*`. This PR restricts the setup to avoid the wildcard profile
for that particular test.

Closes: elastic#117782
n1v0lg added a commit to n1v0lg/elasticsearch that referenced this issue Dec 5, 2024
@n1v0lg
Fix ProfileIntegTests (elastic#117888)
52f039e 
The test setup for `ProfileIntegTests` is flawed, where the full name of
a user can be a substring of other profile names (e.g., `SER` is a
substring of `User <random-string>-space1`) -- when that's passed into
suggest call with the `*` space, we get a match on all profiles, instead
of only the one profile expected in the test, since we are matching on
e.g. `SER*`. This PR restricts the setup to avoid the wildcard profile
for that particular test.

Closes: elastic#117782
elasticsearchmachine pushed a commit that referenced this issue Dec 5, 2024
@n1v0lg
Fix ProfileIntegTests (#117888) (#118060)
fbb42f1 
The test setup for `ProfileIntegTests` is flawed, where the full name of
a user can be a substring of other profile names (e.g., `SER` is a
substring of `User <random-string>-space1`) -- when that's passed into
suggest call with the `*` space, we get a match on all profiles, instead
of only the one profile expected in the test, since we are matching on
e.g. `SER*`. This PR restricts the setup to avoid the wildcard profile
for that particular test.

Closes: #117782
elasticsearchmachine pushed a commit that referenced this issue Dec 16, 2024
@n1v0lg
Fix ProfileIntegTests (#117888) (#118061)
3a249f2 
The test setup for `ProfileIntegTests` is flawed, where the full name of
a user can be a substring of other profile names (e.g., `SER` is a
substring of `User <random-string>-space1`) -- when that's passed into
suggest call with the `*` space, we get a match on all profiles, instead
of only the one profile expected in the test, since we are matching on
e.g. `SER*`. This PR restricts the setup to avoid the wildcard profile
for that particular test.

Closes: #117782
elasticsearchmachine pushed a commit that referenced this issue Dec 16, 2024
@n1v0lg
Fix ProfileIntegTests (#117888) (#118059)
187aecf 
The test setup for `ProfileIntegTests` is flawed, where the full name of
a user can be a substring of other profile names (e.g., `SER` is a
substring of `User <random-string>-space1`) -- when that's passed into
suggest call with the `*` space, we get a match on all profiles, instead
of only the one profile expected in the test, since we are matching on
e.g. `SER*`. This PR restricts the setup to avoid the wildcard profile
for that particular test.

Closes: #117782
maxhniebergall pushed a commit to maxhniebergall/elasticsearch that referenced this issue Dec 16, 2024
@n1v0lg @maxhniebergall
Fix ProfileIntegTests (elastic#117888) (elastic#118061)
cb94a31 
The test setup for `ProfileIntegTests` is flawed, where the full name of
a user can be a substring of other profile names (e.g., `SER` is a
substring of `User <random-string>-space1`) -- when that's passed into
suggest call with the `*` space, we get a match on all profiles, instead
of only the one profile expected in the test, since we are matching on
e.g. `SER*`. This PR restricts the setup to avoid the wildcard profile
for that particular test.

Closes: elastic#117782
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@n1v0lg n1v0lg

Labels
low-risk An open issue or test failure that is a low risk to future releases :Security/Profile User Profiles & Personalisation :Security/Security Security issues without another label Team:Security Meta label for security team >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Avoid un-intended wildcard matches in ProfileIntegTests n1v0lg/elasticsearch
2 participants
@n1v0lg @elasticsearchmachine