GH-6114: Static path matching

[dantleech]

This (early stage) WIP PR introduces a static path matcher which intends to emulate the behavior of the PHPUnit FileIterator in order to prevent PHPUnit traversing the filesystem when a deprecation is triggered.

The PHPUnit FileIterator uses glob to find directories and we therefore need to support the glob patterns - which can vary according to the platform. This PR uses https://man7.org/linux/man-pages/man7/glob.7.html as a reference in addition to testing the behavior locally to confirm assumptions.

The webmozart/glob provides a similar feature however it's behavior is different as it supports curly braces, and * is restricted to a single directory level, while * in PHPUnit will return all descendants and I'm sure there are other differences - however I've used that as a starting point.

TODO:

• Escaping unhandled regex characters, or escaping the regex before parsing the pattern.
• Character classes [:alnum:] etc.
• Collating symbols
• Equivalence class expressions
• Emulating glob behavior of unterminated [ character groups.
• Seems to be a PHPUnit globstar bug whereby /a** will match /b and all other directories, where as /ab* will not match anything. We can either copy that behavior or "fix" it.

and maybe writing the implementation from scratch if regex turns out to be a bad fit.

Usages on Github:

• wildcard: 1.8k https://github.com./search?q=directory+%22*%22+path%3A*.xml+path%3A**%2Fphpunit.xml+language%3AXML&type=code
• globstar: 378: https://github.com./search?q=directory+%22**%22+path%3A*.xml+path%3A**%2Fphpunit.xml+language%3AXML&type=code
• Character range/class (directory.*\[): 0: https://github.com./search?q=%2Fdirectory.*%5C%5B%2F+path%3A*.xml+path%3A**%2Fphpunit.xml+language%3AXML&type=code
• ? any char: 0 https://github.com./search?q=%2Fdirectory.*%5C%5B%2F+path%3A*.xml+path%3A**%2Fphpunit.xml+language%3AXML&type=code

[dantleech]

this will/could include the $suffix, $prefix and $exclude also

[dantleech]

or could simply remove this and handle the prefix/suffix independently of the "file matcher"

[dantleech]

Have refactored to tokenize the glob string, while less performant it's easier to reason about and we only need to compile the regex for each <include or <exclude - not for each path.

[dantleech]

this handles any "left over tokens" - including T_ASTERISK - maybe all tokens should be handled explicitly?

[dantleech]

COLON and BACKSLASH are not tested

[dantleech]

@sebastianbergmann this PR is indicative of the approach but needs the final 20% of work:

• Refactor the source filter/mapper tests to create the paths they need in a temporary directory? this would make it easier to test edge cases without contaminating other tests and grant mode confidence.
• Implement the exclude/include logic (either by compiling that into the regex or as a separate concern).
• We can implement or forbid the use of [[:character-ranges:]] and other "esoteric" glob patterns.

But I'm keen for any thought you have on the approach so far - the biggest concern for me is providing confidence that it will be compatible with at least 99.99% of configuration files.

[dantleech]

this will need to change to account for include/exclude matching on the basename either in the regex or otherwise.

[sebastianbergmann]

@sebastianbergmann this PR is indicative of the approach but needs the final 20% of work:

* Refactor the source filter/mapper tests to create the paths they need in a temporary directory? this would make it easier to test edge cases without contaminating other tests and grant mode confidence.

* Implement the exclude/include logic (either by compiling that into the regex or as a separate concern).

* We can implement or forbid the use of `[[:character-ranges:]]` and other "esoteric" glob patterns.

Thank you for your work so far. I am afraid that I might not be able to look into this before the PHPUnit Code Sprint in April in Berlin. There I will be able to look into this, for sure, and together with other developers. I hope I can look into this before then, but I cannot promise that.

But I'm keen for any thought you have on the approach so far - the biggest concern for me is providing confidence that it will be compatible with at least 99.99% of configuration files.

We could delay this until PHPUnit 13 and break backward compatibility (for edge cases, "esoteric" glob patterns, etc.).

[sebastianbergmann]

Is it expected that the tests are currently failing?

[dantleech]

Yes - the prefix/suffix filtering hasn't been added yet and the tests fail due to the tests we added upstream. The PR is at an inflection point:

• Do we want to commit to supprting all the glob features? or could we simplify it.
• Do we want to refactor the source-finding/filtering tests to be more dynamic?[1]
• Are we happy with the general approach in this PR?

And then I was also considering how to implement the suffix/prefix logic - as it probably makes sense to include them in the compiled regex - but then it intrudes on the "copy the behavior of glob" approach" - so if we decide to simplify the globbing behavior it makes that decision easier too.

[1] currently all the source mapping/filtering tests share the same fixture which makes it hard to test edge cases without breaking other tests - I think it could generate the file tree instead.