Skip to content

GODRIVER-3173 Complete pending reads on conn checkout #1977

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 16 commits into
base: master
Choose a base branch
from

Conversation

prestonvasquez
Copy link
Member

@prestonvasquez prestonvasquez commented Mar 5, 2025

GODRIVER-3173

Summary

Move pending response logic into the foreground.

To test locally you will have to check out the branch here for testdata/specifications: mongodb/specifications#1675

Background & Motivation

From the specifications update:

When a connection is checked out after a network timeout, the driver now attempts to resume and complete reading any pending server response (instead of closing and discarding the connection). This may require multiple checkouts.
Each pending response read is subject to a cumulative 3-second static timeout. The timeout is refreshed after each successful read, acknowledging that progress is being made. If no data is read and the timeout is exceeded, the connection is closed.

To reduce unnecessary latency, if the timeout has expired while the connection was idle in the pool, a non-blocking single-byte read is performed; if no data is available, the connection is closed immediately.
This update introduces new CMAP events and logging messages (PendingResponseStarted, PendingResponseSucceeded, PendingResponseFailed) to improve observability of this path.

@mongodb-drivers-pr-bot mongodb-drivers-pr-bot bot added the priority-3-low Low Priority PR for Review label Mar 5, 2025
Copy link
Contributor

mongodb-drivers-pr-bot bot commented Mar 5, 2025

API Change Report

./v2/event

compatible changes

ConnectionPendingResponseFailed: added
ConnectionPendingResponseStarted: added
ConnectionPendingResponseSucceeded: added
PoolEvent.RequestID: added

./v2/x/mongo/driver/topology

incompatible changes

BGReadCallback: removed
BGReadTimeout: removed

compatible changes

PendingResponseTimeout: added

@prestonvasquez prestonvasquez changed the title (POC V2) DRIVERS-2868 Complete pending reads on conn checkout (POC V2) GODRIVER-3173 Complete pending reads on conn checkout Mar 5, 2025
@prestonvasquez prestonvasquez requested a review from Copilot March 19, 2025 17:45
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request implements changes to support completing pending reads on connection checkout. Key changes include:

  • Adding new YAML tests for client‐side operation timeouts related to pending reads.
  • Updating connection and pool logic to handle pending reads with context values (e.g. maxTimeMS and requestID) and synchronizing background reads.
  • Introducing new monitoring events and updating event verification and test instrumentation to track pending read events.

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
testdata/client-side-operations-timeout/pending-reads.yml New YAML tests for handling operation timeouts and pending read scenarios.
internal/integration/client_test.go Updated concurrent operation tests to account for pending reads.
internal/driverutil/context.go Added context helper functions for maxTimeMS and requestID propagation.
x/mongo/driver/topology/pool.go Updated pool checkout logic to await completion of pending reads and renamed timeout constants.
x/mongo/driver/topology/pool_test.go Revised pool tests to verify new pending read behaviors.
internal/integration/unified/event_verification.go Added fields and tests for verifying new pending read pool events.
event/monitoring.go Extended pool event monitoring with pending read event support.
x/mongo/driver/operation.go Propagated context values in operation execution for pending read handling.
internal/integration/csot_test.go Updated CSOT tests to ensure connection closure conditions with pending reads.
Comments suppressed due to low confidence (2)

internal/integration/unified/event_verification.go:61

  • Field name 'ConnectionPendingreadSucceeded' should be 'ConnectionPendingReadSucceeded' to maintain consistent capitalization with the other pending read event fields.
ConnectionPendingreadSucceeded *struct{} `bson:"connectionPendingReadSucceeded"`

x/mongo/driver/topology/pool.go:908

  • Typo in comment: 'alawys' should be corrected to 'always'. Consider removing or clarifying this inline question to avoid confusion.
if size == 0 { // Question: Would this alawys equal to zero?

@prestonvasquez prestonvasquez changed the title (POC V2) GODRIVER-3173 Complete pending reads on conn checkout GODRIVER-3173 Complete pending reads on conn checkout Apr 30, 2025
@prestonvasquez prestonvasquez marked this pull request as ready for review April 30, 2025 22:40
@prestonvasquez prestonvasquez requested a review from a team as a code owner April 30, 2025 22:40
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements pending response handling by moving the pending read logic into the foreground and replacing the legacy BGReadCallback mechanism. Key changes include renaming the test function from TestBackgroundRead to TestAwaitPendingRead, replacing the awaitRemainingBytes field with a new pendingResponseState (with corresponding context values), and updating pool and connection logic to support pending response events.

Reviewed Changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.

File Description
x/mongo/driver/topology/pool_test.go Renamed test functions and updated assertions to use pendingResponseState instead of awaitRemainingBytes
x/mongo/driver/topology/pool.go Replaced BGReadTimeout with PendingResponseTimeout and integrated awaitPendingResponse logic with new event publications
x/mongo/driver/topology/connection.go Replaced awaitRemainingBytes with pendingResponseState and updated read methods accordingly
Other test and integration files Adjusted test expectations and event verifications for pending read events
Comments suppressed due to low confidence (1)

x/mongo/driver/topology/connection.go:929

  • [nitpick] Consider removing or clarifying the inline comment about whether size would always equal zero to avoid confusion in future maintenance.
if size == 0 { // Question: Would this alawys equal to zero?

_, err = conn.readWireMessage(ctx)
regex := regexp.MustCompile(
`^connection\(.*\[-\d+\]\) incomplete read of message header: context deadline exceeded: read tcp 127.0.0.1:.*->127.0.0.1:.*: i\/o timeout$`,
)
assert.True(t, regex.MatchString(err.Error()), "error %q does not match pattern %q", err, regex)
assert.Nil(t, conn.awaitRemainingBytes, "conn.awaitRemainingBytes should be nil")
close(errsCh) // this line causes a double close if BGReadCallback is ever called.
assert.Nil(t, conn.pendingResponseState, "conn.awaitRemainingBytes should be nil")
Copy link
Preview

Copilot AI Apr 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the error message to refer to 'conn.pendingResponseState' instead of 'conn.awaitRemainingBytes' for clarity and consistency with the new field name.

Suggested change
assert.Nil(t, conn.pendingResponseState, "conn.awaitRemainingBytes should be nil")
assert.Nil(t, conn.pendingResponseState, "conn.pendingResponseState should be nil")

Copilot uses AI. Check for mistakes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority-3-low Low Priority PR for Review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant