Don't send an exception until retries are exhausted if the exception is retryable with Sidekiq

[brettburley]

We tend to have jobs that raise exceptions on occasion due to issues like network timeouts, etc. that we expect to happen regularly. Given a list of retryable_exceptions, we might want jobs that raise one of these exceptions to not send an error notification until their last retry is exhausted. This way we can eliminate extra noise from errors that we expect to happen, but still allow these jobs to retry (and eventually succeed), while still receiving immediate notifications for unexpected errors.

This could be implemented in the Sidekiq integration to not send an error report if the exception is in the list of retryable_exceptions and the job has remaining retries.

I'm happy to take a stab at a PR that integrates this, but curious if this seems like it is within the scope of an integration, or if the integrations are intended to be very simple, and if there's a precedence for adding configuration options like retryable_exceptions that's specific to a particular integration.

[nateberkopec]

Agreed. I think this is within the scope of the Sidekiq integration.

[kornate]

@st0012 could you give a status update on this? Is it expected to be integrated? As a new customer, we have the same issue whereby timeouts and other commonly retried errors, are costing us a fortune!

[st0012]

@brettburley @kornate sorry for the delay. I've implemented this feature in #1532, which is scheduled to be a part of 4.7.0 release 🙂

[travisp]

@st0012 this update is useful, but would you consider something that also includes some of the original request to be able to specify which exceptions should be sent? There are many cases where it's useful to ignore certain exceptions that will be retried (until exhaustion), but still report other exceptions.

[st0012]

@travisp I can understand the use case, but I won't put that in high priority for now. but if anyone can propose an implementation for it, I'll consider it.