The following logic has been implemented to manage the delivery of alerts:
For alert webhooks, if a payload is unable to go through, all new notifications for that same alert on the same webhook will be blocked until a retry either succeeds or times out.
For example: If an alert goes through on a webhook, but the improvement notification fails to go through, further alerts will not be sent until the improvement notification retry either succeeds or times out.
A failed payload will continue to retry for up to one hour before timing out. An email will be sent after the third retry failure, which will generally be around 7-15 minutes after the initial failed attempt.
We will be continuing to improve this area of the product in future releases to reduce this blocking behavior while still giving our customers correct ordering of alerts.