Child Item

Webhook Returning Empty Response Error

I have 3 apps that are configured identically. 2 Development and 1 production.

Each has a webhook URL specified. The live production one keeps erroring out with the below error. I can immediately re-enable it. It looks like a connectivity issue, however, I don't experience this error anywhere else. That endpoint terminates at Cloudflare's edge, so there should never be an empty response. Once I re-enable it, it says enabled for a few minutes (but doesn't deliver any messages) and then errors out again.

I temporarily solved this problem by adding the same endpoint twice, one errored out, and one worked successfully. Still no webhook delivery but no errors on the duplicate. Eventually, I removed the first one and its been working since. Again, while the production app is error out, the dev apps are humming along. They have the same endpoints.

What's also concerning is, I don't get any notification or email when this occurs, it just stops working. Are there no notifications that are sent out when a problem like this occurs?

Actual Error:

"Error: Did not receive a response from the server.

Request:
POST <URL>

Response:
HTTP/1.1 N/A Unknown

Response Body (First 256 bytes):
(No response body)"

Find more posts tagged with

Developers

Comments

Greg-DB

Thanks for the report. If/when a webhook URI is automatically disabled, we do send an email notifying you of that. Please make sure that the email address on the account that owns the app is up to date and can receive email from the official Dropbox email domains.

It sounds like this the webhooks functionality is working for you now, so it may have been a transient network disruption. If you see this again though, please open an API ticket with the relevant app key(s) and webhook URI(s) so we can investigate this specifically. Thanks!

redm

Hi Greg,

Thanks for the quick response. I did update my email earlier, although it was previously using a google email, and it wasn't coming into spam or trash. That's good to know though, hopefully, I will get an email if it comes up again. I will review what you linked.

On the webhook issue, it was doing this for the last 24 hours. My attempts to fix included removing and re-adding. It only started working when I added it twice, waited 10 minutes, and then removed the errored one. It's very odd. No problems since. That does not fit the profile of a network issue.

Thanks again!

Derek

redm

Hi Greg,

This indeed did happen again, and I indeed did NOT get an email. We checked all the logs. We are using G-Suite, and no email was ever sent. As such, functionality was broken for a long period before we realized it.

1) The email system for Webhooks is not working or reliable. I'm not sure what to do??

2) Why are you disabling after a failure, why not automatically re-enable. It's not clear what the original issue was if it occured from the DB side as we have no events that line up with the issue. Regardless, problems occur, and so to shutoff webhooks for an entire app and all its users look like a poor choice; why not re-enable automatically with exponential backoff.

Here is the latest error we got, all we had to do was disable, and re-enable, which could have been done programmatically by DB:

Error: Internal Dropbox error

Request:
POST <redacted>

Response:
HTTP/1.1 N/A Unknown

Response Body (First 256 bytes):
(No response body)

Thank you for your help @Greg-DB

Greg-DB

Thanks for following up. In that case, please do please open an API ticket with the relevant app key(s) and webhook URI(s) so we can investigate this failure for you. We can also check our logs for see what happened with the email. Please make sure to open the ticket while signed in to the account that owns the app in question.

And thanks for the feedback regarding the disabling behavior. I can't speak to why the system was originally designed in this way, but I'll pass this feedback along to the team.

redm

Hi @Greg-DB

Thanks for the quick followup; we did open a ticket and got a quick response. There is no clear resolution however we got some additional information and will be able to monitor more closely for a future incident.

They did confirm the email was sent, so it's not clear why it wasn't delivered via G-Suite. We are getting more details on the address the email was sent from and will investigate further. They are also passing along to engineering the request to make this mechanism more robust.

If the issue occurs again and we get further information, we will update this thread.

Thanks again!

Derek

redm

Hi Greg,

I have had this same problem again.

No email again, although after re-adding it and then disabling again, I did finally get an email. The email I needed of course would have been 3 days before when it started.

The problem represents itself the same way as before; 1) disabling and enabling again works, but returns the same error, originally the exact same response from 3 days before, so it really didn't do anything. 2) Adding the same URL again verifies just fine, becomes enabled, but still no events. 3) removing both and re-adding the URL results in an HTTP failure effectively:

HTTP/1.1 N/A Unknown
Response Body (First 256 bytes):
(No response body)"

4) I was finally able to get this working, by adding the same exact URL but adding a query string of `?qs` (a garbage string but enough to slightly change the URL). It also verified fine, and now is working again.

I opened an API ticket (#10716976) but this is the same basic problem we've seen 4 or 5 times earlier and that presents itself every few months.

Derek

Greg-DB

@redm Thanks for the report! I see your support ticket was received. We'll look into it.