Child Item

OTA update fails in CC3200

Hi All,

We are using the Dropbox API to do an OTA upgrade. We have a webclient application running on the TI microcontroller CC3200. We are using the Drop Box API from 2017 onwards, and it is working fine. When we tested the OTA functionality yesterday, it was unable to receive the CDN file URL.

It is able to receive the file information from Dropbox but is unable to get the temporary link from Dropbox.

Do you know what is causing this issue? Is there any recent API upgrade that has failed to respond?

Please help us to fix this issue ASAP.

TI CC3200 logs:

sl_extLib_OtaRun: call OtaClient_ConnectServer OTA server=api.dropbox.com
OtaClient_ConnectServer: http_connect_server api.dropbox.com
0 OTA run = 0
sl_extLib_OtaRun: OtaClient_UpdateCheck, vendorStr=Vid01_Pid00_Ver0302100000
OtaClient_UpdateCheck: call http_build_request /1/metadata/auto/
CdnDropbox_SendReqDir: uri=/2/files/list_folder
metadata file=/Vid01_Pid00_Ver0302100000/f80_sys_mcuimgA.bin, size=142888
sl_extLib_OtaRun: OtaClient_UpdateCheck, numUpdates=1
0 OTA run = 0
sl_extLib_OtaRun: OtaClient_GetNextUpdate: file=/Vid01_Pid00_Ver0302100000/f80_sys_mcuimgA.bin, size=142888
OtaClient_ResourceMetadata: call http_build_request /1/media/auto
OtaClient_ResourceMetadata: file flags=80,metadata flags=80
CdnDropbox_SendReqFileUrl: uri=/2/files/get_temporary_link
0 OTA run = 0
sl_extLib_OtaRun: ResourceMetadata CDN file URL = f=
CdnClient_ConnectByUrl: ERROR, http_extract_domain_by_url, status=-1
sl_extLib_OtaRun ERROR: Failed on CdnClient_ConnectByUrl
0 OTA run = -6
OTA run = -6
OTA: Error with OTA server

Regards,

Sundar

Find more posts tagged with

API

Abuse

Comments

This output doesn't show the actual request/response for the API call. Are you able to retrieve the actual request and response showing the issue?

I'm not aware of a bug on the Dropbox servers that should be causing this. This sounds like this earlier report we received, but we weren't able to identify or reproduce the actual issue there.

Hi Greg,

Thank you for reply. We have many WIFI module chipsets like CC3200, CC3220, CC3235. All of them failed to receive the data from Dropbox. Texas instruments claims that the chipset could not able to send and receive data using their low level HTTP Client library. Did you modified anything related API (TCP/IP layer changes) for last 2 to 3 months? It was worked for past 5 several years. It is now impacting our business and our clients. Could you please help us to resolve the issue.

https://e2e.ti.com/support/wireless-connectivity/wi-fi-group/wifi/f/wi-fi-forum/1337191/cc3200-ota-error--6-for-dropbox/5149726#5149726

Regards,

Sundar

While there may have been updates to the Dropbox servers in this time, I'm not aware of anything that should be causing the servers to incorrectly fail on API calls like this.

If there seems to be an issue with the Dropbox API, please share the actual raw HTTP request and response (showing both the headers and bodies for both, but redact the access token) so we can reproduce the issue and look into it. Thanks!

Hi Greg,

Thank you so much for the reply. TI is saying it is a TLS handshake issue. request/response structure looks good. But there is a performance related issue.

Please refer to the link in the in the attached thread.

https://e2e.ti.com/support/wireless-connectivity/wi-fi-group/wifi/f/wi-fi-forum/1337191/cc3200-ota-error--6-for-dropbox/5160405#5160405

Here is the answer from TI:

It looks like following the first request+response (list folder) the Dropbox server sends a TLS alert from some reason (downgrading the connection to unsecure).
I don't understand the reason (maybe the Dropbox contact can explain this).
If you delay the 2nd request (get-temporary-link) by few seconds (e.g "sleep(5)") the TLS connection will be reestablished and OTA will work.

Regards,

Sundar

Hi @sundares80,

It's good idea to ask the TI support how exactly they reach to that conclusion. More steps to reproduce that inability to execute 2 consecutive requests would be useful - i.e. exactly what's the request content and so on. I just executed 2 requests - 1 list and 1 get temporary link - sticked together. Using either single stream or 2 independent streams in borders of single connection (i.e. HTTP1 or HTTP2) work fluently. Even more, some time ago I tested 2 overlapped requests in borders of single connection (sending second request before wait for the first to end) - again flawless. I posted Python sample code here, but likely removed already.

In short - you don't need to wait between requests and their explanation sounds like they're trying to rip you off. 🤷

Ask them for a way to reproduce for whatever they stated or will state. For example - curl. This command can reproduce almost everything. Can they offer a way for reproduction of their observations using 'curl'? The same using any other popular tool or program code...

Good luck.

@sundares80 I'm not aware of any TLS issues on the Dropbox API servers that should be causing this. We'd be happy to look into it, but we'd need some more specific information. As Здравко said, we'd need to know exactly how to reproduce the issue you're seeing, so please share the specific steps and code to reproduce it.

Also, the output you shared here seems to show some logging specific to the device platform you're using. Can you share any network logging showing the actual TLS issue you're referring to? For instance, a tcpdump showing the issue would be helpful.

Hi Greg,

Thank you both of them for checking this.

Here is the packet capture I got from TI.

This is what I have (the data is encrypted under the TLS):

https://e2e.ti.com/support/wireless-connectivity/wi-fi-group/wifi/f/wi-fi-forum/1337191/cc3200-ota-error--6-for-dropbox/5162901#5162901

If the screenshot not clear, Please use the above link.

The Alert is in frame 55151

Please let me know if you have any fix for this.

Regards,

Sundar

@sundares80 Thanks for sharing that screenshot. Can you share the whole dump though so we can inspect it? Thanks in advance!

Hi Greg,

Please download the wireshark pcap file from below link.

https://e2e.ti.com/cfs-file/__key/communityserver-discussions-components-files/968/cc32xx_2D00_dropbox_2D00_2.pcapng

Regards,

Sundar

Thanks! We'll look into it.

@sundares80 I had the team look at this, but unfortunately, this dump appears to only contain the WiFi frames. Can you instead share a dump with the actual data (TCP + TLS frames) so we can inspect that particular alert, like shown in the earlier screenshot? Thanks!

@Greg-DB, Do you think they are so stupid not to understand what's on the screen and what they send (if they are, no any chance)? As I said, they just try push the case away, while demonstrating something is doing.

The only way is @sundares80 to reproduce it and send the results. One advice received there and ignored sound good though:

... you can change the target to a Dump HTTP server (google for it, e.g. see https://httpdump.app/) and check the log there.

...despite of possible TLS issue cannot be reproduced in such a way, but may be confirmed at least if any at all or completely rejected.

@sundares80, Try do the same what your TI supported is doing (or is trying at least). Catch the packets flowing between Dropbox and your device while using Wireshark. To filter only packets needed, put in the filer something like ip.addr == 162.125.80.19. The IP address is from the screenshot (just for the example), but make sure it's the actual one at the time of meas.

Good luck.

Hi Greg,

Below is the answer provided by TI

no. we can't. I was using an air sniffer log where the TLS content is encrypted.

On the application side you can only publish the socket payload (i.e. if you print the data passed to sl_Send - it will include the HTTP content) but not the entire TCP/IP packets (i.e. it will be text file with http text and not a pcap file). I think one of our customers already provided this to them.

I'm quite sure they can capture the connection log on their server

As per my knowledge, the issue is not with the HTTP (curl request/response). Dropbox servers have some latency with the TLS handshake. Did you upgrade https certificates with different cipher suits?

Regards,

Sundar

@sundares80 Unfortunately this isn't something I can just retrieve from logs on our side. If you don't have a sample showing the issue you can share, please share specific steps, e.g., using curl, to reproduce the issue so we can investigate it that way. Thanks!

@Greg-DB, just a reminder on this topic.

In meantime, I reproduced the issues @sundares80 observes. Yes, despite they are minor deviations, but there are such that appear regularly (regular is maybe not the most correct). Most of the regular clients (including curl, web browsers, etc.) ignore such types of errors, but using diagnostic tools (openssl test suite, for instance) they can be seen. Wow, they appear pretty often - at least once per every 2~3 API calls (in my case). Some very sensitive client may confused (most probably that's why TI clients fail).

Interesting thing, that I noticed, is when higher level error appears (broken API call formatting and so on), encrypt errors are almost missing. For correct API calls, they are raining though. 🤷 I hope this can show some trace to possible reason.

Again minor (almost invisible) bugs, but existing! Just try inspect with openssl. They appear inconsistently, but often enough to reproduce.

@Здравко Thanks for the note! I'll ask the team if they can look into it that way.

@sundares80 We wanted to ask again, since it would be helpful to the team, if you could get and share an unencrypted dump (that is, without the WiFi encryption). The screenshot you shared on 4/24/2024 seems to be of the format we would want to see, but that's not what was shared in the dump you shared the next day (as that one included WiFi encryption). I understand you or your vendor may not want to do so because that would involve effectively sharing the WiFi password, so that could instead be saved and shared as a separate dump, e.g., using the "Export Packet Dissections" "As Plain Text" option, so as not to share the WiFi password.

@Здравко Thank you so much for recreating the issue.

@10 Please find my new packet capture in the google drive link. Please let me know if it works

https://drive.google.com/file/d/1v4dUI2lMIRJ-1Qg-u2Phl10IhSpVuCJr/view?usp=sharing

Regards,

Sundar

@sundares80 Thank you! We'll look into it.

@sundares80 The team is asking if you could also share some sample IP addresses and corresponding timestamps for failed requests so they can try to correlate this. Thanks in advance!

@Greg-DB, There is an easy way to reproduce the same (ok maybe not the same, but almost the same) on a fetching from dropbox.com. Your main site is affected too.

First determine the host IP, next prepare traffic dump and at the end just fetch it. Follow the next sequence:

host www.dropbox.com
sudo tcpdump -v host IP_FROM_PREVIOUS_COMMAND -w dropbox.pcap &
DUMPID=$!
curl -v https://www.dropbox.com/ 2> dropbox.http > /dev/null
kill $DUMPID

In advance close all browser tabs that may affect the records (i.e. with open Dropbox sites) or other tools.

Now you should have complete encrypted and unencrypted records to the same traffic. 🙂

dropbox.pcap - encrypted,

dropbox.http - unencrypted.

You can do the same while calling some API.

Hope this helps.

@Здравко Thanks for sharing this! I'll share this with the team.

Also, can you clarify why the Encrypted Alert in this case is problematic? From my understanding, TLS Alerts like this can be expected in standard use of the TLS protocol. For example, they can be used to close the connection after successfully sending data. It would be helpful if you could elaborate on why the alert would be wrong in this case and how you're checking that. Thanks!

Hi Greg,

As per TI, this extra TLS alert stopping the client code execution. Could you please remove this.

Regards,

Sundar

@Greg-DB wrote:
..., TLS Alerts like this can be expected in standard use of the TLS protocol. For example, they can be used to close the connection after successfully sending data. ...

Definitely NO!

Such a message (code 21) means incorrect encryption! Yes, it's known that many servers misbehave and most SSL/TLS libraries are ignoring this error and treat such as a break (as you said - even more, it's something mandatory with such a code), but this is definitely not something that should be expected. This is rather known bug in many implementation, but definitely not something that should be treated as "expected". The connection should be canceled exactly because the state is unexpected! By the way, most present day web servers (and not only) have corrected this issue; servers like yours are rather exceptions at present.

@sundares80 wrote:
...
As per TI, this extra TLS alert stopping the client code execution. ...

😁 Hi @sundares80,

This is pure try to excuse their own inability to handle such cases. Such cases are something that should be expected in the code and handled appropriately! Usually falling in any unexpected state (including the one discussed here, but not only) should lead to connection break and recreation, definitely not confusion! 😉

Hi @Здравко,

We are in a critical situation. We have installed 100K IOT devices throughout the US. We cannot access the device now. The only thing we can do is perform an OTA update, which is currently not working due to your changes. Please remove this TLS alert message for 6–8 months. During that period, we will do an over-the-air update and fix this issue from our side.

Regards,

Sundar

@Здравко Can you clarify where/how you're seeing code 21 exactly and what type that is? I initially thought I had reproduced an AlertDescription 21, but I believe what I saw was actually just a record ContentType 21, which just means "alert", not a specific alert. In that case, it wouldn't necessarily indicate incorrect encryption or not. The actual alert content was still encrypted.

I was able to configure curl with SSLKEYLOGFILE and point Wireshark at the file so that it could decrypt the alerts though, in which case when I performed a /2/files/get_temporary_link Dropbox API call, I only saw TLS records with Content Type 21 and Alert Description 0, which is just Close Notify. I did not observe any Alert Description 21.

Yes, I may got confused about the exact code. I tried to reproduce it right now - unsuccessful.

There is other strange situation though. Dropbox server refuses to renegotiate! Despite this activity support is optional, it's highly recommended (according to RFC 5246). About Client Hello:

When this message will be sent:
   When a client first connects to a server, it is required to send
   the ClientHello as its first message. The client can also send a
   ClientHello in response to a HelloRequest or on its own initiative
   in order to renegotiate the security parameters in an existing
   connection.

About "TLS protocol issues":

While renegotiation is an optional feature, supporting it is
highly recommended.

When some client tries renegotiation, Dropbox response is:

Received Record
Header:
  Version = TLS 1.2 (0x303)
  Content Type = Alert (21)
  Length = 18
    Level=fatal(2), description=no renegotiation(100)

Hm...🤔 According documentation, such a message should be warning, not fatal! Again, RFC 5246 states:

no_renegotiation
   Sent by the client in response to a hello request or by the server
   in response to a client hello after initial handshaking. Either
   of these would normally lead to renegotiation; when that is not
   appropriate, the recipient should respond with this alert. At
   that point, the original requester can decide whether to proceed
   with the connection. One case where this would be appropriate is
   where a server has spawned a process to satisfy a request; the
   process might receive security parameters (key length,
   authentication, etc.) at startup, and it might be difficult to
   communicate changes to these parameters after that point. This
message is always a warning.

This is a step aside from OP, but I just discovered it and that's why I'm mentioning. Such a message doesn't let a chance to client to decide whether to proceed further (without renegotiation) or not! This is not an issue in particular situation since there is no preceding handshake in the shared pcap, so that alert cannot be such a type (despite no way to be certain since encrypted).

There is another thing that may be interesting in context of OP. As can be seen (if we assume the alert is "close notify"), it comes from server side, not from client side (as it usually does). Such thing is possible only when in the request is header "Connection: close" or I was not able to reproduce it in other situation at least. So to be possible confirmation or rejection of this supposing (inappropriate header), @sundares80 should send plain request to pseudo-server (like netcat, for instance) and see if there is some unintentional header. I suppose wouldn't be possible SSLKEYLOGFILE usage since most probably the application doesn't support it.

@Здравко Thanks for writing that up! I'll pass it along to the team.

Featured Links

Tags

Child Item