~(TLIDE) in Japanese Shift-JIS text changes to ‾(OVERLINE)

nyanta

When saving a .txt file with Shift_JIS text editor encoding in Windows 11, ~(TLIDE) is modified to ‾(OVERLINE) in Dropbox on iOS. This is a Dropbox bug specific to Japanese.

 

I would like you to fix it so that the sameness is maintained in the iOS app even after saving with Shift_JIS.

zacw

I believe this is authoritatively correct depending on how you are inserting the tilde. For example, given this file:

これはテストです。 これらはいくつかのランダムな文章です。
~ <- tilde

With this hex representation:

┌────────┬─────────────────────────┬─────────────────────────┬────────┬────────┐
│00000000│ 82 b1 82 ea 82 cd 83 65 ┊ 83 58 83 67 82 c5 82 b7 │×××××××e┊×X×g××××│
│00000010│ 81 42 20 82 b1 82 ea 82 ┊ e7 82 cd 82 a2 82 ad 82 │×B ×××××┊××××××××│
│00000020│ c2 82 a9 82 cc 83 89 83 ┊ 93 83 5f 83 80 82 c8 95 │××××××××┊××_×××××│
│00000030│ b6 8f cd 82 c5 82 b7 81 ┊ 42 0a 7e 20 3c 2d 20 74 │××××××××┊B_~ <- t│
│00000040│ 69 6c 64 65 0a          ┊                         │ilde_   ┊        │
└────────┴─────────────────────────┴─────────────────────────┴────────┴────────┘

Running this through iconv produces:

> iconv -f shift-jis -t utf-8 file.txt
これはテストです。 これらはいくつかのランダムな文章です。
‾ <- tilde

However, given a UTF-8 encoded version (note this has an actual ~ in it, not the output above):

┌────────┬─────────────────────────┬─────────────────────────┬────────┬────────┐
│00000000│ e3 81 93 e3 82 8c e3 81 ┊ af e3 83 86 e3 82 b9 e3 │××××××××┊××××××××│
│00000010│ 83 88 e3 81 a7 e3 81 99 ┊ e3 80 82 20 e3 81 93 e3 │××××××××┊××× ××××│
│00000020│ 82 8c e3 82 89 e3 81 af ┊ e3 81 84 e3 81 8f e3 81 │××××××××┊××××××××│
│00000030│ a4 e3 81 8b e3 81 ae e3 ┊ 83 a9 e3 83 b3 e3 83 80 │××××××××┊××××××××│
│00000040│ e3 83 a0 e3 81 aa e6 96 ┊ 87 e7 ab a0 e3 81 a7 e3 │××××××××┊××××××××│
│00000050│ 81 99 e3 80 82 0a 7e 20 ┊ 3c 2d 20 74 69 6c 64 65 │×××××_~ ┊<- tilde│
│00000060│ 0a                      ┊                         │_       ┊        │
└────────┴─────────────────────────┴─────────────────────────┴────────┴────────┘

This produces an error when trying to convert:

> iconv -f utf-8 -t shift-jis test-utf8.txt
����̓e�X�g�ł��B �����͂����‚��̃����_���ȕ��͂ł��B
iconv: test-utf8.txt:2:0: cannot convert

According to this StackOverflow post, it seems like there is not a canonical tilde representation of the ~ character in Shift_JIS, so the app is doing its best to try and preserve as much of your content as possible, even if it can't be encoded. This is also my experience when using other macOS tools that can interpret Shift_JIS, though some will preserve the invalid character I'm not sure that's always the desired behavior.