Daniel was referring to the internal string / char mapping used by the firmware.SA0ASM wrote: ↑Wed Dec 27, 2023 8:39 pmI'm unsure if we are looking at the same data, so I downloaded the user.csv file from the same URL taht the OpenGD77 CPS uses and inspected it. The whole file is utf-8 encoded, I checked it with the iconv utility. When you say "it's a mess", what exactly do you mean? That rows can have different character encodings? Or something else? Could you give an example of a "messed up" entry in that file?
73 de SA0ASM
The display font is 8 bit (bitmap), i.e only 255 chars, so the language strings used in the firmware are not a standard 'code page'
Its effecively a custom code page, but because there are only 255 characters, its not possible to support even all accented characters, hence why its not possible to even produce versions for eastern European character sets.
And, the Japanese version has to be a completely different firmware build, with completely differnt bitmap fonts.
This has not changed in several years. The published source code for the GD77 and UV380 has all the bitmap fonts including coments about which characters are supported.
If you look you'll see there is almost no space for additional characters in the font.
For the DMR ID database; it is compressed to a form of 6 bit ascii, becasue originally the DMR ID database did not contain any non (A-Z,a-z,0-9,\.) characters, and the compression greatly increased the number of ID's that would fit into the small memory of a lot of radios.
Its only in the last year that the UV380 (and clones) have been supported, which have more flash memory to store the DMR ID database.
However, simply switching back to uncompressed data for these has other problems, because the data transfer is quite slow and not 100% reliable.
So changing to uncompressed on the UV380 / DM1701 etc, will increase the upload time noticably and also make the uploads less reliable.
Also, I can't remember if the code to handle uncompressed DMR ID's is still in the firmware codebase, because there is a problem of ROM size on all radios, especially the older radios like the GD77/RD5R/DM1801 etc , with almost no ROM space remaining.
So there is no room for unused code, and we generally remove it
The UV380 etc have much more ROM however, its not that simple, because a large portion of the ROM map is taken up by the binary blob of the official firmware, which has to be merged duing DFU, by the CPS, because of the legal / IP problems with the AMBE codec.
So even though the UV380 etc have 2 x the ROM size of the GD77, it does not mean that there is 2 x the available ROM space for the OpenGD77 firmware, becuase the binary blob of the official UV380 firmware , that we need to merge is probably larger
Possibly in one of the old versions of the CPS we may have had a checkbox to enable / disable compression, but we definitely don't have that now.
Anyway. I suggest that you start by taking a look at the firmware source code.