Read/write GD-77 codeplug in Linux in Python 3 (experimental)

LZ1CK
Posts: 12
Joined: Sun Jul 12, 2020 9:50 am

Re: Read/write GD-77 codeplug in Linux in Python 3 (experimental)

Post by LZ1CK » Thu Jul 16, 2020 2:27 pm

MainForm.DataToByte(); initializes an empty codeplug blob, and I'm having some trouble cleanly replicating the same in my script.

First, are these regions really needed? Can we do without them? Can we read them from the radio instead?

If they have to be initialized on the client side, where is the authoritative source for "initial" settings data and locations? I could just hardcode them in my script, but to do so I need to extract the addresses and sizes of parts from the Settings class (Settings() constructor does this), and then extract those regions from a file saved from the OpenGD77CPS tool. It's not trivial for me to setup an OpenGD77CPS build environment ...


EDIT: looking at WRITE_CODEPLUG and READ_CODEPLUG code in worker_DoWork(), it looks like regions read and written are identical. So I'm skipping the extra blobs for now.

VK3KYY
Posts: 2996
Joined: Sat Nov 16, 2019 3:25 am
Location: Melbourne, Australia
Contact:

Re: Read/write GD-77 codeplug in Linux in Python 3 (experimental)

Post by VK3KYY » Thu Jul 16, 2020 10:00 pm

LZ1CK wrote:
Thu Jul 16, 2020 2:27 pm
MainForm.DataToByte(); initializes an empty codeplug blob, and I'm having some trouble cleanly replicating the same in my script.

First, are these regions really needed? Can we do without them? Can we read them from the radio instead?

If they have to be initialized on the client side, where is the authoritative source for "initial" settings data and locations? I could just hardcode them in my script, but to do so I need to extract the addresses and sizes of parts from the Settings class (Settings() constructor does this), and then extract those regions from a file saved from the OpenGD77CPS tool. It's not trivial for me to setup an OpenGD77CPS build environment ...


EDIT: looking at WRITE_CODEPLUG and READ_CODEPLUG code in worker_DoWork(), it looks like regions read and written are identical. So I'm skipping the extra blobs for now.
I never looked at the DataToByte, I just transfer we sections of the byte array to the radio.

The only reason the entire data is not sent to the radio is because I needed space in the EEPROM for the nonvolatile settings in the radio e.g. the brightness and kcurrent channel etc.

So I don't transfer a small part if the codeplug at the end of the predefined SMS messages area, and use that memory for the settings.

The official firmware does some other optimisation by analysing the data bytes so that it does not need to transfer all the channel or contacts data etc. But I did not have time to write code to inspect the data bytes, and the upload is already fast enough, and also its a crazy way to optimise by analysing the bytes after they are encodes from the data structures

LZ1CK
Posts: 12
Joined: Sun Jul 12, 2020 9:50 am

Re: Read/write GD-77 codeplug in Linux in Python 3 (experimental)

Post by LZ1CK » Sat Jul 18, 2020 8:12 am

I am one quarter of the way there -- the script parses Zones, Contacts, Channels and TG Lists (a.k.a RX Groups) into Python dataobjects. In retrospect, I should have implemented the objects as "views" of the binary blob data, but ... well, maybe I'll refactor this at some point.

While I'm at it, I'll add backup/restore calibration, eeprom, flash commands (basically looking at the handlers in FW and implementing the counterpart).
I wish I could paste my Python terminal here, but the forum software breaks when I try to, so here is a very clumsy screengrab:

https://terminalizer.com/view/bd7f79ac4113

There's a lot more to be done, and while I have a clear plan, I don't have as much time for this as I would like to. If anyone wants to join I can share my plan.

LZ1CK
Posts: 12
Joined: Sun Jul 12, 2020 9:50 am

Re: Read/write GD-77 codeplug in Linux in Python 3 (experimental)

Post by LZ1CK » Mon Jul 20, 2020 7:05 am

I can now export to YAML, the german codeplug posted on this forum is attached. Halfway there.
Attachments
Codeplug_RD-5R_by_DL9SEC_3v2_untested.g77.yml.gz
(12.51 KiB) Downloaded 20 times

VK3KYY
Posts: 2996
Joined: Sat Nov 16, 2019 3:25 am
Location: Melbourne, Australia
Contact:

Re: Read/write GD-77 codeplug in Linux in Python 3 (experimental)

Post by VK3KYY » Mon Jul 20, 2020 7:18 am

Thanks for the update.

BTW. Why did you decide to use YAML?

I've never used it, and I don't think support for it is as good as XML or JSON

I just looked at the wikipedia entry for YAML and the format looks very similar to JSON

dl9sec
Posts: 8
Joined: Sun Jun 28, 2020 11:32 am

Re: Read/write GD-77 codeplug in Linux in Python 3 (experimental)

Post by dl9sec » Mon Jul 20, 2020 7:22 am

Looks interesting.
There are the last two zones, they shouldn't be there.
My zones end with BY-RO (Bavaria region Rottach).
Maybe some index overruns, the two zones after BY-RO seems to be more a channel, at least the second one.

Maybe a bug?

73, Thorsten

LZ1CK
Posts: 12
Joined: Sun Jul 12, 2020 9:50 am

Re: Read/write GD-77 codeplug in Linux in Python 3 (experimental)

Post by LZ1CK » Mon Jul 20, 2020 7:49 am

There are the last two zones, they shouldn't be there.
Yes, zones use a funky addressing scheme, and I may have gotten it wrong. I triple-checked, but a few zones at the end of the codeplug fail to parse correctly. I think I've implemented the same algorithm as the one used here https://github.com/rogerclarkmelbourne/ ... lug.c#L262 but ... yeah, it's either a bug, or a broken codeplug. And I vote for the former.
BTW. Why did you decide to use YAML?
It allows for concise representation of the data in a single, plain-text file. Multiple CSV files can be added easily, but I'd rather use my code editor for the codeplug instead of a heavy LibreOffice Calc or some paid alternative, and have it all in one file. Actually, I've done a few apps where I spared myself from implementing a UI by integrating the app with Google Sheets, via their APIs. This worked out nicely, and can be done for import/exporting codeplugs too. But until everything works well I'll be using YAML.
I just looked at the wikipedia entry for YAML and the format looks very similar to JSON
YAML is a superset of JSON, any valid JSON is also valid YAML. It's pretty neat, both human and machine-readable and writable.
Last edited by LZ1CK on Mon Jul 20, 2020 1:09 pm, edited 1 time in total.

VK3KYY
Posts: 2996
Joined: Sat Nov 16, 2019 3:25 am
Location: Melbourne, Australia
Contact:

Re: Read/write GD-77 codeplug in Linux in Python 3 (experimental)

Post by VK3KYY » Mon Jul 20, 2020 8:59 am

LZ1CK wrote:
Mon Jul 20, 2020 7:49 am
...
YAML is a superset of JSON, any valid JSON is also valid YAML. It's pretty neat, both human and machine-readable and writable.
LOL

I have to read XML, JSON etc as part of the day job

User avatar
Zl1XE
Posts: 82
Joined: Fri Jul 24, 2020 12:25 am

Re: Read/write GD-77 codeplug in Linux in Python 3 (experimental)

Post by Zl1XE » Thu Aug 27, 2020 3:41 am

I'm just trying to check that I can write what I can read.
Would that be:

Code: Select all

gd77xfer --port COM3 read export dm1801-02.yaml
and then:

Code: Select all

gd77xfer --port COM3 write import dm1801-02.yaml
I can get the binary with:

Code: Select all

gd77xfer --port COM3 read dm1801-02.bin
and presumably write with:

Code: Select all

gd77xfer --port COM3 write dm1801-02.bin
But how can I create a binary file so i can compare the read bin with the through-the-loop bin to make sure they are the same ?
Or is this all working and I don't really need to check ?

I just want to check before possibly bricking my radio :)
DM-1801

LZ1CK
Posts: 12
Joined: Sun Jul 12, 2020 9:50 am

Re: Read/write GD-77 codeplug in Linux in Python 3 (experimental)

Post by LZ1CK » Thu Sep 03, 2020 9:31 pm

The intended usage of the tool looks like this:

Code: Select all

# read from radio into a file
gd77xfer read myradio.gd77 # in the original binary format, or straight
gd77xfer read --format yaml myradio.yml

# export binary into yaml
gd77xfer export input.gd77 output.yaml

## not yet implemented
gd77xfer import edited-codeplug.yaml existing-binary.gd77 


Post Reply