Firmware analysis - Potential to load all EXPs w/ hw mod

[jimmy]

Since the release of the v2 firmware online, this has allowed me to take a look at the inner workings of the Roland FA and figure out some interesting things. If you're not interested in the technical side of this keyboard, this topic may not be for you.

The main reason for this research was to figure out about the EXP packs, how they work and whether it would ever be possible to modify them or create our own EXPs for custom combinations or completely custom sounds.

With some assistance over at another forum, I was able to completely decompress the firmware files. If you want more info on how to do this yourself, or want to contribute to this project, head here.

Inside the firmware files we can see that all the EXPs are actually already "known" by the FA, and this also shows what packs correspond to their equivalent specific SRX release:

Code: Select all

EXP-01..Dance Trax Collection...tdrf1/srx/SRX08_S...
EXP-02..Keys Collection.tdrf1/srx/SRX07_S...
EXP-03..World Collection....tdrf1/srx/SRX09_S...
EXP-04..Concert Piano Collection....tdrf1/srx/SRX02_S...
EXP-05..Electric Piano Collection...tdrf1/srx/SRX12_S...
EXP-06..Studio Collection...tdrf1/srx/SRX03_S...
EXP-07..Brass Collection....tdrf1/srx/SRX10_S...
EXP-08..Strings Collection..tdrf1/srx/SRX04_S...
EXP-09..Complete Piano Collection...tdrf1/srx/SRX11_S...
EXP-10..Orchestra Collection....tdrf1/srx/SRX06_S...
EXP-11..Vintage Synth Collection....tdrf1/srx/SRX13_S

This means that as of the current firmware, only these EXP packs are supported. The keyboard couldn't handle a new EXP-12 for example without a firmware patch. It also suggests that EXP-11 might be the last one Roland releases, but they could always release another update for more.

After this is a dictionary of all the waveform names:

Code: Select all

Phrase MENU1....Phrase MENU2....Phrase MENU3....61:Slow Grv ....75:BoomRvBel....76:TrpHpKikn....78:NinjaBrek....80:Thump Grv

These are the waveform names included in the EXP-01 pack. It's possible we might be able to modify the waveform data inside the EXP (though that format has still yet to be decoded), but not show this inside the keyboard editor as all the waveform names in the EXPs are hardcoded in the firmware, not in the EXP pack.

Anyway, you can see that on EXP-01, there is what appears to be a file path - "tdrf1/srx/SRX08_S" - I believe this is the EXP file located on the internal filesystem of the FA. My theory is that at boottime, the Firmware bios simply looks for what SRX files exist in the tdrf1/srx folder, and enables/disables them on the keyboard. This leads to the possibility of modifying the FA hardware to fit a larger storage device into the main board, formatting it with a larger filesystem (an extra 288MB) and coping all the EXP files onto that folder. This is of course completely untested, and I haven't even been able to view the filesystem on the device, but if true, this would be a massive boost for all owners of the FA. I'm just a software guy and unfortunately haven't got much experience with electronics modification, but thought maybe someone would be interested in this possibility as this is probably as far as I can take this, at least for now.

I can foresee the potential for an "all EXPs" FA modchip or modding service based on this.

[AnalogHero]

Very intresting. Maybe tdrf1 is already big enough to handle more than 2 EXPs. Problem i see is that even if you could copy more data to this location, it looks like it can hold only 2 EXPs anyway. If you check menu and then goto the screen with the version info it shows what 2 EXPs are loaded.

Im really intrested in having more expansions loaded, but im scared to load a modded firmware.

[jimmy]

Very intresting. Maybe tdrf1 is already big enough to handle more than 2 EXPs. Problem i see is that even if you could copy more data to this location, it looks like it can hold only 2 EXPs anyway. If you check menu and then goto the screen with the version info it shows what 2 EXPs are loaded.

Im really intrested in having more expansions loaded, but im scared to load a modded firmware.

Yeah modifying the keyboard is a bit scary considering how much they are to buy, I agree. I wouldn't want to try loading a hacked firmware for the device because that could go very wrong very fast. Loading a hacked EXP file for some custom sounds might be a bit safer, even if the waveform names would show up wrong on the keyboard, though I've not been able to figure out how that all works yet.

I think the tdrfl path is probably a storage chip on the board somewhere, and I expect it's only big enough to hold 2 EXPs simply because fast RAM/ROM chips, even small ones, are still quite expensive today. I doubt they would spend the extra money on a bigger chip if they didn't need to. But you are right - the firmware might only support two loaded, even if the files are present there, it’s not easy to tell.

[AnalogHero]

Now we see why Roland cant provide us with a simple EXP editor. Why did they put the EXP names AND waveform names in the FIRMWARE?!?. Btw where are patch names stored?

So if you find out how the pcm data is compressed and /or encrypted you could replace them with your own data, but names will be original. Would be very confusing in the end if you dont keep track of what you changed.

But now that you see what they did to block users out, i bet they put checksums in the firmware for the EXPs.

What if you alter 1 byte in one exp bin? I guess it wont be loaded anymore.

[jimmy]

Now we see why Roland cant provide us with a simple EXP editor. Why did they put the EXP names AND waveform names in the FIRMWARE?!?

Yeah, I expected them to be truly modular, much like the original SRX cards. It seems that they built the keyboard and firmware with all the EXP packs in mind, then split them out. The good thing about this is I suspect that during QA/Testing of the FA software, they probably have all the EXP packs installed, which is why I think it's likely that putting all the packs at that location will work. I may be wrong, but to me I think it's likely that the developers would need the ability to test all the packs at once.

Btw where are patch names stored?

That part I haven't figured out. None of the built in patch names or EXP patch names are visible in the firmware file, so they're either encoded in that file, or in the WaveROM on the keyboard and inside each EXP file. I haven't been able to decode it yet.

So if you find out how the pcm data is compressed and /or encrypted you could replace them with your own data, but names will be original. Would be very confusing in the end if you dont keep track of what you changed.

Yes, it would be confusing when you tried to edit them on the keyboard, but if you had a computer nearby to "translate" what it shows on the keyboard then it would be OK. It is possible the Firmware data is just a duplicate, maybe the wave names are in the EXP and I just haven't been able to decode it yet. Until I decode the EXPs we won't know for sure, but they are going to be a tough nut to crack. I was hoping the firmware itself might shine some light on how to decode/encode the EXP data but no luck yet.

But now that you see what they did to block users out, i bet they put checksums in the firmware for the EXPs.

What if you alter 1 byte in one exp bin? I guess it wont be loaded anymore.

I hadn't even thought of that, good point. Don't know until you try. I could try modifying one byte, but that might break a checksum/integrity check in the EXP file itself, it wouldn't necessarily indicate a firmware check.

Thanks for your thoughts btw! It's exciting to think of the possibilities of really expanding the potential of this keyboard.

[husker]

Fascinating Jimmy, thank you.

[AnalogHero]

In theory, before you alter the firmware, you should find a way back if something goes wrong. Nowadays they often have testpoints on the mainboard, which is sometimes just a usb port or jtag. Then you have a way to flash back the original firmware.

With such a setup, including a working connection to the bootloader, you have atleast a possible way back. Im sure the safety measures from roland are very slim here, as you found out its *nix based.

If therer are no solder points, or no one is brave enough poking around on the hardware (i'm not:) you can still try to mod the firmware, flash it, and have a good adrenalin rush during next powerup.

Note, this is all theory.

[jimmy]

I think the USB testpoint is just the main USB port on the back! I dismantled my FA-08 (man, there's a lot of screws on the bottom in the IKEA-esque bottom board) and managed to take a picture of the motherboard. I couldn't move it too much because the LCD screen ribbon cable runs over the metal casing for it, so I could only get a semi-clear shot of the top. I really didn't want to ruin the LCD ribbon cable.





So some interesting info can be gathered from this, there are 4 RAM chips (two on the top, two on the bottom), as opposed to the 9 in the Integra-7, so it has half the waverom, based on the chips. Each IS42S16160D chip is 256MB (16MBx16) so the FA has 1GB of Waverom. The Integra has 9 chips (https://adriangin.wordpress.com/2016/10 ... re-review/) so 2.3GB Waverom on the integra.

What is very interesting is the SRX/EXP. On the Integra-7, it has an internal micro SD card with all the SRX cards on, which it loads into the 3 available S29GL01GP (1gbit/128mb) chips on the motherboard. I could only see two S29GL128 (128mbit/16MB) chips. So the EXP data is possibly obfuscated to produce a file double the size than the true EXP size on the FA, unless there were two more EXP flash ROMs I couldn't see underneath (EDIT: this is the case, there are 2 more chips underneath the board, so 4x S29GL128 (128mbit/16MB) chips in total). I was hoping to see a different filesystem type chip on the FA instead of regular flash, so that was a disappointment. However it is a useful clue.

Also interesting, is the FA has an additional SH2 co-processor that the Integra doesn't have, so while it has less waverom, it has more horsepower behind it, probably to drive the additional arranger software and possibly assist with running 16 channels - I know the integra-7 had some rhythm issues with lots of SN-A stuff loaded. I have some other technical thoughts that I'll probably put all online on a website somewhere to get all this information compiled together.

interesting stuff how it all works. I checked and was able to also download the old firmware v1.03 from the roland site. Uncompressed has no mention of the EXP11 pack in the raw data. It only has up to the EXP10. So I tried loading EXP11 on my FA-08 which still has the old firmware. It showed an error, so this shows that the EXP and firmware are linked unfortunately. It looks like custom sounds might just be too difficult to accomplish.

[desernauta]

very very interesting.
Thanks

[AnalogHero]

What is very interesting is the SRX/EXP. On the Integra-7, it has an internal micro SD card with all the SRX cards on, which it loads into the 3 available S29GL01GP (1gbit/128mb) chips on the motherboard. I could only see two S29GL128 (128mbit/16MB) chips. So the EXP data is possibly obfuscated to produce a file double the size than the true EXP size on the FA, unless there were two more EXP flash ROMs I couldn't see underneath. I was hoping to see a different filesystem type chip on the FA instead of regular flash, so that was a disappointment. However it is a useful clue.

I looked at the exp.bin files with a hexeditor and there is some epmty space in them, but not 16MB. Maybe there are two more chips somewhere.

There is a lot of stuff on the mainboard. IIRC a user on this board has posted some images of the inside of a roland FA06.

Maybe on boot time the FA looks for some files on the USB stick (as it does with the EXPs), and then boots into a testmode or something. Maybe there are some clues in the firmware.

[jimmy]

I looked at the exp.bin files with a hexeditor and there is some epmty space in them, but not 16MB. Maybe there are two more chips somewhere.

There is a lot of stuff on the mainboard. IIRC a user on this board has posted some images of the inside of a roland FA06.

Maybe on boot time the FA looks for some files on the USB stick (as it does with the EXPs), and then boots into a testmode or something. Maybe there are some clues in the firmware.

I looked through the pictures I took and think I can see the other two SRX/EXP RAM chips underneath across a couple of pictures so there is the 2x 32MB EXP virtual "slots" as expected. That annoying ribbon cable prevented me getting a clear look or photo to share, but I really don't want to push my luck and dismantle it again! If we could get ahold of the service manual for the FA, that would provide a circuit diagram - I haven't been able to find that leaked anywhere, and not sure if Roland even release service manuals anymore.

I couldn't find the thread where someone else took apart their FA06 to take pictures, so I'd be interested in seeing that. There are a lot of boards inside other than the main board though.

[Skijumptoes]

I would imagine that the EXP data, such as waveform list etc. exists in the firmware to support the FA's super quick startup time.

It's basically an index, just as any database operates, i.e. you display the index only to the user as quick as you can, and when they select an item from that index, it's index address alone gets loaded. Having the index in the firmware, i.e. local to the OS, and on the quickest area accessible will greatly help startup. The times that the FA takes, you're really look at saving milliseconds too when optimising.

So it makes sense, plus all they have to do is update the firmware and make the EXP available, so it's not at all a bad thing, in general use.

Thanks for the post though, based on what you originally posted i foresee the problem with custom EXP's now would be if those indexes references actual memory start/end addresses - if so then custom EXP's will be tricky without firmware mods.

If they don't reference memory locations, and it's purely an index id that ties in with the data with the exp files, then first step would be to swap the data over on the EXP files and see if different samples can be loaded up in place of the indexed named samples. Trouble is, no-one has got in to the EXP format as far as i'm aware?, and there's a chance that Roland will come down on you if you try to reverse engineer them.

[AnalogHero]

I couldn't find the thread where someone else took apart their FA06 to take pictures, so I'd be interested in seeing that. There are a lot of boards inside other than the main board though.

Found it!
viewtopic.php?f=55&t=49953

[jimmy]

Thanks AnalogHero! Their photos are kind of blurry though sadly, and it also looks like they had the same issue I did of the LCD ribbon cable stopping them moving the metal plate out the way.

I would imagine that the EXP data, such as waveform list etc. exists in the firmware to support the FA's super quick startup time.

It's basically an index, just as any database operates, i.e. you display the index only to the user as quick as you can, and when they select an item from that index, it's index address alone gets loaded. Having the index in the firmware, i.e. local to the OS, and on the quickest area accessible will greatly help startup. The times that the FA takes, you're really look at saving milliseconds too when optimising.

So it makes sense, plus all they have to do is update the firmware and make the EXP available, so it's not at all a bad thing, in general use.

Good point, having everything already in the firmware would speed up loading, that's true. It really is like the EXPs are just the wave-data. The original SRX cards weren't like that at all though, as far as I know, they really did pass the patches and wave list to the host device, so it's a shame that they scrapped that in the name of speed.

Thanks for the post though, based on what you originally posted i foresee the problem with custom EXP's now would be if those indexes references actual memory start/end addresses - if so then custom EXP's will be tricky without firmware mods.

Yeah, it makes it much harder. The firmware would have to be disassembled, modified and reassembled to give us the possibility of dynamic EXP packs if that is the case, which would be many hours of reverse engineering work. At least we know the CPU and co-processor are SuperH based, so if someone wanted to do that, they'd at least know where to start.

If they don't reference memory locations, and it's purely an index id that ties in with the data with the exp files, then first step would be to swap the data over on the EXP files and see if different samples can be loaded up in place of the indexed named samples. Trouble is, no-one has got in to the EXP format as far as i'm aware?, and there's a chance that Roland will come down on you if you try to reverse engineer them.

Yeah, they are clearly encoded in the downloads, so essentially it would require figuring out how to decode them, and after all that, you then have to figure out the actual format itself.

The one other final thought was at least the possibility of improving the EXP swapping performance. The time it takes to copy from the USB stick to the internal EXP slot is very slow, around 1-2 mins, and I wish this could be reduced. In the Integra-7, it loads into the slots from an internal microSD card, and works much faster: https://youtu.be/thuWi9berfM?t=1m38s

It makes me wonder if the firmware could at least be modified to read the contents of the SD card instead of the USB stick to allow faster hotswapping of EXPs - since it uses the same CPU as the integra I think it's likely it has the same interface as the microsd slot on that, which clearly has more bandwidth and speed than the USB. The speed of the USB seems to be 1.1 rather than 2.0. (Also the fact the firmware has the EXP information built in explains why the Wave Update mode has to be done before the main firmware boots up)

Of course the perfect system could show ALL the patches on the FA, then when one from an EXP one is selected, it pauses for a couple of seconds to load the EXP then it's available. Heck, even cooler would be is if it could dynamically load the patches you select into the RAM from the EXP files individually.

EDIT: I am compiling all my findings over at http://rolandfatools.mooo.com/ - stay tuned there for updates

[Aldan]

Hi Jimmy
Hacking firmware is very interesting activity. And very time consuming.

Would like to share one interesting thing which I have noticed few years ago while building arranger app for my FA-06 (the app, still left unfinished).
I have occasionally sent program change message (MSB+LSB+PC) to select EXP tone from the library which was not installed at that point of time.
Anyway, the tone was selected. All patch parameters were on the screen and could be operated.
But, of cause, due to absence of EXP waveform the selected tone was silent.

So, the complete list of all EXP tones as well as patch parameters for them are already inside FA.
If you will manage to compile new EXP library file based on waveforms extracted from existing EXPs that should work.

But, it seems to me, you probably don't need to disassemble main firmware file.
You need to understand the structure of EXP lib file, only.

All these library file are of the same size=32768 kb.
Think, that is due to hardware expansion slots had historical restriction in size.
I believe that if to make new EXP file containing only one waveform and fill-in the remaining space with zeros that should work as well.

What else.
About five years ago I've managed to extract bytes with audio info(backing tracks) from roland's eBand JS-10 firmware.
That was very specific (proprietory roland's) format of audio compression. Think, Roland applies this format for any audio data.
Then I found in inet one ancient utility which converted file with extracted bytes to normal .WAV. Unfortunately, don't remember right now how I did it.

But, conversion itself is not required now.
You don't need to really understand roland's audio format.
You just need to know how few header bytes look like before audio data starts and read few bytes with size of the following audio data.
That should be enough to extract bytes with compressed waveforms from EXP file.

Will dig deep in my archives next week and hopefully will find my investigations on the matter.