Reading Jupiter-80 sound sets on a PC/Mac

Forum for JUPITER-80
User avatar
Relayer
Posts: 15
Joined: 01:12, 13 June 2014

Re: Reading Jupiter-80 sound sets on a PC/Mac

Post by Relayer »

Hi everyone,
New to the board, although I've been reading it on and off for years. Interesting thread on sound management, and congrats on the advancements you are making. Just getting back into playing after a 15 year layoff. Currently have JP80 and Access Virus Ti2. Loved the Roland's I've had in the past-Juno 106, Super JX, D70. It's amazing that Roland doesn't have a package to open up in your DAW like the Virus does. A buddy of mine just bought a Nord Stage 2 and says the editing and management software are second to none. He has, among other things, an Oasis, Motif and a Triton. Said he hooked the Nord up to his computer and the workflow was so intuitive that he really didn't have to read much of the manual. And that keyboard will read and play all sounds from every keyboard in the Nord line. So a couple of questions - is it possible to get a Roland engineer interested in your project? Would make things easier. Could the Nord program be deconstructed and then reconstructed to work within Roland parameters? Could there be a work around, or a way to convert sounds from other Roland boards so they can be played on the Jupiter? Questions may be over the top, but sometimes thinking outside the box creates new opportunities. Thanks and looking forward to joining in on the conversations.
Attachments
IMG_0033a.jpg
IMG_0033a.jpg (88.82 KiB) Viewed 5228 times
moogmaniac
Posts: 25
Joined: 10:01, 5 May 2013

Re: Reading Jupiter-80 sound sets on a PC/Mac

Post by moogmaniac »

Many thanks kimsnarf for all your hard work on this project.

I have just caught up with the thread again, tried the latest version of the librarian with the updated demo of Renoise and it is reading .SVD files fine. Yes, the relationships view is messy, but converting it to a .csv soon sorted it out.

Obviously I would like a full blown librarian so that I can swap files around easily, but this piece of work is a great step forward.

Many Thanks.

David
Dewdman42
Posts: 137
Joined: 00:42, 14 January 2013

Re: Reading Jupiter-80 sound sets on a PC/Mac

Post by Dewdman42 »

Has the SVD file format been posted anywhere? perhaps I can take a crack at a few things...

Also, my JP is in storage for a while, if someone can PM me an SVD to use for trying this out, I would appreciate it.
Dewdman42
Posts: 137
Joined: 00:42, 14 January 2013

Re: Reading Jupiter-80 sound sets on a PC/Mac

Post by Dewdman42 »

I spent the afternoon today analyzing kimsnarf's renoise code to see if I could understand how to parse an SVD file, using synalyzeitpro. I was able to make sense out of most of what he did, still analyzing the bit shifting stuff and need to analyze what he did to find dependencies. Amazing work to spot the patterns, I bought that synalyzeit and I couldn't see any patterns at all when I stared at it, baffles me how he was able to see the patterns, but anyway, he seems to have done so, which is great for all of us.

I may take a crack at writing a little librarian that uses this knowledge. It will take a while, due to time constraints, I'm gonna try. I can envision something where a massive database of registrations, livesets and tones can be kept on the computer, the user can step through them one at a time and each one can be sent to the JP-80 temp area for auditioning. Dependencies can be monitored and duplicates cleaned out.

when you're ready to create a bank to gig with or whatever, then it should be possible to construct a collection of reg, live, tone into the order you want, with the names you want, and overwrite an existing SVD file, which can then be loaded into a blanked JP80. I don't think we can create a new SVD from scratch yet because there is still some bytes in the file we don't know the purpose. But we'll see.

In my mind, the ability to audition everything I have in a large database is key. The ability to construct collections for gigging is key. I think we have the needed info to do that much. I will see what I can do. There might be yet some more reverse engineering to do on the SVD file format and maybe later the BIN file too.

Thanks again to kimsnarf.
User avatar
cello
Posts: 1487
Joined: 11:47, 1 August 2011
Location: Glasgow, UK

Re: Reading Jupiter-80 sound sets on a PC/Mac

Post by cello »

Fantastic dewdman2! Thanks to both you and kimsnarf we might have a independent librarian after all :) Let me know if I can help with anything - and testing when the time comes.

It's not just gigging of course that to have collections is key. For me, as a studio only guy, having pads etc in one collection - or perhaps by studio project - is key; but this is exactly the same functionality.

And yes, being able to remove duplicates and examining dependencies - which for the first time will allow the user to delete sounds without damaging ones you want to keep - is key too.

Good luck with progress with grateful thanks and do let me know if I can help.
arock
Posts: 9
Joined: 19:01, 17 July 2014

Re: Reading Jupiter-80 sound sets on a PC/Mac

Post by arock »

Regarding the .BIN file. I did investigate that file a bit. It's actually just a renamed .TAR file with a two files and a folder inside. Here's the structure of the files inside:

Code: Select all

|
-> ARP_STYLE.BIN
-> MOTION
      |
      -> play_mms.bin
It seems clear that the Names and MIDI data for the user arps are stored in the ARP_STYLE.BIN file. That file appears to be fixed length and has entries for the 128 user arp slots.

The encoding of the "play_mms.bin" file is less clear to me. I have examined several backups and this file appears to be variable length, but I can't determine what settings on the JP80 it depends on. By its name, I thought it might have to do with the settings you can use to customize playback of files on a USB stick (e.g. speed, chaining of files, offset, length, etc) but I could not see those settings cause any change in a backup set I took before and after those changes.
Dewdman42
Posts: 137
Joined: 00:42, 14 January 2013

Re: Reading Jupiter-80 sound sets on a PC/Mac

Post by Dewdman42 »

Hmm, tarred, interesting.
Dewdman42
Posts: 137
Joined: 00:42, 14 January 2013

Re: Reading Jupiter-80 sound sets on a PC/Mac

Post by Dewdman42 »

Arock does play_mms actually hold MP3 or soundfile data of some kind perhaps?
arock
Posts: 9
Joined: 19:01, 17 July 2014

Re: Reading Jupiter-80 sound sets on a PC/Mac

Post by arock »

Dewdman42 wrote:I may take a crack at writing a little librarian that uses this knowledge. It will take a while, due to time constraints, I'm gonna try. I can envision something where a massive database of registrations, livesets and tones can be kept on the computer, the user can step through them one at a time and each one can be sent to the JP-80 temp area for auditioning. Dependencies can be monitored and duplicates cleaned out.
The goal of being able to fully parse an SVD file and "live audition" tones, live sets, and registrations in the temp area is laudable. I would just caution that the full understanding of every setting stored in the SVD file will take an enormous amount of time. The task of creating an offline librarian that can manipulate the structure and dependencies within and between SVD files is a much simpler task. The librarian can basically ignore the details of the tone/liveset/registration storage structure and just copy them around as opaque blocks of bytes.

In another post somewhere here on Roland Clan, the guy who built the librarian for the Korg series of keyboards (e.g. Kronos) made this same point, but it still took him thousands of hours to build just the librarian piece.
Dewdman42 wrote:when you're ready to create a bank to gig with or whatever, then it should be possible to construct a collection of reg, live, tone into the order you want, with the names you want, and overwrite an existing SVD file, which can then be loaded into a blanked JP80. I don't think we can create a new SVD from scratch yet because there is still some bytes in the file we don't know the purpose. But we'll see.
Actually, I think creating a new SVD file from scratch is not that hard. Roland did a nice job of making the file structure itself very easy to parse and understand. The only caveat here is that we would need to copy the system level settings from a user specified "template" backup file that was correct for that user's needs. Those sections are VCL (Visual Control), SYS (system settings) and RBN (Registration Bank Names). The RBNs could be set to defaults and would be pretty easy to modify offline. SYS contains the default and alternate tones and livesets for the colored upper/lower/perc/solo buttons, but again we can allow editing of those offline. I have learned how to parse that part of the SYS section.
Dewdman42 wrote:In my mind, the ability to audition everything I have in a large database is key. The ability to construct collections for gigging is key. I think we have the needed info to do that much. I will see what I can do. There might be yet some more reverse engineering to do on the SVD file format and maybe later the BIN file too.
Perhaps you, kimsnarf, and I should get an email conversation going to fully collect up everything we know about parsing and file structure. I believe my understanding of the details of the file structure is beyond what kimsnarf's grammar file for SynalizeIt knows how to parse.

I also have some C# code that can parse and report details of the tones/livesets/registrations at various levels of granularity, and can write a "minimal, compacted" version of the SVD file that contains just the tones and live sets that are actually referenced by registrations present in the file. This would be useful for taking a backup file and making it much easier to load stuff from other SVDs into it. Common problem for folks is getting an SVD from Axial and not having enough space to load its components.
arock
Posts: 9
Joined: 19:01, 17 July 2014

Re: Reading Jupiter-80 sound sets on a PC/Mac

Post by arock »

Dewdman42 wrote:Arock does play_mms actually hold MP3 or soundfile data of some kind perhaps?
If it did, that would explain the variable length nature of the file. But how would a person actually get that kind of data stored in the storage of the JP80 itself?
Dewdman42
Posts: 137
Joined: 00:42, 14 January 2013

Re: Reading Jupiter-80 sound sets on a PC/Mac

Post by Dewdman42 »

Arock it sounds like you've made some advances on deciphering the file format so yes let's put our heads together on that. That will help a lot.
arock
Posts: 9
Joined: 19:01, 17 July 2014

Re: Reading Jupiter-80 sound sets on a PC/Mac

Post by arock »

Dewdman42 wrote:Arock it sounds like you've made some advances on deciphering the file format so yes let's put our heads together on that. That will help a lot.
Sent you a PM with my email address!
User avatar
flyingace
Posts: 264
Joined: 22:11, 31 August 2011
Location: Central Arkansas, USA
Contact:

Re: Reading Jupiter-80 sound sets on a PC/Mac

Post by flyingace »

I really appreciate all the efforts you guys are taking to come up with an editor/librarian that is so desperately needed and neglected by Roland on their "flagship" synths JP80 & JP50!!

I emailed Roland US a few times over the past few months, each time they say that it's out of their hands and in the hands of the japanese engineers.

I'm thinking about emailing them repeatedly every week from now on. I doubt it will do any good though. :(

Hope you guys succeed, I will pay money for at least a librarian to manage sounds!
Dewdman42
Posts: 137
Joined: 00:42, 14 January 2013

Re: Reading Jupiter-80 sound sets on a PC/Mac

Post by Dewdman42 »

Hello everyone. I have created a github repository where I intend to work on some JP80 librarian stuff. Thanks to the work by kimsnarf and some more work by arock, users from this forum; we have some preliminary information about the SVD file format which I have documented there at github in wiki format. This document is still in progress as we continue to decipher it.

Anyone who is inclined to mess around with this is welcome to join this github project, contribute in any way possible or download anything they see there. Source code snippets or any information about the SVD or BIN file formats will all be welcome there. Be patient, it will be a while before we have a librarian, but those folks made some excellent progress on deciphering much of the SVD file format already and I feel that a librarian WILL come out of all of this eventually.

The name of the project will change later when it becomes more substantial. For now, here it is on github:

https://github.com/steveschow/Jupe

If you just want to read the SVD file format document from the wiki, which is the most interesting thing to find at this point, its here:

https://github.com/steveschow/Jupe/wiki/SVD-File-Format

contact me through github if you would like to become a formal contributor or PM me with any questions you may have. More than one software tool can easily end up on this github repository eventually. For now it will be a repository of information gathering.
Dewdman42
Posts: 137
Joined: 00:42, 14 January 2013

Re: Reading Jupiter-80 sound sets on a PC/Mac

Post by Dewdman42 »

New Main Page for GitHub Librarian development found here: http://steveschow.github.io/Jupe/
Post Reply