Pulpissimo: Using JTAG and Flash for larger sets of data

[LPLA]

I couldn't think of a more descriptive title so let me please explain what i am trying to achieve.

I'm using a hardware accelerator and have quite some amount of data (weights, inputs, etc) that i need to send to the accelerator.
The idea was to use the SPI flash model to store the flash_stim (and the other data) in the flash memory, without using the UserPreload feature.
Now there are a few things that aren't clear to me which i couldn't resolve by reading the issues on github or reading the forum.

Here is my approach so far:

First i wrote an application to send data to the flash memory, similar to the send example.
I created an array that contains some data with the intend of having this array contain the data from the flash_stim.slm file (without the addresses).
This worked for smaller data sets where i defined an array with random sample data to send to the flash.
At some point the array (and therefore the whole code) was simply too big to fit in the Pri banks of the L2 memory and i was stuck in "Waiting for end of computation".

So now i am using JTAG to load my data into the non Pri banks of the L2 memory.
The goal is to send this data to the flash memory, effectively making the flash memory contain the same data as if i had been using the UserPreload feature of the S25FS256S model.
Sending data to the L2 via JTAG of course works fine but also raises the first question.
There are different stimuli files for either JTAG (stim.txt) or STANDALONE (flash_stim.slm).
Effectively they should both do the same thing in the end, but are different in size and format.
For example i can't use readmemh to load the flash_stim.slm into the stimuli array.

As far as i understand there is no way to directly load the flash memory via JTAG because it's not memory mapped, correct?
If this is true, then would the correct way to do this be:
1) Use JTAG to load the image (in this case the stim.txt) into L2 just as you would do when booting from L2 but use a different address (e.g. 0x1C030000).
2) Use SPI to transfer that data to the flash memory (this i do by code, similar to the send example i mentioned, using pulp_read32 to get the data from L2, but this seems rather inefficient).
3) Now i should be able to boot from ROM even tho i didn't use the UserPreload feature.

What i am wondering about are the following things:
- Is the way i described above the correct or rather most efficient way to do this?
- Is there a way to load the flash_stim.sml to flash without using the UserPreload feature or converting the format so readmemh can be used?
- This issue mentions the way to do this would be to write data from JTAG to L2 (like i am doing too) and then to engage the udma to transfer this data to the flash. Unfortunately i'm not sure how you would do this and i couldn't find a documentation or guide on this.

Thank you for your help,
LPLA

[bluewww]

> There are different stimuli files for either JTAG (stim.txt) or STANDALONE (flash_stim.slm).
stim.txt is a list of commands for the PULP jtag TAP while flash_stim.slm is a memory dump of the executable to be preoloaded during simulation

> As far as i understand there is no way to directly load the flash memory via JTAG because it's not memory mapped, correct?

The flash chip is connected to SPI which in turn is accessible over the UDMA (only afaik).

> - Is the way i described above the correct or rather most efficient way to do this?
It certainly should work like this if you get the layout of the data in the flash right

> Is there a way to load the flash_stim.sml to flash without using the UserPreload feature or converting the format so readmemh can be used?
Do you want to use readmemh on the L2 or on the flash memory? Readmemh and L2 is not supported because its quite annoying to properly split the executable among the memory banks we have, but certainly doable.

> This issue mentions the way to do this would be to write data from JTAG to L2 (like i am doing too) and then to engage the udma to transfer this data to the flash. Unfortunately i'm not sure how you would do this and i couldn't find a documentation or guide on this.
You could have a look at the udma driver in the PULP sdk.

[LPLA]

Thank you for your reply.

Do you want to use readmemh on the L2 or on the flash memory? Readmemh and L2 is not supported because its quite annoying to properly split the executable among the memory banks we have, but certainly doable.

The idea is to simulate the following scenario: I have some data for the accelerator on some external memory device. This could be the weights, the actual program, etc. Now i want to load that data to the internal flash memory. After that i want to boot from the internal flash memory (STANDALONE). This would also need a linker script later on so that my weights, etc. are loaded into L2 at the correct addresses. So in a nutshell create an image (program+weights, etc.) and load it into flash using JTAG.

It seems that i can't do [Source] -> [JTAG] -> [SPI] -> [Flash Memoy] but rather [Source] -> [JTAG] -> [L2] -> [SPI] -> [Flash Memoy] or at least in the way i'm currently trying to simulate the scenario.

The steps i am taking at the moment are the following:
1) Create an image file, lets call it example_image.txt which is in the same format as the other JTAG stimulus files.
2) In the testbench: Set the UserPreload of the SPI flash model to 0.
3) In the testbench: Create another array for the flash image (simlar to the array for the stimulus vectors).
4) In the testbench: Use readmemh to load the example_image.txt into the array for the flash image.
5) In the testbench: Use debug_mode_if.load_L2 to load the array into L2 via JTAG (choose the address so that i won't interfere with the address of other potential data).
6) Create a C program which uses the rt_spim_send function to load the data from L2 to the flash memory.
7) Run that program with boot from JTAG.
8) ToDo: After the program has finished "boot" from rom to run the program that has been loaded into the flash memory.

Now there are a few steps where i'm unsure if this idea even makes sense.
Step 1) I would rather load the image in .slm format into flash but i can't use readmemh to load it into the array, therefore i can't use JTAG to load it into the L2.
Step 6) This requires the core do to work. I'd rather load the flash image while the core is halted if that is possible.

I hope my intentions are clear, if not please tell me so i can try to go into more detail.
My biggest worry is that this won't work at all because i'm not loading the correct image (.txt vs .sml).

Thank you for your help,
LPLA