Yet another Pulpissimo (on Zedboard) bring-up thread

[stefanct]

Hi,

I don't want to hijack one of the other current threads on getting started with Pulpissimo and my problem does not seem to be related to any of them anyway.
I am trying to get Pulpissimo to run on a Zedboard and would like to ask anybody who successfully done this to share his/her exact git revisions of the various related repositories. I think this might be the cause of my problems.

What (presumably) works:

1. Synthesis: I was able to synthesize the design and download it to the zedboard. When I do so the hardware sets the LD0 LED.
   
2. Compilation: I have built the PULP-SDK and I am able to compile binaries using pulp-rt. So far I have only tested the hello pulp-rt-example.
   
3. Debugging: Since I don't have the Digilent JTAG adapter I am using an old Altera USB Blaster attached to the PMOD pins. I have changed the OpenOCD configuration accordingly and are able to connect, load and control execution of the RISC-V core with GDB. On first connect it is stopped at 0x1c008080, i.e. _start.
   

*However*, when I load and run the application that way, it does not work as expected. If we assume that we can trust the debugger then it seems like it gets stuck in rt_user_alloc(). pt and  size get initialized to 0 and 144 (0x90), respectively. Thus, the function returns to rt_alloc() where it is retried. On the second, try pt gets initialized to 0x4c4b40 because a equals the address of __rt_freq_domains and it tries to get its first_free field which is completely bogus!?

Since rt_event_alloc() uses the result of rt_cluster_id() for the values eventually ending up in rt_user_alloc() I presume this might be one of the problems that changing the core ID has as discussed in https://github.com/pulp-platform/pulpissimo/issues/107

KR

[bluewww]

While I'm not exactly aware what all the things are that break I know that the sdk and runtime assume a certain pattern in the coreid to figure out if a core is a cluster core (cluster doesn't exist in PULPissimo though) or a fabric controller core. Indeed the lower coreids are used to indicate that we have a cluster id.

You could try to do the bring-up with https://github.com/pulp-platform/pulp-runtime/ which has been recently developed to have a simple bare bones runtime. This could help you with your debugging efforts.

[stefanct]

While I'm not exactly aware what all the things are that break I know that the sdk and runtime assume a certain pattern in the coreid to figure out if a core is a cluster core (cluster doesn't exist in PULPissimo though) or a fabric controller core. Indeed the lower coreids are used to indicate that we have a cluster id.

You could try to do the bring-up with https://github.com/pulp-platform/pulp-runtime/ which has been recently developed to have a simple bare bones runtime. This could help you with your debugging efforts.

Will do, thanks a lot for the fast reply. BTW I have tried to use the bare option of rt/mode of pulp-sdk with exactly that intent but it seems completely broken in the current build system (cf. pulp_opt_rt.mk).

One more thing I forgot to mention in the original post about rt_user_alloc: GDB (built from the pulp toolchain repository) can't decode an instruction apparently:

Code:

 0x1c008a2c  rt_user_alloc+0  lw    a5,0(a0)
 0x1c008a2e  rt_user_alloc+2  addi    a1,a1,7
 0x1c008a30  rt_user_alloc+4  0xc405b5b3
 0x1c008a34  rt_user_alloc+8  li    a4,0

I did not try to decode it manually but I wonder why this might happen since I am using a "matching" GCC and GDB/binutils specifically for pulp/ri5cy.

---

pulp-runtime is not working either, but differently :)
It seems to get stuck in pos_soc_init() and unlike with pulp-sdk breaking execution does not work correctly:

Code:

Breakpoint 1, pos_init_start () at .../pulp-runtime/kernel/init.c:59
59      pos_soc_init();
>>> n
^Cunable to halt hart 0
  dmcontrol=0x80000001
  dmstatus =0x00030c82
Could not read registers; remote failure reply 'E0E'
Could not read registers; remote failure reply 'E0E'

I need to start over completely in that case, i.e. re-download the bitstream, restart openocd and gdb.

When looking more closely what happens I reckon there is either a bug or I am not supposed to use pulp-runtime's pulpissimo configuration the way I do.
ARCHI_NB_FLL is set to 2 for pulpissimo. This define is used to define the size of the pos_freq_domains array.
However, in pos_soc_init() the code tries to write to pos_freq_domains[PI_FREQ_DOMAIN_PERIPH] where PI_FREQ_DOMAIN_PERIPH equals 2, i.e. this is out of bounds. *shrug*

The readme of pulp-runtime is unclear to me. I tried following it as follows to create the binary:

1. set all the environment variables as for pulp-sdk
   
2. cd pulp-rt-examples/hello (which has a modified makefile already setting e.g. -g3)
   
3. source .../pulp-runtime/configs/fpgas/pulpissimo/genesys2.sh
   
4. make clean all (which seems to work as intended, output below)
   

Code:

RM  .../pulp-rt-examples/hello/build
CC  test.c
CC  .../pulp-runtime/kernel/fll-v1.c
CC  .../pulp-runtime/kernel/freq-domains.c
CC  .../pulp-runtime/kernel/chips/pulpissimo/soc.c
CC  .../pulp-runtime/lib/libc/minimal/io.c
CC  .../pulp-runtime/lib/libc/minimal/fprintf.c
CC  .../pulp-runtime/lib/libc/minimal/prf.c
CC  .../pulp-runtime/lib/libc/minimal/sprintf.c
CC  .../pulp-runtime/kernel/init.c
CC  .../pulp-runtime/kernel/kernel.c
CC  .../pulp-runtime/kernel/alloc.c
CC  .../pulp-runtime/kernel/alloc_pool.c
CC  .../pulp-runtime/kernel/irq.c
CC  .../pulp-runtime/kernel/soc_event.c
CC  .../pulp-runtime/kernel/bench.c
CC  .../pulp-runtime/drivers/uart.c
CC  .../pulp-runtime/kernel/crt0.S
CC  .../pulp-runtime/kernel/irq_asm.S
LD  .../pulp-rt-examples/hello/build/test/test

[bluewww]

Are you running this with the mhartid set to 0? If not, do you use openocd master (atleast with this patch https://github.com/riscv/riscv-openocd/c...52438fec6a).
Regarding the instruction decode error, what is your output of "riscv32-unknown-elf-gcc --verbose --version"?
Unfortunately I don't have a zedboard so I can't really help you debug. Internally we use genesys2 and the other FPGA ports are contributions, which besides running synthesis I can't verify.

[stefanct]

Are you running this with the mhartid set to 0? If not, do you use openocd master (atleast with this patch https://github.com/riscv/riscv-openocd/c...52438fec6a).
Regarding the instruction decode error, what is your output of "riscv32-unknown-elf-gcc --verbose --version"?
Unfortunately I don't have a zedboard so I can't really help you debug. Internally we use genesys2 and the other FPGA ports are contributions, which besides running synthesis I can't verify.

Yes, as described in the issue and below. I don't think that openocd's handling of the ID is to blame. I am using e03dd199 (from 2019-11-28) of the UCB riscv-openocd. I have emailed Marek (the original zedboard porter) to chime in... let's see. Apart from the hart ID modification the hardware is unchanged so I don't think this is necessarily a Zedboard-specific problem.

Code:

-    localparam FC_Core_CLUSTER_ID    = 6'd31;
+    localparam FC_Core_CLUSTER_ID    = 6'd0;
-    localparam NrHarts                               = 1024;
+    localparam NrHarts                               = 1;

The gcc version output as used by the hello makefile:

Code:

.../pulp-rt-examples/hello$ riscv32-unknown-elf-gcc --verbose --version
Using built-in specs.
COLLECT_GCC=riscv32-unknown-elf-gcc
COLLECT_LTO_WRAPPER=.../pulp-riscv-gnu-toolchain-build/libexec/gcc/riscv32-unknown-elf/7.1.1/lto-wrapper
riscv32-unknown-elf-gcc (GCC) 7.1.1 20170509
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


Target: riscv32-unknown-elf
Configured with: .../pulp-riscv-gnu-toolchain/riscv-gcc/configure --target=riscv32-unknown-elf --prefix=.../pulp-riscv-gnu-toolchain-build --disable-shared --disable-threads --enable-languages=c,c++ --with-system-zlib --enable-tls --with-newlib --with-headers=.../pulp-riscv-gnu-toolchain-build/riscv32-unknown-elf/include --disable-libmudflap --disable-libssp --disable-libquadmath --disable-libgomp --disable-nls --enable-checking=yes --enable-multilib --with-abi=ilp32 --with-arch=rv32imc 'CFLAGS_FOR_TARGET=-Os  -mcmodel=medlow'
Thread model: single
gcc version 7.1.1 20170509 (GCC)
COLLECT_GCC_OPTIONS='-v' '--version' '-march=rv32imc' '-mabi=ilp32'
 .../pulp-riscv-gnu-toolchain-build/libexec/gcc/riscv32-unknown-elf/7.1.1/cc1 -quiet -v help-dummy -quiet -dumpbase help-dummy -march=rv32imc -mabi=ilp32 -auxbase help-dummy -version --version -o /tmp/ccek0sdN.s
GNU C11 (GCC) version 7.1.1 20170509 (riscv32-unknown-elf)
    compiled by GNU C version 8.3.0, GMP version 6.1.2, MPFR version 4.0.2, MPC version 1.1.0, isl version isl-0.20-GMP

GGC heuristics: --param ggc-min-expand=30 --param ggc-min-heapsize=4096
COLLECT_GCC_OPTIONS='-v' '--version' '-march=rv32imc' '-mabi=ilp32'
 .../pulp-riscv-gnu-toolchain-build/lib/gcc/riscv32-unknown-elf/7.1.1/../../../../riscv32-unknown-elf/bin/as -v --traditional-format -march=rv32imc -mabi=ilp32 --version -o /tmp/ccnINujK.o /tmp/ccek0sdN.s
GNU assembler version 2.28.0 (riscv32-unknown-elf) using BFD version (GNU Binutils) 2.28.0.20170505
GNU assembler (GNU Binutils) 2.28.0.20170505
Copyright (C) 2017 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or later.
This program has absolutely no warranty.
This assembler was configured for a target of `riscv32-unknown-elf'.
COMPILER_PATH=.../pulp-riscv-gnu-toolchain-build/libexec/gcc/riscv32-unknown-elf/7.1.1/:.../pulp-riscv-gnu-toolchain-build/libexec/gcc/riscv32-unknown-elf/7.1.1/:.../pulp-riscv-gnu-toolchain-build/libexec/gcc/riscv32-unknown-elf/:.../pulp-riscv-gnu-toolchain-build/lib/gcc/riscv32-unknown-elf/7.1.1/:.../pulp-riscv-gnu-toolchain-build/lib/gcc/riscv32-unknown-elf/:.../pulp-riscv-gnu-toolchain-build/lib/gcc/riscv32-unknown-elf/7.1.1/../../../../riscv32-unknown-elf/bin/
LIBRARY_PATH=.../pulp-riscv-gnu-toolchain-build/lib/gcc/riscv32-unknown-elf/7.1.1/:.../pulp-riscv-gnu-toolchain-build/lib/gcc/riscv32-unknown-elf/7.1.1/../../../../riscv32-unknown-elf/lib/:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-v' '--version' '-march=rv32imc' '-mabi=ilp32'
 .../pulp-riscv-gnu-toolchain-build/libexec/gcc/riscv32-unknown-elf/7.1.1/collect2 -plugin .../pulp-riscv-gnu-toolchain-build/libexec/gcc/riscv32-unknown-elf/7.1.1/liblto_plugin.so -plugin-opt=.../pulp-riscv-gnu-toolchain-build/libexec/gcc/riscv32-unknown-elf/7.1.1/lto-wrapper -plugin-opt=-fresolution=/tmp/ccMIfxqH.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgloss -plugin-opt=-pass-through=-lgcc -melf32lriscv -dT riscv.ld --march=rv32imc --version .../pulp-riscv-gnu-toolchain-build/lib/gcc/riscv32-unknown-elf/7.1.1/../../../../riscv32-unknown-elf/lib/crt0.o .../pulp-riscv-gnu-toolchain-build/lib/gcc/riscv32-unknown-elf/7.1.1/crtbegin.o -L.../pulp-riscv-gnu-toolchain-build/lib/gcc/riscv32-unknown-elf/7.1.1 -L.../pulp-riscv-gnu-toolchain-build/lib/gcc/riscv32-unknown-elf/7.1.1/../../../../riscv32-unknown-elf/lib /tmp/ccnINujK.o -lgcc --start-group -lc -lgloss --end-group -lgcc .../pulp-riscv-gnu-toolchain-build/lib/gcc/riscv32-unknown-elf/7.1.1/crtend.o
collect2 version 7.1.1 20170509
.../pulp-riscv-gnu-toolchain-build/lib/gcc/riscv32-unknown-elf/7.1.1/../../../../riscv32-unknown-elf/bin/ld -plugin .../pulp-riscv-gnu-toolchain-build/libexec/gcc/riscv32-unknown-elf/7.1.1/liblto_plugin.so -plugin-opt=.../pulp-riscv-gnu-toolchain-build/libexec/gcc/riscv32-unknown-elf/7.1.1/lto-wrapper -plugin-opt=-fresolution=/tmp/ccMIfxqH.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgloss -plugin-opt=-pass-through=-lgcc -melf32lriscv -dT riscv.ld --march=rv32imc --version .../pulp-riscv-gnu-toolchain-build/lib/gcc/riscv32-unknown-elf/7.1.1/../../../../riscv32-unknown-elf/lib/crt0.o .../pulp-riscv-gnu-toolchain-build/lib/gcc/riscv32-unknown-elf/7.1.1/crtbegin.o -L.../pulp-riscv-gnu-toolchain-build/lib/gcc/riscv32-unknown-elf/7.1.1 -L.../pulp-riscv-gnu-toolchain-build/lib/gcc/riscv32-unknown-elf/7.1.1/../../../../riscv32-unknown-elf/lib /tmp/ccnINujK.o -lgcc --start-group -lc -lgloss --end-group -lgcc .../pulp-riscv-gnu-toolchain-build/lib/gcc/riscv32-unknown-elf/7.1.1/crtend.o
GNU ld (GNU Binutils) 2.28.0.20170505
Copyright (C) 2017 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.
COLLECT_GCC_OPTIONS='-v' '--version' '-march=rv32imc' '-mabi=ilp32'

[stefanct]

Thanks to a hint from Marek I was able to get it working. The FC ID is hard-coded in include/archi/chips/pulpissimo/properties.h located in the archi git repository (i.e., runtime/archi/include/archi/chips/pulpissimo/properties.h within the sdk repository). When ARCHI_FC_CID there is changed to 0 debugging with the upstream OpenOCD can be used. There are some remaining issues and debugging (interactions between the HW, OpenOCD and GDB) seems more fragile than I would have hoped but main() is reached. I have not verified UART output or any peripherals yet though.

[stefanct]

The UART is also working. I have created a pull request for some documentation and build updates. Thanks everyone.