为啥直接关注riscv-arch-test，是因为RISCOF 测试框架使用的是riscv-arch-test
1. The architectural test
架构测试是一个单一的测试，代表了可编译和运行的最小测试代码。它是用汇编代码编写的，其产品是test signature。一个架构测试可能由多个测试用例组成。
2. The RISC-V architectural test pool
RISC-V 架构测试池由所有经批准的架构测试组成，这些测试可由测试框架编译，形成架构测试套件。RISC-V 架构测试库必须与测试目标无关（因此，应能在任何符合要求的目标上正确运行）。请注意，这种非功能测试不能替代验证或器件测试。
2.1 Test pool structure
architectural-tests-suite (root)
|-- <architecture>_<mode>/<feature(s)>, where
<architecture> is [ RV32I | RV64I | RV32E ]
<mode> is [ M | MU | MS | MSU ], whereM Machine mode tests - tests execute in M-mode onlyMU Machine/User mode tests - tests execute in both M- & U-modes (S-mode may exist)MS Machine/Supv mode tests - tests execute in both M- & S-modes (not U-mode)MSU All mode tests - tests execute in all of M-, S-, & U-Modes
<feature(s)> are the lettered extension [A | B | C | M ...] or subextension [Zifencei | Zam | ...] when the tests involve extensions, or more general names when tests cut across extension definitionss (e.g. Priv, Interrupt, Vm). The feature string consists of an initial capital letter, followed by any further letters in lower case.
机器模式处理器指令集特性寄存器（MISA）
misa = 0x800000000094112f
二进制表示:1000000000000000000000000000000000000000100101000001000100101111
riscv-arch-test/riscv-test-suite$ ls
env/ Makefile.include README.md rv32e_m/ rv32i_m/ rv64i_m/
riscv-arch-test/riscv-test-suite/rv64i_m$ ls
A/ B/ C/ CMO/ D/ F/ I/ K/ M/ privilege/ P_unratified/ Zfh/ Zfinx/ Zicond/ Zifencei/
3. The RISC-V architectural test suite
RISC-V 架构测试套件是从架构测试池中选出的一组测试，用于测试特定 RISC-V 配置的一致性。测试结果以 test suite signature的形式获得。测试的选择基于目标的断言配置、规范、执行环境或平台要求。合规的处理器或处理器模型应显示与所测试的特定配置的黄金参考测试套件签名相同。
4. The test case
测试用例是架构测试的一部分，只测试规范的一个功能。
注意：一个测试可以包含多个测试用例，每个测试用例都有自己的测试包含条件（由 RVTEST_CASE 宏的 cond_str 参数定义）。
4.1 Test命名
<test objective>-<test number>.S
riscv-arch-test/riscv-test-suite/rv64i_m/I/src$ ls
add-01.S and-01.S bge-01.S bne-01.S lb-align-01.S lhu-align-01.S misalign1-jalr-01.S sd-align-01.S slliw-01.S sltiu-01.S sraiw-01.S srliw-01.S sw-align-01.S
addi-01.S andi-01.S bgeu-01.S fence-01.S lbu-align-01.S lui-01.S or-01.S sh-align-01.S sllw-01.S sltu-01.S sraw-01.S srlw-01.S xor-01.S
addiw-01.S auipc-01.S blt-01.S jal-01.S ld-align-01.S lw-align-01.S ori-01.S sll-01.S slt-01.S sra-01.S srl-01.S sub-01.S xori-01.S
addw-01.S beq-01.S bltu-01.S jalr-01.S lh-align-01.S lwu-align-01.S sb-align-01.S slli-01.S slti-01.S srai-01.S srli-01.S subw-01.S
5. Assembly Test Infrastructure (以./rv32i_m/I/src/add-01.S为例)
//
// This assembly file tests the add instruction of the RISC-V I extension for the add covergroup.
//
#include "model_test.h"
#include "arch_test.h"
RVTEST_ISA("RV32I").section .text.init
.globl rvtest_entry_point
rvtest_entry_point:
RVMODEL_BOOT
RVTEST_CODE_BEGIN#ifdef TEST_CASE_1RVTEST_CASE(0,"//check ISA:=regex(.*32.*);check ISA:=regex(.*I.*);def TEST_CASE_1=True;",add)RVTEST_SIGBASE( x3,signature_x3_1)inst_0:
// rs2 == rd != rs1, rs1==x4, rs2==x24, rd==x24, rs1_val > 0 and rs2_val > 0, rs2_val == 1, rs1_val == (2**(xlen-1)-1), rs1_val != rs2_val, rs1_val == 2147483647
// opcode: add ; op1:x4; op2:x24; dest:x24; op1val:0x7fffffff; op2val:0x1
TEST_RR_OP(add, x24, x4, x24, 0x80000000, 0x7fffffff, 0x1, x3, 0, x18)inst_1:
// rs1 == rs2 != rd, rs1==x10, rs2==x10, rd==x28, rs1_val > 0 and rs2_val < 0, rs2_val == -257, rs1_val == 131072
// opcode: add ; op1:x10; op2:x10; dest:x28; op1val:0x20000; op2val:0x20000
TEST_RR_OP(add, x28, x10, x10, 0x40000, 0x20000, 0x20000, x3, 4, x18)inst_2:
// rs1 == rs2 == rd, rs1==x21, rs2==x21, rd==x21, rs1_val < 0 and rs2_val < 0, rs1_val == -16777217
// opcode: add ; op1:x21; op2:x21; dest:x21; op1val:-0x1000001; op2val:-0x1000001
TEST_RR_OP(add, x21, x21, x21, 0xfdfffffe, -0x1000001, -0x1000001, x3, 8, x18)
......
......
......
RVTEST_CODE_END
RVMODEL_HALTRVTEST_DATA_BEGIN
.align 4rvtest_data:
.word 0xbabecafe
.word 0xbabecafe
.word 0xbabecafe
.word 0xbabecafe
RVTEST_DATA_ENDRVMODEL_DATA_BEGIN
rvtest_sig_begin:
sig_begin_canary:
CANARY;signature_x3_0:.fill 0*(XLEN/32),4,0xdeadbeefsignature_x3_1:.fill 17*(XLEN/32),4,0xdeadbeefsignature_x8_0:.fill 16*(XLEN/32),4,0xdeadbeefsignature_x1_0:.fill 512*(XLEN/32),4,0xdeadbeefsignature_x1_1:.fill 43*(XLEN/32),4,0xdeadbeef#ifdef rvtest_mtrap_routinetsig_begin_canary:
CANARY;
mtrap_sigptr:.fill 64*(XLEN/32),4,0xdeadbeef
tsig_end_canary:
CANARY;#endif#ifdef rvtest_gpr_savegpr_save:.fill 32*(XLEN/32),4,0xdeadbeef#endifsig_end_canary:
CANARY;
rvtest_sig_end:
RVMODEL_DATA_END
简单对上面的汇编代码解释如下：
- Header to inlcude comments
#This assembly file tests the add instruction of the RISC-V I extension for the add covergroup.
- Includes header files
#include “model_test.h”
#include “arch_test.h”
每个测试应只包括以下头文件：
model_test.h - 定义特定于目标的宏，包括必填的宏和可选的宏：（如 RVMODEL_xxx）
arch_test.h - 定义预定义的测试宏，包括必填的宏和可选的宏：（如 RVTEST_xxx）
- Set the TVM of the test
RVTEST_ISA(“RV32I”)
- Test target specific boot-code
RVMODEL_BOOT
- Start of GPR initialization routine and test code
RVTEST_CODE_BEGIN
- Define the RVTEST_CASE string and conditions
#ifdef TEST_CASE_1
// this test is meant for devices implementing rv32I extension and requires enabling the compile
// macro TEST_CASE_1. This test will contribute to the “add” coverage label.
RVTEST_CASE(0,“//check ISA:=regex(.32.);check ISA:=regex(.I.);def TEST_CASE_1=True;”,add)
- Initialize pointer to the signature region
RVTEST_SIGBASE( x16,signature_x16_1) // x16 will point to signature_x16_1 label in the signature region
- Define the test cases
inst_0:
// rs2 == rd != rs1, rs1==x4, rs2==x24, rd==x24, rs1_val > 0 and rs2_val > 0, rs2_val == 1, rs1_val == (2**(xlen-1)-1), rs1_val != rs2_val, rs1_val == 2147483647
// opcode: add ; op1:x4; op2:x24; dest:x24; op1val:0x7fffffff; op2val:0x1
TEST_RR_OP(add, x24, x4, x24, 0x80000000, 0x7fffffff, 0x1, x3, 0, x18)inst_1:
// rs1 == rs2 != rd, rs1==x10, rs2==x10, rd==x28, rs1_val > 0 and rs2_val < 0, rs2_val == -257, rs1_val == 131072
// opcode: add ; op1:x10; op2:x10; dest:x28; op1val:0x20000; op2val:0x20000
TEST_RR_OP(add, x28, x10, x10, 0x40000, 0x20000, 0x20000, x3, 4, x18)
...
...
//Tests for a instructions with register-register operand
#define TEST_RR_OP(inst, destreg, reg1, reg2, correctval, val1, val2, swreg, offset, testreg) \TEST_CASE(testreg, destreg, correctval, swreg, offset, \LI(reg1, MASK_XLEN(val1)) ;\LI(reg2, MASK_XLEN(val2)) ;\inst destreg, reg1, reg2 ;\)
#define TEST_CASE(testreg, destreg, correctval, swreg, offset, code... ) ;\code ;\RVTEST_SIGUPD(swreg,destreg,offset) ;\RVMODEL_IO_ASSERT_GPR_EQ(testreg, destreg, correctval)/* automatically adjust base and offset if offset gets too big, resetting offset *//* RVTEST_SIGUPD(basereg, sigreg) stores sigreg at offset(basereg) and updates offset by regwidth *//* RVTEST_SIGUPD(basereg, sigreg,newoff) stores sigreg at newoff(basereg) and updates offset to regwidth+newoff */
#define RVTEST_SIGUPD(_BR,_R,...) ;\.if NARG(__VA_ARGS__) == 1 ;\.set offset,_ARG1(__VA_OPT__(__VA_ARGS__,0)) ;\.endif ;\CHK_OFFSET(_BR, REGWIDTH,0) ;\SREG _R,offset(_BR) ;\.set offset,offset+REGWIDTHRVMODEL_IO_ASSERT_GPR_EQ 定义在target的model_test.h中```在咱们得model_test.h中将RVMODEL_IO_ASSERT_GPR_EQ 宏定义如下：比较错误的话，往0xF0000080写1```c
#define RVMODEL_IO_ASSERT_GPR_EQ(_S, _R, _I) \li _S, 0xF0000080; \mv t0, _R; \li t3, _I; \beq t0, t3, 1f; \li t2, 1; \sw t2, 0(_S); \j 2f; \
1: \li t2, 0; \sw t2, 0(_S); \
2: \nop;```在tb.v中加入监测对AXI 写地址总线，地址0xF0000080的监测，如果出现fail_cnt >0，可以判断该testcase错误```c
// RVMODEL_IO_ASSERT_GPR_EQ(testreg, destreg, correctval) used to check destreg == correctval
// destreg != correctva write testreg 1, else write testreg 0
always @(posedge `CPU_CLK) beginif ((cpu_awaddr[31:0] == 32'hF000_0080) && cpu_wvalid && `clk_en && (cpu_wstrb[15:0] == 16'hf)) beginif(`SOC_TOP.biu_pad_wdata == 1'b1) beginfail_cnt ++;endend
end
- Change signature base register
// this will change the signature base register to x3. x3 will not point to signature_x3_0 in
// the signature region
RVTEST_SIGBASE( x3,signature_x3_0)// continue with new test cases ..
TEST_RR_OP(add, x4, x24, x27, 0x55555955, 0x00000400, 0x55555555, x3, 0, x5)
...
...
- End the test and halt the test-target
RVTEST_CODE_END
RVMODEL_HALT
- Create test input data section
RVTEST_DATA_BEGIN
rvtest_data:
.word 0xbabecafe
RVTEST_DATA_END
- Create pre-loaded signature region
RVMODEL_DATA_BEGIN
rvtest_sig_begin:
sig_begin_canary:
CANARY;signature_x3_0:.fill 0*(XLEN/32),4,0xdeadbeefsignature_x3_1:.fill 17*(XLEN/32),4,0xdeadbeefsignature_x8_0:.fill 16*(XLEN/32),4,0xdeadbeefsignature_x1_0:.fill 512*(XLEN/32),4,0xdeadbeefsignature_x1_1:.fill 43*(XLEN/32),4,0xdeadbeef#ifdef rvtest_mtrap_routinetsig_begin_canary:
CANARY;
mtrap_sigptr:.fill 64*(XLEN/32),4,0xdeadbeef
tsig_end_canary:
CANARY;#endif#ifdef rvtest_gpr_savegpr_save:.fill 32*(XLEN/32),4,0xdeadbeef#endifsig_end_canary:
CANARY;
rvtest_sig_end:
RVMODEL_DATA_END
6. The test case signature
测试用例signature由单个或多个值表示。值将以 RVMODEL_DATA_BEGIN 指定的地址为起点，以 RVMODEL_DATA_END 指定的地址为终点写入内存。使用 RVTEST_SIGUPD 宏很容易生成signature。
7. The test signature
测试 signature 是由architectural test运行生成的特征值。测试 signature可能由多个测试用例 signature 组成，前缀是一个单独的行，其中包含测试的名称和表示其版本的唯一值。测试target负责从内存中提取值并适当地格式化它们，使用由框架提供的元数据，使用 RVMODEL_DATA_BEGIN 和 RVMODEL_DATA_END 宏。测试用例 signature 值按行写入，从最左边的最高有效字节开始，格式为 <hex_value>，其中值的长度将为 32 位（因此为 8 个字符），而实际测试计算的值长度不考虑。文件应从 signature 的最低地址处的值开始存储（即从 RVMODEL_DATA_BEGIN 到 RVMODEL_DATA_END）。此外，signature 应始终从 16 字节（128 位）边界开始，signature的大小应为 4 字节的倍数（即也应以 4 字节边界结束）。
8. The test suite signature
测试套件 signature 被定义为一组对于给定的架构测试套件有效的测试 signature。它代表了选择的特定 RISC-V 配置的架构测试套件的测试 signature。
9. RISCOF 测试框架
RISCOF - RISC-V 兼容性框架是一个基于 Python 的框架，它使得可以使用一套 RISC-V 架构测试集来测试 RISC-V 目标（硬件或软件实现）与标准的 RISC-V 黄金参考模型的兼容性。
RISC-V Configuration Validator : RISCV-Config
RISC-V Compliance Test Generator : RISC-V CTG
RISC-V ISA Coverage : RISC-V ISAC
为了RISCOF能正常运行测试，需要提供以下内容：
- config.ini：这个文件是一个基本的配置文件，遵循 ini 语法。这个文件将捕获信息，比如：DUT/reference 插件的名称，插件的路径，基于 riscv-config 的 YAML 文件的路径等。
- dut-plugin 目录：RISCOF 要求测试的 DUT 模型以 Python 插件的形式提供。Python 插件实际上就是一个包含某些标准和定义函数的 Python 文件，用于执行测试编译、执行和签名提取的活动。这个 Python 文件的名称需要以 riscof_ 为前缀，并且必须存在于 dut-plugin 目录中。可以参考 Python 插件文件部分，了解如何编写这个 Python 文件。
该目录还需要包含基于 riscv-config 的 isa 和 platform YAML 文件，这些文件提供了 DUT 的定义。这些 YAML 文件将用于过滤需要在 DUT 上运行的测试。
最后，在 dut-plugin 目录中还需要存在一个 env 目录，其中包含环境文件，如 model_test.h，这是编译和运行测试所需的文件。请参考 TestFormat 规范，了解可以在 model_test.h 文件中使用的宏的定义。env 目录还可能包含其他文件，如链接脚本、用户可能需要的后处理脚本。 - reference-plugin 目录：与 DUT 插件类似，RISCOF 也需要一个参考模型插件。目录和文件的结构与 DUT 的相同。但是，不需要 isa 和 platform YAML 文件，因为 RISCOF 将始终从 DUT 插件中选择所有目的的 YAML 文件。
为了简化操作，RISCOF 通过设置命令为用户生成标准的 DUT 和参考模型预置模板，如下图所示：
$ riscof setup --dutname=spike
上述命令将在当前目录下生成以下文件和目录：
├──config.ini # configuration file for riscof
├──spike/ # DUT plugin templates
├── env
│ ├── link.ld # DUT linker script
│ └── model_test.h # DUT specific header file
├── riscof_spike.py # DUT python plugin
├── spike_isa.yaml # DUT ISA yaml based on riscv-config
└── spike_platform.yaml # DUT Platform yaml based on riscv-config
├──sail_cSim/ # reference plugin templates
├── env
│ ├── link.ld # Reference linker script
│ └── model_test.h # Reference model specific header file
├── init.py
└── riscof_sail_cSim.py # Reference model python plugin.
将上面的spike改为C920的plugin就可以了，当然需要修改各个配置文件和python文件
c920_isa.yaml
hart_ids: [0]
hart0:ISA: RV64IMAFDCVZicsr_Zicbom_Zicbop_Zicboz_Zihintpause_Zfh_Zca_Zcb_Zcd_Zba_Zbb_Zbc_Zbsphysical_addr_sz: 40User_Spec_Version: "2.2"Privilege_Spec_Version: "1.10"hw_data_misaligned_support: falsepmp_granularity: 4supported_xlen: [64]
c920_platform.yaml
mtime:implemented: trueaddress: 0xBFF8
mtimecmp:implemented: trueaddress: 0x4000
nmi:label: nmi_vector
reset:address: 0x000000000
执行下面的命令，会从test_list.yaml 提取case列表跑各个case，可以注释掉其中一些，只跑部分case
riscof run --config=config.ini
–suite=riscv-arch-test/riscv-test-suite/
–env=riscv-arch-test/riscv-test-suite/env
–testfile=riscof_work/test_list.yaml
执行下面的命令会生成test_list.yaml 并且跑regression
riscof run --config=config.ini
–suite=riscv-arch-test/riscv-test-suite/
–env=riscv-arch-test/riscv-test-suite/env
10. 测试结果
错误原因分析：
第一种：
是因为编译的时候遇到错误
INFO | Compiling test: /ssd_fes/jiongz/desktop/github/c920_riscof1/riscv-arch-test/riscv-test-suite/rv64i_m/I/src/beq-01.SERROR | /opt/picocom/ThirdParty_Libs/T-head/C920_R2S0P21/C920_R2S0_manuals_and_tools/manuals_and_tools/08_toolchain_900_series_cpu_toolchain/V2.8.0/Xuantie-900-gcc-elf-newlib-x86_64-V2.8.0/bin/../lib/gcc/riscv64-unknown-elf/10.4.0/../../../../riscv64-unknown-elf/bin/ld: main.elf section `.text' will not fit in region `MEM1'
collect2: error: ld returned 1 exit status
第二种：
就是c920默认没使能某个指令
11. rv64i_m/I/src/add-01.S 波形
model_test.h中还有一个宏，用来dump signature和finish simulation
// This will dump the test results (signature) via the testbench dump module.
#define RVMODEL_HALT \signature_dump: \la a0, begin_signature; \la a1, end_signature; \li a2, 0xF0000040; \signature_dump_loop: \bge a0, a1, signature_dump_end; \lw t0, 0(a0); \sw t0, 0(a2); \addi a0, a0, 4; \j signature_dump_loop; \signature_dump_end: \nop; \terminate_simulation: \li a0, 0xF0000000; \li a1, 0xCAFECAFE; \sw a1, 0(a0); \j terminate_simulation
对应tb.v中有
always @(posedge `CPU_CLK or negedge `CPU_RST) beginif (!`CPU_RST) beginmsi <= 1'b0;mei <= 1'b0;mti <= 1'b0;end else begin//if ((wb_cpu.cyc == 1'b1) && (wb_cpu.stb == 1'b1) && (wb_cpu.we == 1'b1) && (cpu_awaddr[31:0] == 32'hF000_0000)) beginif ((cpu_awaddr[31:0] == 32'hF000_0000) && cpu_wvalid && `clk_en && (cpu_wstrb[15:0] == 16'hf)) begincase (`SOC_TOP.biu_pad_wdata[31:0])32'hCAFE_CAFE: begin // end simulation$display("Finishing simulation.");#100;if(fail_cnt >0) begin$error("This case test failed!");end$finish;end......end// Signature Dump
int dump_file; // Declare file handle
always @(posedge `CPU_CLK) beginif ((cpu_awaddr[31:0] == 32'hF000_0040) && cpu_wvalid && `clk_en && (cpu_wstrb[15:0] == 16'hf)) beginif (!dump_file) begin // Check if file is already opendump_file = $fopen("DUT-c920.signature", "w"); // Open file if not already openedend//for (int i = 7; i >= 0; i--) begin// $fwrite(dump_file, "%h\n", `SOC_TOP.biu_pad_wdata[i*4 +: 4]); // Write data//end$fwrite(dump_file, "%h\n", `SOC_TOP.biu_pad_wdata[31:0]); // Write dataendelse if((cpu_awaddr[31:0] == 32'hF000_0000) && cpu_wvalid && `clk_en && (cpu_wstrb[15:0] == 16'hf)) beginif (dump_file) begin // If file is open, close it$fclose(dump_file);dump_file = 0; // Reset file handle to 0 indicating file is closedendend
end// RVMODEL_IO_ASSERT_GPR_EQ(testreg, destreg, correctval) used to check destreg == correctval
// destreg != correctva write testreg 1, else write testreg 0
always @(posedge `CPU_CLK) beginif ((cpu_awaddr[31:0] == 32'hF000_0080) && cpu_wvalid && `clk_en && (cpu_wstrb[15:0] == 16'hf)) beginif(`SOC_TOP.biu_pad_wdata == 1'b1) beginfail_cnt ++;endend
end
#include "model_test.h"
#include "arch_test.h"
RVTEST_ISA("RV64I").section .text.init
.globl rvtest_entry_point
rvtest_entry_point:
RVMODEL_BOOT
RVTEST_CODE_BEGIN#ifdef TEST_CASE_1RVTEST_CASE(0,"//check ISA:=regex(.*64.*);check ISA:=regex(.*I.*);def TEST_CASE_1=True;",add)RVTEST_SIGBASE( x8,signature_x8_1)inst_0:
// rs1 == rs2 != rd, rs1==x0, rs2==x0, rd==x20, rs1_val > 0 and rs2_val > 0, rs1_val == 4, rs1_val==4 and rs2_val==6148914691236517206, rs1_val != rs2_val
// opcode: add ; op1:x0; op2:x0; dest:x20; op1val:0x0; op2val:0x0
TEST_RR_OP(add, x20, x0, x0, 0x0, 0x0, 0x0, x8, 0, x16)inst_1:
// rs2 == rd != rs1, rs1==x2, rs2==x26, rd==x26, rs1_val > 0 and rs2_val < 0, rs2_val == -1073741825
// opcode: add ; op1:x2; op2:x26; dest:x26; op1val:0x5; op2val:-0x40000001
TEST_RR_OP(add, x26, x2, x26, 0xffffffffc0000004, 0x5, -0x40000001, x8, 8, x16)inst_2:
// rs1 == rs2 == rd, rs1==x22, rs2==x22, rd==x22, rs1_val < 0 and rs2_val < 0, rs1_val == -8388609
// opcode: add ; op1:x22; op2:x22; dest:x22; op1val:-0x800001; op2val:-0x800001
TEST_RR_OP(add, x22, x22, x22, 0xfffffffffefffffe, -0x800001, -0x800001, x8, 16, x16)
“riscof_work/rv64i_m/I/src/add-01.S/dut/DUT-c920.signature” 内容如下：
e7d4b281
6f5ca309
00000000
00000000
c0000004
ffffffff
fefffffe
ffffffff
ffffffbf
007fffff
00000080
00000000
66666665
e6666666
00000001
00000000
0001ffff
80000000
10000001
00000000
fffffeff
