VUnit/vunit

VUnit is a unit testing framework for VHDL & SystemVerilog

Rafael Pereira

@rnp:matrix.org

:point_up: Edit: I am playing with VUnit + AXI Stream example at (https://github.com/VUnit/vunit/blob/master/examples/vhdl/array\_axis\_vcs/src/test/tb\_axis\_loop.vhd). I replaced the vunit_lib.axi_stream_master by one I wrote. However I am getting an error (Questa 2021.04, VUnit master branch, on Linux):

# \*\* Fatal: (SIGSEGV) Bad handle or reference.

# Time: 0 ns  Iteration: 0  Process: /axi\_stream\_master\_min\_tb/\<implicit\_\_87\_3> File: /home/user/src/vunit/vunit/vhdl/verification\_components/src/axi\_stream\_slave.vhd

# Fatal error in Subprogram data\_length at /home/user/src/vunit/vunit/vhdl/verification\_components/src/axi\_stream\_pkg.vhd line 497

How could I instantiate the axi_stream_master I wrote properly?

The testbench code: https://gist.github.com/rafaelnp/b8c906ae71a8e0b0daa2d5fe98e6b895

# Attempting stack trace sig 11
# Signal caught: signo [11]
# vsim_stacktrace.vstf written
# Current time Sun Nov 20 22:03:56 2022
# Program = vsim
# Id = "2021.2"
# Version = "2021.04"
# Date = "Apr 14 2021"
# Platform = "linux_x86_64"
# Signature = 4180c34bc81bf15e13c03b3939ce2075
# 0    0x00007feff0de7a01: '/home/rnp/src/vunit/vunit/vhdl/verification_components/src/axi_stream_pkg.vhd:499'
# 1    0x00007feff0fc44b0: '/home/rnp/src/vunit/vunit/vhdl/verification_components/src/axi_stream_slave.vhd:32'
# 2    0x000000000345e8ea: '<unknown (@0x345e8ea)>'
# 3    0x00000000004b7b01: '<unknown (@0x4b7b01)>'
# 4    0x00000000006e8153: '<unknown (@0x6e8153)>'
# 5    0x0000000000c02e55: '<unknown (@0xc02e55)>'
# 6    0x0000000000c07d13: '<unknown (@0xc07d13)>'
# 7    0x0000000000c09a6e: '<unknown (@0xc09a6e)>'
# 8    0x0000000000ed3c0d: '<unknown (@0xed3c0d)>'
# 9    0x00000000038eccbd: '<unknown (@0x38eccbd)>'
# 10   0x00000000038f1116: '<unknown (@0x38f1116)>'
# 11   0x00000000038f2801: '<unknown (@0x38f2801)>'
# 12   0x00000000038f2b66: '<unknown (@0x38f2b66)>'
# 13   0x00000000011456c9: '<unknown (@0x11456c9)>'
# 14   0x0000000003992a6f: '<unknown (@0x3992a6f)>'
# 15   0x00000000039e65e7: '<unknown (@0x39e65e7)>'
# 16   0x00000000039a94d7: '<unknown (@0x39a94d7)>'
# 17   0x00000000039a97b9: '<unknown (@0x39a97b9)>'
# 18   0x000000000378ef5d: '<unknown (@0x378ef5d)>'
# 19   0x0000000000bced3c: '<unknown (@0xbced3c)>'
# End of Stack Trace
# \*\* Fatal: (SIGSEGV) Bad handle or reference.
# Time: 0 ns  Iteration: 0  Process: /axi\_stream\_master\_min\_tb/\<implicit\_\_87\_3> File: /home/user/src/vunit/vunit/vhdl/verification\_components/src/axi\_stream\_slave.vhd
# Fatal error in Subprogram data\_length at /home/user/src/vunit/vunit/vhdl/verification\_components/src/axi\_stream\_pkg.vhd line 497

How could I instantiate the axi_stream_master I wrote properly?

The testbench code: https://gist.github.com/rafaelnp/b8c906ae71a8e0b0daa2d5fe98e6b895

Rafael Pereira

@rnp:matrix.org

[m]

I took a look at the example again, and the supported use case is a DUT with a axi stream slave and master interface, and the VC Bus controls the BFMs, and the use case where the DUT is and start/end point (either the axi stream master or slave present) is not supported, e.g. axi-stream to HDMI. Is this statement correct?

Jim Lewis

@JimLewis

@rnp:matrix.org What have you done to debug your VC? I recommend using /* */ to comment out all of your code and try to load it. Then in the fashion of a binary search add pieces back in until you find the code that is crashing the tool. It should not take too long if you are methodical. Done this many times developing OSVVM VC.

1 reply

Lars Asplund

@LarsAsplund

@rnp:matrix.org You mentioned that your starting point is one of our examples. Does that work before you started to modify? I also noted that you created a handle master_axi_stream that is used when pushing data with push_axi_stream. However, the new master VC, axi_stream_master_min, doesn't use that handle so how will it receive the pushed data? Maybe it's easier to start with the reason for writing your own master? What is the goal of that master? In general, I think you would have to provide all code such that the problem can be reproduced

2 replies

okhajut

@okhajut

I am looking for an old message but am not able to find it. Maybe the answer is in a reply somwhere and so is not visible. Gitter does not appear to have a method to search for text in chat.

The basic problem was as follows:

A design has two variants, one for Xilinx and another for Microsemi. There are certain IP modules in the design for which the architecture changes depending on what variant is chosen but the port map remains the same. I need to be able to tell VUnit which variant to simulate at a given time where the default variant is Microsemi. How should I do this?

Lars Asplund

@LarsAsplund

@okhajut There are several approaches depending on what is most suitable. The starting point would be to have an if-generate statement selecting the correct entity based on a generic which than can be controlled by a VUnit configuration. There is also a development branch that you might be interesting in: https://github.com/VUnit/vunit/tree/add-configuration-support. It adds support for dealing with VHDL configurations. Within that there is a blog to be released along with the feature: https://github.com/VUnit/vunit/blob/add-configuration-support/docs/blog/vhdl_configurations.rst. It has been sitting there for a while but what's left is really to make sure it works under XCelium which I can't test myself.

okhajut

@okhajut

See the issue here is that there are two modules that are completely identical in their port map but have different architectures. The architectures refer to libraries that are vendor specific. Therefore, the approach I have taken is to use VUnitCLI, pass in an argument that selects one or the other variant (default is Xilinx) and then when the file tree is being searched, exclude any file that ends with _xilinx.vhd or _microsemi.vhd depending on the chosen variant.

Now another thing I am trying to find but can't is the name of the testbench that is going to be run inside the run.py Python script and also inside a TCL script that would be passed to the simulator program. The former is very important to work out actually.

Lars Asplund

@LarsAsplund

@okhajut That works. A drawback is that the run script doesn't see the full set of tests so it limits what you can do with a single call of the script. You say that port maps are identical but the architectures differ. Are the entity and architecture names also identical such that only one can be compiled into the same library at the same time?

The set of tests that will be run is the same set of tests listed if you make the same call to the script but add the -l option. Is that the list you're looking for?

okhajut

@okhajut

The testbenches used and everything is same. In one variant I am using Xilixn DPRAM, in the other I am using Microsemi RTG4 DPRAM. The DPRAM behave identical except the EDAC signals which are grounded in the Xilinx version but not in the RTG4 version. But the EDAC signals are just registered and driven to top level ports and not used internally by the design.

Across both variants, the tests remain the same. The design fuctionality remains the same, except the EDAC signals.

Anyway, this topic can be closed.

The other question I posted which is a different topic but that message got merged into a single message that I posted before was that, the run.py needs to decide what simulation waveform (.do file) to tell the simulator to load it when it starts in GUI mode. In order to select the correct .do file, it is important for the run.py to know what specific testbench is being used in the current test. For this reason, I need to know how it is possible to find what testbench is going to be run in VUnitwhen it is started in GUI mode so that I can load the correct simulation waveform.

Lars Asplund

@LarsAsplund

@okhajut Are you using set_sim_option with

modelsim.init_file.gui
A user defined TCL-file that is sourced after the design has been loaded in the GUI. For example this can be used to configure the waveform viewer. During script evaluation the vunit_tb_path variable is defined as the path of the folder containing the test bench. Must be a string.

Note that the set_sim_option method can be used globally, per testbench or per test

For example

prj.set_sim_option(...) # globally
tb = prj.test_bench("tb_something")
tb.set_sim_option(...) # On testbench
test = tb.test("my test case")
test.set_sim_option(...) # On test case

okhajut

@okhajut

I have found this thing called 'set sim option'.

However, the method is not clear. In your example you have strings "tb_something" "my test case". The basic idea is to fetch the test bench name as a string. If I need to provide the name myself then it creates a catch-22.

Lars Asplund

@LarsAsplund

So are you saying that you don't have convenient access to the testbench names in advance but if you would be given a name you have the knowledge to select what do file to run with that?

okhajut

@okhajut

OK, here is what I am trying to do. The simulation testbench names start with tb_ while the simulation waveform .do file has the same name as testbench followed by _wave.do. Therefore, e.g when VUnit starts in GUI mode to run tb_jpeg_dct.vhd, the waveform file is tb_jpeg_dct_wave.do. Now I can only tell the simulator what waveform .do file to execute if I know the actual name of the testbench inside the run.py.

I need to know the name of the specific testbench that has been chosen to run in GUI mode and based on that the simulation waveform file name shall be chosen. What other way is there to specify simulation waveform? Simulation waveform won't actually be loaded when running in batch mode. It is only ever needed when running a specific testbench inside GUI mode.

Lars Asplund

@LarsAsplund

@okhajut Something like this maybe (assuming your VUnit object is called prj):

for lib in prj.get_libraries():
    for tb in lib.get_test_benches(allow_empty=True):  # Don't fail on libs w/o testbenches. vunit_lib, for example
        do_path = tb.name + ".do"  # Probably need some additional work for a full path.
        tb.set_sim_option("modelsim.init_file.gui", do_path)

okhajut

@okhajut

I think I have misunderstood something here so far. What your code shows is that we do set_sim_options for individual testbenches and tests. Then, depending on what is being run, VUnit will ensure that the approprite setting in set_sim_options for that testbench test gets executed. This means that, I do the set_sim_options per testbench or per test e.g for tb_jpeg_fdct and then if I run a different testbench or test, then VUnit will not pass that setting to the simulator. Is this correct?

As far as I know, we pass a script as the set_sim_option. Does this mean that the run.py needs to dynamically create a script and save it as .tcl file and then pass it to VUnit? Or does this mean we create the TCL script as a string, and then pass the string into the set_sim_option? Certainly a mere path to a file is not a TCL command.

Lars Asplund

@LarsAsplund

@okhajut modelsim.init_file.gui takes a string but the string contains a file path (see https://vunit.github.io/py/opts.html#simulation-options). The file then contains things like add wave my_signal. Maybe you can show your script?

Lars Asplund

@LarsAsplund

@/all There has been a number of issues related to Questa lately. For you who haven't created an issue I would appretiate if you do and include example, Questa version and error messages. I've created a Questa label such that it should be easier to spot this issues. I'm working on getting access to Questa but meanwhile the best approach is for you with simulator access to help each other out.

okhajut

@okhajut

See the only thing that requires clarification is this, since the set_sim_option method exists for testbench object and also test object, this means that can be have a whole set of set_sim_option for each testbench and/or test in the run.py and only a specific one that applies to the current testbench & test being run will be executed and the others will be ignored. In other words, the set_sim_option is local to testbench and tests, it is not a global thing.

Is this correct?

Tom

@halfbit:matrix.org

[m]

are there alternatives to writing a custom tcl script or edalize for taking my plain vhdl project and building for a part/board with a constraints file?

I tried edalize and it works ok but then to load it in something like vivado feels quirky

quartus lite sadly doesn't seem to support vhdl08 which was surprising, annoying, and has now eliminated targetting the parts there I think... unless I downgrade to an older quartus

Lars Asplund

@LarsAsplund

@okhajut The VUnit configuration concept includes a range of settings to use with testbenches and test cases. What simulation options to use is one such setting but there are others, for example what values to assign to testbench generics.

For every testbench and test case there is a default configuration. When you use set_sim_option you're updating that default configuration. If you use that method on the VUnit object, the default configuration of every testbench and test case is modified. You can also set it on a testbench to affect only test cases within and you can set it on a single test case. Such a setting overrides what you've done previously on a higher level.

You can also create named configurations (the default configuration has no name) for a testbench and a test case by using add_config. The purpose of that is to run the same tests with several different settings.

Your run script contains configurations for all your tests but only the configuration associated with the running testbench or test case is active.

I recommend that you have a look at https://github.com/VUnit/tdd-intro/blob/master/tutorial/exercise_03/instructions.md to get the hang of it.

Carlos Alberto Ruiz Naranjo

@qarlosalberto

do you know why VUnit doesn't read my system environment variable?

from vunit.sim_if.factory import SIMULATOR_FACTORY

print(SIMULATOR_FACTORY.select_simulator())

(.venv) carlos@fpga-vivado:~/Pictures$ export VUNIT_SIMULATOR=modelsim
(.venv) carlos@fpga-vivado:~/Pictures$ python run.py 
None

Lars Asplund

@LarsAsplund

@qarlosalberto What happens if you do

print(os.environ["VUNIT_SIMULATOR"])

Misspelling of the environment variable or its value could be the cause

Though I see that you haven't misspelled...

Anyway, if Python can't get the value then the problem isn't located to VUnit

Carlos Alberto Ruiz Naranjo

@qarlosalberto

ahh okok, got it!

ModelSim isn't installed in that server

kvantumnuly

@kvantumnuly

Hi all,
I in my DUT, I am use the fixed_pkg_2008 library and it seems that works well with Vunit. I am trying integrate whit DUT into my testbench but still without success. It seems that there is a "collision" between subprogram defined in std_logic_1164 and subprorgam in fixed_pkg_2008. Please, could you suppose some solution?

** Error: c:\projects\g4hw\iau-mixer\hdl\fixed_pkg_2008.vhd(7188): (vcom-1602) Subprogram "to_ostring" is ambiguous.
   Feasible subprograms are:
      (explicit) fixed_pkg.to_ostring[STD_LOGIC_VECTOR, std.TEXTIO.SIDE, NATURAL return std.STANDARD.STRING] at c:\projects\g4hw\iau-mixer\hdl\fixed_pkg_2008.vhd(7022)
      (explicit) std_logic_1164.TO_OSTRING[STD_ULOGIC_VECTOR return std.STANDARD.STRING] at $MODEL_TECH/../vhdl_src/ieee/stdlogic.vhd(263)
** Error: c:\projects\g4hw\iau-mixer\hdl\fixed_pkg_2008.vhd(7190): (vcom-1602) Subprogram "to_ostring" is ambiguous.
   Feasible subprograms are:
      (explicit) fixed_pkg.to_ostring[STD_LOGIC_VECTOR, std.TEXTIO.SIDE, NATURAL return std.STANDARD.STRING] at c:\projects\g4hw\iau-mixer\hdl\fixed_pkg_2008.vhd(7022)
      (explicit) std_logic_1164.TO_OSTRING[STD_ULOGIC_VECTOR return std.STANDARD.STRING] at $MODEL_TECH/../vhdl_src/ieee/stdlogic.vhd(263)
** Error: c:\projects\g4hw\iau-mixer\hdl\fixed_pkg_2008.vhd(7215): (vcom-1602) Subprogram "to_hstring" is ambiguous.
   Feasible subprograms are: ...

Thanks in advance!

Richard Head

@trickyhead_gitlab

what is fixed_pkg_2008?

seems your simulation tool already has fixed_pkg in the IEEE library?

kvantumnuly

@kvantumnuly

@trickyhead_gitlab
fixed_pkg_2008 is package thath allows use fixed point features under Vivado:
https://docs.xilinx.com/v/u/2018.1-English/ug901-vivado-synthesis (page 212).
And yes- in the Vivado I have compiled it into IEEE library.

Richard Head

@trickyhead_gitlab

It seems the newest version of Vivado you dont need to add the library externally anymore, like in synthesis: https://docs.xilinx.com/r/en-US/ug900-vivado-logic-simulation/Fixed-and-Floating-Point-Packages

kvantumnuly

@kvantumnuly

@trickyhead_gitlab oh, it chanching the situation. I will check it. Thanks for your fast response!

dominikdomanski

@dominikdomanski

Hi, I'm trying to use a VUnit fork https://github.com/kmtaylor/vunit to use VUnit with Vivado (xsim), but having problem with setting Vunit up to use xsim - I got Modelsim running but want to switch to Vivado (try it out). Where can I find settings for it?

Lars Asplund

@LarsAsplund

@dominikdomanski Maybe @ludli505_gitlab can help you out with an example.

1 reply

curiousengineer

@curiousengineer

When running tests in GUI by using the --gui switch, does the GUI need to be closed each time before starting the next test in a testbench? In other words, is it possible to run through all tests in a testbench without having to close the GUI at all? This will help me do some analysis with the code coverage among others things. The alternative to this is that I copy paste all the different tests into a single test in the VHDL testbench and then just run that 1 test.

Lars Asplund

@LarsAsplund

@curiousengineer Have a look at the all_in_same_sim attribute here: https://vunit.github.io/run/user_guide.html#running-test-cases-independently

curiousengineer

@curiousengineer

The attribute "run_all_in_same_sim" is actually being added anywhere. What I can see is that there is a loop starting with "while test_suite loop". Where is the attribute specified or used?

Lars Asplund

@LarsAsplund

It's a VUnit attribute which is a comment starting with vunit:. See top line of referenced example.

Where communities thrive

People

Repo info

Activity