VUnit/vunit

VUnit is a unit testing framework for VHDL & SystemVerilog

Lars Asplund

@LarsAsplund

@peteruran You can also do std_logic_vector'("CA")

sschmitz86

@sschmitz86

i wonder , is someone using vunit with ghdl and code coverage ? I use the ghdl/vunit:gcc-master docker container. It should support code coverage. After i got 0% code coverage on all files in my project i did try the coverage example from vunit. Same here, 0% code coverage

Gitter1510

@Gitter1510

Hi guys
for a student project about verification I wanted to do some tests of a DUT using vunit. My goal is to perform the results of the stimulation with a testbench with the measurement results of a LogicAnalyser of the same system. For this I am looking for a way to generate some kind of result table with the corresponding states and timings of my variables when testing with vunit. Does anyone have an idea ?
I hope I have formulated my problem halfway clearly. I am still completely new in the area of HDL and testing, should I say something wrong please excuse me :)

Translated with www.DeepL.com/Translator (free version)

dpaul

@dpaul24

@Gitter1510 , No not completely clear in what you want to do, but this is what VUnit does - "It features the functionality needed to realize continuous and automated testing of your HDL code. VUnit doesn’t replace but rather complements traditional testing methodologies by supporting a “test early and often” approach through automation." Your testbench is still dictates what you want to do and how do you want to project the simulation results.

Having said that, from what I have understood from your post, do you desire to create a simulation log file containing some specifics?

If it is so, then you might as well use the file operations features of the RTL language you are using. First set it correctly by running normal simulation and later you can automate the test using VUnit.

Aaron Panella

@a-panella

@dpaul24 sounds like @Gitter1510 also wants to compare the output of the analyser to the vunit testbench. In this case they will need to use the logic analyser API to produce a log file in the same format... At least that is how I interpreted the question

Gitter1510

@Gitter1510

Aaron Panella nailed it! Thats exactly what i want to do :)!

is there a planned approach for this use case?

Lars Asplund

@LarsAsplund

@Gitter1510 A testbench typically checks values and states at specific points in time to determine if the DUT operates correctly or not. What you're looking for is a trace of all intermediate values. This is something that you could sample with VUnit but I think it would be better to dump these waveforms using simulator capabilities. For example save the waveform to a VCD file and then compare that to the logic analyzer output. VUnit can still setup the simulator to save the VCD and you can use a Python post check to parse and compare VCD and logic analyzer files.

Gitter1510

@Gitter1510

@LarsAsplund Thank you! Is this described in the documentation or is there an example file ?

Carlos Alberto Ruiz Naranjo

@qarlosalberto

I want to run X parallels simulations (-p argument) but I want to add a delay between launches. Could you point me to the code where I can introduce it? I would like to do a local modification in VUnit to test it

Carlos Alberto Ruiz Naranjo

@qarlosalberto

ok, got it

Lars Asplund

@LarsAsplund

@qarlosalberto What is your use case?

Carlos Alberto Ruiz Naranjo

@qarlosalberto

I'm using the delay for xsim

before xsim I run xelab. It works fine, but xelab have a memory peak at the end. If I run N testbenches in parallel I will have N memory peaks, then Vivado crashes. So I have introduced a configurable delay between launches

vruizescribano

@vruizescribano

Hi everybody. I am using VUnit+GHDL. I am trying to display with the logging library the value of a ufixed signal. In order to do that, I use "info("Fixed Value: " & to_string(to_integer(unsigned(n1))));" where "n1<=to_ufixed(6.2,n1);" The error I got in vunit says "bound check failure at" and then vunit points at the exact line where the info() is. I have read the logging library to find more information about the types that accept but I have not seen anything related to fixed points neither float point. I am also doubting about GHDL (and me) and how it handles the fixed_pkg because when I use the signal "n1" in a comparison, vunit+ghdl says it contains a metadata ('z','x','-' etc ...). Is someone working with vunit+ghdl and the fixed_pkg or float_pkg? Thanks everybody

1 reply

nfrancque

@nfrancque

@vruizescribano The "bound check failure" is coming from ghdl, not vunit. Usually means some math operation you're doing is overflowing.

vruizescribano

@vruizescribano

@vruizescribano The "bound check failure" is coming from ghdl, not vunit. Usually means some math operation you're doing is overflowing.

You are right. It looks like I am having an issue with GHDL because even a simple std_logic_vector is giving me a metadata value. This code was run using Modelsim previously without having any issue. I had big expectation with GHDL but Modelsim makes things easier. I will give a second try to GHDL if someone points out what it could be, otherwise I will switch back to Modelsim after Xmas time. Thanks!

nfrancque

@nfrancque

You might want to move this to the ghdl gitter channel but it could be that modelsim was masking an overflow that is actually occurring. If not, the ghdl people are pretty responsive to bug reports.

Is there an existing flag to vunit to just return after completing rather than exit? Right now I am just catching SystemExit which does work. Use case is for prototyping the design I have a software model of it and want to swap out software functions with running a vunit sim as a "function" with returned data. Ideally for me it would just take the place of the post_run and return the Results as it does now.

nfrancque

@nfrancque

And FYI @vruizescribano I'm using ghdl+vunit+fixed_pkg and things seem to work okay

Lars Asplund

@LarsAsplund

@Gitter1510 Dumping the waveform to a file is a rather simulator specific operation so you have to check how that is done from the command line. VUnit allow you pass such extra command line options. See http://vunit.github.io/py/vunit.html?highlight=set_sim#vunit.ui.VUnit.set_sim_option. Once you have that on file you need to compare that to what you have from the logic analyzer. If you put such Python code in a post check function and return True or False depending on the result of the comparison you will get a fully automated solution. See http://vunit.github.io/py/ui.html#pre-and-post-hooks and http://vunit.github.io/py/vunit.html?highlight=post_check#vunit.ui.testbench.TestBench.set_post_check

vruizescribano

@vruizescribano

@nfrancque you're right. Thanks for the responses. I am new in GHDL and I installed it on my personal Macbook - at work I use Modelsim. I do not think it is a bug in GHDL. It is something related to the configuration and/or set up on my personal laptop because even a check_equal() value of a std_logic_vector fails. I will read more about how to set it up and ask in the vhdl gitter channel. Thanks!

Gitter1510

@Gitter1510

@LarsAsplund thank you very much! that should helpf me a lot :)

Karthik Selvan

@amkichu

I have installed Python and GHDL, GTK are properly added to the environment variables in my windows 10. When I run python run.py for vunit examples, I am encountering the following error.

C:\Projects\vunit\vunit-master\examples\vhdl\user_guide>python run.py
Compiling into vunit_lib: ............\Users\vpallichadath\AppData\Roaming\Python\Python310\site-packages\vunit\vhdl\string_ops\src\string_ops.vhd failed
=== Command used: ===
"C:\Program Files (x86)\Ghdl\bin\ghdl" -a --workdir=C:\Projects\vunit\vunit-master\examples\vhdl\user_guide\vunit_out\ghdl\libraries\vunit_lib --work=vunit_lib --std=08 -PC:\Projects\vunit\vunit-master\examples\vhdl\user_guide\vunit_out\ghdl\libraries\vunit_lib -PC:\Projects\vunit\vunit-master\examples\vhdl\user_guide\vunit_out\ghdl\libraries\lib C:\Users\vpallichadath\AppData\Roaming\Python\Python310\site-packages\vunit\vhdl\string_ops\src\string_ops.vhd

=== Command output: ===
C:\Program Files (x86)\Ghdl\bin\ghdl.exe:command-line: cannot find "std" library

Compile failed

Please note that my python installation folder is C:\Program Files\Python310, however, the "Compiling into vunit_lib: " is pointing to another folder. How can I correct this?

embed-me

@embed-me

Hi there! I have been using VUnit for quite some time now and I really love it,
however, as an Emacs user, I always felt like this seamless integration
is still missing, and switching between terminal and editor just did not
feel right. Therefore I decided to write a trivial minor mode that simply acts as a
wrapper around the VUnit Python Script and allows you to interact with
VUnit from within Emacs.
The package is now even available on MELPA and I would appreciate your
feedback on it. Here you can find a short post on my blog https://embed-me.com/vunit-mode-emacs-vunit and this is the official github repository https://github.com/embed-me/vunit-mode.

Loester Franco

@LoesterFranco

Carlos Alberto Ruiz Naranjo

@qarlosalberto

Good job @embed-me

embed-me

@embed-me

@qarlosalberto Thank you! If you actively use the package please let me know your thoughts ;-)

Dominic

@abaebae

@embed-me that looks really nice. What keeps me from using emacs for VHDL is the refactoring feature Sigasi provides that currently no LSP can match (afaik)

Carlos Alberto Ruiz Naranjo

@qarlosalberto

is there a number of characters limit in the test names?

Lars Asplund

@LarsAsplund

Not that I can think of. There is a file path length limit in Windows that you can hit. We name the test output directory after the test name but we limit that length to allow for some amount of directory hierarchy before that. Depending on the depth of that hierarchy you can still run into length problems. What are you seeing?

Carlos Alberto Ruiz Naranjo

@qarlosalberto

my fail, I was generating a random name. So I had a problem with the test pattern

-.-'

embed-me

@embed-me

@abaebae thank you! Btw, I have never used Sigasis refactoring feature and always thought of it as a "smarter version of query&replace". Is it actually that useful in practice?

1 reply

Peter Uran

@peteruran

Hello. I'm getting a "vunit_lib:avalon_pkg - FAILURE - Writing to address 16382 out of range 0 to 2046" error when using the avalon_slave_t with the avalon_slave verification component. The address bus connected to the VC is 14 bit, so I'm generating random 14 bit addresses with OSVVM. This is not a huge concern, but I found it somewhat limiting and unexpected. I'm having a hard time tracking down exactly where the address range constraint is coming from. Should I file this as an issue in GitHub?

2 replies

Carlos Alberto Ruiz Naranjo

@qarlosalberto

@hipolitoguzman is creating a series of practices for his student (Seville University, Spain). He uses VUnit and GHDL with TerosHDL. https://gitlab.com/edcmit/edc-practicas/-/tree/main/practica2 They are in Spanish, but I share them in case you find them interesting :)

Hipólito Guzmán-Miranda

@hipolitoguzman

Thanks for sharing @qarlosalberto ! you can find the link to all the lab lessons here: https://gitlab.com/edcmit/edc-practicas . It's a work in progress but should be finished in the next three weeks

Hipólito Guzmán-Miranda

@hipolitoguzman

Hi, I was wondering if I could integrate Symbiyosys executions with VUnit, so I can have both functional verification tests and formal verification tests managed with the same framwork (VUnit). Is this possible?

Hipólito Guzmán-Miranda

@hipolitoguzman

Since VUnit needs the testbenches and formal verification by definition doesn't use testbenches, what I think I would like to do is to be able to define 'custom' tests in the run.py file that would call a command I specify (in this case sby), and have those tests added to the auto-discovered tests

Lars Asplund

@LarsAsplund

@hipolitoguzman No, VUnit was not designed for this although it has been used in similar fashion by people familiar with the internals. I guess you could create an empty testbench which serves no purpose other than giving your test a name and create an entry point into VUnit. Then you call your formal test from a post check function. Might be interesting to see where that could lead but what I typically do in situations like these is that I have the "other thing" in a completely different script. The thing that binds everything together in one automated flow is either pytest or my CI tool.

Hipólito Guzmán-Miranda

@hipolitoguzman

Thanks for the answer @LarsAsplund , that makes a lot of sense

Carlos Alberto Ruiz Naranjo

@qarlosalberto

do you know a way to merge different JUnit XML files?

Lars Asplund

@LarsAsplund

@qarlosalberto Have a look åt junitparser

Carlos Alberto Ruiz Naranjo

@qarlosalberto

Thank you very much!

Göran Pettersson

@gopetter_gitlab

Hi,
I get a lot of output whenever a check fails, namely:

# Stack trace result from 'tb' command
#  C:/Python/Python39/Lib/site-packages/vunit/vhdl/core/src/core_pkg.vhd 84 return [address 0xff135acf] Subprogram core_failure
# called from  C:/Python/Python39/Lib/site-packages/vunit/vhdl/logging/src/logger_pkg-body.vhd 1238 return [address 0xff1426ce] Subprogram final_log_check
# called from  C:/Python/Python39/Lib/site-packages/vunit/vhdl/run/src/run.vhd 119 return [address 0xff4bd3c6] Subprogram test_runner_cleanup
# called from  C:/Users/nisse/adesign/testbenches/tb_some_module/tb_some_module.vhd 100 return [address 0xff4c213b] Process main
#
#
# Surrounding code from 'see' command
#   79 :   begin
#   80 :     if core_failure_is_mocked then
#   81 :       set(core_failure_mock_state, core_failure_called_idx, 1);
#   82 :       reallocate(to_string_ptr(get(core_failure_mock_state, core_failure_message_idx)), msg);
#   83 :     else
# ->84 :       report msg severity failure;
#   85 :     end if;
#   86 :   end;
#   87 :
#   88 :   procedure check_core_failure(msg : string := "") is
#

I think this makes it hard to read the output or logs and see what test actually failed.
Is there a way to turn this particular output off?

Lars Asplund

@LarsAsplund

@gopetter_gitlab Maybe I should start by explaining why we want that information in the first place. Whenever an error is detected we obviously want the error message to be seen but we also want to provide help to locate where the error occured. If you're using VUnit's checks for error detection you can automatically have the location of the failing check in the error message. In this case the call stack provides redundant location information. Note however that the coloring of VUnit error messages are there for the purpose of being easily spotted in a longer log.

Where communities thrive

People

Repo info

Activity