regression run in own testcase directory

[asicnet]

I use protected models from the suppliers.
They always write the same files. In parallel running simulation they are always overwritten. Thus a reference comparison is not possible.

I am looking for a simple solution to run the simulation not in the directory ...../vunit_out/modelsim/ with the libraries and modelsim.ini but in ...../vunit_out/modelsim/testxyz.

or run it in the output_path

but unfortunately I can't find anything suitable

The other threads could not help

Maybe a easy solution exists

BR
Helmut

[asicnet]

I have a small solution -n vsim_simulator_mixin.py
add the test_suite_name as parameter in _run_batch_file and define a env-variable VUNIT_UNIQUE_DIR
then i can run all tests without overwriting some files.

call is
return self._run_batch_file(str(batch_file_name),test_suite_name)

def _run_batch_file(self, batch_file_name, test_suite_name, gui=False):
    """
    Run a test bench in batch by invoking a new vsim process from the command line
    """
    env = self.get_env()
    try:
        args = [
            str(Path(self._prefix) / "vsim"),
            "-gui" if gui else "-c",
            "-l",
            str(Path(batch_file_name).parent / "transcript"),
            "-do",
            f'source "{fix_path(batch_file_name)!s}"',
        ]
        sim_path=str(Path(self._sim_cfg_file_name).parent)
        if env['VUNIT_UNIQUE_DIR']:
            sim_path=sim_path+'/'+test_suite_name
            os.system("mkdir -p "+sim_path)
        proc = Process(args, cwd=sim_path)
        proc.consume_output()
    except Process.NonZeroExitCode:
        return False
    return True

Maybe it can be done with an argument --unique_dir.

[LarsAsplund]

A side effect of #785 is that we need to run every thread in a private directory. Try to check out that branch and see if it is solving your problem.

[asicnet]

Ok, I have tested it. Now I get a separate directory for each process. Option -u seems to have no influence anymore. My problem is: How do I get in the post_run() and pre_config() and post_check() functions the simulator_output_path?? the test_suite_name and other information concerning the test, so I can check the correct reference data.

VG
Helmut

[asicnet]

I need a possibility to copy all generated files from the simulation_output_path to the output_path with the current test_suite_name.
BR
Helmut

[LarsAsplund]

Today simulator_output_path is available to pre_config only. What you can do to share it with post_check is something like this.

class TestbenchInfo:
    def __init__(self, tb_name):
        self.tb_name = tb_name
        self.output_path = None
        self.simulator_output_path = None

def make_pre_config(info):
    def pre_config(output_path, simulator_output_path):
        info.output_path = output_path
        info.simulator_output_path = simulator_output_path

        # Your code

        return True

    return pre_config

def make_post_check(info):
    def post_check(output_path):
        # Your code will have all information available through the info object
        return True

    return post_check

info = TestbenchInfo("my_testbench")

tb.set_pre_config(make_pre_config(info))
tb.set_post_check(make_post_check(info))

You can make a post_run the same way the post_check was made

[LarsAsplund]

If you're not running in parallel threads then you will still have files being overwritten. You can always copy files from simulator_output_path to the test unique output_path to avoid that.

[asicnet]

Ahh
def pre_config(output_path, simulator_output_path):
has a second parameter.

Will this important possibility of parallization be taken over into the master branch in the future?

[asicnet]

I mean simulator_output_path and own subdir for each thread

[LarsAsplund]

simulator_output_path is already on master branch but private subdirs is an update in the near future. If you do not use parallel threads it will probably possible for you to copy data between simulator_output_path and output_path to avoid that tests overwrite each other

[asicnet]

Only outputpath is in the docu in Pre and post simulation hooks¶

[LarsAsplund]

Yeah, I see that it is missing from the documentation. I will add that. It is supported though:

vunit/vunit/configuration.py

Line 152 in a66d757

"simulator_output_path": simulator_output_path,

[asicnet]

maybe the same for the undocumented "output" in line 170
-- stopp-- is defined in the docu :)

we use only master branch so we wait for the future support

my workaround for now:

    vu = VUnit.from_argv()
    if vu._args.num_threads > 1:             # -p NUM_THREADS, --num-threads NUM_THREADS 
      os.environ["VUNIT_UNIQUE_DIR"]   = 'True'
    self.unique = os.getenv("VUNIT_UNIQUE_DIR",None)

....

def pre_config(self,output_path,simulator_output_path):
if self.unique:
res = re.search(r'test_output/(.*)_',output_path)
test = res.group(1)
self.run_path = simulator_output_path +"/"+ test
else:
self.run_path = simulator_output_path
return True

in def simulate (...
...
if self._gui:
return self._run_batch_file(str(gui_file_name), gui=True)

    if os.getenv("VUNIT_UNIQUE_DIR",None):
        return self._run_batch_file(str(batch_file_name),test_suite_name=test_suite_name)

....
def _run_batch_file(self, batch_file_name, gui=False, test_suite_name=''):
"""
Run a test bench in batch by invoking a new vsim process from the command line
"""
env = self.get_env()
try:
args = [
str(Path(self._prefix) / "vsim"),
"-gui" if gui else "-c",
"-l",
str(Path(batch_file_name).parent / "transcript"),
"-do",
f'source "{fix_path(batch_file_name)!s}"',
]
sim_path=str(Path(self._sim_cfg_file_name).parent)
if env['VUNIT_UNIQUE_DIR']:
sim_path=sim_path+'/'+test_suite_name
os.system("mkdir -p "+sim_path)
proc = Process(args, cwd=sim_path)
proc.consume_output()
except Process.NonZeroExitCode:
return False
return True