Distributed simulations?

alexg · Senior Joined: Jan 07, 2004 Posts: 586 Location: Ottawa

I am looking at distributed simulations as the one of possible ways to speed up simulations for large designs. As a simple example, design and testbench may run in parallel on 2 computers, using data structures to communicate with each other. It would be interesting to know if such an expertize exists in the industry.

Regards,
-Alex

qwk000 · Posted: Fri Feb 11, 2011 10:29 am **Post subject:**

I don't know of any simulators that can support multi-machine computing. VCS supports multi-processor (parallel) computing, but you are looking beyond just multi-processor right?

alexg · Senior Joined: Jan 07, 2004 Posts: 586 Location: Ottawa

Few more words about distributed simulations as I see it.

There is no need for specific simulators - any simulator will work fine.
To further clarify my intent, I'll give another example.

Assume, we have 2 design blocks. These blocks communicate with each other using data frames, and, together, perform some function F which has to be verified. Assume also, that both blocks are quite large, and, together with their block-level testbenches, already consume significant amount of simulation time. In order to verify function F, there can be 2 solutions:

1. Put them together and combine their testbenches
2. Let 2 block-level testbenches run in parallel on 2 different machines, supplying output transactions of the first block to the input of the second one.

2-nd approach is obviously faster than the 1-st one. Block-level testbenches remain almost intact. There is just a need to implement data communication between two testbenches. Using untimed transactions for communication may significantly reduce communication frequency.

-Alex

srini · Posted: Fri Feb 11, 2011 3:31 pm **Post subject:**

How about using Socket based communication across the 2 machines (running the 2 blocks)? If this is an option, I recall VCS had an example perhaps in $VCS_HOME, we even ported to use SV-DPI.

Srini
www.cvcblr.com/blog
_________________
Srinivasan Venkataramanan
Chief Technology Officer, CVC www.cvcblr.com
A Pragmatic Approach to VMM Adoption
SystemVerilog Assertions Handbook
Using PSL/SUGAR 2nd Edition.
Contributor: The functional verification of electronic systems

pavanshanbhag · Posted: Sat Feb 12, 2011 11:38 am **Post subject:**

AXIOM-EDA : Has the solution for your problem. It has a Multi CPU architecture - graphical debugging using single kernel.
MPSim is the simulator which is been designed by Axiom folks, that was designed from the beginning to address RTL, testbenches, assertion, coverage and debugging in a single kernel architecture for maximum performance productivity and predictability.

Its better to check with them :
info@axiom-da.com
_________________
-Pavan K Shanbhag

“The difference between genius and stupidity, genius knows his limits.” - Albert Einstein

peterpb · Junior Joined: Mar 18, 2004 Posts: 9

You might take a look at SimCluster from Avery Design:
http://www.avery-design.com/files/docs/SimClusterDS2010.pdf

FYI.
I don't (and didn't) work for Avery-Design. I only know that, a few years back, some engineers tried it. And I heard positive comments about the tool.

pavanshanbhag · Posted: Sun Feb 13, 2011 12:28 pm **Post subject:**

alexg · Senior Joined: Jan 07, 2004 Posts: 586 Location: Ottawa

Peter, Pavan,
Thank you for the link to the Avery Design Solution.

I have 2 problems with it:
1. I don't want to use automated partition of design/testbench. In other words - I would prefer to partition design and testbench by myself.
2. I would like to create my own means to compress/send/decompress signal-level data. I believe, I can do it better than any tool can do Wink

So, I am thinking more about the method mentioned by Srini.

Recently, I was playing with Verilog fileIO and i looks that it provides relatively clean solution to transfer SV packet structures between 2 simulations. Here is the basic idea:

1. Simulators connect with "channels", being able to transfer 1 structure at a time ("blocking" type of communication)
2. Each "channel" is just a file. Using functions and tasks :can_get, get, can_put and put, 2 simulators can communicate through the file.
You can download working example of such communication (there is README file in the tarball).

Here is the link to the file:
http://www.box.net/shared/u7ssqrk2an

It would be good to know your opinion about this method.

Regards,
-Alex

srini · Posted: Sun Feb 13, 2011 11:55 pm **Post subject:**

Hi Alex,

chm · Posted: Mon Feb 14, 2011 4:06 am **Post subject:**

Hi Alex,

chrisspear · Posted: Mon Feb 14, 2011 10:58 am **Post subject:**

Parallel simulation has been tried for decades, and partitioning is always the biggest problem. You need multiple partitions with the following requirements:
-Parallel activity. Dividing the design in two blocks does no good if block B can only run after block A completes
-Equal sizes: If block A is more than 3x block B, you'll get little benefit from running them on separate processors
-Low communication: If the blocks are constantly sharing large amounts of data (a serial activity), you'll get little benefit from running them in parallel

The best designs today for parallel simulations are multi-core CPUs. Equal size, parallel activity, but still a lot of share resources, so may still not give a great speedup.

VCS has automatic partitioning, and the ability to run common activities such as waveform dumping in parallel with the rest of simulation. Look into this tried and tested solution before you go off and try to build your own. (Yes, I do work for Synopsys.)[/list][/list]
_________________
Chris Spear
Co-Author: SystemVerilog for Verification - 3rd edition!
http://chris.spear.net/systemverilog

alexg · Senior Joined: Jan 07, 2004 Posts: 586 Location: Ottawa

Hi Chris,

Thank you for your answer. Please see my comments for the requrements you've mentioned:

cabriggs · Posted: Wed Feb 23, 2011 4:22 pm **Post subject:**

Ed Arthur of Cisco talked about this at a DV Club meeting a few years ago: http://www.dvclub.org/images/Presentations/Arthur_Q207.pdf

They built a custom layer on on top of MPI and the result is somewhat similar to Avery's SimCluster. This shows you one way to do it yourself.

alexg · Senior Joined: Jan 07, 2004 Posts: 586 Location: Ottawa

Thank you. It's an interesting presentation.

-Alex.

jmcneal · Posted: Tue Mar 01, 2011 2:56 pm **Post subject:**

Alex -

As has been pointed out, your NFS solution would be vastly slower than the supported options already listed.

Many years ago I worked at Avery Design and installed SimCluster at several customer sites. You do not have to go with automatic partitioning, but that often makes partitioning much easier, especially for flat gate level designs. If you already have several blocks that consume roughly the same compute resources, and communication between them is slow or infrequent, you could see some significant speed up.

I've used VCS's parallel simulation recently as well (Currently working at Synopsys).

You get the best bang for buck when you have a process that is really large, or simulations that are really slow. These tools aren't a way to take a 2-hr simulation and make it run in 30 mins, but more of a way to take a 6 day simulation and run it in 12 hrs. Several times at Avery we'd get an N-1 (N= # of processors) speedup for very large/very slow simulations.