Why Be Concerned With Simulation Performance?

postgenerate · Senior Joined: Jan 18, 2004 Posts: 31 Location: Phoenix

Is cycle-per-second ASIC simulation performance one of your concerns? If so, maybe you’re doing something wrong!

I can remember back in the early days of RTL languages (1990) when we needed to transition from using Verilog to VHDL as our tool of choice for developing ASICs for our new F-22 combat system. Uncle Sam was telling us that VHDL was going to be the new emerging standard.

We put together a list of features we needed to make the new simulator compatible with our planned environment and then we proceeded to call in the vendors. The main theme the vendors pushed was “Its all about performance”. Our response was “Wait Mr. Vendor, what good is performance if your simulator does not work with LSI Logic MDE tools, or with the old Logic Modeling Hardware Modeler, or the LM software models? What about your 1076 compliance? What does your roadmap look like?” It was clear that choosing a simulator because of a 10 or 20% performance advantage would be a disastrous decision if it did not easily fit in our environment.

Today it’s almost an entirely different picture. Simulators today have reached a mature status, so vendors are left with little but to still rant over their outstanding benchmark performance. I maintain that the importance (or lack of) of this remains the same. The word today is “Smart” Verification. Today it’s about using the simulation cycles you have wisely and not measuring your worth as a verification engineer by how many cycles you can throw at the device you are verifying. It’s about truly understanding the device you are trying to verify and writing structured constrained random stimulus to target all corner cases as efficiently as possible. It’s about using metrics to determine value of your tests with detailed functional coverage code, and using temporal declarative code to evaluate protocol behavior in an understandable fashion. This is “New Generation” testbench development and provides breakthroughs in verification quality and development cycle schedules regardless of simulator cycle per second performance.

Certainly better performance is nice and there are always exceptions that absolutely require the highest performance simulator and days worth of stimulus. On the other hand, with the maturity of simulators today, and the wealth of inexpensive Linux hardware coupled with transaction based HVL/simulator interfaces, in most situations isn’t your time better spent invested in your verification methodology rather than getting that last CPS from your simulator?

I am a strong advocate of using SystemC for “new generation” testbench development. My methodology stretches the OSCI simulator as far as it will go, uses PLI interfaces to stimulate and probe the device under test along with simulator specific kernel interfaces. My methodology involves running everything on Linux servers, and structuring my environments to gain portability by switching on and off vendor specific features such as assertions. We switch in vender tools as desired because they offer immediate value in accomplishing specific tasks, not because we are handcuffed to them for legacy reasons.

I for one am tired of listening to performance shootouts, performance pitches, benchmarks etc. My performance is measured by how fast and how well I can meet the project verification goals not how may cycles per second I can get out of my simulator. Or, maybe my tools of choice also just happen to be good performing tools.

Tom West
Phoenix AZ
See more at www.openverificationfoundation.org

z · Senior Joined: Jan 09, 2004 Posts: 92

Dan Joyce presented a specific approach to understand the device for better verification (http://www.deepchip.com/items/0410-07.html). This DVCon'03 paper and slides can be downloaded at http://img.cmpnet.com/deepchip/downloads/Dan_Audit.zip.

Darren · Posted: Fri Jan 23, 2004 9:50 am **Post subject:**

This is where the verification process gets philosophical. My view as a verification engineer is that to a large degree I do not want to know about the internals of the design I am verifying - the target specification document should be adequate enough for me to verify to and to generate my functional coverage points from. If it is not, then the concept engineer has not done his job properly, as he has not fully thought through the implications of what he is doing. If I have to understand the internals of the design to verify it, then I run the risk of falling under the same set of assumptions that the designer did - what I am trying to do is verify that what the designer did matches the target specification, not that the designer has designed what he thought he did.

Although simulator performance isn't the be all and end all, and of course you should get as much intelligent verification per cycle as possible, it is still of importance. If one simulator is 50% faster than another, then what may be a days regression becomes half a day, allowing more rapid respin and rechecking over-night.

To further answer Z's point - verification engineers should be involved in the specification phase, as they should ensure that the specification is (a) verifiable, and (b) good enough to verify from. If the spec does not match either of these, then it shouldn't be released to the design phase.

vhdlcohen · Posted: Fri Jan 23, 2004 5:51 pm **Post subject:**

z · Senior Joined: Jan 09, 2004 Posts: 92

My original queston can be rephrased as whether RTL provides "Documentation with unambiguous definition of interface and design properties" for verification engineers. RTL clearly provides "Documentation with unambiguous definition of interface and design properties" for verification tools as well as synthesis tools. RTL is a valid supplement to the specification document according to some people but not others. Hardly anybody would use RTL as a replacement of the specification document.

Now we also have assertions in PSL (or another language). If the assertions are available, is there any need of the specification document any more? Does the specification document still have to be as complete as before? Is there any reason to still use RTL as a supplement to the specification document at all? In short term, assertions likely will only supplement RTL and the specification document in its documentation role for verification engineers. How about long term? Would some people write assertions based on the specification document and/or RTL? Would some people write RTL and/or the specification document based on the assertions? Would the same person write assertions as well as RTL and/or the specification document?

The first paper at last summer's DAC presented how Intel people used assertions. If I understand it correctly, Intel people wrote assertions based on the specification document. They intended to start discussing whether to replace some testbenches with assertions. In addition to using assertions to verify designs, they also mathematically proved the correctness of the assertions against some higher-level (more readable) assertions, which ABV vendors have not promoted. So, some assertions are more readable than others, and people apparently need the less readable assertions.

Each assertion is likely concise, but the entire collection of assertions may not. Before any tool (or, at least, any guideline) is available to check the redundancy and the completeness of an assertion collection, people can write too many or too few assertions. As reported in the Intel paper, a complete redundant collection (maybe many of these collections?) of assertions can be useful. The completeness and the redundancy are not important if assertions do not replace anything. However, assertions has only limited values if they are only extra work for extra benefit.

In reality, assertions can be redundant easily. On one side of an interface, an engineer makes an assertion "the output has to follow this rule". On the other side of the same interface, another engineer makes another assertion "the input has to follow this rule". Both assertions do exactly the same thing. If people make only 1 assertion for each interface, which side? Block-level verification needs assertions in the block for all of its interfaces, but the neighboring blocks also need assertions on the other side for the same reason.

BTW, the assertions providing "Documentation with unambiguous definition of interface and design properties" probably should not be automatically generated. All ABV vendors sell tools to automatically generate assertions from RTL source code.

vhdlcohen · Posted: Fri Jan 23, 2004 10:57 pm **Post subject:**

z · Senior Joined: Jan 09, 2004 Posts: 92

vhdlcohen · Posted: Sat Jan 24, 2004 2:00 am **Post subject:**

srini · Posted: Sat Jan 24, 2004 7:21 am **Post subject:**

z · Senior Joined: Jan 09, 2004 Posts: 92

vhdlcohen · Posted: Mon Jan 26, 2004 5:34 pm **Post subject:**

alain · Junior Joined: Jan 11, 2004 Posts: 5

A few years from now, you'll have both speed and assertions: RTL running at hundreds of kilohertz using emulation, and synthesized assertions to stop whenever something goes wrong. The third pillar of DV will be directed-random generators to feed such a beast.

Mark my words Smile

Alain Raynaud, Technical Director, EVE-USA