| Login | | Don't have an account yet? You can create one. As a registered user you have some advantages like theme manager, comments configuration and post comments with your name. | |
| Who's Online | There are currently, 62 guest(s) and 1 member(s) that are online.
You are Anonymous user. You can register for free by clicking here | |
 | |
|
Verification Guild: Forums |
|
| View previous topic :: View next topic |
| Author |
Message |
SAHO
Senior
Joined: Oct 16, 2004
Posts: 24
|
 Posted: Sat Oct 16, 2004 5:13 pm Post subject: Bug finding using purely dynamic simulation... Clarify this! |
 |
|
To verification experts:
Reading customer testimonials on formal tools in verification (for example, 0-in (http://www.0-in.com) and Safelogic (http://www.safelogic.se) suggests that it is "quite" impossible to detect functional bugs in HDL code using dynamic simulation (without the use of assertion.) This observation caused by the fact the in a dynamic simulation environment, bugs are required to be propagated to the primary outputs for their presence (of bugs) to be detected. In some occasion, bugs may never get "output" and resulting bugs not found in HDL simulation during the lengthy simulation hours.
Can someone please certify these claims with real example?
regards,
SAHO |
|
| Back to top |
|
 |
vhdlcohen
Industry Expert
Joined: Jan 05, 2004
Posts: 1237
Location: Los Angeles, CA
|
| Posted: Sat Oct 16, 2004 7:40 pm Post subject: |
|
|
| Quote: | Reading customer testimonials on formal tools in verification (for example, 0-in (http://www.0-in.com) and Safelogic (http://www.safelogic.se) suggests that it is "quite" impossible to detect functional bugs in HDL code using dynamic simulation (without the use of assertion.) This observation caused by the fact the in a dynamic simulation environment, bugs are required to be propagated to the primary outputs for their presence (of bugs) to be detected. In some occasion, bugs may never get "output" and resulting bugs not found in HDL simulation during the lengthy simulation hours.
Can someone please certify these claims with real example? |
I'll use a hypothetical example to demonstrate the concept.
Suppose I have a design with inputs "i1, i2" and output "q1".
I also have internal signals "a, b" representing FSM states.
My internal FSM works as follows:
property P1; @ (posedge clk) i1 |=> a ##[1:2] b; endproperty : P1
property P2; @ (posedge clk) b |-> ##[1:5] q1; endproperty : P2
property P3 ; @ (posedge clk) i2 |-> ##[1:4] q1; endproperty : P3
Property P1 states that if "i1" at cycle 1 (cy1), then my FSM goes to "a" at cy2 and to "b" at either cy3 or cy4.
Property P2 states that if in state "b", then starting from the next cycle up to 5 cycles later, output "q1" is ttrue.
Property P3 states that if input "i2", then within 4 cycles, output "q1" is true.
Suppose that the internal design is such that property P1 fails, and "b" is never asserted within the expected range after "i1" is asserted. However, if "i1" and "i2" are simultaneously asserted in a blackbox test, output "q1" (resulting from hardware meeting P3 property) will mask the error that the machine implementing properties P1 and P2 operate correctly. Thus, the verification code, in the testbench will think that in response to "i1" output "q1" works OK.
Also, signal "b" is an internal, and does not propagate. Internal state "b" may control data entering a big buffer, that is later processed under other control signals. Thus, if the hardware that implements property P1 is incorrect, the error may not appear for a long time in simulation, and may be masked by other conditions. Debugging will be more difficult because the effect of an error in P1 machine (for lack of a better name) will not be seen till much later, if at all.
I recall an old design case that drove me insane! It was a DMA controller design where the data load should not have been written in the microprocessor program code region.
My design had no interlocks, and testing in those days was through a breadboard. Needless to say, as soon as a DMA transaction was initiated, my machine went into "Neverland " -- no relations to Michael Jackson!
Assertions would have helped then.
Ben Cohen
_________________
Ben Cohen http://www.systemverilog.us/
* SystemVerilog Assertions Handbook, 3rd Edition, 2013
* A Pragmatic Approach to VMM Adoption
* Using PSL/SUGAR ... 2nd Edition
* Real Chip Design and Verification
* Cmpt Design by Example
* VHDL books |
|
| Back to top |
|
|
confused
Senior
![]()
Joined: Jan 09, 2004
Posts: 185
|
| Posted: Sun Oct 17, 2004 8:56 am Post subject: Re: Bug finding using purely dynamic simulation... Clarify t |
|
|
These are not lies, but they can mislead.
| SAHO wrote: | | "quite" impossible to detect functional bugs in HDL code using dynamic simulation (without the use of assertion.) |
Every serious project detected some functional bugs in HDL code using dynamic simulation (without the use of assertion.) It is "quite" impossible to detect all functional bugs in HDL code using formal tools as well.
| SAHO wrote: | | This observation caused by the fact the in a dynamic simulation environment, bugs are required to be propagated to the primary outputs for their presence (of bugs) to be detected. |
This is true for many projects (perhaps all projects known to these customers). However, some dynamic simulation environments allow watching internal signals (especially if the HDL is Verilog). You may exclude them because they use the spirit of assertions.
| SAHO wrote: | | In some occasion, bugs may never get "output" and resulting bugs not found in HDL simulation during the lengthy simulation hours. |
This can be true. It is possible for a dynamic simulation environment to access any internal signals that assertions or formal tools can access. If this is a smart trick, why people did not do it? What's the big difference between accessing internal signals from a dynamic simulation environment and the same from an assertion or from a formal tool?
These tools are probably very good tools. These customers may have good reasons to like these tools. Their reasons may not be true to all projects, and they may or may not be true to your project.
Does any reader claim that it is impossible for a dynamic simulation environment to access any internal signals?
If you know how to do it but your dynamic simulation environment does not access any internal signals, why?
It is more convincing if they have compared with a dynamic simulation environment that access internal signals. Did they know this way to be more convincing? If they did, why did they pick a less convincing way? |
|
| Back to top |
|
|
vhdlcohen
Industry Expert
Joined: Jan 05, 2004
Posts: 1237
Location: Los Angeles, CA
|
| Posted: Sun Oct 17, 2004 1:29 pm Post subject: |
|
|
| Quote: | | it is "quite" impossible to detect functional bugs in HDL code using dynamic simulation (without the use of assertion.) T |
I believe that the bottom line here is that assertions help in locating bugs early because they are close to the DUT at the white-box level or interface level, and define properties that the design must meet. The point that those vendors make is that ABV languages like PSL and SVA, and verification IPs or modules, like OVL or those designed by vendors (e.g., 0-In, Synopsys, or other) significantly help in the verification process because they become automatic built-in checkers.
Sure, we, as engineers, can verify the design with waveforms without any assertions. This is because the design properties are all in our heads, and we can (theoretically) catch the errors by analyzing the waveforms, with the internal signals of a design. We probably all have experienced that process, and understand our limitations.
The beauty that assertion languages and pre-built IP assertions have brought is the ease of defining and integrating the design properties for a more thorough verification process.
Those language allow to define multiple relevant design properties that not only verify the design, but often clarify to the design engineer the requirements. Not ethat properties should not be written in a manner that emulate RTL, but rather express the reuqirements.
Is assertion-based verification solution THE SOLUTION? No, other factors enters into the verification process of a design -- that should be another topic good for this forum.
Ben
_________________
Ben Cohen http://www.systemverilog.us/
* SystemVerilog Assertions Handbook, 3rd Edition, 2013
* A Pragmatic Approach to VMM Adoption
* Using PSL/SUGAR ... 2nd Edition
* Real Chip Design and Verification
* Cmpt Design by Example
* VHDL books |
|
| Back to top |
|
|
|
|
You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
| |
|
|
Forum FAQ
Search
Usergroups
Profile
Digest
Log in
