What about expression coverage?

Newsletter · Original Contribution Joined: Dec 08, 2003 Posts: 1107

(Originally from Issue 4.15, Item 2.0)

From: Graeme Cunningham

How important a metric to people feel expression coverage is?

If possible would people be able give an indication of the length of
time and the effort involved in achieving a high level of expression
coverage.

Also do people find that identifying logically uncoverable code in
designs is a problem.

Newsletter · Original Contribution Joined: Dec 08, 2003 Posts: 1107

(Originally from Issue 4.16, Item 3.0)

From: Adam Krolnik

It depends on your coding style. If your code uses case/if/else
statement with very simple expressions, you may not need expression
coverage. If your coding style is mostly assigns with large
expressions or block style with large expressions in the code then it
could be very important.

I've seen designs that have expression coverage 20x the number of
blocks.

It also depends on your use of functional coverage. One could match
expression coverage with a lot of functional coverage (though its
probably easier to use expression coverage and use functional coveage
for what is not seen in expression.)

> If possible would people be able give an indication of the length of
> time and the effort involved in achieving a high level of expression
> coverage.

Without good support for the question below (identifying uncoverable
points) it takes a good amount of time. Toward the end of the project
you may spend hours creating a testcase for specific missing
expression points. Note, that functional coverage can be just as time
consuming. It depends on the rarity of the points being considered.

> Also do people find that identifying logically uncoverable code in
> designs is a problem.

YES! uncoverable code comes from:

- Replicated modules that have signals or ports tied to constants.

- Suboptimal expressions that contain redundant expressions. [ Note we
have seen expressions that don't need certain variables from this
kind of analysis. The variables factor out! - a bug?? ]

- Ooverengineered code - earlier code is already gated by the same
signal

The result of high expression coverage is a well tested design! But
there can still be bugs at all three levels (architectura, design,
implementation)

- Adam Krolnik, Verification Mgr.
LSI Logic Corp.

Newsletter · Original Contribution Joined: Dec 08, 2003 Posts: 1107

(Originally from Issue 4.16, Item 3.1)

From: Dan Joyce

We have found expression coverage to be a very important part of our
code coverage analysis. In my group we have been using Code Coverage
Analysis for over 10 years. We started with line coverage, but there
were important conditions (especially state transitions) that were not
tracked with simple line coverage. However, tracking all the
combinations of a boolean expression can get overly burdensome. The
number of coverage points explodes with the number of terms in the
boolean equation.

But even more importantly, we found that when tracking exhaustive
coverage on boolean equations, many of the combinations were wild
goose chases (not possible in the system). Time gets wasted tracking
these down. But we did find a very effective boolean tracking method
that gave us a smaller number of coverage points, eliminated all the
impossible ones, and generally kept all the real ones. We asked
Verisure (and other tools have followed) for a "Focused Expression"
option. Synopsys calls it "Sensitized Condition Coverage".

This takes a 4-term boolean which would give 16 possible combinations
for the 4 terms, and generates 5 - 8 coverage combinations, depending
on the equation. These combinations of terms include all the
combinations where each of the 4 terms in the boolean are controlling
the output of the equation to a true or false result. For instance

if (A || B || C || D) begin
state <= foo;
end

The following 5 combinations are the "Sensitized Conditions" for which
we want coverage:

A && !B && !C && !D ---- A controlling the output to "true"
!A && B && !C && !D ---- B controlling the output to "true"
!A && !B && C && !D ---- C controlling the output to "true"
!A && !B && !C && D ---- D controlling the output to "true"
!A && !B && !C && !D ---- A, B, C, and D all controlling the output to
"false"

Additionally we have prioritized our Code Coverage efforts to maximize
our results with the limited time we have. The 5 priorities are as
follows:

1) Hand inserted custom coverage points (inserted by design or
verification engineer) - Any hand inserted coverage is the most
powerful because thought has been put into it.

2) Line Coverage in State Machines

3) Sensitized Condition Coverage in State Machines - 2 and 3 give us
all of the arcs in all state machines.

4) Line Coverage outside State Machines

5) Sensitized Condition Coverage outside State Machines

Unfortunately our present coverage tool does not give us Line and
Sensitized Condition Coverage for only state machine code. But we do
all 5 at this time anyway.

- Dan Joyce, Hewlett Packard