Question about ISA verification

Newsletter · Original Contribution Joined: Dec 08, 2003 Posts: 1107

(Originally from Issue 4.12, Item 7.0)

From: Aarti Mohan

What is ISA verification? How does one measure completeness of the
same? What are the various strategies one could use to effectively
validate the ISA of a processor??

[Note from the editor: I assume "ISA" stands for Instruction Set
Architecture"]

Newsletter · Original Contribution Joined: Dec 08, 2003 Posts: 1107

(Originally from Issue 4.13, Item 2.0)

From: Jeff Cartwright

ISA verification insures the processor behaves as specified. It is a
pre-silicon stage showing the functional behavior of instructions,
registers, addressing modes, and other architectural features of the
processor, such as memory maps, caches and paging are correct.

To measure completeness, a verification plan is developed listing the
features and crosses between features. The crosses between features
are commonly called grid points. Populating the grids, or ensuring
that each point is tested, is accomplished by generating a test and
monitoring the test to check the grid point is hit. Monitors plug into
your testbench. Usually, large companies have well-defined API's for
writing monitors, for data collection and a database for storing
monitor information. This allows you to do multiple types of analysis
on the data.

All modern processors are verified using a combination of randomly
generated and handwritten directed tests. There may be other
strategies, but I am not aware of any.

All Intel processors, all ARM architectures, all IBM processors, all
Motorola PowerPC and DSP's, all AMD processors, TI processors, such as
C5x & C6x, all MIPS architectures, and so on are verified using
dynamic random test generators. I have a relatively complete list of
processors and the dynamic random test generator associated with it.

The main goals of random test generation are: 1) hit 80+% of the RTL
code in a very short time period, both in terms of calendar and
simulation time, and 2) do this with a minimum of labor, and 3) allow
tests to roam into areas (corner cases) that may not have been
foreseen by the designer or verification engineer.

For simple architectures, random static generators are sufficient. A
simple architecture has a limited number of instructions, few
addressing modes, little if any state information between instructions
(e.g., no pipeline, no parallelism, no caching). A static generator
creates streams of assembly code with no regard for the state of the
machine. Generated tests will be correct because there is little to no
interaction between the instructions, so they can be laid down one
after the other. Common tools for writing static generators include:
DGL & Perl.

There are two major issues w.r.t. simulation time for verifying
sophisticated processors: 1) simulation licenses are scarce, and 2)
simulation time is precious. If the test generator creates bad tests,
tests that will fail during simulation, then you are wasting scarce
resources, and you will not be able to verify the processor on
schedule.

For complex architectures, like the ones TI uses (ARM, MIPS, C5x and
C6x), dynamic generators are needed. Dynamic generators know about the
state of the machine. They will not create invalid tests. TI has a
world-wide license to a dynamic random test generator that supports
all of its architectures.

Because dynamic generators know about the state of the machine, they
can reuse values, do register forwarding, and create complex loops
among other things. This allows for the creation of more compact
tests, which hit more grid points and use less simulation time.

A particular value may have a low probability of being generated
randomly. Often, random generators are supplemented with a small
number of directed or handwritten tests. Because the cost of failure
can be huge, some verification teams use multiple random test
generators (e.g., Itanium, Pentium, Sun).

Another common practice is to keep tests which fail, or which
demonstrate regression.

There are two commercially available dynamic random test generators:
Genesys-Pro from IBM's R&D lab in Haifa, and RAVEN from Obsidian
Software. With languages, like vera and e, you can write a test
generator.

I hope this helps. Please let me know if you have any other questions.

- Jeff Cartwright, Obsidian Software

Newsletter · Original Contribution Joined: Dec 08, 2003 Posts: 1107

(Originally from Issue 4.14, Item 1.0)

From: David Ljung Madison

Let's not forget formal.

I found that my most successful verification was done with
a combination of formal and directed-random tests.
Two blocks with no bugs found post-silicon yet!