Processor Verification Methodolgies

fredinashed · Junior Joined: Oct 19, 2009 Posts: 9

Hi ,
I am looking any pointers to any good articles, white papers, books or websites that discuss verification methodologies for processor cores.
Have a new processor project to start and only have VHDL, a simulator and code coverage to complete verification.
At the moment it is looking to costly to move to system verilog, assertions or any formal tools.
So looking for new ideas to add to current methods or more justification to support a move to new tools.

qwk000 · Posted: Tue Jun 21, 2011 10:55 am **Post subject:**

Sorry I don't have any links or white papers to give you. Just a few suggestions.

Maybe that was all trivial. Not sure how much experience or time you have...

pavanshanbhag · Posted: Fri Jun 24, 2011 2:02 am **Post subject:**

Inbetween I have written a whitepaper on "Verification Flow Guidelines", I can circulate you, if you pass me ur mail address.
_________________
-Pavan K Shanbhag

“The difference between genius and stupidity, genius knows his limits.” - Albert Einstein

Hanken · Senior Joined: Aug 02, 2005 Posts: 55

Hello Pavan, if your document is not endorsed, you can just upload to share it here http://www.verificationguild.com/modules.php?name=Downloads&d_op=AddDownload.

tblackmore · Posted: Mon Jun 27, 2011 10:32 am **Post subject:**

Hi fredinashed,

there must be a lot of (very busy) processor verification engineers out there, but not so many replies as yet. Perhaps they were a little put off by the 'At the moment it is looking too costly ...' since processor verification is probably not something you want to do quick and dirty.

I can point you to a paper published in 2006 at DesignCon written by Fabio Bruno and myself titled something like 'Verifying the TriCore2 multithreaded processor' but this is now quite old and things have moved on somewhat from there.

When you say simulator I guess you mean ISS? You're sure to need one of these. How will you simulate the ISS and VHDL - each standalone with offline comparison, or together checking each instruction (and all intervening asynchronous events) as it executes? You'll need to look at memory and bus accesses as well as register updates. And where comparison with the model and scoreboarding don't provide suitable checking assertions may prove suitable (e.g. to check that bus protocols are adhered to).

I would also think that, on top of your directed tests, you would need a pseudo-random Instruction Stream Generator, which you would probably want to use to drive interrupts and other external stimuli as well as generate instruction streams. I think code coverage will not be enough - e.g. pipeline effects mean that much coverage will not propogate to anywhere interesting - so you'll need functional coverage of some kind (and lots of it). You'll want to pay particular attention to areas that are notoriously problematic, such as atomic instructions. With all this stimulation you'll want to think carefully how you'll debug all the failures. Think of triaging these, at least by failure type, and you'll need some kind of tool to compare the traces from your VHDL processor and your ISS.

It would be good to do some formal e.g. just formally verify the arithmetic works correctly - this is likely to be quicker than getting a decent of level of coverage here as well as being exhaustive. But formal verification of e.g. cache controllers is probably well advised (these can hide killer bugs) and not easy.

Of course this all just functional verification. Performance is likely to be important (careful analysis of benchmarks may help here), as is power. But this may be beyond the scope of your task.

And this is just the tip of the iceberg since it is really in the details as to how succesful the verification will be. And of course none of it is about tools - it can all be done in many languages with many tools. In fact, when recruiting your verification team I would look closely at their experience with processors as much as at their experience with any particular verification methodologies or languages.

All the best - processor verification is an interesting challenge,
Tim.

fredinashed · Junior Joined: Oct 19, 2009 Posts: 9

Hi All,
Thanks for the reply's so far, to answer some of the points raised.

pavanshanbhag - are you able to upload the verification flow guide lines as suggested ?

tblackmore - Thank you for the paper reference its exactly the kind of thing I was looking for, I have downloaded it and also started looking at the paper references.

In regard to you comment about 'too costly', I agree with you whole heartedly, I currently think the verification proposal put to me is woefully lacking and cost is no excuse. Perhaps I should have said I want to adopt new tools and methods but am not being backed up.
Not having worked on a processor verification before I am looking to see what more experienced processor people have done, such that I can strengthen my arguments for adopting new methods and tools.
Also as you point out 'none of it is about tools' so I am interested in what I could achieve with out a budget for new tools.

tblackmore · Posted: Wed Jun 29, 2011 5:04 am **Post subject:**

I'm glad the reference was useful.

It seems to me that your single most important task is to convince your management just how woefully inadequate their proposal is, which is really what was driving the tone of my reply. Perhaps the best way to do this is to start finding bugs. There certainly will be plenty if the processor has not been verified properly. A well written Instruction Stream Generator (ISG) is likely to find plenty. And each new feature you add to this, or new methodology you adopt, is likely to find more.

My reference to 'none of it' being about tools or languages was really meant to say that none of it was about specific tools or languages. I believe it doesn't matter too much if your ISG is written by an expert in SV or e testbenches, but they will need to understand the processor architecture well and previous experience with processor verification will be invaluable - they will have a sense of where the killer bugs will be. Clearly tools can be written from scratch - and no doubt some of your tools will have to be written from scratch (such as the debug tools) - but this can be as expensive or more than buying a 3rd party tool, and no doubt more time consuming. I really wouldn't suggest writting a constraint solver for pseudo-random generation, although this is of course possible.

pavanshanbhag · Posted: Thu Jun 30, 2011 10:13 am **Post subject:**

miket · Posted: Mon Jul 04, 2011 8:11 am **Post subject:**

A very interesting paper on this topic was published at San Jose SNUG in 2007. The title of the paper is "Applying CRV to Microprocessor Verification". It was written by Jason Chen of Synopsys.

Its a good paper and the concepts in it are sound. Some work we did in our shop is loosely based on Chen's ideas.

---mike

fredinashed · Junior Joined: Oct 19, 2009 Posts: 9

Hi mike,
Thank you for the paper reference it looks good, ( haven't read it fully yet!)

JimLewis · Senior Joined: Jun 29, 2004 Posts: 21 Location: Tigard, OR

> Have a new processor project to start and only have VHDL, ...

We have developed open source VHDL packages for constrained random verification and functional coverage. Check them out at:
http://www.synthworks.com/downloads/index.htm

Best,
Jim
_________________
SynthWorks VHDL Training
1-800-505-VHDL
jim@synthworks.com

war_isbest · Senior Joined: Dec 05, 2007 Posts: 23

miket · Posted: Sun Aug 07, 2011 1:02 pm **Post subject:**

I can't speak for Mr. Blackmore, I would strongly recommend a CRV approach. I referred a paper earlier in this thread that should get you started.

---mike

Hans · Posted: Thu Aug 11, 2011 12:42 pm **Post subject:**

As a shameless plug, another paper co-authored by myself was published at DVCon 2009:

"Model-Based Instruction Stream Generation For Processor Verification"

See if you can dig it up - you may find it useful. If you can't find it, drop me a line so I can send it to you.

Hans

war_isbest · Senior Joined: Dec 05, 2007 Posts: 23

Hi,
Thanks for the links.
Can any one please guide me in deciding about the reference model?
I have given a proposal where the verification environment contains an ISG which feeds the RTL and reference model.Their architectural states are compared to verify the RTL.

I have also proposed to use assertions to verify the small blocks and their interfacing.In addition i have told that we will also write directed tests to focus upon block level verification.

Here one question which is asked to me is: "What is the level of details required by the reference model and what should be tested in directed tests."

One level is that model can be pure sequential and need not reflect pipeline behavior. Is that correct?

Second level is where we model the pipeline and cache too.

Plz help.

Thanks for your replies.