Parting Thoughts, and Farewell

dave_whipp

After 20 years working in hardware verification I’ve recently changed course to focus more on software. It was suggested to me that I might like to leave some parting thoughts.

One of my early tasks, as an intern in the summer of 1990, was to take part in an LVS review. Back then, this check -- that the layout matches the schematic -- was a manual process. We had the printout of a SPICE netlist (that had been simulated), and we had a large plot of the chip’s layout (a few square meters in size). One person would read the netlist: “base of transistor Q125 to emitter of Q134”. The other person, me in this case, would find this route on the plot and trace over it with a marker pen (on tracing paper). A year later I was testing a SW tool that attempted automate this tedious process.

Over time I have observed a recurring pattern of EDA tool development. First, a tedious manual process is automated by custom in-house tools, in multiple chip companies. Once such tools becomes pervasive, then EDA companies attempt to sell tools that do the same thing: the value proposition is that, by relieving the chip companies of the burden of maintaining their own version of tools that offer little competitive advantage (because everyone has such tools), the in-house engineers are able to concentrate on solving newer problems that are not yet ripe for standardization.

In the second half of the 1990s I was working for one of the early adopters of ARM. At the time, testing methodology was dominated by self-checking directed tests. It was becoming increasing clear that this approach didn’t scale and that successful SoC methodology would require something better. In one study we analyzed several hundred directed tests that had been written by junior engineers: we found that some 20% of tests did not test what they claimed: even when they covered the intended lines of code in the DUT, they failed to check even simple things that could go wrong.

A methodology was needed that would tell us what was being tested, independent of the tests themselves. Furthermore it was realized that, if we didn’t rely on the quality of the tests, then it would be much cheaper to generate them randomly. This observation was not unique to any one company, and so naturally there soon sprung up a number of EDA solutions: the HVL ecosystem was born, and came to be dominated by E and Vera.

Then something changed. For reasons that remain murky, big EDA companies decided to collapse this ecosystem. A meme was introduced, that they could create better tools if they didn’t have competition. One big monolithic language, they said, would be superior to an ecosystem of smaller, more focused, fragments of solutions. It would be simpler to build verification tools if both the RTL and HVL components were written in the same language. We hear the same thing today: we should merge VHDL and Verilog into one language; and we should merge VMM and OVM into one library. As you may gather, I do not favor these ambitions.

Fortunately, the very idea that we should write our testbench in the same language as our DUT is itself undermining the strategy of borgification. As architects increasingly use C++ models of components (or sometimes Python, Perl, Matlab, etc), there is a natural inclination to reuse tests of those models (written in SW languages) against the final design (written in Verilog/VHDL). In this environment, HVLs that are targeted to HDLs become less relevant. There is a pressing need for testing tools that are language-agnostic – and such tools are indeed emerging. The shadow of the SystemVerilog steamroller is lifting.

One such tool, that I have been using successfully over the past few years, is Breker’s Trek. Trek randomly generates directed tests using a constrained random walk of a graph (constructed by verification engineers) that describes how an environment interacts with the DUT. This is a step back from the purist approach of SAT-based constraint solvers, but it does provide an effective platform for exploring deep sequences of interactions. It would be good to see panel discussions of SAT-solving Vs graph walking as methodologies for finding deep bugs.

Another tool that has recently emerged is Certitude, now owned by Springsoft. It takes on the problem of verification quality. Most people realize that a regression suite that hits 100% coverage with a 100% pass rate provides no guarantee of completeness: tests may be covering a feature without checking the results; and things may be broken that lead to failing tests being incorrectly reported as passing. Certitude demonstrates that significant advances in verification methodology do not require new languages.

But returning to the question of languages, one of the disappointments over recent years has been the focus on implementation efficiency. 10 years ago I was writing testbenches using Perl. In dynamic languages it is incredibly easy to create transactions, add “post-it” annotations, set up ad-hoc callbacks, and a whole bunch of other things that are the basis of powerful testbenches. SystemVerilog shines brightest when used for the BFM layer. For higher layers I find that productivity is sapped by the rigidity of the language, and readability is sapped by the boilerplate required for even simple tasks.

In moving from the world of hardware verification to the realm of software it is interesting to see how the two disciplines have different targets for their verification methodologies. In HW verification the imperative is to find any bug that could result in a respin (i.e. that does not have a SW workaround). The focus in SW is more on the quality of the code, to manage complexity. Testability is used to force clean dependency structures so as to maintain a flexible codebase. Maintainability is the dominant constraint. In contrast, the dominant constraint of a HW design is the laws of physics: power, timing, and area. It is fascinating to see how these constraints drive methodology, and to think about a future in which HW designs will become increasingly complex.

For the next few years I will focus on SW. After that, I cannot predict what the future may hold. For the time being at least, I bid you all farewell.

Dave.

vhdlcohen · Posted: Sun Feb 06, 2011 6:04 pm **Post subject:**

krupan · Senior Joined: Mar 18, 2010 Posts: 11

Thanks, Dave, for the insight. A very interesting read. Some thoughts I had in response.

You say the push to standardized languages and libraries is a shame, but so is investing all your effort into a proprietary solution (e, vera). Since the tool licenses are all negotiated contracts with the EDA companies, once you have invested heavily in their proprietary solution, they suddenly have more leverage in those price negotiations. That's why it's good that Verilog and VHDL are standards, and that's why there was a push for SystemVerilog and UVM standards.

That being said, I have to agree with you that SystemVerilog is not the ideal solution and that dynamic languages like Perl or (even better, if you ask me) Python make a whole lot more sense for writing testbenches. All the design patterns employed to make VMM/OVM/UVM more flexible are a testament to the fact that the strictness of the language is in the way. For reference, see how the software world has turned from C, C++, and Java to the likes of Ruby and Python for applications where speed of writing code and code flexibility are greater concerns than execution speed: web applications. Testbenches have the same priorities.

I recently came from the software world back into verification, and I too noticed the big emphasis in the software world on code quality and maintainability and how it compares to hardware design. It makes complete sense in the software world, because code is never done. There are always new features to add and bugs to fix, and the faster you can accomplish that, the more money there is to be made. Your code better be readable and easy to change. Hardware has had the difficulty, but also the luxury, of tape-out: at some point the code really is frozen and you can drop it and move on. Except, not really. Modules get re-used, and more and more, the code turns into an FPGA instead of an ASIC, and then it's basically the same as software, it absolutely needs to be maintainable and changeable to meet customer needs. I expect to see software and hardware focuses converge more as time goes on.

One other big thing that I noticed coming from software world, is how expensive our tools are here in EDA land. Software design has gotten to the point that you can create really good software with all free, open source tools. In fact, I'd be willing to bet good money that the companies creating the software that we use don't pay the kind of license fees for their tools that we pay them for our tools. Why is that? Why don't we have an open source simulator that competes with the expensive commercial ones the way gcc and Linux did to put commercial compilers and workstation operating systems almost completely out of business? Imagine if Dave's description of how EDA tools came about was changed to this:

"Over time I have observed a recurring pattern of EDA tool development. First, a tedious manual process is automated by custom in-house tools, in multiple chip companies. Once such tools becomes pervasive, then the chip companies release the tools as open source: the value proposition is that, by relieving the chip companies of the burden of each maintaining their own version of tools that offer little competitive advantage (because everyone has such tools), the in-house engineers are able to concentrate on solving newer problems that are not yet ripe for standardization."

Alas, instead we have the Pepsi vs. Coke world of expensive and basically equivalent EDA tools from either Cadence, Synopsys, or Mentor.

dave_whipp · Posted: Mon Feb 07, 2011 10:09 pm **Post subject:**

krupan · Senior Joined: Mar 18, 2010 Posts: 11

True, it's easier to let internal tools die on the vine than it is to open source them, and there are risks and downsides to doing it, yet successful companies like Facebook, Google, IBM, Intel, and even Microsoft, have released open source code. Why?

As for a business model, there are successful companies that release *all* their products as open source. Red Hat is a prime example, and I'm sure the founders of MySQL made out pretty well when Sun bought their company. And think, if chip makers released their code as open source, they might stop losing their star tool developers to start-ups. Wink

It's true that the size of the EDA market is much smaller than the general software world, and that is probably the biggest reason open source hasn't taken off here like it has in the software world. The chances of a Linus Torvalds emerging from amongst the EDA ranks is a lot slimmer.

pavanshanbhag · Posted: Sat Feb 12, 2011 11:28 am **Post subject:**

Dave Wrote:

Sckoarn · Posted: Mon Mar 07, 2011 9:50 am **Post subject:**

I have enjoyed reading your posts Dave.

All the best in your new direction.

Sckoarn ;(
_________________
VHDL Testbench GNU publication
http://opencores.org/project,vhld_tb