Verification Horizons BLOG
Verification Horizons BLOG RSS
Redefining Verification Performance (Part 2)
In my last blog, I gave a few examples of different ways of thinking about getting more work done by finding solutions that increase amount of work accomplished per cycle, instead of just a brute-force approach to the problem. Before I talk about advanced verification solutions, I want to talk about why performance even matters.
First, we all intuitively know that the sooner we find a bug, the cheaper it is to fix. Doug Josephson and Bob Gottlieb attempt to quantify this notion in their chapter “Silicon Debug,” from the book Advances in Electronic Testing: Challenges and Methodologies (Springer, 2006). Figure 1 summarizes their findings in terms of the relative cost of finding bugs within a typical design cycle. Notice that a functional bug that prevents us from achieving first silicon success can cost us 10,000 X or more to fix than if it was found during the initial design phase.
Figure 1: Relative Cost of Finding Bugs
Obviously, speed, accomplishment, efficiency, and quality of results are all important attributes to getting more work done, finding bugs sooner, and thus reducing cost.
Let’s look and see how the industry as a whole is doing in achieving first silicon success. Figure 2 shows the 2002 and 2004 Ron Collett International functional verification studies and the late 2007 FarWest Research functional verification study. You can see that there is a continual downward trend in achieving first silicon success.
Figure 2: Industry Trends in Achieving First Silicon Success
Figure 3 list the types of bugs that caused a respin, where functional bugs account for the largest contributor.
Figure 3: Flaws That Caused a Respin
So this is the state of the industry today. But what about tomorrow’s design? What additional performance requirements will be required to meet tomorrow challenges? Will brute force approaches to achieving verification performance really enable us to get more work done?
Figure 4 shows the International Technology Roadmap for Semiconductors’ projected growth of transistors on a chip. Let’s focus on the ten-year span from 2008 to 2018.
Figure 4: International Technology Roadmap for Semiconductors Trends
You can see that, within ten years, there is about a 10x increase in the number of transistors on a chip. Now obviously, not everyone will be creating behemoth designs that take advantage of all the available transistors in 2018. Yet, for the sake of argument, it is interesting to calculate what theoretical maximum increase in verification effort would be required to verify a large design in 2018 compared to 2008, as shown in Figure 5. The verification effort grows at a double-exponential rate with respect to the Moore’s Law curve. Hence, if the number of transistors per chip increases 10X between 2008 and 2018, then the verification effort would increase 1024X.
Figure 5: Verification Effort Trends
Obviously, verification performance matters! Certainly, we as an industry can’t afford a 1000x increase in verification teams. Nor will brute force verification approaches to the problem scale.
In my next blog, I’ll discuss ideas on ways to improving verification performance.
Making formal property checking easy to use
For years one of the objectives in EDA has been to make formal property checking easy to use and its results easy to understand. With the Automatic formal check feature in the June release of the 0-In Formal tool version 3.0, I think we have made significant progress in this area.
The feature, which predefines a set of assertion rules to look for design issues automatically, makes formal technology accessible to users who are not yet ready to write properties in System Verilog Assertion (SVA) or Property Specification Language (PSL). To make it easier to comprehend problems in the design, the tool highlights the violations back to the RTL code.
Automatic formal check focuses on three areas inadequately addressed by dynamical simulation:
The first area is functional coverage. Today, when constrained random simulation fails to achieve the targeted coverage goal, engineers have to fine tune the environment or add new tests. These efforts, often attempted relatively late in the verification cycle, can consume vast amounts of time and resources while still failing to reach parts of the design. In contrast, automatic formal check can be used to identify unreachable code early in the verification cycle. These targets can be eliminated from the coverage model. As a result, the coverage measurement is more accurate and you know when you are done.
The next area is design initialization. If a design cannot be initialized reliably in silicon, it will not function correctly. An obvious precursor then is making sure all the registers are initialized correctly at RTL. If X’es are used, we need to monitor the X creation, propagation and usage cycle. Dynamic simulation does not interpret X’es accurately as in silicon, which has only 1s and 0s. Automatic formal check is ideal in verifying register initialization under different modes or configurations. Then, with internal assertions and formal technologies, we can check that although X’es are created, they are not used by downstream registers.
The final area is corner case design issues. Time and time again, designers unintentionally write code that violates logical correctness. Examples include combinational loops, full case violations, parallel case violations, undriven logic, finite-state machine (FSM) deadlocks and FSM livelocks. Unless tests are written to specifically target these corner case design issues are, such issues are difficult to exercise. On the other hand, by formally analyzing the design semantics, automatic formal check identifies these design issues statically and creates the stimuli to highlight them to the users.
If you are interested to know more about the automatic formal check feature in 0-In Formal, please feel free to register for our upcoming seminar in San Jose.
Tags: Assertion-Based Verification, automatic formal check, formal property checking, formal verification, functional coverage, RTL, SVA
Redefining Verification Performance (Part 1)
What does the word performance mean to you?
Speed? Well, obviously speed is an important characteristic. Yet, if the team is running in the wrong direction, it really doesn’t matter how fast they are going.
How about accomplishment? After all, we do assess an employee’s or project team’s accomplishments using a process we refer to as a performance review.
What about efficiency, which is a ratio comparing the amount of work accomplished to the effort or cost put into the process? Certainly, from a project perspective, effort and cost should be an important consideration.
Finally, perhaps quality of results is a characteristic we should consider. After all, poor results are of little use.
From a verification perspective, I think it is necessary to focus on the real problem, that is, the project’s verification objectives:
- Reduce risk-find more bugs sooner
- Know when we are done-increase confidence
- Improve project productivity and efficiency-get more work done
Now, whenever I hear the phrase “get more work done,” I’m often reminded of Henry Ford, who was the founder of the Ford Motor Company. Henry is probably best know as the father of modern assembly lines used in mass production, and he revolutionized transportation specifically, and American industry in general. Henry once said, “If I had asked people what they wanted, they would have said faster horses.” This quote provides a classic example of the importance of focusing on the real problem, and thinking outside the box.
In fact, Henry Ford’s faster horses example is often used in advanced courses on product marketing and requirements gathering. The typical example of focusing on the real problem generally involves a dialogue between Henry and a farmer, as follows:
Henry: So, why do you want faster horses?
Farmer: I need to get to the store in less time.
Henry: And why do you need to get to the store in less time?
Farmer: Because I need to get more work done on the farm.
As you can see, the farmer really didn’t need faster horses-he needed a solution that would allow him to get more work done on the farm. Faster horses are certainly one solution, but thinking outside the box, there are other more efficient solutions that would yield higher quality results.
Now, before I move on to discuss ways to improve verification performance, I would like to give one more example of thinking outside the box to improve performance. And for this example, I’ve chosen the famous Intel 8088 microprocessor. I was just an engineering student when the 8088 was released in 1979, and like so many geeks of my generation, I couldn’t wait to get my hands on one.
The 8088 had a maximum clock speed of approximately 5Mhz. It took multiple clock cycles to complete an instruction (on average about 15). Furthermore, a 16-bit multiplication required about 80 clock cycles. So the question is, how could we improve the 8088 performance to get more work done?
Well, one approach would be to speed up the clock. However, this would only provide incremental improvements compared to what could be achieved by thinking outside the box and architecting a more clever solution that took advantage of Moore’s Law. In fact, over time, that is exactly what happened.
First, the multiplier performance can be improved by moving to a single-cycle multiplier, such as a Wallace Tree, Baugh-Wooley, or Dadda architecture. These architectures calculate multiple partial products in parallel. Second, the average number of clock cycles per instruction can be reduced by moving to pipelined architectures, where multiple instruction executions overlap, giving a net effect of one instruction completing every clock cycle (as an ideal case example).
The point is that we have moved to solutions that get more work done by “increasing the amount of work per cycle,” instead of just a brute force approach to the problem.
In my next blog, I’ll discuss why performance even matters, followed by thoughts on improving verification performance.
Tags: Add new tag, formal verification, functional verification, Simulation
SystemVerilog Coding Guidelines: Package import versus `include
Another frequently asked question: Should I import my classes from a package or `include them? To answer this properly, you need to know more about SystemVerilog’s type system, especially the difference between its strong and weak typing systems.
In programming languages, weak typing is characterized by implicit or ad-hoc conversions without explicit casting between values of different data types. Verilog’s bit vectors, or integral types, represent these weak typing aspects by implicitly padding and truncating values to be the proper bit lengths – at least proper by Verilog standards. If you perform a bitwise AND of a 7-bit and 8-bit vector, Verilog implicitly zero pads an 8th bit to the 7-bit operand and returns an 8-bit result. In contrast using VHDL, you would have to explicitly state whether you wanted the 7-bit operand to be padded, or the 8-bit operand to be truncated so that you have an expression with operands of equal size.
With a few exceptions, all other types in SystemVerilog follow strong typing rules. Strong typing rules require explicit conversions or casts when assigning or expressing operands of unequal types. And understanding what SystemVerilog considers equivalent types is key to understanding the effect of importing a class from a package versus including it from a file.
Inheritance aside, SystemVerilog uses the name of a type alone to determine type equivalence of a class. For example, suppose I have these two class definitions A and B below:
class A; |
class B; |
SystemVerilog considers these two class definitions unequal types because they have different names, even though their contents, or class bodies, are identical. The name of a class includes more than just the simple names A and B; the names also include the scope where the definition is declared. When you declare a class in a package, the package name becomes a prefix to the class name:
package P; |
package Q; |
Now there are two definitions of class A, one called P::A and the other called Q::A. And the variables P::a1 and Q::a1 are type incompatible referencing two different class A’s. Re-writing the above example using an include file creates the same situation – two incompatible class definitions.
| File A.sv | File P.sv | File Q.sv |
class A; |
package P; |
package Q; |
After `including class A into each package, you wind up with two definitions of class A. Using `include is just a shortcut for cut and pasting text in a file. Importing a name from a package does not duplicate text; it makes that name visible from another package without copying the definition.
| File A.sv | File P.sv | File R.sv | File S.sv |
class A; |
package P; |
package R; |
package S; |
Class A is declared in package P, and only in package P. The variables R::a1 and S::a1 are type compatible because they are both of type P::A. The fact that class A was `included from another file once it is expanded is no longer relevant once you consider the placement of the text from the file.
When you get compiler errors claiming that two types are incompatible even though they appear to have the same name, make sure you consider the scope where the types are declared as part of the full name. Class names declared in a module are prefixed by the module instance name, so the same module instantiated multiple times will create unique class names, all incompatible types.
For further information about packages, check out the June Verification Horizons article entitled “Using SystemVerilog Packages in Real Verification Projects”.
Dave Rich
Tags: IEEE 1800, S, SystemVerilog, Verification
The reports of OVM’s death are greatly exaggerated (with apologies to Mark Twain)
Now that the Accellera VIP-TSC has released UVM-EA, effectively narrowing the choice of verification methodologies to UVM or OVM, many people are asking which way to go — OVM or UVM? The answer depends a lot on where you are in code development and what your risk tolerance is. The good news is that neither is a bad choice. One thing is certain: OVM is not dead yet. It will be around for a long time. Across the industry a lot of IP has been built and testbenches have been put into production all with OVM. These must remain operational and supported for a similarly long time. Right now, OVM is the lowest risk choice overall. There is tons of tool support and lots of IP available along with training and other material from a variety of vendors. We know OVM is stable and production-worthy just by how widespread its use is.
In its current state UVM is for the more adventurous. By and large it is OVM with the Os changed to Us. That’s why we can say it is stable and not necessarily a bad choice. The risk comes not from whether or not UVM itself works, but in how much support of various kinds is currently available. Vendors are now updating their tool sets to support UVM, but that work is far from complete. I imagine as the Accellera committee gets close to releasing UVM-1.0 we’ll see announcements from vendors around their tool support for UVM. Training, examples, and other material is under development as well and will probably be available from their respective sources shortly after UVM-1.0 is out.
Another aspect to the risk involves 3rd party IP. IP vendors are also working on converting their IP to be UVM compatible. If you use 3rd party IP you will need to know when your vendor will make available the components you need converted to run under UVM. The biggest risk comes from the potential of UVM not being entirely backward compatible with OVM (factoring out the simple syntactic name changes). The VIP-TSC has stated that backward compatibility is a goal, but not a hard requirement. For example, the VIP-TSC is now looking deeply at phasing. Proposals are being considered that enable phases to run in parallel and to add additional default phases. If done properly these changes would add significant capability to UVM and be entirely backward compatible. If not, UVM could require architectural changes in drivers and monitors to accommodate new phases. Exactly which way this will go is not clear right now. Mentor, of course, is making a case to retain backward compatibility.
The availability of UVM-EA in advance of the standard affords a prime opportunity to kick the tires and to start some early planning. Go ahead and download it and start evaluating it. Try it out on small- to medium-sized pieces of code. The UVM-EA includes a script to change the Os to Us. It’s the same script that was used to import OVM as the seed for UVM development. You can use UVM-EA to figure out what you need to do to convert your environment from OVM to UVM.
Eventually there will be a tipping point and UVM will become the obvious choice for a testbench methodology. That day is still in the distance. In the mean time OVM is around and provides all the features necessary to build sophisticated testbenches.
Tags: accellera, OVM, UVM, UVM E.A., UVM Early Adopter, UVM-1.0, vip-tsc
New Verification Academy Advanced OVM (&UVM) Module
I’ve always loved the Chinese proverb, “Give a man a fish and you feed him for a day. Teach a man to fish and you feed him for a lifetime.” Yet, why merely settle for fish when you can have sushi! My point is, to remain strategically relevant in today’s competitive landscape, it is necessary to constantly reinvent ourselves and evolve our technical skills. To that end, we have created the Verification Academy to help you evolve your advanced functional verification skills.
Since we launched the Verification Academy, we have had numerous requests for training on the Open Verification Methodology (OVM). Hence, in February we launched a new module titled Basic OVM that has been received with overwhelming enthusiasm. It’s currently our top viewed module. The Basic OVM module consists of 2.5 hours of content, and is divided into eight 20-minute sessions. The module is primarily aimed at existing VHDL and Verilog engineers who recognize they have a functional verification problem, but have little or know experience with constrained-random verification or object-oriented programming. Our goal for releasing the Basic OVM module is to raise your skill level to the point where you have sufficient confidence in your own technical understanding. In turn, you will have the confidence required to start the process of adopting advanced functional verification techniques.
This month, we are excited to announce the next step in evolving your Open Verification Methodology skills. Our new module is titled Advanced OVM (&UVM) and provides a higher level of OVM understanding beyond what is presented in our previously released Basic OVM module. What’s particularly exciting about this release is that we are addressing the numerous requests from you concerning advanced functional verification and creating contemporary testbenchesusing the OVM. The Advanced OVM module is presented by our own subject matter expert, Tom Fitzpatrick, who has been a driving force behind the OVM development and standardization. Tom is one of my favorite technical presenters. He is both informative and entertaining, so I’m sure you will really enjoy our new Advanced OVM module.
Now, as shown in Table 1, the Verification Academy covers a wide variety of topics, which enables you to start evolving your advanced functional verification skills.
Table 1. Verification Academy Modules
| Module Name |
Number of |
Description |
|
Evolving Capabilities |
1 |
This module provides a framework for all the modules within the Verification Academy, while introducing a tool for assessing and improving an organization’s advanced functional verification capability |
|
Assertion-Based Verification |
11 |
This module provides a comprehensive introduction to ABV techniques, include an introduction to SystemVerilog Assertions |
|
CDC Verification |
7 |
This module provides an understanding of the clock-domain crossing problem, in terms of metastability and reconvergence, and then introduces verification solutions |
|
FPGA Verification |
8 |
This module, although targeted at FPGA engineers, provides an excellent introduction to anyone interested in learning various functional verification techniques |
|
Basic OVM |
8 |
This module provides a step-by-step introduction to the basics of OVM |
|
Advanced OVM |
5 |
This module provides the next level of understanding beyond the skills introduced in the Basic OVM module |
Another exciting announcement is that we have added a language capture option to most (and eventually all) Verification Academy modules. The language options include: Chinese Simplified and Traditional, Japanese and Russian.
In the next few weeks we have another exciting announcement related to the Verification Academy. Stay tuned!
I would like to encourage you to check out all our new and existing content at the Verification Academy by visiting www.verificationacademy.com.
Tags: Add new tag, OVM
OVM/UVM @DAC: The Dog That Didn’t Bark
In the classic Sherlock Holmes story, “Silver Blaze,” Holmes realizes that the family dog didn’t bark when the suspect entered the house to commit the crime. This leads Holmes to deduce that the suspect was familiar to the dog and, from there, he of course unravels the rest of the mystery (which I won’t spoil for you). Since then, “the dog that didn’t bark” has been used to denote something seemingly innocuous that reveals a lot more under the surface.
I was reminded of “the dog that didn’t bark” this week at DAC. I did a number of presentations on OVM and UVM in the OVMWorld booth and participated in a panel discussion on UVM at the Accellera breakfast on Tuesday. It was great to see the amount of interest shown by the audiences at all of these events. As you may know, Accellera chose OVM as the basis for UVM, and a lot of the discussion was on where we go from here. One question that kept coming up was whether Mentor and Cadence would continue development of OVM, now that UVM is coming closer to reality. The implication clearly was that it would somehow be a bad thing if we did, although I’m not sure exactly why. Our answer was that Mentor will continue supporting OVM as long as our customers are using it (which could be for some time given how long it sometimes takes for standards to come to fruition), but that we would put our energy into developing new functionality in UVM. It didn’t occur to me until later that there was a dog that didn’t bark at DAC.
The question that I never heard asked nor answered, especially at the Accellera breakfast, is this. If everyone is so concerned that continuing OVM development outside of UVM is somehow a bad thing, will Synopsys show a similar commitment to the success of UVM by suspending any further VMM development? After all, if further OVM development will somehow deter adoption of UVM, further VMM development would as well. So I think Synopsys should make the same commitment to the success of UVM that Mentor and Cadence have.
DAC: Day 1; An Ode to an Old Friend
Denali Finale
While I ponder the hundreds of partners I work with to support a vibrant ecosystem of ModelSim and Questa users (thank you to all of you, by the way!) tonight was a special night for a long standing partner: Denali.
Looking back into the archives, the first communication from Sanjay (Srivastava), Mark (Gogolewski) and David (Lin) pondering our possible collaborative efforts in the late nineties to the first of your party invitations to the Bourbon Street Bash held at Patout’s Bourbon Vieux Room give me pause. Last in the archive is, of course, the invitation to the Monday Denali Party at the House of Blues. (And it is amazing what Jeannette Zelaya can do in two weeks! My hats off to you!)
In the frivolity before the musical festivities, the EDA Idol judges gathered for the fourth (and final) Denali EDA Idol competition. I must say, there are a lot of very talented musicians from within EDA and our customers. This year’s crop of contestants were very good. But fellow judges Simon Davidmann and Karen Bartleson reminisced about the past contests and an end of an era. Karen Bartleson even out did her wardrobe of prior years and was stunning as our EDA Idol “Paula.” I sure hope she can change her hair color back overnight. The picture to the left is actually Karen. Can you recognize her?
One can only hope the “music” continues on. The Mentor ModelSim team has made a small dent in EDA music with the creation of a“ModelSim” Fender Stratocaster guitar this year. (No, I don’t play, but maybe one day we can share it with those who can.)
If the contestants tonight prove anything, hope rises eternal for more music, fun and camaraderie in the future. We can hope.
On an endnote, the start of tonight’s party was a moving tribute from Mark Gogolewski to fellow Denali employees, customers, partners and his best treasure, his “better half” as he said, his wife. You touched a chord with many.
To all of Denali (present and past), thank you for the opportunity to collaborate with you for more than a decade of mutual customer success. I wish all of you well, always.
UVM: Joint Statement Issued by Mentor, Cadence & Synopsys
DAC Attendees Invited to Accellera’s Breakfast sponsored by Mentor, Cadence & Synopsys
The full statement can be read at EDA Cafe, click here.
The Big-3 EDA companies point out in the statement the work within Accellera to create an interoperability guide and kit to ensure verification IP and testbenches written in either the Verification Methodology Manual (VMM) or the Open Verification Methodology (OVM) can work together. This preserves the investments made to date by users of those two methodologies.
The joint statement also says the Accellera Universal Verification Methodology (UVM) is based on OVM 2.1.1 and firmly rooted in SystemVerilog. While we know today UVM is OVM 2.1.1 with a few small changes or additions, it is made clear that Accellera has just begun. What happens next is the topic of the Accellera breakfast meeting. (Have you registered yet for it?)
The joint statement asked these questions:
- If we fast forward by a year, what would UVM base class release X look like?
- What features should it have to solve the problems faced a year from now? 3 years from now?
- Are we looking at adding more of the same or make a quantum leap in our ability to deal with much larger and significantly more complex designs?
- What specifically are we doing to improve our ability to find bugs in the design and then fix them?
What questions do you have? If you want to share them here, please do. If you cannot attend the breakfast in person, I’ll bring your questions along to ask and report back after DAC on what happened at the Accellera breakfast.
Tags: accellera, OVM, Standards, SystemVerilog, UVM, Verification, verification IP, VIP, vmm
Static Verification
After spending years verifying ASICs with dynamic simulation, I started working on static verification 10 years ago in a startup called 0-In Design Automation. I firmly believe that static verification can complement dynamic simulation. Static verification uses synthesis and formal technologies to find bugs in the design. It does not rely on simulation stimulus. You do not need to exercise the bugs, propagate the results, and check the outputs to detect them.
Static verification includes RTL lint, static checks, formal checks, automated and assertion-based formal property checking. To read more on static verification, you can take a look at my white paper: Getting Started With Static Verification. If you are interested in formal methods, you can take a look at Harry Foster’s white paper: Why Now for Formal Property Checking. Both can be found in the Knowledge Center of DAC.com.
In the future, we are going to talk about individual static verification technologies and its application in areas such as RTL verification, clock domain crossing verification, low power verification, timing constraint verification, etc. Your feedback and comments are most welcome.
Tags: 0-In, Assertion-Based Verification, CDC, dac, formal, RTL, Verification
About Verification Horizons BLOG
This blog will provide an online forum to provide weekly updates on concepts, values, standards, methodologies and examples to assist with the understanding of what advanced functional verification technologies can do and how to most effectively apply them. We're looking forward to your comments and suggestions on the posts to make this a useful tool.
@dennisbrophy tweets
@dave_59 tweets
- @yaron_think_ver Good luck with that! #
- is back from three vacations in two weeks. Left behind: one son (UCSD) and one iPhone (road kill) #
Latest Posts
- Redefining Verification Performance (Part 2)
- Making formal property checking easy to use
- Redefining Verification Performance (Part 1)
- SystemVerilog Coding Guidelines: Package import versus `include
- The reports of OVM’s death are greatly exaggerated (with apologies to Mark Twain)
- New Verification Academy Advanced OVM (&UVM) Module
Subscribe to the Verification Horizons Newsletter