Archive for July, 2011
Who Doesn’t Like Faster?
In my last blog post I introduced new technology called Intelligent Testbench Automation (“iTBA”). It’s generating lots of interest in the industry because just like constrained random testing (“CRT”), it can generate tons of tests for functional verification. But it has unique efficiencies that allow you to achieve coverage 10X to 100X faster. And who doesn’t like faster? Well since the last post I’ve received many questions of interest from readers, but one seems to stick out enough to “cover” it here in a follow up post.
Several readers commented that they like the concept of randomness, because it has the ability of generating sequences of sequences; perhaps even a single sequence executed multiple times in a row. 1 And they were willing to suffer some extra redundancy as an unfortunate but necessary trade-off.
While this benefit of random testing is understandable, there’s no need to worry as iTBA has you covered here. If you checked out this link - http://www.verificationacademy.com/infact2 - you found an interactive example of a side by side comparison of CRT and iTBA. The intent of the example was to show comparisons of what happens when you use CRT to generate tests randomly versus when you use iTBA to generate tests without redundancy.
However in a real application of iTBA, it’s equally likely that you’d manage your redundancy, not necessarily eliminate it completely. We’ve improved the on-line illustration now to include two (of the many) additional features of iTBA.
Coverage First – Then Random
One is the ability to run a simulation with high coverage non-redundant tests first, followed immediately by random tests. Try it again, but this time check the box labeled “Run after all coverage is met”. What you’ll find is that iTBA achieves your targeted coverage in the first 576 tests, at which time CRT will have achieved somewhere around 50% coverage at best. But notice that iTBA doesn’t stop at 100% coverage. It continues on, generating tests randomly. By the time CRT gets to about 70% coverage, iTBA has achieved 100%, and has also generated scores of additional tests randomly. You can have the best of both worlds. You can click on the “suspend”, “resume”, and “show chart” buttons during the simulation to see the progress of each.
Interleave Coverage and Random
Two is the ability to run a simulation randomly, but to clip the redundancy rather than eliminate it. Move the “inFact coverage goal” bar to set the clip level (try 2 or 3 or 4), and restart the simulation. Now you’ll see iTBA generating random tests, but managing the redundancy to whatever level you chose. The result is simulation with a managed amount of redundancy that still achieves 100% of your target coverage, including every corner-case.
iTBA generates tons of tests, but lets you decide how much to control them. If you’re interested to learn more about how iTBA can help you achieve your functional verification goals faster, you might consider attending the Tech Design Forum in Santa Clara on September 8th. There’s a track dedicated to achieving coverage closure. Check out this URL for more information about it. http://www.techdesignforums.com/events/santa-clara/event.cfm
1 – By the way, if achieving your test goals is predicated on certain specific sequences of sequences, our experts can show you how to create an iTBA graph that will achieve those goals much faster than relying on redundancy. But that’s another story for another time.
In my early days of standards development, I was intrigued how a standard went from the development phase to use phase. New standards were heralded with great fanfare but were also followed very quickly with books and other material to allow the “mere mortal” to understand what the IEEE standards prose meant and how best to use it. Everyone had their favorite VHDL book and I think I have them all!
What was clear to me was the IEEE standard was not sufficient to practice or understand the standard. After all, examples were few and far between in the standard. And even if there were examples in the standard, you were reminded that they are not part of the official standard – or in standards-speak – they are nonnormative.
User groups were popular too and continue to be today. VHDL International (now Accellera) had this notion of local VHDL user group chapters. When it came time to drive adoption of the VHDL gate-level library standard (known as VITAL), I attended several user group meetings to share details on how to use the new standard. I even solicited the support of a VHDL notable to put together a seminar series that would help ASIC library makers build their libraries. We took the seminar around the world and met with all the top ASIC suppliers. We even took our product that implemented the standard to the Cloud – while we did not call it the Cloud at the time. We had a model validation service in the early days of the internet that could be used to run training examples to validate ones own understanding or even test models and concepts to see if they would work. Free evaluation software was still a thing of the future then. As one byproduct of that work, we did have one competitor inundate us with the 1000’s of VHDL tests. We did throttle back their access to be fair to the others. But at that time, we left few ideas unexplored on how to drive global use and adoption of that standard.
What I understood was crossing the chasm from standards development to practicing the standard meant we had to build the knowledge, expertise and confidence in the user community to help them accept the standard and adopt it. I also learned that the standards developing organizations were not the best equipped to help practice the standard. The simple reason for this is the SDO is in place to bring together competitors to collaborate on the development of the standard but not foster competition on algorithms to best use the standard. This is perhaps better said by Synopsys’ Karen Bartleson in her “First Commandment for Effective Standards: Cooperate on Standards; Compete on Tools.”
Today’s Challenges with UVM & OVM
We are at that chasm with Open Verification Methodology (OVM) and the Universal Verification Methodology (UVM) today. While some may suggest OVM & UVM sit in a homogenous world where it works the same everywhere, the effective practice of the standards is anything but that. There are competitive options for users to explore and they are not ideas best promoted by a standards group. Mentor’s Mark Olen points out the value of an advanced method to generate stimulus rather than relying on the methods built into OVM & UVM in his recent blog post. Mark shows how a user gains 10x-100x in efficiency all the while doing this from within their OVM or UVM testbench.
Mentor has thought long and hard about how to best get this information to users and how to help them practice OVM and UVM better than they can if they only had access to the lowest common denominator of information. We first did a blind survey to see what methodology the design and verification community was using now and what they were going to use 12 months from now to validate our focus on OVM and UVM. Mentor’s Harry Foster has shared a lot of detailed information on this already. If you have not read his blog postings on this yet, you should start with his prologue that outlines the survey.
The survey clearly showed that UVM was in its ascendency and OVM was going to maintain strong and growing domination into 2012. Other survey results also clearly point out that SystemVerilog is the language of choice. While the survey shows what the user is doing, the standards developers were all collaborating on UVM and giving little time to OVM.
A Little Attention Goes a Long Way
While users were focused on continued use of OVM and planning for major move to UVM in 2012, the community developing standards had all but shifted to UVM, seemingly abandoning OVM. OVM was in need of care and attention given its dominant position in planned and future use.
Mentor stepped into the breach and has brought OVM into a strong, user-centric home that preserves the OVM World openness and augments it with several levels of additional user benefits in the Verification Academy. It also joins OVM and UVM in a single location that would not be appropriate in a standards body. After all, UVM is the standard from Accellera, not OVM. The Verification Academy also opens the cross pollination of ideas between the OVM and UVM users so one group can learn from another. We also brought the SystemVerilog User Group (SVUG) into the forum as well since OVM and UVM are based on the SystemVerilog language.
As we brought all these groups together, we did get many questions about Verification Academy Access Levels. First off, we dropped the OVM World requirement to register to download OVM. UVM and VMM were allowing anonymous downloads, so we made it the same for OVM. Of the 15,000+ OVM World registrants, most registered to download OVM. Just as OVM can now be downloaded without registration, the forums can be accessed in read-only mode without registration as well.
For those who used their OVM World registration to post on the forum, we moved them to “Forum Only Access” members so they could continue their posting privileges. The highest level of membership is “Academy Total Access.” Membership at this level is restricted to those who give a valid business profile. It enables access to training material, courses and lessons to help build SystemVerilog, OVM and UVM skills. It also allows users to gain knowledge about the advance algorithms that can help them get the 10x-100x or more out of OVM and UVM over conventional use. Below is a table of Verification Academy membership levels and privileges:
|Observer||Read-Only Forum Access. Free OVM/UVM kit download. No registration required.|
|Forum Only Access||Post to Forum and contributions area. Registration with any credentials required.|
|Academy Total Access||Total access. All academy areas open for free use. Registration with valid business profile.|
The response to this has been outstanding. While we strongly urge those who wish to develop the UVM standard to visit www.accellera.org and its www.uvmworld.org site to monitor that work, Verification Academy seems to have a much larger community of users with which to interact. And we will keep the Verification Academy current with the most recent versions of OVM and UVM. As of late July 2011 we recorded the following statistics.
|Verification Academy Forum||5,476|
|UVM World Forum||685|
|VMM Central Forum||696|
We look forward to continue to develop the site and add to the richness of its content and continue to improve your experience with it. Your comments on how we can improve it are always welcome.
I am pleased to announce that Mentor Graphics has recently expanded Verification Academy to provide a “one-stop shop” for all your UVM/OVM and general verification information needs. As you may have noticed, the OVMWorld.org website has now been redirected to the Verification Academy and all of the content has been preserved, albeit reorganized a bit.
Of particular importance, the OVMWorld Forum discussion group has been preserved, in its entirety, on the new Verification Academy Forum, which also includes threads for UVM as well as OVM discussions, as well as general SystemVerilog discussions and discussions about other Academy content. All users of the OVMWorld forum will be able to access the Academy forum using their OVMWorld login information. Of course, current Academy users may also access the forum with their current Academy login.
By combining the expanded discussion forum with the UVM/OVM Online Methodology Cookbook and on-line video tutorials, the Verification Academy is now the premier resource for verification engineers wishing to get the most out of UVM and OVM. The Cookbook will continue to provide the latest methodology and how-to information, let you comment and make suggestions, and also let you generate PDF books directly from the website. You can print out one of our predefined books or customize to make one of your own. Of course, you can also download code examples to go along with the articles.
All OVM and UVM download kits, including UVM1.1, OVM2.1.2 and the new standalone UVM Register kit for use with OVM, are available on the Academy with no login required. In addition, unregistered users may also preview the Cookbook and read the discussion threads in the forum. To participate in the forum, you’ll need to register (forum users are allowed to use free email addresses), and to access the premium content, you’ll need to use your “work” email address to register.
So, go take a look at the new Verification Academy. I think you’ll like what you see.
About Verification Horizons BLOG
This blog will provide an online forum to provide weekly updates on concepts, values, standards, methodologies and examples to assist with the understanding of what advanced functional verification technologies can do and how to most effectively apply them. We're looking forward to your comments and suggestions on the posts to make this a useful tool.
- Part 1: The 2012 Wilson Research Group Functional Verification Study
- What’s the deal with those wire’s and reg’s in Verilog
- Getting AMP’ed Up on the IEEE Low-Power Standard
- Prologue: The 2012 Wilson Research Group Functional Verification Study
- Even More UVM Debug in Questa 10.2
- IEEE Approves New Low Power Standard
- May 2013 (2)
- April 2013 (2)
- March 2013 (2)
- February 2013 (5)
- January 2013 (1)
- December 2012 (1)
- November 2012 (1)
- October 2012 (4)
- September 2012 (1)
- August 2012 (1)
- July 2012 (6)
- June 2012 (1)
- May 2012 (3)
- March 2012 (1)
- February 2012 (6)
- January 2012 (2)
- December 2011 (2)
- November 2011 (2)
- October 2011 (3)
- September 2011 (1)
- July 2011 (3)
- June 2011 (6)
- Intelligent Testbench Automation Delivers 10X to 100X Faster Functional Verification
- Part 9: The 2010 Wilson Research Group Functional Verification Study
- Verification Horizons DAC Issue Now Available Online
- Accellera & OSCI Unite
- The IEEE’s Most Popular EDA Standards
- UVM Register Kit Available for OVM 2.1.2
- May 2011 (2)
- April 2011 (7)
- User-2-User’s Functional Verification Track
- Part 7: The 2010 Wilson Research Group Functional Verification Study
- Part 6: The 2010 Wilson Research Group Functional Verification Study
- SystemC Day 2011 Videos Available Now
- Part 5: The 2010 Wilson Research Group Functional Verification Study
- Part 4: The 2010 Wilson Research Group Functional Verification Study
- Part 3: The 2010 Wilson Research Group Functional Verification Study
- March 2011 (5)
- February 2011 (4)
- January 2011 (1)
- December 2010 (2)
- October 2010 (3)
- September 2010 (4)
- August 2010 (1)
- July 2010 (3)
- June 2010 (9)
- The reports of OVM’s death are greatly exaggerated (with apologies to Mark Twain)
- New Verification Academy Advanced OVM (&UVM) Module
- OVM/UVM @DAC: The Dog That Didn’t Bark
- DAC: Day 1; An Ode to an Old Friend
- UVM: Joint Statement Issued by Mentor, Cadence & Synopsys
- Static Verification
- OVM/UVM at DAC 2010
- DAC Panel: Bridging Pre-Silicon Verification and Post-Silicon Validation
- Accellera’s DAC Breakfast & Panel Discussion
- May 2010 (9)
- Easier UVM Testbench Construction – UVM Sequence Layering
- North American SystemC User Group (NASCUG) Meeting at DAC
- An Extension to UVM: The UVM Container
- UVM Register Package 2.0 Available for Download
- Accellera’s OVM: Omnimodus Verification Methodology
- High-Level Design Validation and Test (HLDVT) 2010
- New OVM Sequence Layering Package – For Easier Tests
- OVM 2.0 Register Package Released
- OVM Extensions for Testbench Reuse
- April 2010 (6)
- SystemC Day Videos from DVCon Available Now
- On Committees and Motivations
- The Final Signatures (the meeting during the meeting)
- UVM Adoption: Go Native-UVM or use OVM Compatibility Kit?
- UVM-EA (Early Adopter) Starter Kit Available for Download
- Accellera Adopts OVM 2.1.1 for its Universal Verification Methodology (UVM)
- March 2010 (4)
- February 2010 (5)
- January 2010 (5)
- December 2009 (15)
- A Cliffhanger ABV Seminar, Jan 19, Santa Clara, CA
- Truth in Labeling: VMM2.0
- IEEE Std. 1800™-2009 (SystemVerilog) Ready for Purchase & Download
- December Verification Horizons Issue Out
- Evolution is a tinkerer
- It Is Better to Give than It Is to Receive
- Zombie Alert! (Can the CEDA DTC “User Voice” Be Heard When They Won’t Let You Listen)
- DVCon is Just Around the Corner
- The “Standards Corner” Becomes a Blog
- I Am Honored to Honor
- IEEE Standards Association Awards Ceremony
- ABV and being from Missouri…
- Time hogs, blogs, and evolving underdogs…
- Full House – and this is no gamble!
- Welcome to the Verification Horizons Blog!
- September 2009 (2)
- July 2009 (1)
- May 2009 (1)