ABV and being from Missouri…

The last industry project I worked on, before joining EDA, was an advanced chip set for a very large, high-end server product line. The project consisted of a large team, spanning multiple years, with numerous physical, design, and verification challenges. During the project’s postmortem, where all the various engineering teams get together to discuss what worked well and what did not, I overhead one of the design engineers say that he would never do another design project without assertions.  In fact, his opinion was universally shared among all the project designers. Now, wait a minute, notice I said designers. What gives?  When questioning the design team further, I heard them say that assertions actually made their life easier.  The team claimed that the extra time it took them to add assertions during the RTL coding stage was more than made up for by the reduction in debugging time during verification. Not only that, they claimed that often the act of adding an assertion forced them to think about the design in a different way and even exposed a design error prior to any form of verification. Now that’s productivity!

All right, so if this stuff is so great, then why isn’t everyone doing it? In fact, a large industry study conducted by Farwest Research in conjunction with Mentor Graphics late in 2007 revealed that only 37 percent of the industry had adopted and integrated ABV techniques into their flow. This intrigues me—particularly since there have been numerous case studies published by best in class companies over the past 15 years that quantitatively demonstrate the benefits of adopting ABV into the project flow.  So I decided to dig a little deeper into this situation, and this is what I found:

  1. Project teams need help in understanding the methodological aspects of integrating ABV into their existing flow.
  2. There are a number of myths held by non-believers that need to be addressed before adoption can proceed.

If you look at the myriad of material that has been published on ABV over the past few years, what you will find is that most of the discussions focus on value propositions and specific ABV tools, or the discussion delves into details of a particular assertion language.  However, what I hear from various teams trying to adopt ABV often takes a more methodological bent, particularly related to the required steps for successfully integrating ABV into existing flows.

To address these concerns, at DAC 2008, with the sponsorship of Accellera, I organized a successful workshop titled: Beyond Syntax and Semantics: Industry Experiences with OVL/SVA/PSL.

I recognized that successful application of these assertion language standards in an industrial setting requires the development of project team member skills and verification process maturity beyond a simple understanding of assertion language syntax and semantics. Hence, the workshop was organized so that folks from multiple industry projects would share their experiences of applying ABV on real projects—with a focus on answering these questions:

 

  • What is required to mature a project team’s ABV skills for successful adoption?
  • What needs to be considered in terms of a project’s ABV infrastructure (beyond commercial tools)?
  • What metrics need to be defined (and gathered) to measure progress?
  • What benefits are real-industry projects seeing using OVL/SVA/PSL?

Going forward, what I plan to do in the next few weeks is create a set of related blogs where I address both the methodology aspects of integrating ABV into a project flow and a number of commonly held myths about ABV.

I’d like to hear from you.  Who has successfully integrated ABV into the flow?  For those who have not started, what are the obstacles you see to adoption?

Post Author

Posted December 6th, 2009, by

Post Tags

Post Comments

6 Comments

About Verification Horizons BLOG

This blog will provide an online forum to provide weekly updates on concepts, values, standards, methodologies and examples to assist with the understanding of what advanced functional verification technologies can do and how to most effectively apply them. We're looking forward to your comments and suggestions on the posts to make this a useful tool. Verification Horizons BLOG

@dennisbrophy Tweets

  • Loading tweets...

@dave_59 Tweets

  • Loading tweets...

@jhupcey Tweets

  • Loading tweets...

Comments

6 comments on this post | ↓ Add Your Own

Commented on December 8, 2009 at 9:03 am
By Srinivasan Venkataramanan

>> What are the obstacles you see to adoption?

Major one I hear from RTL folks often is the verbosity associated (as of SystemVerilog 2005) with using OVL-like libraries. Especially existing users of 0-in checkerware are so much pampered by the ease of use and the value it adds – though their management may have the extra $$ as concern – it is hard for them to appreciate the need to type-type-type the “clock, reset” mundane stuff! It was all being “inferred” so far and suddenly come a standard language/implementation such as SVA and that takes them back in history! Refer to AMD’s excellent presentation on OVL TC for a proof! True, the new (very new I must say) “checker” construct along with $inferred* takes care of it (sigh… it lacks $inferred_enable). We cover these in our recently published SVA Handbook 2nd edition (http://www.systemverilog.us/sva_info.html) and also in upcoming DVCon 2010 paper.

Cheers
Srini
http://www.cvcblr.com

Commented on December 8, 2009 at 10:28 am
By Harry Foster

Oh, a shameless plug for your new book! ;-) Seriously, congrats on the new publication. Concerning 0-in checkerware…back before I joined EDA, I was one of the earliest users of the technology, and I also liked being pampered by its ease of use. ;-) The 0-in engineering team obviously spent a lot of engineering $$ to develop the technology in such a way that it would be easy to adopt and use. Not only did it abstract away the details of clocks and resets, but also the complexity associated with temporal logics.

The somewhat slow adoption of ABV in the industry is due more to process issues, and less about language features (or lack of). The reality is that there are different stakeholders of ABV requiring each different use models. A one-size-fits-all methodology will not serve all stakeholders. The demands and needs of the designer focusing on low-level implementation assertions are different from those of a verification engineer who is focused on interface protocols or higher-level, end-to-end behaviors. The demands and needs of an IP provider dealing with multiple customer simulation languages and environments are different than an IP consumer. I plan to blog about these specific methodological needs in a set of future blogs.

Thanks for your feedback and I look forward to seeing your upcoming DVCon paper!

Commented on December 22, 2009 at 9:14 pm
By ben cohen

Whenever people talk about assertions, and their benefits, they always seem to address the benefits during the verification of the design. However, I find that there is another very important aspect of assertions that really comes into play way before the design verification of the RTL. That aspect is the clarification and specification of the requirements. The SVA/PSL can be written before a single line of RTL is written. The assertions DO clarify the requirements to a level of abstraction higher and clearer than English. The review process of such SVA/PSL code can be a real payoff before an RTL is written. I even see a payoff in adding such requirement properties in a requirement document, along with English. Harry, did you ever address that key point n your papers?

A plug, we demonstrate that concept, by example, in our SVA 2nd Edition book.
ben@SystemVerilog.us

Commented on December 22, 2009 at 10:45 pm
By Harry Foster

Yes, I have been making this claim for many years related to both verification planning and the creation of assertions (see DAC Panel 2006 http://www.eetimes.com/conf/dac/showArticle.jhtml?articleID=191600034&kc=2443). It turns out that I observed this first hand over 10 years ago on the very same project that I reference in this blog (before there was even SVA or PSL, and we used proprietary assertion languages). I had an engineer walk into my office one day and ask me for help on how to write a certain type of assertion for his design. I then asked the engineer to go to the white board and draw a timing diagram for his block. While he was at the board describing his design a strange look came over his face. He looked at me and said “You know, I’ve got a bug!” It turns out that just by thinking about the design intent in a different way he uncovered a bug. Talk about improved verification efficiency! I’ve said for years that there is no substitute for thinking.

Commented on January 6, 2010 at 10:12 am
By Howard MERIN

Harry, it’s good to see that someone who worked with me right out of school is now a well known name in EDA. In fact when I started to learn about ABV, the book you co-authored was my first reference.

As you know Zocalo has “bet the farm” on enabling ABV and I’m from Missouri. If I had known we were in a “missionary market” I may not have jumped in so readily. Anyway here we are and we have really tried to get our arms around what’s needed.

What we see is assertion use is on an ad hoc basis primarily by designers. Most designers can write simple one or two cycle assertions and since it for their own use, they are added inline without documentation. The 37% usage that you define most likely reflects ad hoc use.

We know of only a few companies (large ones) that are making a serious effort to implement ABV as a methodology. They appear to be focusing at the designer level and via comments inserted in the RTL related to OVL assertions. The comments then trigger automated software that handles the bind files and documentation…two important housekeeping tasks of ABV. OVL provides a consistent structure that enables the process.

At the verification level the norm for useful assertions is complex temporal SVAs. SVA is a difficult language that the broader base of verification engineers is not proficient in. This level of SVA must also be debugged representing a major effort. We see very little assertion use at the system level even on an ad hoc basis.

Metrics as you point out is a necessity. To even consider a full scale ABV methodology, management needs a continuous assessment of what assertion use is buying them. Also required is a rigid assessment of where assertions will provide the most leverage. Ad hoc use of assertions seems better than nothing, but is it? They may not even be making a dent in the real need.

Zocalo’s White Paper entitled Enabling Assertion Based Verification describes how Zocalo’s Zazz™ product set addresses the preceding issues and how a company or project can incrementally move into ABV on a cost effective basis. Go to http://www.zocalo-tech.com to download the White Paper.

Best regards,
Dave Stevens, Vice President Operations
Zocalo Tech, Inc.

Commented on March 18, 2010 at 5:52 pm
By temporal logics

[…] systems for group decision making under the latter. … Mail (will not be published) (required) …ABV and being from Missouri… Verification Horizons BLOGNot only did it abstract away the details of clocks and resets, but also the complexity associated […]

Add Your Comment

Archives

November 2014
  • SystemVerilog Testbench Debug – Are we having fun yet?
  • ARM® Techcon Paper Report: How Microsoft Saved 4 Man-Months Meeting Their Coverage Closure Goals Using Automated Verification Management & Formal Apps
  • Preparing for the Perfect Storm with New-School Verification Techniques
  • On-Demand Webinar: UVM Sequences in Depth
  • October 2014
  • DVCon India: A Smashing Hit!
  • September 2014
  • Portable and Productive Test Creation with Graph-Based Stimulus
  • Supporting A Season of Learning
  • August 2014
  • DVCon Goes Global!
  • Better Late Than Never: Magical Verification Horizons DAC Edition
  • July 2014
  • Accellera Approves UVM 1.2
  • May 2014
  • Getting More Value from your Stimulus Constraints
  • The FPGA Verification Window Is Open
  • April 2014
  • UVM DVCon 2014 Tutorial Video Online
  • Mentor Enterprise Verification Platform Debuts
  • March 2014
  • New Verification Academy ABV Course
  • DVCon 2014 Issue of Verification Horizons Now Available
  • February 2014
  • DVCon–The FREE Side
  • More DVCon–More Mentor Tutorials!
  • UVM 1.2: Open Public Review
  • DVCon 2014: Standards on Display
  • Just because FPGAs are programmable doesn’t mean verification is dead
  • January 2014
  • Managing Verification Coverage Information
  • November 2013
  • Epilogue: The 2012 Wilson Research Group Functional Verification Study
  • New Verification Horizons Issue Available
  • October 2013
  • Happy Halloween from ARM TechCon
  • IEEE Standards Association Symposium on EDA Interoperability
  • STMicroelectronics: Simulation + Emulation = Verification Success
  • September 2013
  • A Decade of SystemVerilog: Unifying Design and Verification?
  • Part 12: The 2012 Wilson Research Group Functional Verification Study
  • August 2013
  • Part 11: The 2012 Wilson Research Group Functional Verification Study
  • Part 10: The 2012 Wilson Research Group Functional Verification Study
  • Part 9: The 2012 Wilson Research Group Functional Verification Study
  • Part 8: The 2012 Wilson Research Group Functional Verification Study
  • July 2013
  • Part 7: The 2012 Wilson Research Group Functional Verification Study
  • Walking in the Desert or Drinking from a Fire Hose?
  • Part 6: The 2012 Wilson Research Group Functional Verification Study
  • A Short Class on SystemVerilog Classes
  • Part 5: The 2012 Wilson Research Group Functional Verification Study
  • Part 4: The 2012 Wilson Research Group Functional Verification Study
  • June 2013
  • Part 3: The 2012 Wilson Research Group Functional Verification Study
  • Part 2: The 2012 Wilson Research Group Functional Verification Study
  • May 2013
  • Texas-Sized DAC Edition of Verification Horizons Now Up on Verification Academy
  • IEEE 1801™-2013 UPF Standard Is Published
  • Part 1: The 2012 Wilson Research Group Functional Verification Study
  • What’s the deal with those wire’s and reg’s in Verilog
  • April 2013
  • Getting AMP’ed Up on the IEEE Low-Power Standard
  • Prologue: The 2012 Wilson Research Group Functional Verification Study
  • March 2013
  • Even More UVM Debug in Questa 10.2
  • IEEE Approves New Low Power Standard
  • February 2013
  • Verification Horizons DVCon Issue Now Available
  • Get your IEEE 1800-2012 SystemVerilog LRM at no charge
  • IEEE 1800™-2012 SystemVerilog Standard Is Published
  • See You at DVCon 2013!
  • Get Ready for SystemVerilog 2012
  • January 2013
  • VHDL Update Comes to Verification Academy!
  • December 2012
  • IEEE Approves Revised SystemVerilog Standard
  • November 2012
  • Coverage Cookbook Debuts
  • October 2012
  • IoT: Internet of Things
  • Check out the October, 2012 Verification Horizons
  • Improving simulation results with formal-based technology
  • Introducing “Verification Academy 2.0”
  • September 2012
  • OVM Gets Connected
  • August 2012
  • OpenStand & EDA Standardization
  • July 2012
  • Synthesizing Hardware Assertions and Post-Silicon Debug
  • Virtual Emulation for Debugging
  • Verification Academy: Up Close & Personal
  • SystemC Standardization Cycle Completes
  • Verification Standards Take Another Step Forward
  • New UVM Recipe of the Month: Scoreboarding in UVM
  • June 2012
  • Intelligent Testbench Automation – Catching on Fast
  • May 2012
  • Two Articles You Need to Check Out
  • Off to DAC!
  • Dave Rich Featured on EEWeb
  • March 2012
  • How Did I Get Here?
  • February 2012
  • Expanding the Verification Academy!
  • Get on the Fast Track to Advanced Verification with UVM Express
  • Introducing UVM Connect
  • Tornado Alert!!!
  • UVM: Some Thoughts Before DVCon
  • UVM™ at DVCon 2012
  • January 2012
  • SystemC 2011 Standard Published
  • Verification solutions that help reduce bug cost
  • December 2011
  • Instant Replay for Debugging SoC Level Simulations
  • 2011 IEEE Design Automation Standards Awards
  • November 2011
  • Getting started with the UVM – Using the Register Modeling package
  • TLM Becomes an IEEE Standard
  • October 2011
  • Worlds Standards Day 2011
  • VHS or Betamax?
  • Verification Issues Take Center Stage
  • September 2011
  • New UVM Recipe-of-the-Month: Sequence Layering
  • July 2011
  • Combining Intelligent Testbench Automation with Constrained Random Testing
  • Going from “Standards Development” to “Standards Practice”
  • Verification Academy Now Includes OVMWorld Content
  • June 2011
  • Intelligent Testbench Automation Delivers 10X to 100X Faster Functional Verification
  • Part 9: The 2010 Wilson Research Group Functional Verification Study
  • Verification Horizons DAC Issue Now Available Online
  • Accellera & OSCI Unite
  • The IEEE’s Most Popular EDA Standards
  • UVM Register Kit Available for OVM 2.1.2
  • May 2011
  • Part 8: The 2010 Wilson Research Group Functional Verification Study
  • Getting Your Standards Update @ DAC 2011
  • April 2011
  • User-2-User’s Functional Verification Track
  • Part 7: The 2010 Wilson Research Group Functional Verification Study
  • Part 6: The 2010 Wilson Research Group Functional Verification Study
  • SystemC Day 2011 Videos Available Now
  • Part 5: The 2010 Wilson Research Group Functional Verification Study
  • Part 4: The 2010 Wilson Research Group Functional Verification Study
  • Part 3: The 2010 Wilson Research Group Functional Verification Study
  • March 2011
  • Part 2: The 2010 Wilson Research Group Functional Verification Study
  • Part 1: The 2010 Wilson Research Group Functional Verification Study
  • Prologue: The 2010 Wilson Research Group Functional Verification Study
  • Language Transitions: The Dawning of Age of Aquarius
  • Using the UVM libraries with Questa
  • February 2011
  • DVCon: The Present and the Future
  • Free at Last! UVM1.0 is Here!
  • Parameterized Classes, Static Members and the Factory Macros
  • IEEE Standards in India
  • January 2011
  • Accellera Approves New Co-Emulation Standard
  • December 2010
  • New Verification Horizons: Methodologies Don’t Have to be Scary
  • The Survey Says: Verification Planning
  • October 2010
  • Towards UVM Register Package Interoperability
  • IEC’s 47th General Assembly Meeting Opens
  • UVM: Giving Users What They Want
  • September 2010
  • UVM Takes Shape in the Accellera VIP-TSC
  • Accellera VIP-TSC Selects RAL for UVM 1.0 Register Package
  • OVM Cookbook Available from OVMWorld.org
  • UVM Register Package Candidate News
  • August 2010
  • Redefining Verification Performance (Part 2)
  • July 2010
  • Making formal property checking easy to use
  • Redefining Verification Performance (Part 1)
  • SystemVerilog Coding Guidelines: Package import versus `include
  • June 2010
  • The reports of OVM’s death are greatly exaggerated (with apologies to Mark Twain)
  • New Verification Academy Advanced OVM (&UVM) Module
  • OVM/UVM @DAC: The Dog That Didn’t Bark
  • DAC: Day 1; An Ode to an Old Friend
  • UVM: Joint Statement Issued by Mentor, Cadence & Synopsys
  • Static Verification
  • OVM/UVM at DAC 2010
  • DAC Panel: Bridging Pre-Silicon Verification and Post-Silicon Validation
  • Accellera’s DAC Breakfast & Panel Discussion
  • May 2010
  • Easier UVM Testbench Construction – UVM Sequence Layering
  • North American SystemC User Group (NASCUG) Meeting at DAC
  • An Extension to UVM: The UVM Container
  • UVM Register Package 2.0 Available for Download
  • Accellera’s OVM: Omnimodus Verification Methodology
  • High-Level Design Validation and Test (HLDVT) 2010
  • New OVM Sequence Layering Package – For Easier Tests
  • OVM 2.0 Register Package Released
  • OVM Extensions for Testbench Reuse
  • April 2010
  • SystemC Day Videos from DVCon Available Now
  • On Committees and Motivations
  • The Final Signatures (the meeting during the meeting)
  • UVM Adoption: Go Native-UVM or use OVM Compatibility Kit?
  • UVM-EA (Early Adopter) Starter Kit Available for Download
  • Accellera Adopts OVM 2.1.1 for its Universal Verification Methodology (UVM)
  • March 2010
  • The Art of Deprecation
  • OVM 2.1.1 Now Ready for Download
  • February 2010 Verification Horizons Newsletter Now Available
  • IEEE Standards Meetings in India
  • February 2010
  • I Do It …
  • SystemVerilog: A time for change? Maybe not.
  • Partners Offer Support for OVM 1.0 Register Package
  • SystemC Day at DVCon
  • OVM/VMM Interoperability Kit: It’s Ready!
  • January 2010
  • Three Perfect 10’s
  • OVM 1.0 Register Package Released
  • Accellera Adopts OVM
  • SystemC (IEEE Std. 1666™) Comes to YouTube
  • Debugging requires a multifaceted solution
  • December 2009
  • A Cliffhanger ABV Seminar, Jan 19, Santa Clara, CA
  • Truth in Labeling: VMM2.0
  • IEEE Std. 1800™-2009 (SystemVerilog) Ready for Purchase & Download
  • December Verification Horizons Issue Out
  • Evolution is a tinkerer
  • It Is Better to Give than It Is to Receive
  • Zombie Alert! (Can the CEDA DTC “User Voice” Be Heard When They Won’t Let You Listen)
  • DVCon is Just Around the Corner
  • The “Standards Corner” Becomes a Blog
  • I Am Honored to Honor
  • IEEE Standards Association Awards Ceremony
  • ABV and being from Missouri…
  • Time hogs, blogs, and evolving underdogs…
  • Full House – and this is no gamble!
  • Welcome to the Verification Horizons Blog!
  • September 2009
  • SystemVerilog: The finer details of $unit versus $root.
  • SystemVerilog Coding Guidelines
  • July 2009
  • The Language versus The Methodology
  • May 2009
  • Are Program Blocks Necessary?