Verification Horizons BLOG

This blog will provide an online forum to provide weekly updates on concepts, values, standards, methodologies and examples to assist with the understanding of what advanced functional verification technologies can do and how to most effectively apply them. We're looking forward to your comments and suggestions on the posts to make this a useful tool.

9 July, 2014

Accellera has announced the completion of a multi-year effort to update its latest edition of the Universal Verification Methodology (UVM).  In completing this effort, the UVM 1.2 Class Reference Document was approved as an Accellera standard and the UVM Working Group has supplied an accompanying open-source reference implementation.  Questa supports UVM 1.2.

In addition to the resources you can download from Accellera, additional information on UVM 1.2 can be found at the Verification AcademyHTML documentation can easily be found at the Verification Academy too.

If you are a user of UVM 1.1 and have not been part of the UVM 1.2 development effort, you should know your peers have been busy the past few years since the stabilization and completion of UVM 1.1 to drive global adoption of UVM and to add, enhance and extend UVM.  In UVM 1.2 Messaging is now object-oriented, Sequences can automatically raise and drop objects, the register layer can now control transaction order within bursts and numerous bugs in UVM 1.1 have been fixed to improve quality.

Backward Incompatibility

All these changes come with a cost to the current UVM 1.1 user community.  When Accellera announced UVM 1.2 availability, it also disclosed some of the new features introduce backward incompatibility.  To reduce those issues, Accellera is making release notes and a one way conversion script part of the UVM 1.2 kit to ease the migration path forward.

If you follow the Verification Academy Cookbook rules, you will probably not see any impact from the backward compatibility issues.  And if you control your total verification environment, you will probably find it simpler to migrate forward as well.  Those who depend on outside resources will need to make sure those resources (like Verification IP) migrate forward to UVM 1.2 so you can migrate forward to UVM 1.2.  Mixing UVM 1.1 and UVM 1.2 was not considered by the Accellera UVM Working Group and is fraught with unknown issues.  We consider the migration an all or nothing proposition.  If you have multi-division, multi-company projects underway, it would be prudent to plan you move to UVM 1.2 with care at the conclusion of projects and when all suppliers and participating teams can migrate to UVM 1.2.

Public Review Period

Accellera seeks your input and feedback on UVM 1.2.  To support this, a public review forum on the Accellera website has been established to allow users to catalog issues, ask questions and generally offer feedback to help improve UVM 1.2 quality.

The public review process will end on October 1, 2014.  We encourage users to take the time now to test UVM 1.2 in their own environments and share their feedback to expidite the migration to UVM 1.2.

Path to IEEE

Public feedback will be taken into account along with further Accellera member testing to update UVM 1.2 prior to a committed hand-off to the IEEE for further standardization there later this year.  As this path unfolds, I will share updates on the standardization effort in the IEEE.

Verification Academy DAC 2014 UVM 1.2 Presentation

You will find many resources around the world on UVM 1.2.  At DAC 2014, the Verification Academy booth sponsored a session on UVM 1.2 titled  “UVM: What’s New, What’s Next, and Why You Care.”  If you did not attend DAC, you can still download the presentation and watch a video replay of it if you are a Verification Academy “full access” member (free registration required; restrictions apply).

The presentation by Tom Fitzpatrick goes into detail on the UVM 1.2 topic.  Importantly in Tom’s presentation is a discussion about what you should care about today.  You may find that software is a big issue and that his thesis challenges one to ask if UVM 1.2 is stuck in the past rather than addressing what should be addressed next.  I invite you to download the presentation and watch the video and share with me your thoughts. What do you think?

, , , , ,

29 May, 2014

Verification engineers put lots of effort into writing and tuning constraints for random stimulus. It’s critical that the constraints correctly express the valid relationships between the stimulus variables. Otherwise, invalid stimulus will be generated or, worse, important valid combinations of stimulus will not be generated.
When it comes to bug hunting, running open-loop random stimulus is recognized as a good way to ensure that cases are exercised that the verification engineer wouldn’t intuitively think of. However, the very constraints that verification engineers work so hard to perfect get in the way of this goal by introducing random-resistant cases – value combinations that have an extremely low probability of occurring.

Consider the SystemVerilog class shown in Figure 1 below to see just what a dramatic effect a few constraints can have on the cases that a constraint solver produces. One simple constraint skews the entire random distribution!

Figure 1: Constraints skew random distribution

Figure 1: Constraints skew random distribution

This type of skewed distribution is easy to see and adjust for when the variable combinations are monitored by functional coverage. However, let’s face it, the whole premise of using random stimulus to find bugs is that random generation will produce cases that we didn’t think of (and, thus, didn’t create functional coverage for).
What if the very constraints that engineers spend so much time creating and refining could actually help ensure that corner cases are hit? If you’re attending DAC this year, come see a poster paper titled “Strategy-Driven Stimulus Generation: Constraint-Guided Test Selection” that proposes an approach that leverages the constraint description to identify high-value stimulus values and get more value from bug-hunting simulation runs:

Session Title: Designer/IP Track Poster Session – Wednesday
Session Number: 302
Presentation Title: Strategy-Driven Generation: Constraint-Guided Test Selection
Date: Wednesday, 6/4/2014 12:00-1:30PM
Room: 100

How do you ensure that your random simulations continue to provide incremental value, and aren’t just testing the same thing over and over again?

7 May, 2014

My Feb. 4 post introduced Mentor Graphics’ three-step FPGA verification process intended to help design teams get out of the reprogrammable lab more effectively. Since then, I’ve engaged FPGA vendors, design managers and engineers to explain the process, paying special attention to the merits and technical detail for injecting automation into any FPGA verification environment, the hallmark of Mentor’s process. The feedback from these conversations helped me to develop a series of technical webinars, now available for free and on-demand. Check them out and let us know what you think in the comments below. My hope is the webinars might serve as a starting point for your own conversations on verification of FPGAs, demand for which seems to continue to grow as process nodes shrink.

Injecting Automation into Verification – FPGA Market Trends

Injecting Automation into Verification – Code Coverage

Injecting Automation into Verification – Assertions

Injecting Automation into Verification – Improved Throughput

, , , , , , , , ,

25 April, 2014

DVCon 2014 Conference Proceedings Published

2014DVCon_logoWith record attendance announced for DVCon 2014, one might wonder if there is really a need to put some of the “Accellera Day” tutorial videos online.  With more than 1,000 professionals attending in some capacity, it would be easy to conclude that everyone that needs to know about UVM and the developments on the updated version to it, probably know.  Looking at just the LinkedIn design and verification forums one will realize there are 10’s of thousands who would have benefited if they had attended DVCon.  Thus, sharing this information more broadly is in order.

UVM Tutorial Video

UVM – What’s Now and What’s Next” is the tittle of the DVCon 2014 tutorial on UVM.  It covered use cases and pragmatic topics of the current UVM 1.1 standard as well as advanced topics for the next update, UVM 1.2.  The presenters covered sequence creation, register layer use, TLM-based communication, test execution, run-time phases and messaging enhancements.

The tutorial was split into five separate sections delivered by five speakers as follows:

  • Working Group Update: Adam Sherer, Accellera (7 min.)
  • Overview and Library Concepts: John Aynsley, Doulos (36 min.)
  • Stimulus Generation: Shawn Honess, Synopsys (21 min.)
  • UVM Register Layer: Tom Fitzpatrick, Mentor Graphics (36 min.)
  • UVM 1.2 Introduction: Uwe Simm, Cadence Design Systems (25 min.)

You can find out more information about the online tutorial videos hereRegistration is required, but there is no charge for access.  Once you have registered, you will get links to each of the five sections.  You can stream them or download them for offline access as you wish.  They are suitable for viewing on your computer or mobile devices.

DVCon 2014 Proceedings

DVCon 2014 was a full conference; it was more than just the the Accellera Day UVM Tutorial.  And in keeping with DVCon tradition, the conference proceedings are made available to all several months after the conference without charge.  If you visit the DVCon history area, you will find the 2014 proceedings have been published.  What I like about the DVCon proceedings it not only are the papers published, but the slides that were presented at the conference will often accompany the paper.

As an example, if you were interested in the DVCon 2014 Best Oral Presentation paper and presentation (Kelly D. Larson from NVIDIA on , “Determining Test Quality through Dynamic Runtime Monitoring of SystemVerilog Assertions” by the way), you will now find both the paper and presentation available online here.

For all those who did not make it to DVCon 2014, or who were there and could not see everything, the proceedings are now online and the first of the Accellera Day tutorials videos is published. Accellera is busy readying its other tutorial videos.  I’ll share information on their availability as they appear in the weeks and months ahead.

, , , , ,

10 April, 2014

Its always fun to take the wraps off of solutions we have been hard at work developing.  The global team of Mentor Graphics engineers have spent considerable time and energy to bring the next level of SoC design and verification productivity to what seems to be a never ending response to Moore’s Law.  As silicon feature sizes get smaller, design sizes get larger and the verification problem mushrooms.  But you know that.  These changes are the constants that drive the need for continued innovation.  Our next level of innovation for design verification is embodied in the Mentor Enterprise Verification Platform (EVP) which we recently announced.

Gary Smith recently published Keeping Up with the Emulation Market, and lays out the fact that verification platforms are unifying with emulation now a pivotal element, not just for microprocessor design success, but for Multi-Platform Based SoC design success as well.  The need to bring software debug into the loop with early hardware concepts is a verification challenge that must be supported as well.  Pradeep Chakraborty reported on the point made by Anil Gupta of Applied Micro at the UVM 1.2 Day in Bangalore where Anil implored “Think about the block, the subsystem and the top.”  The point made was software is often overlooked or under tested prior to committing to hardware implementation implying that our focus on UVM leaves us to verify no higher than where UVM takes us – and that is not the “top” of the SoC that mandates software be part of the verification plan.

Path to Success

With the Mentor EVP, we do address these issues.  We bring simulation and emulation together in a unified platform.  Software debug on conceptual hardware is supported to address verification at the “top.”  And even as Gary’s report concludes with a wonder about how easy access to emulation will be supported for the masses.  That too is solved in the Mentor EVP using VirtuaLAB that can be hosted in data centers along with the emulator vs. complex, one-off lab setups that lock an emulator to a design and lock out your global team of software developers from collaborating.  The Mentor EVP moves to emulation for the masses in a 24×7 world.

With big designs comes big data and complex debug tasks.  These complex debug tasks are all easily handled by the new Mentor Visualizer Debug Environment that has native UVM and SystemVerilog class-based debug capabilities and low-power UPF debug support to easily pinpoint design errors. All of this works in both interactive and post-simulation modes for simulation and emulation.  To keep the software team productive, and get to SoC signoff sooner, the innovative and new Veloce OS3 global emulation resourcing technology moves software debug think-time offline to Mentor’s Codelink software debug tool.

And there’s more!  But I’ll leave that for you to discover.  When you have time, visit us here, to learn more about the Mentor Enterprise Verification Platform.

Path to Standards

As the move to support Multi-Platform Based SoC evolves, so do the standards that underpin it.  And as I’ve reported on the comments of others in this blog – and the understanding from our experience that UVM can only go so far in Multi-Platform Based SoC verification – we concluded the time is right for the industry to explore the need for new standards.

We announced at DVCon 2014 an offer to take our graph-based test specification into an Accellera committee to help move beyond the limitations today’s standards have.  As our investment in tools, technology and platforms continues, we are keenly aware users want their design and verification data to be as portable as possible.  The Accellera user community members echoed the need to discuss portable stimulus that can take you up and down the design hierarchy from block, to subsystem, to system (“top”) and support the concurrent design of hardware and software.

In support of this, Accellera approved the formation of a Portable Stimulus Specification Proposed Working Group (PWG) to study the validity and need for a portable stimulus specification.  To that end, join me at the kickoff meeting to launch this activity on Wednesday, May 7, 2014 from 10:00am to 4:00pm Pacific time at the offices of Mentor Graphics in Fremont, CA USA.  If you would to attend, or you would  like time on the agenda to discuss technology that would advance the development of a Portable Stimulus Specification or discuss your objectives/requirements for this group, contact me and I will put you in touch with the meeting organizer.  Accellera PWG meetings are open to all and do not require Accellera membership status to attend.

, , , , , , , , , ,

17 March, 2014

As some of you might be aware, the Verification Academy has a video course dedicated to the topic of Assertion-Based Verification (ABV). In fact, this was the first video course we released for the academy almost six years ago. To date, it has remained one of our most popular courses. Yet one of the most common requests for improvement to this course has been adding content that focuses more on the verification process and provide guidelines on how to effectively integrate ABV into an existing flow. At the Verification Academy, we always welcome your feedback on content improvement. But before I talk about our new, updated ABV course, let me explain to you why I’m such a big believer in this technology.

Ensuring functional correctness continues to pose one of the greatest challenges for today’s SoC design teams. Indeed, recent industry studies have found that more time is spent in functional verification than any other project task. [1] But if you dig a little deeper into the trends revealed by these studies, you will find that debugging is the fastest-growing component of the overall verification process, and that on average, it consumes 34 percent of the verification engineer’s total effort. To make matters worse, the industry is witnessing increasing pressure to shorten the overall development cycle. Clearly, new design and verification techniques that improve productivity—combined with a focus on maturing functional verification process capabilities within an organization—are required.

Fig-6-2

Assertion-based verification (ABV), although certainly not an end-all to the verification challenge, directly addresses the debugging challenge. For example, those that have effectively adopted an assertion-based verification (ABV) methodology have seen a significant reduction in simulation debugging time (as much as 50 percent [2][3]) due to improved observability. In addition, organizations that have embraced an ABV methodology are able to take advantage of more advanced verification techniques, such as formal property checking, thus improving their overall verification quality and results.

While the process of writing assertions is fairly well understood by those skilled in the art—or whose skill can be easily acquired through a wealth of published papers and books that focus on language syntax and semantics—the process of creating a repeatable ABV methodology that integrates into an existing verification flow is not. And this is where our new ABV course will help. We’ve added a new session titled “Maturing a Project’s ABV Process Capabilities” that provides a set of actionable guidelines and recommendations that are process focused. These recommendations are based on years of ABV experience from multiple projects in various market segments, and they should help you achieve effective adoption of ABV on your project.

To learn more about the new ABV course, visit the Verification Academy (www.verificationacademy.com).

References

[1]     Prologue: The 2012 Wilson Research Group Functional Verification Study (here)

[2]     Y. Abarbanel, I. Beer, L. Gluhovsky, S. Keidar, Y. Wolfsthal, “FoCs—Automatic Generation of Simulation Checkers from Formal Specifications,” Proc. 12th International Conference Computer Aided Verification, pp. 414-427 (2000)

[3]     B. Turumella, M. Sharma, M., “Assertion-based verification of a 32 thread SPARC™ CMT microprocessor,” In Proceedings of the 45th Design Automation Conference, DAC 2008, pp. 256 – 261, (2008)

3 March, 2014

DVCon is always one of my favorite events in our industry, and I am proud to let you know that the latest issue of Verification Horizons is available “hot off the presses” at the Verification Academy to mark the occasion. For those of you attending the conference, please consider this issue as an addendum to the great technical program being offered (especially paper 8.1, “Of Camels and Committees: Standards Should Enable Innovation, Not Strangle It” by Dave Rich and yours truly). For those of you not able to join us at DVCon this year, consider this your consolation prize.

Although fewer in number, I’m sure you’ll find the articles in Verification Horizons as informational and useful as any you’ll see at DVCon. In particular, I’d like to make sure you check out these articles by our partners:

  • “Don’t Forget the Little Things That Can Make Verification Easier” by our friend Stu Sutherland of Sutherland HDL
  • “Taming Power-Aware Bugs with Questa Ultra” by SmartPlay Technologies
  • “Using Mentor Questa for pre-silicon validation of IEEE 1149.1-2013 based Silicon Instruments” by Intellitech
  • “Dealing With UVM and OVM Sequences” by eInfochips

If you’re at DVCon, please make sure to stop by the Mentor Graphics booth (#501) to say hi. Please join us on Wednesday for our luncheon presentation at noon, right after Session 8, in which I’ll present my paper mentioned above (that’s right. I’m not above shameless self-promotion). And we’ll wrap up the week with two Mentor-sponsored tutorials on Thursday:

Both of these tutorials feature a mix of Mentor presenters and customers to offer some practical examples that will give you some new ideas for improving your verification process. I hope to see you at DVCon.

, , ,

27 February, 2014

DVCon 2014 LogoPsst!  I’ll let you in on some news…

While DVCon calls the free portion of the conference “Exhibits Only,” let me share a little secret for you – You also gain access to the conference panels and the keynote presentation.

For those in Silicon Valley and local to DVCon, I invite you to register for the FREE side of the conference, not just for the conference exhibition that will have (in evening hours) drinks and appetizers, but for the industry conversation that will be offered via panels and CEO keynote.  The two panels will also feature Mentor Graphics speakers so you can learn our opinions on the topics as well.

How do you secure your FREE pass?  That’s the simple part!  Go here and start the registration process by clicking the “REGISTER NOW” button in the upper right.  After entering your contact information and completing a brief survey, you will be asked to select the part of the conference you wish to attend.  Select “Exhibit Only” for no charge.  Then “checkout” to complete your registration and you are done!  Of course, you can just show up and do this onsite.  But why waste time in line when you can do this from your computer or mobile device?

See you there!  You can find us at our Mentor Graphics booth.  We are booth 501.  (P.S., if you cannot spare the time to attend but would like to see a running commentary on the sessions, panels and other happenings follow me on Twitter: @dennisbrophy or look for the conference hashtag #DVCon.)

Now here is what you can get for free:

Panels

Is Software the Missing Piece In Verification?

Moderator Ed Sperling – Semiconductor Engineering
Panelists Tom Anderson – Breker
Kenneth Knowlson – Intel
Steve Chappell – Synopsys
Sandeep Pendharkar – Vayavya Labs
Frank Schirrmeister – Cadence Design Systems
Mark Olen – Mentor Graphics
Location Oak Ballroom
Date & Time Wednesday – 5 March 2014 8:30am – 9:45am

Did We Create the Verification Gap?

Moderator John Blyler – Extension Media
Panelists Janick Bergeron – Synopsys
Jim Caravella – NXP
Harry Foster – Mentor Graphics
John Goodenough – ARM
Bill Grundmann – Xilinx
Mike Stellfox – Cadence Design Systems
Location Oak Ballroom
Date & Time Wednesday – 5 March 2014 1:30pm – 3:00pm

Keynote

An Executive View of Trends and Technologies in Electronics
Lip-Bu Tan, President & CEO Cadence Design Systems
Oak Ballroom
Tuesday – 4 March 2014 2:00pm – 2:30pm

Exhibition

Monday: 5:00pm – 7:00pm (Booth Crawl included; Attendees open to win $500 gift card!)
Tuesday: 2:30pm – 6:00pm (Reception 5:00pm – 6:00pm)
Wednesday: 2:00pm – 6:00pm (Reception 5:00pm – 6:00pm)

, , ,

25 February, 2014

As DVCon expands, we at Mentor Graphics have grown our sponsored sessions as well.  Would you expect less?

In DVCon’s recent past, it was a tradition for the North American SystemC User Group (NASCUG) to sponsor a day of activity before the official start of the conference.  When OSCI merged with Accellera, the day before the official conference start grew to become Accellera Day with a broader set of meetings and activities covering many of Accellera’s standards.  This has all grown into a more official part of the DVCon program.  On Monday at DVCon – or as many still call it – Accellera Day – the tradeshow now joins in opening.  I covered this in detail in an earlier blog, so I won’t repeat myself now.

The pre-conference education and meet-up to discuss the latest in standards development is joined by an end of conference tutorial series that has expanded to allow four parallel sessions from three.  Instead of the one tutorial we at Mentor Graphics would otherwise sponsor at DVCon, we will offer two in this expanded series. Given the impact verification has on design it would seem right that more time be devoted to topics that address this.  One half-day tutorial is just to short to give the subject its due respect.

The two Mentor Graphics sponsored tutorials at DVCon, to be run in series, will devote a day to explore the application of current verification technology by us and users like you.  If you are already attending DVCon, you are making your tutorial selections now.  And for those who might only be interested to attend the tutorials themselves, DVCon offers a tutorials-only package ($145/Tutorial).  Mentor’s two tutorials are:

The first tutorial references “smooth sailing,” not because this will be a “no-pirate zone,” although I can tell you that since International Talk Like a Pirate Day is in late September, one won’t have to worry about a morning of pirate talk! [Interesting Fun Fact: Mentor Graphics’ headquarters in Wilsonville, OR USA is a short 50 miles (~80 km) north of the creators of this parotic holiday.]  The smooth sailing comes from the ability to easily use multiple engines from simulation, formal, emulation, FPGA prototyping to address your block to system-level verification needs.

The second tutorial is all about formal.  Or, in a more colloquial way to say it, we will answer the question: Whatsup with formal?  No, I doubt we will find more slang terms for formal technology being used and created in the tutorial.  But the tutorial will certainly look at more focused applications of formal technology.  As a pioneer in focused formal applications (like clock domain crossing) the creation of these focused formal applications has greatly simplified use and expanded technology access to verification teams with RTL design checks, X-state verification, and more joining the list.  Maybe we should ask Whatsapp with formal! But wait!  That slang question is already taken – and Facebook affirmed ownership with a $19B purchase of it recently.  Oh well, I lament.  Join me at this tutorial and we can explore something suitable and not yet taken as a replacement.  I can’t think of a better way to close DVCon than to see if we can invent another $19B term (or app).

, , , , , ,

23 February, 2014

UVM 1.2 Release is Imminent

As vice chair of DVCon 2014, I can share with you that the Universal Verification Methodology (UVM) remains a topic of great interest.  It sets the pace for tutorials and given the pending release by Accellera, learning what is new in UVM 1.2 is a compelling reason to attend DVCon.

The Accellera Day tutorial series on Monday at DVCon is popular with UVM being a session of great interest.  Aside from the “verification crisis” driving the need to explore this industry standard, the first major update is also a reason to generate this interest.  The UVM tutorial is meant for the novice and expert alike.  UVM experts can expect to walk away with more information on the new UVM 1.2 features and how they might plan to deploy them.

Naturally, I suggest you consider registering for the conference to attend this tutorial.  (There are still a few seats left; but you will need to hurry!)

UVM Working Group Discussions

As a member of the Accellera UVM Working Group, I have asked the team to consider adopting the SystemC development scheme of an open public review of a pending release of open source code.  While the merger of OSCI and Accellera to form Accellera Systems Initiative inherited the OSCI style of public review, Accellera has not fully embraced it for all its projects.

In a disclosure of a bit of insider conversation I had with the UVM WG this last week, I asked the group to confirm that we were going to bypass the “official” public review option and go to an internal 30-day review cycle only – then release to the public.  While the conclusion was to stay on the 30-day internal review path, the group also noted that one who may be familiar with Git might be able to locate the source code (and many have) and do testing.

Since the bleeding-edge users know they can access as it is being developed, why not share the Git commands for everyone to gain access?  So the group has done just this.  When last minute changes for Release Candidate 4 were put in place, the Git script to offer access for early review was shared publicly.  You can find can find this public message here, thanks to UVM WG member Adiel Khan (from Synopsys).

If you are a seasoned UVM user and are attending DVCon the week of March 3rd, I would encourage you to do some testing now so you can connect with the developers first hand.  And even if you are not attending DVCon but want to migrate to UVM 1.2, you might want to get an early start to determine what you might need to do to adopt this release.

If you are not going to attend the DVCon UVM tutorial and want a short update on what this version will offer, the UVM WG secretary, Adam Sherer (from Cadence), put together a brief slide set that he presented at the TVS DVClub event in September 2013 that you can download.  You may find it a useful companion to the download of the open source code.

Even if you are not attending DVCon, the adoption of UVM is globally substantial and it might be good to reflect on the need for broader testing.  In the first releases of UVM, this may not have been as important as few were using it and the number of tests limited to the main developers.  However, as its popularity has grown and adoption increased, it is probably a good idea for the Accellera UVM Working Group to consider the impact of a new release on teams actively using it now.  While the UVM WG drives to closure on its release candidate and the UVM 1.2 standard, you are offered the opportunity to give us feedback.  For those who have time, please do!

Mentor Commentary on Standards Development

Lastly, for those attending DVCon, check out our own Tom Fitzpatrick’s Wednesday morning paper – Of Camels and Committees: Standards Should Enable Innovation, Not Strangle It. His commentary on the development process may shed some additional light into how technology additions, changes and enhancements are judged for inclusion in updates to standards, like UVM.

Resources:
- UVM 1.2 New Feature Presentation (Sept 2013): Download Here (Free)
- UVM 1.2 Public Review Instructions (Feb 2014): Download Here (Free)
- Mentor Commentary at DVCon: Register Here ($)

, , , , , ,

@dennisbrophy Tweets

  • Loading tweets...

@dave_59 Tweets

  • Loading tweets...