Posts Tagged ‘UVM’
Prologue: The 2012 Wilson Research Group Functional Verification Study
This is the first in a series of blogs that presents the results from the 2012 Wilson Research Group Functional Verification Study.
Study Overview
In 2002 and 2004, Ron Collett International, Inc. conducted its well known ASIC/IC functional verification studies, which provided invaluable insight into the state of the electronic industry and its trends in design and verification. However, after the 2004 study, no other industry studies were conducted, which left a void in identifying industry trends.
To address this void, Mentor Graphics commissioned Far West Research to conduct an industry study on functional verification in the fall of 2007. Then in the fall of 2010, Mentor commissioned Wilson Research Group to conduct another functional verification study. Both of these studies were conducted as blind studies to avoid influencing the results. This means that the survey participants did not know that the study was commissioned by Mentor Graphics. In addition, to support trend analysis on the data, both studies followed the same format and questions (when possible) as the original 2002 and 2004 Collett studies.
In the fall of 2012, Mentor Graphics commissioned Wilson Research Group again to conduct a new functional verification study. This study was also a blind study and follows the same format as the Collett, Far West Research, and previous Wilson Research Group studies. The 2012 Wilson Research Group study is one of the largest functional verification studies ever conducted. The overall confidence level of the study was calculated to be 95% with a margin of error of 4.05%.
Unlike the previous Collett and Far West Research studies that were conducted only in North America, both the 2010 and 2012 Wilson Research Group studies were worldwide studies. The regions targeted were:
- North America:Canada,United States
- Europe/Israel:Finland,France,Germany,Israel,Italy,Sweden,UK
- Asia (minusIndia):China,Korea,Japan,Taiwan
- India
The survey results are compiled both globally and regionally for analysis.
Another difference between the Wilson Research Group and previous industry studies is that both of the Wilson Research Group studies also included FPGA projects. Hence for the first time, we are able to present some emerging trends in the FPGA functional verification space.
Figure 1 shows the percentage makeup of survey participants by their job description. The red bars represents the FPGA participants while the green bars represent the non-FPGA (i.e., IC/ASIC) participants.
Figure 1: Survey participants job title description
Figure 2 shows the percentage makeup of survey participants by company type. Again, the red bars represents the FPGA participants while the green bars represents the non-FPGA (i.e., IC/ASIC) participants.
Figure 2: Survey participants company description
In a future set of blogs, over the course of the next few months, I plan to present the highlights from the 2012 Wilson Research Group study along with my analysis, comments, and obviously, opinions. A few interesting observations emerged from the study, which include:
- FPGA projects are beginning to adopt advanced verification techniques due to increased design complexity.
- The effort spent on verification is increasing.
- The industry is converging on common processes driven by maturing industry standards.
My next blog presents current design trends that were identified by the survey. This will be followed by a set of blogs focused on the functional verification results.
Also, to learn more about the 2012 Wilson Reserach Group study, view my pre-recorded Functional Verification Study web-seminar, which is located out on the Verification Academy website.
Quick links to the 2012 Wilson Research Group Study results (so far…)
- Part 1 – Design Trends
Tags: accellera, Assertion-Based Verification, formal verification, functional coverage, functional verification, IEEE, Simulation, Standards, SystemVerilog, UVM, Verification Academy, Verification Methodology, verilog, vhdl
Even More UVM Debug in Questa 10.2
We’re really excited about the recent Questa 10.2 release, and I’m sure you’ll be just as excited when you check it out. For you UVM-philes out there, we’ve extended our industry-leading UVM Debug features to make your life even easier. I’ll present a quick overview of the new features here, but you’ll really want to get your hands on 10.2 and take a more in-depth look for yourself.
The first thing you’ll notice is that we’ve enhanced to Structure Window (usually located in the upper left of the debugger) to show the class type of each UVM component in your testbench. This will make it easier to know exactly what your factory settings and other configuration settings have yielded as you built your testbench.
One of the most common requests we’ve gotten is to provide a way to see what exactly is happening with the configuration database (uvm_config_db). In the UVM Details window, you can now see the values that are available to the selected component, and by right-clicking you can see who wrote the value and where the write occurred.
In the Stream view of the Details window, you can see all of the transaction streams being recorded by the selected component.
Also, when debugging UVM processes, the Processes Window now includes the hierarchical path to the component that initiated the process.
Lastly, for those of you who may not be GUI-centric, we’ve added a new “uvm” command to the command-line interface in the transcript window (or via “.do” files):
uvm subcommand [args...]
where the “subcommand” lets you choose from a number of options. The default subcommand (and in my opinion, the coolest) is the “call” command, which allows you to call UVM functions directly from the command line. You can even call functions in UVM components by referring to the components via their hierarchical name
uvm call test_top.env1.fab.get_full_name
or via a handle provided by Questa (as seen in the Class Instances window).
call @myClassType@223.get_full_name
There are other useful UVM commands that I won’t go into here, but you should definitely check them out. So, what are you waiting for? To find out more information about Questa with this link: http://www.mentor.com/products/fv/questa/
IEEE 1800™-2012 SystemVerilog Standard Is Published
Download the standard now – at no charge!
The IEEE has published the latest update to the SystemVerilog standard. And courtesy of Accellera, the standard is available for download without charge directly from the IEEE.
The latest update to the SystemVerilog standard is now ready for download. It joins other EDA standards, like SystemC in the IEEE Get™ program that grants public access to view and download current individual standards at no charge as a PDF. (If you wish to have an older, superseded and withdrawn version of the standard or if you wish to have a printed copy or have it in a CD-ROM format, you can purchase older and alternate formats from IEEE for a fee.)
Over the years Accellera came to understand that many people continued to use the freely available version that seeded the initial IEEE 1800 SystemVerilog standard. Since it is significantly out of date, Accellera collaborated with the IEEE Standards Association to ensure the latest version of the SystemVerilog standard would be freely available in electronic form to all whom wish to download it. Accellera now hopes all those old 3.1a versions that everyone has and uses can now be placed in the archives.
The new version of standard should be used by the UVM (Universal Verification Methodology) community as the definitive specification of the SystemVerilog standard upon which UVM is built. It goes very well with the UVM Cookbook and the Coverage Cookbook.
From Mentor’s perspective, it also makes a good companion to the Questa verification platform and complements our latest product update in which we announced support for the IEEE 1800-2012 SystemVerilog standard among other things.
If you have not done so already, download your copy now by clicking here.
Tags: Coverage Cookbook, IEEE 1800, IEEE Get, Standards, systemc, SystemVerilog, UVM, UVM Cookbook
IEEE Approves Revised SystemVerilog Standard
IEEE Std. 1800™-2012 Officially Ratified
The IEEE Standards Association (SA) Standards Board (SASB) officially approved the latest SystemVerilog revision, Draft 6, as an IEEE standard. The SASB Review Committee (RevCom) agenda and the SASB agenda include review and formal approval of the latest work by the IEEE Computer Society Design Automation Standards Committee’s (DASC) SystemVerilog Working Group at their December 2012 meeting series.
What’s New?
The new standard has many new features, numerous clarifications and various corrections to improve the standard and keep pace with electronic system design and verification. DVCon 2012 included a session presentation, Keeping Up with Chip – The Proposed SystemVerilog 2012 Standard Makes Verifying Ever-Increasing Design Complexity More Efficient” that detailed the standard. The paper was written by Stuart Sutherland (Sutherland HDL, Inc.) and Tom Fitzpatrick (Mentor Graphics). You can find a copy of the paper here at the DVCon 2012 archive and the presentation can be found at Sutherland HDL’s site here.
For users of Mentor Graphics’ Questa Verification Platform, many of the major SystemVerilog 2012 features can be used today, like multiple inheritance. As Stu and Tom said in their presentation, “This is BIG!” If you read their full paper, they discuss some ways this new feature might be useful for a UVM testbench.
Major work was done to augment the current notion of constraints in SystemVerilog. In past versions of the standard they were known as hard constraints. What this meant was all the conditions of the constraints had to be met otherwise there would be an error. There was no built-in method to relax the need to satisfy the constraints. Given the world of multiple constraints is the norm for testbenches today the potential for conflicts between them is high. To alleviate this the SystemVerilog Working Group introduced soft constraints to the standard. If you are interested in the details of what was proposed to be added the standard, you can reference the full proposal here that is included in the standard. Stu and Tom said that “This is also a big enhancement!”
Availability
IEEE 1800™-2012 has only now been approved. The standard itself is not ready to be published yet. Plans are to have it ready to be published before DVCon 2013, which is scheduled for late February 2013. I will share publication information as it becomes available. And, I hope you join me and attend DVCon 2013 where we can plan to celebrate the unveiling of the published standard.
While the IEEE publication will be the authoritative source on the standard, I have pointed to the presentation and paper by Stu Sutherland and Tom Fitzpatrick for information on the new standard that you can reference now. For those who depend on assertions, you will find SystemVerilog-2012 has a major update with enhancements for properties and sequences in the area of immediate assertions, data type support, argument passing, vacuity definitions, global clock resolution and inferred clocking in sequences and much more. You may find the SystemVerilog Assertions Handbook 3rd Edition by Ben Cohen, et. al. to be of value as well. You can find more information about it on Amazon.com here.
The Story Continues…
There is much more to the SystemVerilog-2012 story I will share more of that in the months ahead. The global team of experts who have put this together has been an outstanding collection of individuals ranging from producers and suppliers of electronic design automation software to consumers of said technology who have ensured the language can be used to design and verify the most demanding of electronic systems.
Stay tuned! For now, I encourage you to get informed!
Tags: 1800, Assertions, Ben Cohen, DASC, dvcon, Hard Constraints, IEEE SASB, IEEE-SA, Muttiple Inheritance, RevCom, Soft Constraints, Standards, Stu Sutherland, SystemVerilog, Tom Fitzpatrick, UVM
Coverage Cookbook Debuts
Verification Academy Adds Major New Technical Resource
The Verification Academy adds another major methodology cookbook to focus on effective coverage adoption. The Coverage Cookbook describes the different types of coverage that are available to track your verification process progress, how to create a functional coverage model from a specification, and provides examples to implement functional coverage for different types of designs.
Verification Academy “full access” members have access to the free Coverage Cookbook and the UVM/OVM Cookbooks as well. Are you a registered full access member? If not, register now to become a full access member. (Restrictions apply.)
Coverage is not a new topic. It was one of major additions to the SystemVerilog (IEEE Std. 1800™-2009) standard. But the SystemVerilog functional coverage extensions were left to the verification engineer to use in such as way to return meaningful measurements of how much of the design specification was being tested. The Universal Verification Methodology (UVM) offers greater structure for coverage over SystemVerilog, but it too, is still only a piece of the puzzle.
As verification teams have come to generate greater amounts of information from use of SystemVerilog, UVM and other verification tools, the data from the verification runs needs to be easily used to drive coverage closure. Within the Mentor Graphics Questa verification platform, this resulted in the development of the Unified Coverage Database (UCDB) and associated verification management and planning features.
Since verification teams use a variety of tools and technology from many sources, it was an imperative that verification information could be easily shared and combined to help drive faster coverage closure across the industry. This is why Mentor Graphics donated its UCDB API to Accellera where it became the Unified Coverage Interoperability Standard (UCIS).
It would be great to think that we are done; but we’re not. Tools and data are just two dimensions of the three dimensions to any IC design project. A comprehensive approach to verification management that handles all of this adds the third dimension. The Mentor Graphics Questa Verification Management features handle all this.
Now the question is how to best adopt and use all the capabilities at hand from the standards to the verification technology at your finger tips.
The Verification Academy Coverage Cookbook is one of the important tools you now have to help pull all the information into a single place where you can learn the theory and put that theory into practice. The Coverage Cookbook is much like the OVM/UVM Cookbooks in that it is web friendly, while supporting the ability for you to generate a PDF file of the whole document in case you want to have a printed copy or have it available for offline reference.
The Theory section covers:
|
The Practice section shows three examples you can use today:
|
The Coverage Cookbook is a live document. You can expect continued extensions and contributions to enhance it. As Harry Foster, Mentor Graphics’ Chief Scientist Verification put it, “Methodology is the bridge between tools and technologies, which creates a productive, predictable, and repeatable solution.” We should expect that our collective use of this technology will help hone the methodology which is the heart of the Coverage Cookbook. And with this use, we should expect the Coverage Cookbook to evolve as we achieve greater verification productivity.
Let us know what you think about the Coverage Cookbook and what we might be able to do to improve it. In the meantime, Happy Coverage Closing!
Tags: accellera, Coverage, Coverage Closure, Coverage Cookbook, functional coverage, Harry Foster, IEEE 1800, OVM, SystemVerilog, UCDB, UCIS, UVM, Verification Academy
Introducing “Verification Academy 2.0”
A new style takes center stage
It was Fashion Week in Portland, Oregon in early October. And while the thought of Portland and fashion might not be believable to many in the world, especially those who look to the design houses of Paris or Milan, it was. What struck me was the blend of fashion with high tech this year. Intel took the opportunity to roll out its fashion inspired campaign (dressing room mirror sized tablets) and Mitsubishi used it to launch its new electric vehicle (named MiEV in case you did not know). Certainly it was more than just your run-of-the-mill runway show. But that was not the only thing “getting some style” here in the Portland area.
The Verification Academy team at Mentor Graphics has been working hard as well to restyle the Verification Academy website, modernize it and make content easily accessible. It made its debut in late September, a few weeks before the Portland Fashion Show. While these two things are a coincidence, the focus on a refreshed style should not to be totally unexpected.
Some of the changes just had to be made given the success of the Verification Academy. When it started a few years back, Harry Foster (the face in the picture of the Verification Academy website above) knew the adoption of advanced technology was hampered by unequal and slow distribution of knowledge. Part of the Verification Academy’s thrust was to bring information about advanced verification topics to the whole world in a format that could be easily used. The content comes from respected verification subject matter experts and the first “runaway success” was the Open Verification Methodology (OVM) training by John Aynsley from Doulos for the “basic” module and Tom Fitzpatrick from Mentor Graphics for the “advanced” module. The Universal Verification Methodology (UVM) course, likewise, has also joined the ranks of the highly watched. Updates to the Academy improve the services to deliver video.
We have moved to the most current web video protocols that allow modern browsers and mobile devices to easily access course content. You can watch courses on the “smaller” smartphone screens to the largest of TV displays with SD and HD video to support your viewing preferences. Since content is delivered in native web technologies, users do not have to depend on Flash or other plugins.
We have also migrated the Academy to the leading open source content management system and adopted the use SSL throughout the Verification Academy to make it more secure.
When we first started the Verification Academy, we did not know how large the community would grow nor could we predict the demands the community would place on the resources to support it. Today, there are almost 12.5K users making it the largest single site to support the verification professional. The changes we have made to the internals of the site show a speed improvement of over 400% by exploiting a commercial content delivery network to handle large media.
And for many members, where English is a second language, the video captions, when offered, are in plain text. Registered users can click on the picture to the right to see the UVM Introduction and enable closed caption to see how the text appears right below the video. (Or, from reading the text below video in the picture to the right, you can see John is introducing himself at the moment of this screen capture.)
We have also made big improvements to searches. The searching facility now scans across all content at once, from the forums, to the UVM/OVM Cookbook and presents the information to you in an improved way to allow you to filter the results to focus on just that you want to know.
Want to experience the new Verification Academy 2.0 style? Click here to go to the Verification Academy to see these changes and discover these and other changes yourself. Share your comments with me on what you think. Have we made it better for you? And if not, what more can we do to improve your experience even more?
Tags: OVM, UVM, UVM Cookbook, Verification, Verification Academy
OVM Gets Connected
OVM Bridges SystemVerilog and SystemC Languages
When UVM Connect was first released, the multilingual connection between IEEE Std. 1800™ (SystemVerilog) and IEEE Std. 1666™ (SystemC) standards bridged the two languages to allow design and verification engineers to access UVM from SystemC or SystemVerilog to exploit native languages advantages. OVM users wondered if it was possible to support them as well since OVM is a derived from UVM.
It is possible and UVM Connect has been extended to allow OVM users to enjoy the same benefits. An update to UVM Connect now allows it to be compiled to run with the OVM. And since the extensions are based on IEEE standards, they can be used in your simulator of choice.
OVM Thrives
The thriving OVM community is of no surprise. Last year, Harry Foster blogged about research on the use and adoption of verification methodologies. The research was done after UVM was established as an Accellera standard, and showed OVM continued its leading position as shown in one of the charts from Harry’s blog (see below). The chart even showed OVM was predicted to have a modest growth in adoption as well.
Mentor continues to bring many of the UVM additions back to the OVM user community in a way that does not disturb the upgrade path from OVM to UVM. The major addition to UVM in the first round of Accellera standardization was the addition of a register and memory package. This was back ported to OVM. (The OVM register and memory kit can be found here, if you are interested.) Now, UVM Connect has been extended to provide full OVM use.
Download
The UVM Connect 2.2 kit supports multilingual use of OVM and can be found at the Verification Academy and the Accellera UVM World contributions download site.
If you find issues or have other suggestions that we should consider, you can always share your input at the OVM Forum or UVM Forum. In addition to interacting with other users, the Verification Academy is a good site for online resources like the UVM/OVM Cookbook, basic and advanced OVM/UVM training, and more.
Tags: 1666, 1800, accellera, IEEE, Multilanguage OVM, Multilanguage UVM, OVM, Standards, systemc, SystemVerilog, UVM, UVM Connect
Verification Academy: Up Close & Personal
Live & In-Person at DAC 2012!
Verification Academy, the brain child of Harry Foster, Chief Verification Scientist at Mentor Graphics, was live from the Design Automation Conference tradeshow floor this year. Harry is pictured to the right giving an update on his popular verification survey from the DAC tradeshow floor.
The Verification Academy, predominantly a web-based resource is a popular site for verification information with more than 11,000 registered members for forum access on topics ranging from OVM/UVM, SystemVerilog and Analog/Mixed-Signal design. The popular OVM/UVM Cookbook, which used to be available as a print edition, is now a live online resource there as well. A whole host of educational modules and seminars can also be found there too.
If you know about the Verification Academy, you know all about the content mentioned above and that there is much more to be found there. For those who don’t know as much about it, Harry took a break from the being at the Verification Academy booth at DAC to discuss the Verification Academy with Luke Collins, Technology Journalist, Tech Design Forum. (Flash is required to watch Harry discuss Verification Academy with Luke.)
The Verification Academy at DAC was a great venue to connect in person with other Verification Academy users to discuss standards, methodologies, flows and other industry trends. Each hour there were short presentations by Verification Academy members that proved to be a popular way to start some interesting conversations. While we realize not all Verification Academy members were able to attend DAC in person, we know many have expressed an interest to some of the presentations. Verification Academy “Total Access” members now have access to many of the presentations.
|
ARM |
||
|
Doulos |
||
|
Thales Alenia Space |
||
|
Test & Verification Solutions |
||
|
Willamette HDL |
||
|
Sunburst Design |
||
|
Mentor Graphics |
Total Access members can also download all the presentations in a .zip file. Happy reading to all those who were unable to visit us at DAC and thank you to all who were able to stop by and visit.
Tags: ABV, ACE, ams, ARM, Assertion-Based Verification, Coverage Closure, dac, Doulos, formal, IEEE, iTBA, Low Power, OVM, SystemVerilog, Tech Design Forum, Thales, upf, UVM, UVM Express, Verification Academy, Verification Trends
SystemC Standardization Cycle Completes
Open-Source Proof-of-Concept Library Released
Accellera Systems Initiative has released for general industry use an open-source proof-of-concept library as a companion to the recently minted IEEE Std. 1666™-2011, SystemC Language Reference Manual standard
In November 2011, the IEEE Standards Association approved IEEE Std. 1666-2011. The completed and published standard was made available to the community as a whole for free in an agreement between Accellera Systems Initiative and the IEEE Standards Association in February 2012. As a reminder, you can download your personal copy of IEEE 1666 here for free.
In the nearly 6 months since this version of the standard has been available about 7,000 copies have been downloaded under the IEEE Get program.
The previous version was also made available for free download and was just as popular as this version of the standard is.
While the approved standard was being made ready for publication, Accellera Systems Initiative was also busy completing the open-source proof-of-concept library. After taking comments and feedback from a public review process, version 2.3.0 of the library was completed and is now available.
IEEE 1666-2011 added a number of important new features, including support for transaction-level modeling (TLM) that has proven to be an important element to enable high-level design and is a key component upon which the Universal Verification Methodology (UVM) is built from.
For those who want to used the SystemC library directly, it is now available for wide industry access.
Download Resources
The downloads from the IEEE and Accellera Systems Initiative will require some license agreement approvals. The links are not one-click access to the material below.
- IEEE 1666-2011 Standard
All who download will need to agree to a click-through IEEE license and declare a “user type.” - SystemC 2.3.0 Open-Source Proof-of-Concept Library
Non-members need to register as a “community participant” in order to download the library and regression tests.
Tags: 1666, Accellera Systems Initiative, IEEE, open-source, proof-of-concept library, Standards, systemc, UVM
Intelligent Testbench Automation – Catching on Fast
Graph-Based Intelligent Testbench Automation
While intelligent testbench automation is still reasonably new when measured in EDA years, this graph-based verification technology is being adopted by more and more verification teams every day. And the interest is global. Verification teams from Europe, North America, and the Pacific Rim are now using iTBA to help them verify their newest electronic designs in less time and with fewer resources. (If you haven’t adopted it yet, your competitors probably have.) If you have yet to learn how this new technology can help you achieve higher levels of verification, despite increasing design complexity, I’d suggest you check out a recent article in the June 2012 issue of Verification Horizons titled “Is Intelligent Testbench Automation For You?” The article focuses on where iTBA is best applied and where it will help you most by producing optimal results, and how design applications with a large verification space, functionally oriented coverage goals, and unbalanced conditions can often experience a 100X gain in coverage closure acceleration. For more detail about these and other considerations, you’ll have to read the article.
Fitzpatrick’s Corollary
And while you’re there, you might also notice that the entire June 2012 issue of Verification Horizons is devoted to helping you achieve the highest levels of coverage as efficiently as possible. Editor and fellow verification technologist Tom Fitzpatrick succinctly adapts Murphy’s Law to verification, writing “If It Isn’t Covered, It Doesn’t Work”. And any experienced verification engineer (or manager) knows just how true this is, making it critical that we thoughtfully prioritize our verification goals, and achieve them as quickly and efficiently as possible. The June 2012 issue offers nine high quality articles, with a particular focus on coverage.
Berg’s Proof
Another proof that iTBA is catching on globally, is the upcoming TVS DVClub event being held next Monday 2 July 2012, in Bristol, Cambridge, and Grenoble. The title of the event is “Graph-Based Verification”, and three industry experts will discuss different ways you can take advantage of what graph-based intelligent testbench automation has to offer. My colleague and fellow verification technologist Staffan Berg leads off the event with a proof of his own, as he will present how graph-based iTBA can significantly shorten your time-to-coverage. Staffan will show you how to use graph-based verification to define your stimulus space and coverage goals, by highlighting examples from some of the verification teams that have already adopted this technology, as I mentioned above. He’ll also show how you can introduce iTBA into your existing verification environment, so you can realize these benefits without disrupting your existing process. I have already registered and plan to attend the TVS DVClub event, but I’ll have to do some adapting of my own as the event runs from 11:30am to 2:00pm BST in the UK. But I’ve seen Staffan present before, and both he and intelligent testbench automation are worth getting up early for. Hope to see you there, remotely speaking.
Tags: functional verification, Intelligent Testbench Automation, iTBA, testbench, UVM, Verification, Verification Academy
About Verification Horizons BLOG
This blog will provide an online forum to provide weekly updates on concepts, values, standards, methodologies and examples to assist with the understanding of what advanced functional verification technologies can do and how to most effectively apply them. We're looking forward to your comments and suggestions on the posts to make this a useful tool.
Latest Posts
- Part 1: The 2012 Wilson Research Group Functional Verification Study
- What’s the deal with those wire’s and reg’s in Verilog
- Getting AMP’ed Up on the IEEE Low-Power Standard
- Prologue: The 2012 Wilson Research Group Functional Verification Study
- Even More UVM Debug in Questa 10.2
- IEEE Approves New Low Power Standard
Subscribe to the Verification Horizons Newsletter