Posts Tagged ‘1800’

24 February, 2017

My last blog post was written a few years ago before attending a conference when I was reminiscing about the 10-year history of SystemVerilog. Now I’m writing about going to another conference, DVCon, and being part of a panel reminiscing about the 15-year history of SystemVerilog and envisioning its future. My history with SystemVerilog goes back much further.

Soul of a New MachineMy first job out of college was working with a group of Data General (DG) engineers made public in the book, Soul of a New Machine, by Tracy Kidder in 1981. Part of my job was writing a program that simulated the microcode of the CPUs we designed. Back then, there were no hardware description languages and you had to hard code everything for each CPU. If you were lucky you could reuse some of the code for the user interface between projects. Later, DG came up with a somewhat more general-purpose simulation language. It was general-purpose in the sense that it could be used for a wider range of projects based on the way DG developed hardware. But getting it to work in another company’s environment would have been a challenge.  By the way, Badru Agarwala was the DG simulation developer I worked with who later founded the Verilog simulation companies Frontline and Axiom. He now manages the Calypto division at Mentor Graphics.

Many other processor companies like DEC, IBM and Intel had their own in-house simulation languages or were in the process of developing one because no commercially viable technologies existed. Eventually, Phil Moorby at Gateway Design began developing the Verilog language and simulator. One of the benefits of having an independent language, although not an official standard yet, was you could now share or bring in models from outside your company. This includes being able to hand off a Verilog netlist to another company for manufacturing. Another benefit was that companies could now focus on the design and verification of their products instead of the design and verification of tools that design and verify their products.

I evaluated Verilog in its first year of release back in 1985/1986. DG decided not to adopt Verilog at that point, but I liked it so much I left DG and joined Gateway Design as one of its first application engineers. Dropping another name, Karen Bartleson was one of my first customers as a CAD manager working at UTMC. She recently took the office of President and CEO of the IEEE.

IEEEFast forward to the next decade, when Verilog became standardized as IEEE 1364-1995. But by then it had already lost ground in the verification space. Companies went back to developing their own in-house verification solutions. Sun Microsystems developed Vera and later released it as a commercial product marketed by Systems Science. Arturo Salz was one of its developers and will be on the DVCon panel with me as well. Specman was developed for National Semiconductor and a few other clients and later marketed by Verisity. Once again, we had the problem of competing independent languages and therefore limiting the ability to share or acquire verification models. So, in 1999, a few Gateway alums and others formed a startup which I joined a year later hoping to consolidate design and verification back into one standard language. That language was SUPERLOG and became the starting point for the Accellera SystemVerilog 3.0 standard in 2002, fifteen years ago.

IEEE Std 1800-2012There are many dates you could pick for the start of SystemVerilog. You could claim it couldn’t exist until there was a simulator supporting some of the features in the standard. For me, it started when I first read an early Verilog Language Reference Manual and executed my first initial block 31 years ago. I’ve been using Verilog almost every day since. And now all of Verilog is part of SystemVerilog. I’ve been so much a part of the development of the language from its early beginnings; that’s why some of my colleagues call me “The Walking LRM”. Luckily, I don’t dream about it. I hope I never get called “The Sleeping LRM”.

So, what’s next for SystemVerilog? Are we going to repeat the cycle of fragmentation and re-consolidation? Various extensions have already begun showing up in different implementations. SystemVerilog has become so complex that no one can keep a good portion of it in their head anymore. It is very difficult to remove anything once it is in the LRM. Should we start over? We tried to do that with SUPERLOG, but no one adopted it until it was made fully backward compatible with Verilog.

The Universal Verification Methodology (UVM) was designed to cut down the complexity of learning and using SystemVerilog. There are now a growing number of sub-methodologies for using the UVM because the UVM itself has exploded in complexity  (UVM Framework and Easier UVM to name a couple). I have also taken my own approach when teaching people SystemVerilog by showing a minimal subset (See my SystemVerilog OOP for UVM Verification course).

DVConI do believe in the KISS principle of Engineering and I hope that is what prevails in the next version of SystemVerilog, whether we start over or just make refinements. I hope you will be able to join the discussion with others and me at the DVCon panel next week, or in the forums in the Verification Academy, or on Twitter.

-Dave

, , , , , , , , , , , ,

31 October, 2016

ASIC/IC Language and Library Adoption Trends

This blog is a continuation of a series of blogs related to the 2016 Wilson Research Group Functional Verification Study (click here).  In my previous blog (click here), I focused on I various verification technology adoption trends. In this blog I plan to discuss various ASIC/IC language and library adoption trends..

Figure 1 shows the adoption trends for languages used to create RTL designs. Essentially, the adoption rates for all languages used to create RTL designs is projected to be either declining or flat over the next year.

BLOG-2016-WRG-figure-10-1

Figure 1. ASIC/IC Languages Used for RTL Design

As previously noted, the reason some of the results sum to more than 100 percent is that some projects are using multiple languages; thus, individual projects can have multiple answers.

Figure 2 shows the adoption trends for languages used to create ASIC/IC testbenches. Essentially, the adoption rates for all languages used to create testbenches are either declining or flat. Furthermore, the data suggest that SystemVerilog adoption is starting to saturate or level off in the mid-70s range.

BLOG-2016-WRG-figure-10-2

Figure 2. ASIC/IC Languages Used for  Verification (Testbenches)

Figure 3 shows the adoption trends for various ASIC/IC testbench methodologies built using class libraries.

BLOG-2016-WRG-figure-10-3

Figure 3. ASIC/IC Methodologies and Testbench Base-Class Libraries

Here we see a decline in adoption of all methodologies and class libraries with the exception of Accellera’s UVM, whose adoption continued to increase between 2014 and 2016. Furthermore, our study revealed that UVM is projected to continue its growth over the next year. However, like SystemVerlog, it will likely start to level off in the mid- to upper-70 percent range.

Figure 4 shows the ASIC/IC industry adoption trends for various assertion languages, and again, SystemVerilog Assertions seems to have saturated or leveled off.

BLOG-2016-WRG-figure-10-4

Figure 4. ASIC/IC Assertion Language Adoption

In my next blog (click here) I plan to present the ASIC/IC design and verification power trends.

Quick links to the 2016 Wilson Research Group Study results

, , , , , , , , , , , , , , , , , , ,

7 October, 2015

ieee-sa-logo2Design and verification flows are multifaceted and predominantly built by bringing tools and technology together from multiple sources.   The tools from these sources build upon IEEE standards – several IEEE standards.  What started with VHDL (IEEE 1076™) and Verilog/SystemVerilog (IEEE 1800™) and their documented interfaces has grown.  As more IEEE standards emerged and tools and technology combined these standards in innovative and differentiated ways the industry would benefit from an ongoing open and public discussion on interoperability.  The IEEE Standards Association (IEEE-SA) continues with this tradition started by my friends at Synopsys with the IEEE-SA EDA & IP Interoperability Symposium.  And for 2015, I’m pleased to chair the event.

Anyone working on or using design and verification flows that depend on tool interoperability as well as design and verification intellectual property (IP) working together will benefit from attending this symposium.  The symposium will be held Wednesday, 14 October 2015, at the offices of Cadence Design Systems in San Jose, CA USA.  You can find more information about the event at the links below:

  • Register: Click here.
  • Event Information: Click here.
  • Event Program: Click here.

A keynote presentation by Dan Armbrust, CEO Silicon Catalyst, opens the event with a talk on Realizing the next growth wave for semiconductors – A new approach to enable innovative startups.  If you are one of the Silicon Valley innovators, you might like to hear what Dan shares on this next growth wave.  From my perspective, I suspect it will include being more energy conscious in how we design.  The work on current and emerging IEEE standards that address those energy concerns will follow.  We will review what the conclusions were from the DAC Low Power Workshop and leadership from the IEEE low power standards groups will discuss what they are doing in context of Low Power Workshop.

We then take a lunch break and celebrate 10 Years of SystemVerilog.  The first IEEE SystemVerilog standard (IEEE Std. 1800™-2005) was published in November 2005.  It seems fitting we celebrate this accomplishment.  Joining many of the participants in the IEEE SystemVerilog standardization effort for this celebration will be participants from the Accellera group that incubated it before it became an IEEE standard.  We won’t stop with just celebrating SystemVerilog.  We will also share information on standards projects that have leveraged SystemVerilog, like UVM, which has recently become a full fledged IEEE standards project (IEEE P1800.2).  With so many people who have worked on completed and successful IEEE standards, Accellera offered to bring its Portable Stimulus Working Group members over for a lunch break during their 3-day face-to-face Silicon Valley meeting to mingle with them, to learn from them and hopefully be inspired by them as well.  Maybe some of the success of building industry relevant standards can be shared between the SystemVerilog participants and Accellera’s newer teams.

We will then return to a focus on energy related issues with our first topic area being on power modeling for IP.  Chris Rowen, Cadence Fellow, will take us through some recent experiences on issues his teams have faced driving even higher levels of power efficiency from design using ever more design IP. Tails from the trenches never get old and offer us insight on what we might do in the development of better standards to help address those issues.  While Chris will point to a lot of issues when it comes to the use of design IP, I believe these issues are only compounded when it comes to the Internet of Things (IoT).  We have assembled a great afternoon panel to discuss if the “ultimate power challenge” is IoT.  I can’t wait to hear what they say.

Lastly, when we pull all these systems together, LSI package board issues pose a design interoperability challenge as well.  The IEEE Computer Society’s (CS) Design Automation Standards Committee (DASC) has completed another standard developed primarily outside of North America.  The DASC has a long history of global participation and significant standards development outside of North America, like is the case for VHDL AMS (IEEE 1076.1).  We will hear from the IEEE 2401™-2015 leadership on their newly minted IEEE standard and the LSI package board issues that have been addressed.

We don’t have time to highlight all the EDA & IP standards work in the IEEE, but our principle theme to address issues of power in modern design and verification led us to focus on a subset of them.  So, if your favorite standard or topic area does not appear in the program, let me know and we can add that to our list to consider next year.  And when I say “we,” the work to put together an event like this takes a lot of people. All of us are interested in what we should do for next year and what your input is to us.  For me, in addition to working to collect this, I also need to thank those who did all the work to make this happen.  I’ve often said, as chair, you let the others do all the work.  It has been great to collaborate with my IEEE-SA friends, my peers at the other two Big-3 EDA companies.  It has also been great to get the input and advice on the Steering Committee from two of the world’s largest silicon suppliers (Intel & TSMC) and to include for the first time, support from standards incubators Accellera Systems Initiative and Si2.

, , , , , , , , , , , , , , , ,

30 July, 2015

Accellera Handoffs UVM to IEEE

It has been a long path from Mentor’s AVM to IEEE P1800.2.  But the moment has arrived: Accellera has formally announced UVM 1.2 will be submitted as a contribution to the IEEE P1800.2™ working group.

Verification Methodology Beginnings

As the IEEE finalized approval of the initial release of SystemVerilog (IEEE Std. 1800™) in 2005, I floated the idea of the need for a methodology that would be a companion to it.  At the time there was little to no industry desire to explore this opportunity in earnest – apart from interest by Mentor Graphics – so we launched our Advanced Verification Methodology (AVM) and set a new direction for an open functional verification methodology.  We built implementations of AVM based on SystemVerilog and SystemC (IEEE Std. 1666™).  We also pioneered an open-source mechanism based on the Apache 2.0 license which is now the accepted license to foster global and rapid open-source adoption in the EDA industry.  And as others joined with us in this journey, AVM grew to become OVM, then UVM.  Now UVM is set to become an IEEE standard.  The IEEE has assigned it project number 1800.2.

imagePath to IEEE

To say we are pleased to see UVM move to the IEEE is an understatement.  We congratulate the Accellera UVM team on its accomplishment and look forward to participate in this phase of UVM’s standardization. Since our first public announcement on May 8, 2006 when we introduced the world to AVM and announced support for it from 19 of our Questa Vanguard Partners, to our announced collaboration with Cadence Design Systems on the development of the Open Verification Methodology (OVM) on August 16, 2007 and the eventual announcement January 8, 2010 that Accellera adopts OVM as the basis of its Universal Verification Methodology, we have guided its development and supported a path for the Big-3 EDA to voice positive public support.  We are thrilled Accellera has announced its delivery of UVM to the IEEE for ongoing standardization and maintenance.

IEEE Standardization

What comes next?  The IEEE P1800.2 (UVM) project has announced a Call for Participation and kickoff meeting to be held August 6, 2015 from 9am – 11am PDT.  The first meeting will be held via teleconference.  In order to attend, you will need to register for the meeting.  Membership in the IEEE project will be “entity-based” with one company, one vote.  The call for participation has details on membership requirements in order to observe or actively participate.  The 1800.2 project will only focus on the written specification and not the open-source base class library (BCL).  The Accellera UVM TSC will continue to update the BCL.  Accellera has committed to keep the BCL implementation current with changes proposed and approved by the IEEE 1800.2 working group.  This is just like the arrangement Accellera has with the IEEE for SystemC.

Join us at the upcoming meeting and remember to register in order to attend!

 

, , , , , , , , , ,

27 July, 2015

ASIC/IC Language and Library Adoption Trends

This blog is a continuation of a series of blogs related to the 2014 Wilson Research Group Functional Verification Study (click here).  In my previous blog (click here), I presented our study findings on various verification technology adoption trends. In this blog, I focus on language and library adoption trends.

As previously noted, the reason some of the results sum to more than 100 percent is that some projects are using multiple languages; thus, individual projects can have multiple answers.

Figure 1 shows the adoption trends for languages used to create RTL designs. Essentially, the adoption rates for all languages used to create RTL designs is projected to be either declining or flat over the next year, with the exception of SystemVerilog.

2014-WRG-BLOG-ASIC-10-1

Figure 1. ASIC/IC Languages Used for RTL Design

Figure 2 shows the adoption trends for languages used to create ASIC/IC testbenches. Essentially, the adoption rates for all languages used to create testbenches are either declining or flat, with the exception of SystemVerilog. Nonetheless, the data suggest that SystemVerilog adoption is starting to saturate or level off at about 75 percent.

2014-WRG-BLOG-ASIC-10-2

Figure 2. ASIC/IC Languages Used for  Verification (Testbenches)

Figure 3 shows the adoption trends for various ASIC/IC testbench methodologies built using class libraries.

2014-WRG-BLOG-ASIC-10-3

Figure 3. ASIC/IC Methodologies and Testbench Base-Class Libraries

Here we see a decline in adoption of all methodologies and class libraries with the exception of Accellera’s UVM3, whose adoption increased by 56 percent between 2012 and 2014. Furthermore, our study revealed that UVM is projected to grow an additional 13 percent within the next year.

Figure 4 shows the ASIC/IC industry adoption trends for various assertion languages, and again, SystemVerilog Assertions seems to have saturated or leveled off.

2014-WRG-BLOG-ASIC-10-4

Figure 4. ASIC/IC Assertion Language Adoption

In my next blog (click here) I plan to present the ASIC/IC design and verification power trends.

Quick links to the 2014 Wilson Research Group Study results

, , , , , , , , , , , , , , , ,

3 June, 2015

FPGA Language and Library Trends

This blog is a continuation of a series of blogs related to the 2014 Wilson Research Group Functional Verification Study (click here). In my previous blog (click here), I focused on FPGA verification techniques and technologies adoption trends, as identified by the 2014 Wilson Research Group study. In this blog, I’ll present FPGA design and verification language trends, as identified by the Wilson Research Group study.

You might note that the percentage for some of the language and library data that I present sums to more than one hundred percent. The reason for this is that many FPGA projects today use multiple languages.

FPGA RTL Design Language Adoption Trends

Let’s begin by examining the languages used for FPGA RTL design. Figure 1 shows the trends in terms of languages used for design, by comparing the 2012 Wilson Research Group study (in dark blue), the 2014 Wilson Research Group study (in light blue), as well as the projected design language adoption trends within the next twelve months (in purple). Note that the language adoption is declining for most of the languages used for FPGA design with the exception of Verilog and SystemVerilog.

Also, it’s important to note that this study focused on languages used for RTL design. We have conducted a few informal studies related to languages used for architectural modeling—and it’s not too big of a surprise that we see increased adoption of C/C++ and SystemC in that space. However, since those studies have (thus far) been informal and not as rigorously executed as the Wilson Research Group study, I have decided to withhold that data until a more formal study can be executed related to architectural modeling and virtual prototyping.

Figure 1. Trends in languages used for FPGA design

It’s not too big of a surprise that VHDL is the predominant language used for FPGA RTL design, although the projected trend is that Verilog will likely overtake VHDL in terms of the predominate language used for FPGA design in the near future.

FPGA Verification Language Adoption Trends

Next, let’s look at the languages used to verify FPGA designs (that is, languages used to create simulation testbenches). Figure 2 shows the trends in terms of languages used to create simulation testbenches by comparing the 2012 Wilson Research Group study (in dark blue), the 2014 Wilson Research Group study (in light blue), as well as the projected verification language adoption trends within the next twelve months (in purple).

Figure 2. Trends in languages used in verification to create FPGA simulation testbenches

FPGA Testbench Methodology Class Library Adoption Trends

Now let’s look at testbench methodology and class library adoption for FPGA designs. Figure 3 shows the trends in terms of methodology and class library adoption by comparing the 2012 Wilson Research Group study (in dark blue), the 2014 Wilson Research Group study (in light blue), as well as the projected verification language adoption trends within the next twelve months (in purple).

Figure 3. FPGA methodology and class library adoption trends

Today, we see a downward trend in terms of adoption of all testbench methodologies and class libraries with the exception of UVM, which has increased by 28 percent since 2012. The study participants were also asked what they plan to use within the next 12 months, and based on the responses, UVM is projected to increase an additional 20 percent.

FPGA Assertion Language and Library Adoption Trends

Finally, let’s examine assertion language and library adoption for FPGA designs. The 2014 Wilson Research Group study found that 44 percent of all the FPGA projects have adopted assertion-based verification (ABV) as part of their verification strategy. The data presented in this section shows the assertion language and library adoption trends related to those participants who have adopted ABV.

Figure 4 shows the trends in terms of assertion language and library adoption by comparing the 2010 Wilson Research Group study (in dark blue), the 2012 Wilson Research Group study (in green), and the projected adoption trends within the next 12 months (in purple). The adoption of SVA continues to increase, while other assertion languages and libraries either remain flat or decline.

Figure 4. Trends in assertion language and library adoption for FPGA designs

In my next blog (click here), I will shift the focus of this series of blogs and start to present the ASIC/IC findings from the 2014 Wilson Research Group Functional Verification Study.

Quick links to the 2014 Wilson Research Group Study results

, , , , , , , ,

19 September, 2013

It’s hard for me to believe that SystemVerilog 3.1 was released just over 10 years ago. The 3.1 version added Object-Oriented Programming features for testbench development to a language predominately used for RTL design synthesis. Making debug easier was one of the driving forces in unifying testbench and design features into a single language. The semantics for evaluating expressions and executing statements would be the same in the testbench and design. Setting breakpoints and stepping through the code would be seamless. That should have made it easier for either a verification or a design engineer to understand a complete verification environment. Or maybe it would enable either one to at least understand enough of the environment to isolate a particular problem.

Ten years later, I have yet to see that promise fulfilled. Most design engineers still debug their simulations the same way they debug in the lab: they look at waveforms. During simulation, they rarely look at the design source code, and certainly never look at the testbench code (unless it’s just basic pin wiggling like a waveform). Verification engineers are not much different. They rely on waveform debugging because that is what they were brought up on, and many do not even realize source-level debugging is available to them. However the test/testbench is more like a piece of software than a hardware description, and there are many things about a modern testbench that is difficult to display in a waveform (e.g. call stacks, local variables, and random constraints). And methodologies like the UVM add many layers of source-level complexity that most users do not have the time to wade through.

Next week I will be presenting as part of an Industry Special Session during the Forum on specification & Design Languages (FDL September 24-26,2013) that will discuss these issues and try to get more involvement from the academic and user communities to help resolve them. Was combining constructs from many languages into one a success? Can tools provide representations of source-level constructs in an easier graphical form? We hopefully will not need another decade.

, , , ,

12 August, 2013

Language and Library Trends (Continued)

This blog is a continuation of a series of blogs that present the highlights from the 2012 Wilson Research Group Functional Verification Study (for a background on the study, click here).

In my previous blog (Part 8 click here), I focused on design and verification language trends, as identified by the Wilson Research Group study. This blog presents additional trends related to verification language and library adoption trends.

You might note that for some of the language and library data I present, the percentage sums to more than 100 percent. The reason for this is that some participants’ projects use multiple languages or multiple testbench methodologies.

Testbench Methodology Class Library Adoption

Now let’s look at testbench methodology and class library adoption for IC/ASIC designs. Figure 1 shows the trends in terms of methodology and class library adoption by comparing the 2010 Wilson Research Group study (in blue) with the 2012 study (in green). Today, we see a downward trend in terms of adoption of all testbench methodologies and class libraries with the exception of UVM, which has increased by 486 percent since the fall of 2010. The study participants were also asked what they plan to use within the next 12 months, and based on the responses, UVM is projected to increase an additional 46 percent.

Figure 1. Methodology and class library trends

Figure 2 show the adoption of testbench methodologies and class libraries for FPGA designs (in red). We do not have sufficient data to show prior adoption trends in the FPGA space, but we anticipate that our future studies will enable us to do this. However, we did ask the FPGA study participants which testbench methodologies and class libraries they were planning to adopt within the next 12 months. Based on these responses, we anticipate that UVM adoption will increase by 40 percent, and OVM increase by 24 percent in the FPGA space.

Figure 2. Methodology and class library trends

Assertion Languages and Libraries

Finally, let’s examine assertion language and library adoption for IC/ASIC designs. The Wilson Research Group study found that 63 percent of all the IC/ASIC participants have adopted assertion-based verification (ABV) as part of their verification strategy. The data presented in this section shows the assertion language and library adoption trend related to those participants who have adopted ABV.

Figure 3 shows the trends in terms of assertion language and library adoption by comparing the 2010 Wilson Research Group study (in blue), the 2012 Wilson Research Group study (in green), and the projected adoption trends within the next 12 months (in purple). The adoption of SVA continues to increase, while other assertion languages and libraries either remain flat or decline.

Figure 3. Assertion language and library adoption for Non-FPGA designs

Figure 4 shows the adoption of assertion language trends for FPGA designs (in red). Again, we do not have sufficient data to show prior adoption trends in the FPGA space, but we anticipate that our future studies will enable us to do this. We did ask the FPGA study participants which assertion languages and libraries they planned to adopt within the next 12 months. Based on these responses, we anticipate an increase in adoption for OVL, SVA, and PSL in the FPGA space within the next 12 months.

Figure 4. Trends in assertion language and library adoption for FPGA designs

In my next blog (click here), I plan to focus on the adoption of various verification technologies and techniques used in the industry, as identified by the 2012 Wilson Research Group study.

, , , , , , , , , ,

5 August, 2013

Language and Library Trends

This blog is a continuation of a series of blogs that present the highlights from the 2012 Wilson Research Group Functional Verification Study (for a background on the study, click here).

In my previous blog (Part 7 click here), I focused on some of the 2012 Wilson Research Group findings related to testbench characteristics and simulation strategies. In this blog, I present design and verification language trends, as identified by the Wilson Research Group study.

You might note that for some of the language and library data I present, the percentage sums to more than one hundred percent. The reason for this is that some participants’ projects use multiple languages.

RTL Design Languages

Let’s begin by examining the languages used for RTL design. Figure 1 shows the trends in terms of languages used for design, by comparing the 2007 Far West Research study (in gray), the 2010 Wilson Research Group study (in blue), the 2012 Wilson Research Group study (in green), as well as the projected design language adoption trends within the next twelve months (in purple) as identified by the study participants. Note that the design language adoption is declining for most of the languages with the exception of SystemVerilog whose adoption continues to increase.

Also, it’s important to note that this study focused on languages used for RTL design. We have conducted a few informal studies related to languages used for architectural modeling—and it’s not too big of a surprise that we see increased adoption of C/C++ and SystemC in that space. However, since those studies have (thus far) been informal and not as rigorously executed as the Wilson Research Group study, I have decided to withhold that data until a more formal blind study can be executed related to architectural modeling and virtual prototyping.

Figure 1. Trends in languages used for Non-FPGA design

Let’s now look at the languages used specifically for FPGA RTL design. Figure 2 shows the trends in terms of languages used for FPGA design, by comparing the 2012 Wilson Research Group study (in red) with the projected design language adoption trends within the next twelve months (in purple).

Figure 2. Languages used for Non-FPGA design

It’s not too big of a surprise that VHDL is the predominant language used for FPGA RTL design, although we are starting to see increased interest in SystemVerilog.

Verification Languages

Next, let’s look at the languages used to verify Non-FPGA designs (that is, languages used to create simulation testbenches). Figure 3 shows the trends in terms of languages used to create simulation testbenches by comparing the 2007 Far West Research study (in gray), the 2010 Wilson Research Group study (in blue), and the 2012 Wilson Research Group study (in green).

Figure 3. Trends in languages used in verification to create Non-FPGA simulation testbenches

The study revealed that verification language adoption is declining for most of the languages with the exception of SystemVerilog whose adoption is increasing. In fact, SystemVerilog adoption increased by 8.3 percent between 2010 and 2012.

Figure 4 provides a different analysis of the data by partitioning the projects by design size, and then calculating the adoption of SystemVerilog for creating testbenches by size. The design size partitions are represented as: less than 5M gates, 5M to 20M gates, and greater than 20M gates. Obviously, we find that the larger the design size, the greater the adoption of SystemVerilog for creating testbenches. Yet, probably the most interesting observation we can make from examining Figure 4 is related to smaller designs that are less than 5M gates. Here we see that 58.8 percent of the industry has adopted SystemVerilog for verification. In other words, it is safe to say that SystemVerilog for verification has become mainstream today and not just limited to early adopters or leading-edge design projects.

Figure 4. SystemVerilog (for verification) adoption by design size

Let’s now look at the languages used specifically for FPGA RTL design. Figure 5 shows the trends in terms of languages used for FPGA design, by comparing the 2012 Wilson Research Group study (in red) with the projected design language adoption trends within the next twelve months (in purple).

Figure 5. Trends in languages used in verification to create FPGA simulation testbenches

In my next blog (click here), I’ll continue the discussion on design and verification language trends as revealed by the 2012 Wilson Research Group Functional Verification Study.

, , , , , , , , , , , , , , , , , ,

29 July, 2013

Testbench Characteristics and Simulation Strategies

This blog is a continuation of a series of blogs that present the highlights from the 2012 Wilson Research Group Functional Verification Study (for background on the study, click here).

In my previous blog (click here), I focused on the controversial topic of effort spent in verification. In this blog, I focus on some of the 2012 Wilson Research Group findings related to testbench characteristics and simulation strategies. Although I am shifting the focus away from verification effort, I believe that the data I present in this blog is related to my previous blog and really needs to be considered when calculating effort.

Time Spent in full-chip versus Subsystem-Level Simulation

Let’s begin by looking at Figure 1, which shows the percentage of time (on average) that a project spends in full-chip or SoC integration-level verification versus subsystem and IP block-level verification. The mean time performing full chip verification is represented by the dark green bar, while the mean time performing subsystem verification is represented by the light green bar. Keep in mind that this graph represents the industry average. Some projects spend more time in full-chip verification, while other projects spend less time.

Figure 1. Mean time spent in full chip versus subsystem simulation

Number of Tests Created to Verify the Design in Simulation

Next, let’s look at Figure 2, which shows the number of tests various projects create to verify their designs using simulation. The graph represents the findings from the 2007 Far West Research study (in gray), the 2010 Wilson Research Group study (in blue), and the 2012 Wilson Research Group study (in green). Note that the curves look remarkably similar over the past five years. The median number of tests created to verify the design is within the range of (>200 – 500) tests. It is interesting to see a sharp percentage increase in the number of participants who claimed that fewer tests (1 – 100) were created to verify a design in 2012. It’s hard to determine exactly why this was the case—perhaps it is due to the increased use of constrained random (which I will talk about shortly). Or perhaps there has been an increased use of legacy tests. The study was not design to go deeper into this issue and try to uncover the root cause. This is something I intend to informally study this next year through discussions with various industry thought leaders.

Figure 2. Number of tests created to verify a design in simulation

Percentage of Directed Tests versus Constrained-Random Tests

Now let’s compare the percentage of directed testing that is performed on a project to the percentage of constrained-random testing. Of course, in reality there is a wide range in the amount of directed and constrained-random testing that is actually performed on various projects. For example, some projects spend all of their time doing directed testing, while other projects combine techniques and spend part of their time doing directed testing—and the other part doing constrained-random. For our comparison, we will look at the industry average, as shown in Figure 3. The average percentage of tests that were directed is represented by the dark green bar, while the average percentage of tests that are constrained-random is represented by the light green bar.

Figure 3. Mean directed versus constrained-random testing performed on a project

Notice how the percentage mix of directed versus constrained-random testing has changed over the past two years.Today we see that, on average, a project performs more constrained-random simulation. In fact, between 2010 and 2012 there has been a 39 percent increase in the use of constrained-random simulation on a project. One driving force behind this increase has been the maturing and acceptance of both the SystemVerilog and UVM standards—since two standards facilitate an easier implementation of a constrained-random testbench. In addition, today we find that an entire ecosystem has emerged around both the SystemVerilog and UVM standards. This ecosystem consists of tools, verification IP, and industry expertise, such as consulting and training.

Nonetheless, even with the increased adoption of constrained-random simulation on a project, you will find that constrained-random simulation is generally only performed at the IP block or subsystem level. For the full SoC level simulation, directed testing and processor-driven verification are the prominent simulation-based techniques in use today.

Simulation Regression Time

Now let’s look at the time that various projects spend in a simulation regression. Figure 4 shows the trends in terms of simulation regression time by comparing the 2007 Far West Research study (in gray) with the 2010 Wilson Research Group study (in blue), and the 2012 Wilson Research Group study (in green). There really hasn’t been a significant change in the time spent in a simulation regression within the past three years. You will find that some teams spend days or even weeks in a regression. Yet today, the industry median is between 8 and 16 hours, and for many projects, there has been a decrease in regression time over the past few years. Of course, this is another example of where deeper analysis is required to truly understand what is going on. To begin with, these questions should probably be refined to better understand simulation times related to IP versus SoC integration-level regressions. We will likely do that in future studies—with the understanding that we will not be able to show trends (or at least not initially).

Figure 4. Simulation regression time trends

In my next blog (click here), I’ll focus on design and verification language trends, as identified by the 2012 Wilson Research Group study.

, , , , , , , , , ,

@dennisbrophy tweets

Follow dennisbrophy

@dave_59 tweets

Follow dave_59

@jhupcey tweets

  • #ARM now hiring formal verification engineers in Austin: exciting tech challenge + Ram is a great guy to work with.…https://t.co/uwIXLHWqvg
  • Attention all SF Bay Area formal practitioners: next week Wednesday 7/26 on Mentor's Fremont campus the Verificatio…https://t.co/9Y0iFXJdYi
  • This is a very hands-on, creative role for a verification expert -- join us! https://t.co/jXWFGxGrpn

Follow jhupcey