Posts Tagged ‘Verification Academy’

21 January, 2015

This blog is a continuation of a series of blogs that present the highlights from the 2014 Wilson Research Group Functional Verification Study (for a background on the study, click here).

In this blog I discuss the issue of study bias, and what we did to address these concerns.

MINIMIZING STUDY BIAS

When architecting a study, three main concerns must be addressed to ensure valid results: sample validity bias, non-response bias, and stakeholder bias. Each of these concerns is discussed in the following sections, as well as the steps we took to minimize these bias concerns.

Sample Validity Bias

To ensure that a study is unbiased, it’s critical that every member of a studied population have an equal chance of participating. An example of a biased study would be when a technical conference surveys its participants. The data might raise some interesting questions, but unfortunately, it does not represent members of the population that were unable to participant in the conference. The same bias can occur if a journal or online publication limits its surveys to only its subscribers.

A classic example of sample validity bias is the famous Literary Digest poll in the 1936 United States presidential election, where the magazine surveyed over two million people. This was a huge study for this period in time. The sampling frame of the study was chosen from the magazine’s subscriber list, phone books, and car registrations. However, the problem with this approach was that the study did not represent the actual voter population since it was a luxury to have a subscription to a magazine, or a phone, or a car during The Great Depression. As a result of this biased sample, the poll inaccurately predicted that Republican Alf Landon versus the Democrat Franklin Roosevelt would win the 1936 presidential election.

For our study, we carefully chose a broad set of independent lists that, when combined, represented all regions of the world and all electronic design market segments. We reviewed the participant results in terms of market segments to ensure no segment or region representation was inadvertently excluded or under-represented.

Non-Response Bias

Non-response bias in a study occurs when a randomly sampled individual cannot be contacted or refuses to participate in a survey. For example, spam and unsolicited mail filters remove an individual from the possibility of receiving an invitation to participate in a study, which can bias results. It is important to validate sufficient responses occurred across all lists that make up the sample frame. Hence, we reviewed the final results to ensure that no single list of respondents that made up the sample frame dominated the final results.

Another potential non-response bias is due to lack of language translation, which we learned during our 2012 study. The 2012 study generally had good representation from all regions of the world, with the exception of an initially very poor level of participation from Japan. To solve this problem, we took two actions:

  1. We translated both the invitation and the survey into Japanese.
  2. We acquired additional engineering lists directly from Japan to augment our existing survey invitation list.

This resulted in a balanced representation from Japan. Based on that experience, we took the same approach to solve the language problem for the 2014 study.

Stakeholder Bias

Stakeholder bias occurs when someone who has a vested interest in survey results can complete an online study survey multiple times and urge others to complete the survey in order to influence the results. To address this problem, a special code was generated for each study participation invitation that was sent out. The code could only be used once to fill out the survey questions, preventing someone from taking the study multiple times or sharing the invitation with someone else.

2010 Study Bias

While architecting the 2012 study, we did discover a non-response bias associated with the 2010 study. Although multiple lists across multiple market segments and across multiple regions of the world were used during the 2010 study, we discovered that a single list dominated the responses, which consisted of participants who worked on more advanced projects and whose functional verification processes tend to be mature. Hence, for this series of blogs we have decided not to publish any of the 2010 results as part of verification technology adoption trend analysis.

The 2007, 2012, and 2014 studies were well balance and did not exhibit the non-response bias previously described for the 2010 data. Hence, we have confidence in talking about general industry trends presented in this series of blogs.

Quick links to the 2014 Wilson Research Group Study results (so far…)

,

21 January, 2015

This is the first in a series of blogs that presents the findings from our new 2014 Wilson Research Group Functional Verification Study. However, unlike my previous Wilson Research Group functional verification study blogs, which focused on the ASIC/IC market, I plan to begin this set of blogs with an exclusive focus on FPGA trends. Why? For the following reasons:

  1. Unlike the traditional ASIC/IC market, there has historically been very few studies published on FPGA functional verification trends. We started studying the FPGA market segment back in the 2010 study, and we now have collected sufficient data to confidently present industry trends related to this market segment.
  2. Today’s FPGA designs have grown in complexity—and many now resemble complete systems. The task of verifying SoC-class designs is daunting, which has forced many FPGA projects to mature their verification process due to rising complexity. The FPGA-focused data I present in this set of blogs will support this claim.

My plan is to release the ASIC/IC functional verification trends through a set of blogs after I finish presenting the FPGA trends.

Introduction

In 2002 and 2004, Collett International Research, Inc. conducted its well-known ASIC/IC functional verification studies, which provided invaluable insight into the state of the electronic industry and its trends in design and verification at that point in time. However, after the 2004 study, no additional Collett studies were conducted, which left a void in identifying industry trends. To address this dearth of knowledge, four studies were commissioned by Mentor Graphics in 2007, 2010, 2012, and 2014, which focused on functional verification. These were world-wide, double-blind, functional verification studies, covering all electronic industry market segments. To our knowledge, the 2014 study was the largest functional verification study ever conducted. This set of blogs presents the findings from our 2014 study and provides invaluable insight into the state of the electronic industry today in terms of both design and verification trends.

Study Background

Our study was modeled after the original 2002 and 2004 Collett International Research, Inc. studies. In other words, we endeavored to preserve the original wording of the Collett questions whenever possible to facilitate trend analysis. To ensure anonymity, we commissioned Wilson Research Group to execute our study. The purpose of preserving anonymity was to prevent biasing the participants’ responses. Furthermore, to ensure that our study would be executed as a double-blind study, the compilation and analysis of the results did not take into account the identity of the participants.

For the purpose of our study we used a multiple sampling frame approach that was constructed from eight independent lists that we acquired. This enabled us to cover all regions of the world—as well as cover all relevant electronic industry market segments. It is important to note that we decided not to include our own account team’s customer list in the sampling frame. This was done in a deliberate attempt to prevent biasing the final results. My next blog in this series will discuss other potential bias concerns when conducting a large industry study and describe what we did to address these concerns.

After data cleaning the results to remove inconsistent or random responses (e.g., someone who only answered “a” on all questions), the final sample size consisted of 1886 eligible participants (i.e., n=1886). To put this figure in perspective, the 2004 Collett study sample size consisted of 201 eligible participants.

Unlike the 2002 and 2004 Collett IC/ASIC functional verification studies, which focused only on the ASIC/IC market segment, our studies were expanded in 2010 to include the FPGA market segment. We have partitioned the analysis of these two different market segments separately, to provide a clear focus on each. One other difference between our studies and the Collett studies is that our study covered all regions of the world, while the original Collett studies were conducted only in North America (US and Canada). We have the ability to compile the results both globally and regionally, but for the purpose of this set of blogs I am presenting only the globally compiled results.

Confidence Interval

All surveys are subject to sampling errors. To quantify this error in probabilistic terms, we calculate a confidence interval. For example, we determined the overall margin of error for our study to be ±2.19% at a 95% confidence interval. In other words, this confidence interval tells us that if we were to take repeated samples of size n=1886 from a population, 95% of the samples would fall inside our margin of error ±2.19%, and only 5% of the samples would fall outside.

Study Participants

This section provides background on the makeup of the study.

Figure 1 shows the percentage of overall study participants by market segment.

2014-WRG-BLOG-P-1

Figure 1: Study participants by market segment

Figure 2 shows the percentage of overall study eligible participants by their job description. An example of eligible participant would be a self-identified design or verification engineer, or engineering manager, who is actively working within the electronics industry. Overall, design and verification engineers accounted for 60 percent of the study participants.

2014-WRG-BLOG-P-2

Figure 2: Study participants job title description

Before I start presenting the findings from our 2014 functional verification study, I plan to discuss in my next blog (click here) general bias concerns associated with all survey-based studies—and what we did to minimize these concerns.

Quick links to the 2014 Wilson Research Group Study results (so far…)

,

14 January, 2015

“Who Knew?” about verification IP (VIP), was the theme of a recent DeepChip post by John Cooley on December 18.  More specifically the article states, “Who knew VIP was big and that Wally had a good piece of it?”  We knew.

We knew that ASIC and FPGA design engineers can choose to buy design IP from several alternative sources or build their own, but that does not help with the problem of verification.  We knew that you don’t really want to rely on the same source that designed your IP, to test it.  We knew that you don’t want to write and maintain bus functional models (BFMs) or more complete VIP for standard protocols.  Not that you couldn’t, but why would you if you don’t have to?

We also knew that verification teams want easy-to-use VIP that is built on a standard foundation of SystemVerilog, compliant with a protocol’s specification, and is easily configurable to your implementation.  That way it integrates into your verification environment just as easily as if you had built it yourself.

Leading design IP providers such as ARM®, PLDA, and Northwest Logic knew that Mentor Graphics’ VIP is built on standards, is protocol compliant, and is easy to use.  In fact you can read more about what Jim Wallace, systems and software group director at ARM; Stephane Hauradou, CTO of PLDA; and Brian Daellenbach, president of Northwest Logic; have to say about Mentor Graphics’ recently introduced EZ-VIP technology for PCIe 4.0 (at this website http://www.mentor.com/company/news/mentor-verification-ip-pcie-4 ), and why they know that their customers can rely on it as well.

Verification engineers knew, too.  You can read comments from many of them (at Cooley’s website http://www.deepchip.com/items/dac14-06.html ), about their opinions on VIP.  In addition, Mercury Systems also knew.  “Mentor Graphics PCIe VIP is fully compliant with the PCIe protocol specification and with UVM coding guidelines. We found that we could drop it into our existing environment and get it up and running very quickly”, said Nick Solimini, Consulting DV Engineer at Mercury Systems. “Mentor’s support for their VIP is excellent. All our technical questions were answered promptly so we were able to be productive throughout the project”.

So, now you know,  Mentor Graphics’ Questa VIP is built on standard SV UVM, is specification compliant, is easy to get up and running and is an integral part of many successful verification environments today.  If you’d like to learn more about Questa VIP and Mentor Graphics’ EZ-VIP technology, send me an email, and I’ll let you in on what (thanks to Cooley and our customers) is no longer the best kept secret in verification.  Who knew?

, , , , , , , ,

24 November, 2014

SystemVerilog Testbench Debug – Are we having fun yet?

Fun

Debug should be fun. Watching waveforms march by, seeing ERRORS and WARNINGS pop out in a transcript file, tracing drivers back to their source, understanding race conditions between simulators and between source code changes – and my favorite – debugging random stability issues. Fun.

Old School – logfiles and interactive

Or at least it should be fun. It used to be fun. I’d setup my collection of scripts to run tests and examine logfiles. Push the button and go for coffee or go home. The next day I’d examine log files and figure out what happened. Usually I’d have to jump into interactive simulation and debug on the fly. Set some breakpoints and watch what happened. That was then. My tests and RTL were all Verilog. Life was good. I was in control of what was going on, and could get my head around it.

New School – logfiles, interactive and class handles

Fast-forward to today. Still have scripts to run tests. Still have log files. Still push the button and get coffee or go home. Still jump into interactive simulation. Still set breakpoints. But now my tests are SystemVerilog class-based – usually UVM. My tests are C code. My tests are constrained random tests. Debug just got harder. I can’t fit the whole testbench + RTL into my head at once. I need help.

Debugging your class based testbench

I prefer to do as much debug as possible in “post-sim” mode. I want to run simulation and capture as much as possible. Then debug my wavefile and source code. What to do about my SystemVerilog class based testbench? Easy. Capture my classes in the wave database. Show them to me in the wave window.

<UVM Testbench class hierarchy window and those same classes in the wave window>

Wave Window

Wave Window

But that’s not possible. Is it? What IS possible?

What? Objects in the wave database? Yes. Objects and their members in the wave database.

Examine the values of class member variables in post-sim mode. Use the waveform window for classes and class member variables just like signals.

What about the handles that are in my classes? Can I chase them to other objects? Yes. Follow class handle “pointers” to other objects – essentially exploring the OBJECT SPACE that existed at THAT time during simulation. But I’m in post sim!

Can I see all the sequence items that hit my driver? Yes. How? Just put the driver “handle” into the wave window and “open” it. You can see the virtual interface handle (if you have one). You can see the transactions that went through the driver (the driver did a ‘get_next_item (t)’ 100,000 times!).

<Transaction handle ‘t’ from the driver in the wave window, with the driver’s virtual interface>

Driver and 't' in Wave Window

Driver and ‘t’ in Wave Window

In the wave window? Yes. All 100,000 of them? Yes.

Now I’m having fun again. That’s great. I can see what’s going on inside my objects. In post-sim mode.

What’s NOT possible?

Will it babysit? No. One thing at a time.

Are you having fun yet?

Find more details in Verification Horizons article – Old School vs. New School – Visualizer and on Verification Academy – Verification and Debug: Old School Meets New School 

You can find all the sessions on New School verification techniques via the following link:

https://verificationacademy.com/seminars/academy-live

, , , , , ,

11 September, 2014

From those just beginning to study electronic systems design to the practicing engineer, this is the time of the year when those taking their first steps to learn VHDL, Verilog/SystemVerilog join the academic “back to school” crowd and those who are using design & verification languages in practice are honing skills at industry events around the world.

A new academic year has started and the Mentor Higher Education Program (HEP) is well set to help students at more than 1200 colleges and universities secure access to the same commercial tools and technology used by industry.  It is a real win-win when students learn using the same tools they will use after graduating.  Early exposure and use means better skilled and productive engineers for employers.

The functional verification team at Mentor Graphics knows that many students would prefer to have a local copy of ModelSim on their personal computer to do their course work and smaller projects as they learn VHDL or Verilog.  To help facilitate that we make the ModelSim PE Student Edition available for download without charge.  More than 10,000 students use ModelSim PE Student Edition around the world now in addition to our commercial grade tools they can access in their university labs.

For the practicing engineer, the Verification Academy offers an online community of more than 25,000 design and verification engineers that exchange ideas on a wide variety issues across the numerous standards and methodologies.  If you are not a member of the Verification Academy, I recommend you join.  You will also find the Verification Academy at DAC for one-on-one discussions and even more recently Verification Academy Live daylong seminars which came to Austin and which will be in Santa Clara – as of the writing of this blog.  There is still time to register for the Santa Clara event and I invite you to attend.

As design and verification is global, Accellera realized that DVCon should explore the needs of the global design and verification engineer population as well.  For 2014, DVCon Europe and DVCon India were born from an already successful running SystemC User Group events.  These user-led conferences will be held so engineers in these areas can more easily come together to share experiences and knowledge to ultimately become more productive.

Students and practicing engineers alike can benefit from fee-free access to some of the popular IEEE EDA standards.   While I don’t think reading them alone is the ultimate way to educate yourself, they make great companions to daily design and verification activities.  Accellera has worked with the IEEE to place several EDA standards in the IEEE Standards Association’s “Get™” program.  Almost 16,000 copies of the SystemC standard (1666) and just about the same number of SystemVerilog standards (1800) have been downloaded as of the end of August 2014.  Have you download your free copies yet?

The chart below shows the distribution of nearly 45,000 downloads which have occurred since 2010.  Stay tuned for breaking news on some updates to the EDA standards in the Get program.  When updated, they will replace the versions available now.  So if you want to have the current versions and the ones to come out shortly, you better download your copies now.  If the electronic version is not sufficient for you, the IEEE continues to sell printed versions.

image

From students to practicing engineers, the season of learning has started.  I encourage you to find your right venue or style of learning and connect with others to advance and improve your design and verification productivity.

, , , , , , , , , ,

9 July, 2014

Accellera has announced the completion of a multi-year effort to update its latest edition of the Universal Verification Methodology (UVM).  In completing this effort, the UVM 1.2 Class Reference Document was approved as an Accellera standard and the UVM Working Group has supplied an accompanying open-source reference implementation.  Questa supports UVM 1.2.

In addition to the resources you can download from Accellera, additional information on UVM 1.2 can be found at the Verification AcademyHTML documentation can easily be found at the Verification Academy too.

If you are a user of UVM 1.1 and have not been part of the UVM 1.2 development effort, you should know your peers have been busy the past few years since the stabilization and completion of UVM 1.1 to drive global adoption of UVM and to add, enhance and extend UVM.  In UVM 1.2 Messaging is now object-oriented, Sequences can automatically raise and drop objects, the register layer can now control transaction order within bursts and numerous bugs in UVM 1.1 have been fixed to improve quality.

Backward Incompatibility

All these changes come with a cost to the current UVM 1.1 user community.  When Accellera announced UVM 1.2 availability, it also disclosed some of the new features introduce backward incompatibility.  To reduce those issues, Accellera is making release notes and a one way conversion script part of the UVM 1.2 kit to ease the migration path forward.

If you follow the Verification Academy Cookbook rules, you will probably not see any impact from the backward compatibility issues.  And if you control your total verification environment, you will probably find it simpler to migrate forward as well.  Those who depend on outside resources will need to make sure those resources (like Verification IP) migrate forward to UVM 1.2 so you can migrate forward to UVM 1.2.  Mixing UVM 1.1 and UVM 1.2 was not considered by the Accellera UVM Working Group and is fraught with unknown issues.  We consider the migration an all or nothing proposition.  If you have multi-division, multi-company projects underway, it would be prudent to plan you move to UVM 1.2 with care at the conclusion of projects and when all suppliers and participating teams can migrate to UVM 1.2.

Public Review Period

Accellera seeks your input and feedback on UVM 1.2.  To support this, a public review forum on the Accellera website has been established to allow users to catalog issues, ask questions and generally offer feedback to help improve UVM 1.2 quality.

The public review process will end on October 1, 2014.  We encourage users to take the time now to test UVM 1.2 in their own environments and share their feedback to expidite the migration to UVM 1.2.

Path to IEEE

Public feedback will be taken into account along with further Accellera member testing to update UVM 1.2 prior to a committed hand-off to the IEEE for further standardization there later this year.  As this path unfolds, I will share updates on the standardization effort in the IEEE.

Verification Academy DAC 2014 UVM 1.2 Presentation

You will find many resources around the world on UVM 1.2.  At DAC 2014, the Verification Academy booth sponsored a session on UVM 1.2 titled  “UVM: What’s New, What’s Next, and Why You Care.”  If you did not attend DAC, you can still download the presentation and watch a video replay of it if you are a Verification Academy “full access” member (free registration required; restrictions apply).

The presentation by Tom Fitzpatrick goes into detail on the UVM 1.2 topic.  Importantly in Tom’s presentation is a discussion about what you should care about today.  You may find that software is a big issue and that his thesis challenges one to ask if UVM 1.2 is stuck in the past rather than addressing what should be addressed next.  I invite you to download the presentation and watch the video and share with me your thoughts. What do you think?

, , , , ,

3 March, 2014

DVCon is always one of my favorite events in our industry, and I am proud to let you know that the latest issue of Verification Horizons is available “hot off the presses” at the Verification Academy to mark the occasion. For those of you attending the conference, please consider this issue as an addendum to the great technical program being offered (especially paper 8.1, “Of Camels and Committees: Standards Should Enable Innovation, Not Strangle It” by Dave Rich and yours truly). For those of you not able to join us at DVCon this year, consider this your consolation prize.

Although fewer in number, I’m sure you’ll find the articles in Verification Horizons as informational and useful as any you’ll see at DVCon. In particular, I’d like to make sure you check out these articles by our partners:

  • “Don’t Forget the Little Things That Can Make Verification Easier” by our friend Stu Sutherland of Sutherland HDL
  • “Taming Power-Aware Bugs with Questa Ultra” by SmartPlay Technologies
  • “Using Mentor Questa for pre-silicon validation of IEEE 1149.1-2013 based Silicon Instruments” by Intellitech
  • “Dealing With UVM and OVM Sequences” by eInfochips

If you’re at DVCon, please make sure to stop by the Mentor Graphics booth (#501) to say hi. Please join us on Wednesday for our luncheon presentation at noon, right after Session 8, in which I’ll present my paper mentioned above (that’s right. I’m not above shameless self-promotion). And we’ll wrap up the week with two Mentor-sponsored tutorials on Thursday:

Both of these tutorials feature a mix of Mentor presenters and customers to offer some practical examples that will give you some new ideas for improving your verification process. I hope to see you at DVCon.

, , ,

30 October, 2013

MENTOR GRAPHICS AT ARM TECHCON

This week ARM® TechCon® 2013 is being held at the Santa Clara Convention Center from Tuesday October 29 through Thursday October 31st, but don’t worry, there’s nothing to be scared about.  The theme is “Where Intelligence Counts”, and in fact as a platinum sponsor of the event, Mentor Graphics is excited to present no less than ten technical and training sessions about using intelligent technology to design and verify ARM-based designs.

My personal favorite is scheduled for Halloween Day at 1:30pm, where I’ll tell you about a trick that Altera used to shave several months off their schedule, while verifying the functionality and performance of an ARM AXI™ fabric interconnect subsystem.  And the real treat is that they achieved first silicon success as well.  In keeping with the event’s theme, they used something called “intelligent” testbench automation.

And whether you’re designing multi-core designs with AXI fabrics, wireless designs with AMBA® 4 ACE™ extensions, or even enterprise computing systems with ARM’s latest AMBA® 5 CHI™ architecture, these sessions show you how to take advantage of the very latest simulation and formal technology to verify SoC connectivity, ensure correct interconnect functional operation, and even analyze on-chip network performance.

On Tuesday at 10:30am, Gordon Allan described how an intelligent performance analysis solution can leverage the power of an SQL database to analyze and verify interconnect performance in ways that traditional verification techniques cannot.  He showed a wide range of dynamic visual representations produced by SoC regressions that can be quickly and easily manipulated by engineers to verify performance to avoid expensive overdesign.

Right after Gordon’s session, Ping Yeung discussed using intelligent formal verification to automate SoC connectivity, overcoming observability and controllability challenges faced by simulation-only solutions.  Formal verification can examine all possible scenarios exhaustively, verifying on-chip bus connectivity, pin multiplexing of constrained interfaces, connectivity of clock and reset signals, as well as power control and scan test signal connectivity.

On Wednesday, Mark Peryer shows how to verify AMBA interconnect performance using intelligent database analysis and intelligent testbench automation for traffic scenario generation.  These techniques enable automatic testbench instrumentation for configurable ARM-based interconnect subsystems, as well as highly-efficient dense, medium, sparse, and varied bus traffic generation that covers even the most difficult to achieve corner-case conditions.

And finally also on Halloween, Andy Meyer offers an intelligent workshop for those that are designing high performance systems with hierarchical and distributed caches, using either ARM’s AMBA 5 CHI architecture or ARM’s AMBA 4 ACE architecture.  He’ll cover topics including how caching works, how to improve caching performance, and how to verify cache coherency.

For more information about these sessions, be sure to visit the ARM TechCon program website.  Or if you miss any of them, and would like to learn about how this intelligent technology can help you verify your ARM designs, don’t be afraid to email me at mark_olen@mentor.com.   Happy Halloween!

, , , , , , , , , ,

19 August, 2013

Verification Techniques & Technologies Adoption Trends

This blog is a continuation of a series of blogs that present the highlights from the 2012 Wilson Research Group Functional Verification Study (for background on the study, click here).

In my previous blog (Part 9 click here), I focused on some of the 2012 Wilson Research Group findings related to design and verification language and library trends. In this blog, I present verification techniques and technologies adoption trends, as identified by the 2012 Wilson Research Group study.

An interesting trend we are starting to see is that the electronic industry is maturing its functional verification processes, whether they are targeting their designs at IC/ASIC or FPGA implementations. This blog provides data to support this claim. An interesting question you might ask is, “What is driving this trend?” In some of my earlier blogs (click here for Part 1 and Part 2) I showed an that design complexity is increasing in terms design sizes and number of embedded processors. In addition, I’ve presented trend data that showed an increase in total project time and effort spent in verification (click here for Part 5 and Part 6). My belief is that the industry is being forced to mature its functional verification processes to address increasing complexity and effort.

Simulation Techniques Adoption Trends

Let’s begin by comparing  non-FPGA adoption trends related to various simulation techniques from the 2007 Far West Research study  (in blue) with the 2012 Wilson Research Group study  (in green), as shown in Figure 1.

Figure 1. Simulation-based technique adoption trends for non-FPGA designs

You can see that the study finds the industry increasing its adoption of various functional verification techniques for non-FPGA targeted designs. Clearly the industry is maturing its processes as I previously claimed.

For example, in 2007, the Far West Research Group found that only 48 percent of the industry performed code coverage. This surprised me. After all, HDL-based code coverage is a technology that has been around since the early 1990’s. However, I did informally verify the 2007 results through numerous customer visits and discussions. In 2012, we see that the industry adoption of code coverage has increased to 70 percent.

In 2007, the Far West Research Group study found that 37 percent of the industry had adopted assertions for use in simulation. In 2012, we find that industry adoption of assertions had increased to 63 percent. I believe that the maturing of the various assertion language standards has contributed to this increased adoption.

In 2007, the Far West Research Group study found that 40 percent of the industry had adopted functional coverage for use in simulation. In 2010, the industry adoption of functional coverage had increased to 66 percent. Part of this increase in functional coverage adoption has been driven by the increased adoption of constrained-random simulation, since you really can’t effectively do constrained-random simulation without doing functional coverage.

Now let’s look at  FPGA adoption trends related to various simulation techniques from the 2010 Far West Research study  (in pink) with the 2012 Wilson Research Group study  (in red).

Figure 2. Simulation-based technique adoption trends for non-FPGA designs

Again, you can clearly see that the industry is increasing its adoption of various functional verification techniques for FPGA targeted designs. This past year I have spent a significant amount of time in discussions with FPGA project managers around the world. During these discussions, most mangers mention the drive to improve verification process within their projects due to  rising complexity of this class of designs. The Wilson Research Group data supports these claims.

In fact, Figure 3 illustrates this maturing trend in the FPGA space, where we saw a 15 percent increase in the adoption of RTL simulation and an 8.5 percent increase in the adoption of code coverage. For complex FPGA designs, the traditional approach of “burn and churn” and debug in the lab is no longer a viable option. Nonetheless, it is still somewhat alarming that 31 percent of the FPGA study participants work on projects that perform no RTL simulation.

Figure 3. FPGA projects maturing their verification processes

Signoff Criteria Trends

We saw earlier in this blog the increased adoption of coverage techniques in the industry. Coverage has become a major component of a project’s verification signoff criteria. In Figure 4, we see how coverage has increased in importance in verification signoff criteria within the past five years, while other decision attributes have declined in terms of importance.

Figure 4. Non-FPGA functional verification signoff criteria trends

We see the same trends for FPGA designs, as shown in Figure 5.

Figure 5. FPGA functional verification signoff criteria trends

In my next blog (click here), I plan to continue the discussion related to adoption of various verification technologies and techniques as identified by the 2012 Wilson Research Group study.

, , , , , , , , , ,

5 August, 2013

Language and Library Trends

This blog is a continuation of a series of blogs that present the highlights from the 2012 Wilson Research Group Functional Verification Study (for a background on the study, click here).

In my previous blog (Part 7 click here), I focused on some of the 2012 Wilson Research Group findings related to testbench characteristics and simulation strategies. In this blog, I present design and verification language trends, as identified by the Wilson Research Group study.

You might note that for some of the language and library data I present, the percentage sums to more than one hundred percent. The reason for this is that some participants’ projects use multiple languages.

RTL Design Languages

Let’s begin by examining the languages used for RTL design. Figure 1 shows the trends in terms of languages used for design, by comparing the 2007 Far West Research study (in gray), the 2010 Wilson Research Group study (in blue), the 2012 Wilson Research Group study (in green), as well as the projected design language adoption trends within the next twelve months (in purple) as identified by the study participants. Note that the design language adoption is declining for most of the languages with the exception of SystemVerilog whose adoption continues to increase.

Also, it’s important to note that this study focused on languages used for RTL design. We have conducted a few informal studies related to languages used for architectural modeling—and it’s not too big of a surprise that we see increased adoption of C/C++ and SystemC in that space. However, since those studies have (thus far) been informal and not as rigorously executed as the Wilson Research Group study, I have decided to withhold that data until a more formal blind study can be executed related to architectural modeling and virtual prototyping.

Figure 1. Trends in languages used for Non-FPGA design

Let’s now look at the languages used specifically for FPGA RTL design. Figure 2 shows the trends in terms of languages used for FPGA design, by comparing the 2012 Wilson Research Group study (in red) with the projected design language adoption trends within the next twelve months (in purple).

Figure 2. Languages used for Non-FPGA design

It’s not too big of a surprise that VHDL is the predominant language used for FPGA RTL design, although we are starting to see increased interest in SystemVerilog.

Verification Languages

Next, let’s look at the languages used to verify Non-FPGA designs (that is, languages used to create simulation testbenches). Figure 3 shows the trends in terms of languages used to create simulation testbenches by comparing the 2007 Far West Research study (in gray), the 2010 Wilson Research Group study (in blue), and the 2012 Wilson Research Group study (in green).

Figure 3. Trends in languages used in verification to create Non-FPGA simulation testbenches

The study revealed that verification language adoption is declining for most of the languages with the exception of SystemVerilog whose adoption is increasing. In fact, SystemVerilog adoption increased by 8.3 percent between 2010 and 2012.

Figure 4 provides a different analysis of the data by partitioning the projects by design size, and then calculating the adoption of SystemVerilog for creating testbenches by size. The design size partitions are represented as: less than 5M gates, 5M to 20M gates, and greater than 20M gates. Obviously, we find that the larger the design size, the greater the adoption of SystemVerilog for creating testbenches. Yet, probably the most interesting observation we can make from examining Figure 4 is related to smaller designs that are less than 5M gates. Here we see that 58.8 percent of the industry has adopted SystemVerilog for verification. In other words, it is safe to say that SystemVerilog for verification has become mainstream today and not just limited to early adopters or leading-edge design projects.

Figure 4. SystemVerilog (for verification) adoption by design size

Let’s now look at the languages used specifically for FPGA RTL design. Figure 5 shows the trends in terms of languages used for FPGA design, by comparing the 2012 Wilson Research Group study (in red) with the projected design language adoption trends within the next twelve months (in purple).

Figure 5. Trends in languages used in verification to create FPGA simulation testbenches

In my next blog (click here), I’ll continue the discussion on design and verification language trends as revealed by the 2012 Wilson Research Group Functional Verification Study.

, , , , , , , , , , , , , , , , , ,

@dennisbrophy Tweets

  • Loading tweets...

@dave_59 Tweets

  • Loading tweets...

@jhupcey Tweets

  • Loading tweets...