Prologue: The 2010 Wilson Research Group Functional Verification Study
In 2002 and 2004, Ron Collett International, Inc. conducted its well known ASIC/IC functional verification studies, which provided invaluable insight into the electronic industry’s state and trends in design and verification at that point in time. However, after the 2004 study, no other industry studies were conducted, which left a void in identifying industry trends.
To address this void, Mentor Graphics commissioned Far West Research to conduct a new industry study on functional verification in the fall of 2007. The study was conducted as a blind study to avoid influencing the results. This means that the survey participants did not know that the study was commissioned by Mentor Graphics. In addition, to support trend analysis on the data, the survey followed the same format and questions that were asked in the 2002 and 2004 Collett studies.
In the fall of 2010, Mentor Graphics commissioned Wilson Research Group to conduct a new functional verification study. This study is a blind study and follows the same format as the Collett and Far West Research studies. The 2010 Wilson Research Group study is one of the largest functional verification studies ever conducted. It is about 3.5 times larger than the Collett studies, and twice as large as the Far West Research study. The overall confidence level of the study was calculated to be 95% with a margin of error of 4.1%.
Unlike the previous Collett and Far West Research studies that were conducted in North America only, the 2010 Wilson Research Group study is a worldwide study. The regions targeted are:
- North America: Canada, United States
- Europe/Israel: Finland, France, Germany, Israel, Italy, Sweden, UK
- Asia (minus India): China, Korea, Japan, Taiwan
The survey results are compiled both globally and regionally for analysis.
Another difference with this study is that it includes FPGA engineers. I decided when I started to process the study results that I would compile both with the combined FPGA and non-FPGA data (when appropriate) and also separately for analysis. Obviously, for trend analysis I can only show the non-FPGA (IC/ASIC) data since no previous study included FPGA participants.
When compiling and analyzing the data from the study, in addition to calculating the mean on various aspects of the data, I decided to calculate the median for trend analysis. My objective in calculating the median is that the resulting value partitions the data into equal halves, which at times is more insightful when discussing trends.
Figure 1 shows the percentage makeup of survey participants by the company type. The grey bars represents the FPGA participants while the green bars represents the non-FPGA (i.e., IC/ASIC) participants.
Figure 1: Survey participants company description
Figure 2 shows the percentage makeup of survey participants by their job description. Again, the grey bars represents the FPGA participants while the green bars represents the non-FPGA (i.e., IC/ASIC) participants.
Figure 2: Survey participants job title description
In a future set of blogs, I plan to present the highlights from the 2010 Wilson Research Group study along with my analysis, comments, and obviously, opinions. A few interesting observations emerged from the study, which include:
- Reuse adoption is increasing.
- The effort spent on verification is increasing.
- The industry is adopting more advanced functional verification techniques.
My next blog (click here) presents current design trends that were identified by the survey. This will be followed by a set of blogs focused on the functional verification results.
Quick links to the 2010 Wilson Research Group Study results (so far…)
- Part 1 – Design Trends
- Part 2 - Design Trends (Continued)
- Part 3 – Reuse
- Part 4 - Effort Spent In Verification
- Part 5 – Effort Spent In Verification (Continued)
- Part 6 - Testbench characteristics and Simulation Strategies
- Part 7 - Testbench characteristics and Simulation Strategies (continued)
- Part 8 – Language and Library Trends
- Part 9 – Verification Techniques and Technologies Adoption Trends
More to come!!!
Posted March 30th, 2011, by Harry Foster
- Loading tweets...
- Loading tweets...
- Loading tweets...
- Mentor Enterprise Verification Platform Debuts
- New Verification Academy ABV Course
- DVCon 2014 Issue of Verification Horizons Now Available
- DVCon–The FREE Side
- More DVCon–More Mentor Tutorials!
- UVM 1.2: Open Public Review
- DVCon 2014: Standards on Display
- Just because FPGAs are programmable doesn’t mean verification is dead
- Managing Verification Coverage Information
- Epilogue: The 2012 Wilson Research Group Functional Verification Study
- April 2014 (1)
- March 2014 (2)
- February 2014 (5)
- January 2014 (1)
- November 2013 (2)
- October 2013 (3)
- September 2013 (2)
- August 2013 (4)
- July 2013 (6)
- Part 7: The 2012 Wilson Research Group Functional Verification Study
- Walking in the Desert or Drinking from a Fire Hose?
- Part 6: The 2012 Wilson Research Group Functional Verification Study
- A Short Class on SystemVerilog Classes
- Part 5: The 2012 Wilson Research Group Functional Verification Study
- Part 4: The 2012 Wilson Research Group Functional Verification Study
- June 2013 (2)
- May 2013 (4)
- April 2013 (2)
- March 2013 (2)
- February 2013 (5)
- January 2013 (1)
- December 2012 (1)
- November 2012 (1)
- October 2012 (4)
- September 2012 (1)
- August 2012 (1)
- July 2012 (6)
- June 2012 (1)
- May 2012 (3)
- March 2012 (1)
- February 2012 (6)
- January 2012 (2)
- December 2011 (2)
- November 2011 (2)
- October 2011 (3)
- September 2011 (1)
- July 2011 (3)
- June 2011 (6)
- Intelligent Testbench Automation Delivers 10X to 100X Faster Functional Verification
- Part 9: The 2010 Wilson Research Group Functional Verification Study
- Verification Horizons DAC Issue Now Available Online
- Accellera & OSCI Unite
- The IEEE’s Most Popular EDA Standards
- UVM Register Kit Available for OVM 2.1.2
- May 2011 (2)
- April 2011 (7)
- User-2-User’s Functional Verification Track
- Part 7: The 2010 Wilson Research Group Functional Verification Study
- Part 6: The 2010 Wilson Research Group Functional Verification Study
- SystemC Day 2011 Videos Available Now
- Part 5: The 2010 Wilson Research Group Functional Verification Study
- Part 4: The 2010 Wilson Research Group Functional Verification Study
- Part 3: The 2010 Wilson Research Group Functional Verification Study
- March 2011 (5)
- February 2011 (4)
- January 2011 (1)
- December 2010 (2)
- October 2010 (3)
- September 2010 (4)
- August 2010 (1)
- July 2010 (3)
- June 2010 (9)
- The reports of OVM’s death are greatly exaggerated (with apologies to Mark Twain)
- New Verification Academy Advanced OVM (&UVM) Module
- OVM/UVM @DAC: The Dog That Didn’t Bark
- DAC: Day 1; An Ode to an Old Friend
- UVM: Joint Statement Issued by Mentor, Cadence & Synopsys
- Static Verification
- OVM/UVM at DAC 2010
- DAC Panel: Bridging Pre-Silicon Verification and Post-Silicon Validation
- Accellera’s DAC Breakfast & Panel Discussion
- May 2010 (9)
- Easier UVM Testbench Construction – UVM Sequence Layering
- North American SystemC User Group (NASCUG) Meeting at DAC
- An Extension to UVM: The UVM Container
- UVM Register Package 2.0 Available for Download
- Accellera’s OVM: Omnimodus Verification Methodology
- High-Level Design Validation and Test (HLDVT) 2010
- New OVM Sequence Layering Package – For Easier Tests
- OVM 2.0 Register Package Released
- OVM Extensions for Testbench Reuse
- April 2010 (6)
- SystemC Day Videos from DVCon Available Now
- On Committees and Motivations
- The Final Signatures (the meeting during the meeting)
- UVM Adoption: Go Native-UVM or use OVM Compatibility Kit?
- UVM-EA (Early Adopter) Starter Kit Available for Download
- Accellera Adopts OVM 2.1.1 for its Universal Verification Methodology (UVM)
- March 2010 (4)
- February 2010 (5)
- January 2010 (5)
- December 2009 (15)
- A Cliffhanger ABV Seminar, Jan 19, Santa Clara, CA
- Truth in Labeling: VMM2.0
- IEEE Std. 1800™-2009 (SystemVerilog) Ready for Purchase & Download
- December Verification Horizons Issue Out
- Evolution is a tinkerer
- It Is Better to Give than It Is to Receive
- Zombie Alert! (Can the CEDA DTC “User Voice” Be Heard When They Won’t Let You Listen)
- DVCon is Just Around the Corner
- The “Standards Corner” Becomes a Blog
- I Am Honored to Honor
- IEEE Standards Association Awards Ceremony
- ABV and being from Missouri…
- Time hogs, blogs, and evolving underdogs…
- Full House – and this is no gamble!
- Welcome to the Verification Horizons Blog!
- September 2009 (2)
- July 2009 (1)
- May 2009 (1)