New vtg documents structure

TGs/verification-task-group/README.md

@@ -1,10 +1,15 @@
-# Verification Documentation
-The sub-directories below this point contain documentation related to verification of the CORE-V family of RISC-V cores.
-* **CV32E40P**: verification documentation specific to the CV32E40P core
-* **MeetingMinutes**: Minutes of various Verification Task Group meetings
-* **MeetingPresentations**: Slides presented at various VTG meetings
-* **GeneralPresentations**: non-VTG-specifc slides
+# Verification-Task-Group management
+The sub-directories below this point contain information about the projects that are coordinated within the OpenHW Verification Task Group (VTG).
+
+* **archive**: where older docs and slides and notes live
+* **documents**: current documents
+* **ecosystem**: information on some discussions with members of the OpenHW verification ecosystem
+* **meetings**: VTG monthly meetings records/reports/slides/docs - by year, and meeting
+* **projects**: Sub-project monthly meetings records/reports/slides/docs - by year, and meeting
 
 ## Looking for the Verification Strategy?
 The CORE-V verification strategy is an overview of the strategy used to verify CORE-V cores.  This documentation is not specific to any one core.
 It has been moved to [core-v-verif](https://github.com/openhwgroup/core-v-verif.git).
+
+## Looking for the verification planning information?
+The CORE-V verification planning document can be found: [VerificationPlanning101.md](projects/core-v-verif/documents/VerificationPlanning101.md).

TGs/verification-task-group/meetings/2022/2022-Oct-17/2022-Oct-17-VTG-Monthly-Meeting-agenda.txt

@@ -0,0 +1,19 @@
+OpenHW: VTG Meeting 17th Oct
+----------------------------
+Intro by new chair
+5 min update for each project from leaders of verification for each core undergoing verification:
+	CV32E40X
+	CV32E40S
+	CV32E40Pv2
+	CVA6
+	CV32E20
+update on core-v-verif projects: Mike
+Introduction to first 3 Advanced RISC-V Verification Methodologies projects
+	ARVM-FunctionalCoverage
+		developing open-source VIPs (such as functional coverage) that can be used for many different core configurations/implementations
+	ARVM-Standards
+		defining and implementing evolving interface standards for test bench components to enable better test bench component reuse and potentially stimulate availability of compatible VIPs
+	ARVM-TestbenchQuality
+		developing quality measurement of test benches - so quality of cores can be predicted (for example defining fault models and tests and relating those to TRL levels)
+
+#

TGs/verification-task-group/projects/ARVM-Advanced-RISCV-Verification-Methodologies-TopProject/meetings/2022/2022-Aug10-ARVM-Advanced-riscv-Verification-Methodologies-OpenHW-Project-Concept-Proposal.(PASSED).md

@@ -0,0 +1,138 @@
+# OpenHW Project Concept Proposal:
+# Advanced RISC-V Verification Methodology (ARVM)
+## Presented at OpenHW TWG 2022-July-25, reviewed 10-Aug-2022, and Approved 06-Sep-2022
+
+## Summary of Project
+This project is to be a project under the VTG.
+
+This project aims to enhance the capabilities and efficiency of the RISC-V verification available to all RISC-V core developers and thereby improve the quality of the available RISC-V cores and reduce the risk of RISC-V market fragmentation, disarray, and slow growth.
+
+## Summary of market
+Today, many RISC-V processor implementations use a relatively simple subset of the RISC-V ISA. However, more advanced features and capabilities are brought forward as implementations, supporting features such as Out-of-Order, multi-issue, multi-core, multi-hart, multi-thread, and new ISA extensions such as vectors.
+
+To support this evolution in processor capabilities, processor verification tools, technology, VIPs and methodologies must continually improve and be adopted for delivery of quality RISC-V processors.
+
+By working on and succeeding with this ARVM project, OpenHW can lead the industry in evolving best-in-class RISC-V processor DV solutions and impact the entire RISC-V community.
+
+### Goals of Project
+The project will address RISC-V processor verification with an initial sub-project list of:
+
+- **ARVM-Methodologies**: improving the capabilities and available methodologies available for verification environments
+- **ARVM-Promotion**: educating / informing processor verification teams of choices and techniques available across the RISCV community and verification ecosystem (for example tutorials and videos)
+- **ARVM-TestbenchQuality**: developing quality measurement of test benches - so quality of cores can be predicted (for example defining fault models and tests and relating those to TRL levels)
+- **ARVM-Standards**: defining and implementing evolving interface standards for test bench components to enable better test bench component reuse and potentially stimulate availability of compatible VIPs
+- **ARVM-FunctionalCoverage**: developing open-source VIPs (such as functional coverage) that can be used for many different core configurations/implementations
+- **ARVM-SoCIntegration** : consider requirements and solutions for SoC core integration verification (for example cache coherency with uncore components)
+- **ARVM-Roadmap**: ongoing roadmap to accommodate new innovations in RISC-V designs, new ratified extensions, and new tool developments
+
+To be clear - this project / group is focused on advancing high quality industrial strength verification and is not targeting specific OpenHW core implementations / teams / groups - but it is expected that members of the various OpenHW core verification teams will participate actively in, and benefit from, this project.
+
+## Project Organization
+This project and its sub-projects will be tracked as follows in OpenHW:
+- ARVM (Advanced riscv Verification Methodologies) will be the "carrier project" within VTG that goes through the Project Concept gate in the OpenHW TWG to setup the basic idea and structure
+- Each of the sub-projects as listed above (or others to be determined) will traverse the Project Launch, Plan Approve, and Project Freeze gates in the OpenHW TWG under the guidance of the ARVM leaders
+- ARVM will work with OpenHW staff to develop the Project Launch, Plan Approve and Project Freeze criteria for the sub-projects as suits their nature
+
+## Who would make use of the developments in this project
+This project is focused on advancing high quality industrial strength verification and is not targeting specific OpenHW core implementations / teams / groups - but it is expected that members of the various OpenHW core verification teams will participate actively in, and benefit from, this project. 
+
+The outputs from this project range from education & methodology, to VIPs and interface standards - all of which will enhance DV teams capabilities and thereby drive RISC-V to more success.
+
+All 'internal' OpenHW core verification teams can participate and benefit and so can external core verification teams, including commercial partners. Nothing will be done which will be specific to OpenHW cores - the focus is ecosystem wide.
+
+OpenHW is a collaborative community / organization and for a sub-project to become an approved ARVM project an OpenHW member must propose it and provide resources to work on the sub-project.
+
+There is no requirement that a sub-project is directly related to an existing OpenHW core / project.
+
+### Other verification projects (not this project, but part of other VTG projects)
+Here are some suggestions about other projects that could be done in VTG if / when member companies come forward to develop project and resource plans for them (but note - these are not currently part of this ARVM project planning):
+- Continued promotion of SV/UVM testbenches for CORE-V cores and SoCs
+- Reduction of the integration effort of new cores into core-v-verif
+- Build on the core-v-mcu-uvm project to develop an SoC verification methodology
+- Review and update of Strategic Goals for core-v-verif
+- Standing up a multi-simulator automatic CI flow for all supported CORE-V cores
+- Creation of an open-source formal verification testbench for at least one CORE-V core
+- Refactoring of CORE-V-VERIF into per-core verification repos
+- Make ISACOV standalone VIP, able to connect to any Instruction Fetch Bus Agent
+- Improved processes for creating, writing and tracking DVplans
+- Toolchain independence
+- Update riscv-dv to make it a true UVM component (and integrate it into core-v-verif)
+- Update the verification coding style guidelines
+- Lint checking to create automated checks for the above guidelines
+
+However these may become part of the ARVM sub projects as appropriate.
+
+## Initial Estimate of Timeline
+ARVM is an ongoing carrier project. The following are initial steps:
+
+August / September 2022
+- Refine and enhance the sub-projects list, develop the sub-projects focus
+- Encourage participation in ARVM sub-projects both among existing OpenHW members and new prospective members
+- for ARVM sub-projects setup recurring meetings on the OpenHW Calendar, chaired by the ARVM Technical Project Leader (named)
+- for ARVM sub-projects setup MatterMost channels
+- for ARVM sub-projects setup GitHub repos
+
+October 2022 - onwards - get on with it
+
+December 2022 - target for an initial progress update at the RISC-V Summit
+
+Provide regular feedback on progress to VTG, TWG etc...
+
+## Explanation of why OpenHW should do this project
+As an open-source industry forum already focusing on industry quality verification of CORE-V processor cores, publishing results in open-source while making use of best available tools, it is natural for the OpenHW Group to foster continued development of verification methodologies specific to that application domain. While the ARVM project would increase the scope of verification methodology within OpenHW to consider requirements beyond those of the CORE-V Processor Cores IP, the increased verification methodology focus will be beneficial for CORE-V Cores IP and for OpenHW members.
+
+## Industry landscape: description of competing, alternative, or related efforts in the industry
+RISC-V has created opportunities for new entrants to develop their own processors.  Few of these new entrance have experience with processor verification.
+
+As there is no industry accepted flow for processor verification - every core verification team is forced to invent & develop their own verification ideas, components, tools, simulators, models, interfaces, frameworks, flows, etc. This is inefficient, costly, error prone, and is unnecessary.
+
+RISC-V International is focused on the ISA and the standardization of the ISA. Yes RVI has a compliance group - but this is solely focused on ISA compliance (architecture) and not, repeat not, on processor implementation verification (micro-architecture).
+
+CHIPS Alliance is focused on open-source EDA tooling, for example Verilator, and RISCV-DV - yes components as part of verification. 
+
+This new OpenHW ARVM group is needed and focused on industrial grade processor verification.
+
+## OpenHW Members/Participants committed to participate
+- Imperas 
+- Codasip (name to be provided)
+- Other EDA tool companies under discussion to join OpenHW
+- Current and future OpenHW members actively verifying one or more CORE-V projects
+
+All OpenHW core projects with verification targeting TRL5 should be participating.
+
+## Project Leader(s)
+### Technical Project Leader(s)
+Simon Davidmann (Imperas)
+
+Each sub-group will have a 'leader' - tbd.
+
+## Resource Requirements
+Imperas will provide leadership for the ARVM carrier project as well as contribute to the engineering resources for the sub-projects
+
+Other members will participate and contribute in the new project and sub-projects
+
+Engineering resources are mainly needed to help drive the sub-projects
+
+## Project license model
+All deliverables including documents and code will be open-source on https://github.com/openhwgroup/ and developed under Apache 2.1 and/or Solderpad 2.1 licenses
+
+## Description of initial code contribution, if required
+It is not currently thought that ARVM will be part of the Eclipse CORE-V Cores project. If required any initial code contributions will follow OpenHW/EF contribution questionnaire rules.
+
+## Repository Requirements
+Github repos for specs, project documents, milestones, reports etc.
+
+## Project distribution model
+Code Releases will be made available on http://downloads.openhwgroup.org/ under an ARVM heading.
+
+Output documentation will be made available on ReadTheDocs at https://docs.openhwgroup.org/projects/
+
+## Preliminary Project plan
+as above
+
+## Risk Register
+Main risk for the carrier project initially is lack of participation and interest during the refinement of the sub-project list. To succeed, ARVM requires a considerable critical mass of expertise on verification. 
+
+To manage this, the ARVM project requires "lobbying" for participation by the current OpenHW processor companies and organizations. 
+
+

TGs/verification-task-group/projects/ARVM-FunctionalCoverage/meetings/2022/monthly-report-2022-Oct-17.md

@@ -0,0 +1,43 @@
+
+[comment]: # "this template is for ARVM projects"
+
+# **ARVM-FunctionalCoverage** Monthly Report for 17-October-2022
+
+Project leader : Simon Davidmann   
+Lead company :  Imperas  
+Participating companies : SiLabs, Dolphin  
+
+## Overview
+There are almost 1,000 instructions in RV64 (inc. all ratified and soon to be ratified extensions).  
+For each instruction somebody will need to write SystemVerilog covergroups and coverpoints…   
+Maybe 10-40 lines of SystemVerilog for each instruction…   
+That is 10,000-40,000 lines of SystemVerilog code to be written… (and be correct and working…)  
+And that is just for the basic un-privilege mode ISA…   
+
+This sub-project is to enable the developing of open-source VIPs (such as functional coverage) that can be used for many different core configurations/implementations.  
+
+## Current Status
+cv32e40p used first generation of SystemVerilog for RV32I generated by Imperas in 2020.  
+Second generation architecture is generated and works directly from RVVI-TRACE core tracer testbench interface.   
+Uses machine readable ISA definition and generates examples as compliance level functional coverage for RV32I, RV32M.   
+Currently soliciting input / requirements on what needs to be covered in verification plans for different ISA extensions.   
+Initial focus is F (FPU) functional coverage for cv32e40pv2 (Dolphin) and Zc for cv32e40s (SiLabs).   
+
+## Key activities / tasks completed this month
+- Topic presented (Oct. 4th) at DVClub Europe/India webinar: https://www.tessolve.com/automated-verification-checks/
+- F (FPU): First SystemVerilog functional coverage code for FPU has been generated and is under review at Dolphin
+- Zc (code-size-reduction): has been coded up into machine readable form and is soon to be generated  
+- existing generated code is under review in SiLabs 
+- output of code generator is sections for verification plan in .csv - currently being reviewed  
+
+## Planned activities / tasks for coming month
+- reviews of resultant functional coverage (RV32I, RV32M, F, Zc)   
+- addition of more requirements for F and Zc
+- addition of more design verification (DV) coverage such as hazards (and micro-architectural)   
+- scheduling of regular sub-project progress meetings (currently they are ad-hoc)  
+- discussions regarding working with isacov
+
+## Issues / items that are slowing progress
+- none - just starting up...
+
+

TGs/verification-task-group/projects/ARVM-Standards/meetings/2022/monthly-report-2022-Oct-17.md

@@ -0,0 +1,32 @@
+
+[comment]: # "this template is for ARVM projects"
+
+# **ARVM-Standards** Monthly Report for 17-October-2022
+
+Project leader : Simon Davidmann   
+Lead company :  Imperas  
+Participating companies : OpenHW, SiLabs, Dolphin, NXP, Intrinsix 
+
+## Overview
+Defining and implementing evolving interface standards for test bench components to enable better test bench component reuse and potentially stimulate availability of compatible VIPs.  
+Easing the adoption of interface to core tracer and test bench (RVVI-TRACE).  
+Easing the adoption interface of test bench to Verification IP that includes the reference model (RVVI-API).  
+Enabling the development of other interfaces, e.g. RVVI-VVP Virtual Verification Peripherals.   
+
+## Current Status
+RVVI-TRACE and RVVI-API are in use in cv32e40x core-v-verif testbench to allow use of Imperas async-lock-step-compare VIPs.  
+Existing test benches use ad-hoc virtual peripherals.  
+
+## Key activities / tasks completed this month
+- OpenHW and Imperas migrated cv32e40x test bench from ad-hoc to RVVI-TRACE and RVVI-API
+- with this integration made several requests for enhancements to RVVI
+
+## Planned activities / tasks for coming month
+- port cv32e40s core-v-verif test bench from ad-hoc to use of RVVI
+- arrange OpenHW discussions on RVVI-VVP
+- discussions with RISC-V International compliance group on RVVI-VVP (Virtual Peripherals) (collaborative paper accepted for RISC-V summit in Dec.)
+
+## Issues / items that are slowing progress
+- none - just starting up...
+
+

TGs/verification-task-group/projects/ARVM-TestbenchQuality/meetings/2022/monthly-report-2022-Oct-17.md

@@ -0,0 +1,31 @@
+
+[comment]: # "this template is for ARVM projects"
+
+# **ARVM-TestbenchQuality** Monthly Report for 17-October-2022
+
+Project leader : Simon Davidmann   
+Lead company :  Imperas  
+Participating companies : Codasip
+
+## Overview
+Developing quality measurement of test benches - so quality of cores can be predicted (for example defining fault models and tests and relating those to TRL levels)
+
+## Current Status
+Currently have TRL   
+early days - just starting to collect information
+
+## Key activities / tasks completed this month
+- reviewed TRL slides (https://docs.google.com/presentation/d/1XrhbHpFRYtAiSvAMPhf7b5oIIrpJKBIv/edit#slide=id.p1)
+
+
+## Planned activities / tasks for coming month
+- arrange first OpenHW discussions
+
+## Issues / items that are slowing progress
+- none - just starting up...
+
+## Other information
+- 2021 12min YouTube Video on ["Better Quality RTL"](https://www.youtube.com/watch?v=wwSEIEfxysc) from Philippe Luc of Codasip 
+
+
+

TGs/verification-task-group/projects/CV32E20/meetings/2022/VTG monthly-report-2022-Oct-12.md

@@ -0,0 +1,40 @@
+
+[comment]: # "this template is for core verification projects"
+
+# **CV32E20 (CVE2)** Monthly Report for 17-October-2022
+
+## Overview
+Targeted core : CV32E20  
+Verification project leader :   Shared responsibility Maarten Arts / Lee Hoff  
+Lead company : NXP / Intrinsix  
+Target date verification complete (RTL Freeze) :  2023-Q1  
+Target verification quality level (TRL 1-5) :    TRL5  
+Verification approach being used (self check, compare signature, compare trace file, lock-step-compare, other) :   step & Compare  
+Reference model used (Imperas, spike, spike-modified, qemu, qemu-modified, other) :   Imperas  
+Test Generator used (riscv-dv, Valtrix, force-riscv, other) :  riscv-dv(?)  
+Formal approach (Jasper, Questa formal, Onespin, other) : not determined, but Jasper as preferred tool  
+
+## Current Status
+Core revision version being tested : 0.1  
+Core specification (link to pdf) :   https://ibex-core.readthedocs.io/en/latest/ (but needs to be ported to CV32E20)  
+Verification plan / specification completeness (%) :  0%  
+Test bench (link GitHub) :   Not Available yet  
+Functional coverage code created completeness (%) :  0%  
+Formal / simulation assertions written completeness (%) :   0%  
+
+## Key activities / tasks completed this month
+- Stood up "Hello World in core-v-verif"  
+
+## Planned activities / tasks for coming month
+- tbd  
+
+## Issues / items that are slowing progress
+- tbd    
+
+## Risks
+- to project timescales    
+	-- tbd   
+- to project quality    
+	-- tbd  
+
+#