Hardware designers evaluate AI differently than most software developers. Beyond code generation, they care about whether a system can understand architectural intent, navigate large RTL projects, respect design constraints, and accelerate verification workflows.
To evaluate these capabilities, ChipNexus partnered with Purdue University’s System-on-Chip Extension Technologies (SoCET) group to assess NEX across increasingly complex RTL design tasks, ranging from a 32-bit ALU to a dual-core cache-coherent RV32I processor. The evaluation involved multiple members of the SoCET program with varying levels of hardware design experience. The most complex design, estimated at roughly 100 hours of manual effort, was completed in approximately 2.5–3.5 hours using NEX.
Key Findings
- Up to 43× productivity improvement on complex RTL design tasks.
- NEX successfully handled projects containing 25–30 RTL files and ~30 assembly test programs.
- Productivity gains increased with design complexity.
- Results across multiple SoCET participants showed that experienced designers achieved the largest gains, suggesting NEX amplifies domain expertise.
- The evaluators identified specification adherence as one of NEX’s strongest differentiators.
Evaluation Methodology
The evaluation covered a progression of increasingly sophisticated hardware designs:
- 32-bit ALU
- Single-cycle RISC-V processor
- 5-stage pipelined RISC-V processor
- Dual-core cache-coherent RV32I processor
Manual effort estimates were derived from historical statistics and experience associated with Purdue University’s SoCET program. NEX measurements included implementation, debugging, verification, and testbench development workflows.
The purpose of the evaluation was not simply to determine whether NEX could generate RTL, but to assess how effectively it could operate inside realistic hardware projects where architectural constraints, debugging, and verification matter as much as code generation.
Results: Measured Designer Productivity Improvements
The bar plot reported below shows the measured productivity improvement relative to a human baseline across four representative RTL design tasks. The gains become more pronounced as design complexity increases, with experienced users achieving between 2.4× and 43× reductions in active design time.
A notable observation is the difference between users with and without prior RTL design experience. While both groups benefited substantially from NEX, experienced designers consistently achieved larger productivity gains. This suggests that NEX amplifies existing engineering expertise rather than replacing it; a theme that appeared repeatedly throughout the evaluation.
The measured improvements were not solely the result of faster code generation. Throughout the evaluation, NEX reduced repetitive engineering work across implementation, verification, debugging, and refactoring workflows. As projects became larger and more interconnected, these workflow efficiencies compounded, producing the largest gains on processor-scale designs.
The Most Complex Design: A Dual-Core Cache-Coherent Processor
The most demanding design in the evaluation was a dual-core cache-coherent RV32I processor consisting of approximately:
- 25–30 RTL files
- Roughly 30 assembly test programs
The challenge extended well beyond generating Verilog. The system needed to remain aligned with architectural requirements while preserving project-specific constraints across a multi-file hardware project.
One of the evaluator’s strongest observations was NEX’s ability to reason across an entire workspace rather than operating on individual files in isolation.
This made it feel less like a single-file code generator and more like a design agent operating across an actual hardware project.
By analyzing dependencies, naming conventions, and module relationships, NEX demonstrated awareness of the broader system context while working through implementation and debugging tasks.
More Important Than Speed: Respecting Architectural Constraints
While the productivity gains were significant, an evaluator repeatedly highlighted a different capability as the most important.
Many general-purpose coding assistants attempt to eliminate errors by modifying whatever code appears related to the problem. In hardware design, this behavior can be dangerous because architectural constraints often exist for specific reasons.
According to the evaluator:
A major positive observation was that when something failed, NEX did not blindly change constrained parts of the design just to make the error disappear.
Instead of optimizing solely for passing a test, NEX generally preserved the intended architecture and worked within the specified design constraints.
This distinction is particularly important in larger RTL projects where maintaining correctness often matters more than producing a quick fix.
How NEX Accelerated Development
Several workflow patterns emerged throughout the evaluation.
Refactoring and Automation
During large-scale renaming and refactoring tasks, NEX recognized repetitive changes and generated automation scripts rather than performing edits one file at a time.
Instead of manually editing many files one by one, NEX recognized the repetitive pattern and wrote a Python script. This showed strong awareness of when automation was more efficient than direct editing.
Testbench Generation
NEX also substantially reduced verification effort.
The evaluators could either provide a desired testbench structure or simply describe testing intent in natural language. NEX then generated testbenches and additional variations designed to exercise corner cases and alternative scenarios.
Iterative Debugging
When issues occurred, NEX was able to identify failures, propose fixes, rerun simulations, and continue iterating toward a solution.
An evaluator observed that for larger tasks NEX would often continue working through multiple debugging stages until the requested objective had been reached.
Effective Prompting Matters
The evaluation also revealed that successful outcomes depend heavily on how engineers interact with the system.
One of the most effective workflows involved asking NEX to clarify requirements before implementation began.
The best workflow I found was to begin by asking NEX to ask me questions. This helped clarify the design intent before implementation began.
Once architectural goals and constraints were clearly established, implementation quality improved substantially.
An evaluator also noted:
The key lesson is that lazy prompting can still get to a result, but it often creates more work later.
In other words, NEX can compensate for incomplete instructions to some degree, but engineering intent remains critical.
Limitations and Areas for Improvement
Evaluators also highlighted several areas where the workflow could be improved.
Several challenges emerged:
- Long-running tasks sometimes lacked sufficient progress visibility.
- Vague prompts occasionally resulted in less structured code or suboptimal naming conventions.
- Users could become overly reliant on automatic debugging without fully understanding changes being made.
An evaluator specifically noted that debugging speed can become a double-edged sword:
Because it can move quickly, it is tempting for the user to repeatedly approve fixes without fully understanding each step.
The report suggests that more structured debugging summaries and clearer progress indicators would further improve usability.
Beyond Productivity: Onboarding and Education
An unexpected finding from the evaluation was NEX’s usefulness as a learning and onboarding tool.
Rather than acting purely as a code generator, the system frequently behaved more like a technical assistant capable of explaining concepts, design decisions, and unfamiliar codebases.
An evaluator described modern AI systems as:
Closer to personalized teaching assistants and workflow accelerators.
Because explanations can be adapted to different levels of expertise, NEX may help both experienced engineers and newcomers understand unfamiliar hardware designs more quickly.
This makes the technology valuable not only for implementation work but also for education, onboarding, and knowledge transfer.
The Human-AI Partnership
The evaluation repeatedly reinforces a broader lesson.
NEX did not eliminate the need for engineering judgment. Instead, it amplified the effectiveness of engineers who could clearly define architecture, constraints, and objectives.
When specifications were ambiguous, NEX often avoided making arbitrary decisions and instead requested clarification.
An evaluator summarized the experience this way:
This made the interaction feel more like working with an engineer or assistant than simply using an autocomplete tool.
The strongest results occurred when experienced engineers guided the process and used NEX to accelerate implementation and verification work.
Why This Matters
The largest productivity improvements were observed on the most complex designs. This suggests that AI assistance may be most valuable not for isolated coding tasks, but for navigating the growing scale and complexity of modern hardware projects.
Conclusion
Across designs ranging from a simple 32-bit ALU to a dual-core cache-coherent RISC-V processor, NEX consistently reduced implementation effort while preserving architectural intent.
The largest benchmark reduced a task estimated at roughly 100 hours of manual effort to as little as ~2.5–3.5 hours while still requiring engineers to guide architecture, validate results, and make design decisions. As shown in bar plot above, productivity improvements ranged from roughly 2× on small RTL blocks to more than 43× on complex processor designs for experienced users.
The evaluation suggests that the most important role for AI in hardware development is not replacing engineers. It is increasing the leverage of engineers while allowing them to spend more time on architecture, verification, and design decisions rather than repetitive implementation work.
