Cooking at Home or Eating Out? - The Pros and Cons of Homegrown VIP

By now, we're all pretty much convinced that reuse is essential in the semiconductor industry. Gone are the days when we built everything from scratch an re-invented the wheel on each project. Today, to build a new SoC we stitch together multiple blocks that we've already used on previous projects, maybe tweaking them a little, while only designing those new things that will help our product differentiate itself from the competition.

The same principle also holds for the testbenches we develop for those blocks and SoCs. It makes sense, right? If we have a block with an AHB interface, do we really need to write a verification component that can drive the AHB protocol for each new iteration of that block? What if we have multiple blocks that communicate through AHB? Why couldn't the same verification component be used within all of those testbenches? The industry recognized this and some time ago new languages appeared that borrowed from general purpose programming languages to make it easier to develop reusable verification intellectual property (VIP).

These days, we have two choices when considering VIP. We can either buy commercial products from established vendors or we can take charge and implement these ourselves. There are many ideas scattered throughout the net on the make or buy dilemma as it applies to VIP. Most of them were written by VIP vendors and really, what are they going to say? I don't claim any authority on the topic with this post, but I want to share my view as both a homegrown VIP developer and subsequent user.

A little while back our team decided to go the way of the warrior and develop our own in-house UVCs for the AMBA protocols we use. This was by no means anything new within the company. We already had a considerable portfolio of eVCs and other UVCs (of varying quality) for various protocols our chips employ, both proprietary and standard. I was lucky enough to have just finished a project around that time and to get a chance to be involved in the development. Now, more than a year later, I've started to use some of them on my current project. This is a good time to reflect on the pros and cons of homegrown VIP.

The most obvious advantage to building versus buying is the financial one. The cost model I've seen up to now for VIP involves subscription fees for licenses. This is money that can be saved. We do have to pay upfront by having to spend engineering time developing the VIP, though. This can lead to somewhat of a ski rental problem because we might not really be able to say how high this initial cost will be, but for simple protocols I'd argue this isn't such a big deal. For example, I'd say that a UVC for a simple protocol like APB could be developed in about one man-week and one for something more complicated like AHB maybe in something of the order of one man-month. Protocols of the complexity order of AXI and beyond are a whole different matter...

While money is an important factor, that wasn't the reason we went down the route of writing our own UVCs. The main one was stability. I think a lot of us have seen the case where VIP from vendor X doesn't work with the simulator from vendor Y. We know that the SystemVerilog standard isn't specific enough and open to interpretation in some places and different simulators can treat a construct in different ways. At the same time, the degree to which different vendors have implemented the standard also varies, with some constructs being supported in one tool, but not in another. Having the source code under our control means that we can tackle such issues, so switching simulator vendors won't impact us so dramatically anymore. But having write access to the source code can be both a blessing and a curse...

We have to admit that more often than not, when starting something new we get very excited and want to jump directly into our text editors and start writing some code. This "code first, think later" mentality can only hurt us because instead of focusing on quality from the beginning we might just say "we can weed out the bugs later". Bugs in a UVC can be disastrous as they can lead to design bugs escaping the net and making it onto silicon. This isn't to say that commercial VIP is 100% bug free (as I've seen my fair share), but vendors are more likely to have good quality control procedures set up, meaning that bugs become rare. Unit and integration testing are the tools we can use to mitigate such problems. If you haven't already I invite you to have a look at SVUnit. I've also written about it in this post. If there's anything that really, really, really needs be properly tested, it's our UVCs!

Another blessing when we have access to the source code of our UVCs (and not jus read-only) is extendability. It's much easier to accommodate deviations from the standard protocol when we can build these directly into the UVC and not have to bolt them onto a standard UVC. How we do this, however, should depend on whether or not we need to implement the standard protocol too.

The curse begins to manifest itself when we would like to have our UVC do some cool thing that we only need in our current project, but that no one else needs. Since we have the source code under our command, we might think that it won't do any harm to add new capabilities to the UVC. They may be useful to others someday too! It's also easier that using OOP concepts like inheritance and/or composition, so the temptation is just too big. What we get with such feature creep, though, is a tangled mess of configuration switches, extra interface signals that don't exist in the specification and spaghetti code for our UVC components. What we need is a clear vision of what is of general interest and what is not. We can use our power over the code for good, by adding hooks to it to facilitate extendability, but not to implement everything on a whim.

Homegrown UVCs will also tend to have ad-hoc maintenance practices. This is also coupled with lax release processes, with no guidelines for deprecating old features and adding new ones. My biggest pet peeve is source code management, where everything will get dumped into the main branch, instead of making use of branches and labels (or whatever the terminology of your SCM is) and other tools that are provided to us. The main question here is "do we release tar files with specific versions or do we point to the repository where the UVC development is done?". This, in my opinion, is the root of all evil. In the latter case, users will get itchy fingers and want to change code to fit their needs. As we saw above, even when done with the best intentions, this isn't a good idea. I tend to avoid this even for fixes, even if I do have access to the source code and the development repository. If I need to tweak the behavior of the UVC I prefer to create extended classes in my verification environment rather than change something in the UVC itself. These can always be incorporated later or be released as a standalone package for others to use.

Last but not least, since documentation is always a second class citizen, it will just fall behind (I plead guilty here!). Even if we spend time to do the documentation properly and to keep it up-to-date, people still won't read it if they have someone to ask in person. To prove this, how many times have you asked your simulator's application engineer about something instead of reading up on it in the manual?

Lately, open source UVCs have started appearing. The guys over at AMIQ Consulting have already released three VIP packages: Ethernet, APB and DCR. I haven't looked at them, so I won't comment, but the company does have an EDA arm hence I'm guessing they understand development concepts much better than I do (and they could probably even add to the points above). I'm hoping to see many more such such releases in the future, from them and from others. I mentioned open source UVCs here since they have the potential of saving us the initial development cost associated with building our own, while still providing all of the advantages. They also, however, carry the same temptations. Remember, having write access to the source code is power and, as we know already, power corrupts!

In conclusion, having homegrown VIP can be a really good thing, as long as we invest the time to do it right. The list ended up having more cons than pros, but these can be mitigated. We just need to plan changes appropriately and not just commit every single thing into the UVC repository without thinking and especially not without testing. It might not seem worth it to spend the extra effort, but if we don't we'll end up using a lot of our time fire-fighting and we may have just been better off buying the VIP. We need to set responsibilities for the UVC's source code and stick with them. We need to know that we have two hats to wear, one as a customer and one as a developer, and we need to know when to wear each.