Security Fractals and the Open Source Supply Chain – SecAdvent Day 4

Open source won.

We aren’t going to talk about if using open source is a good idea. You already have open source everywhere. If you think you don’t have any open source in your organization or project, you’re almost certainly wrong. You’d be wise to go check.

From a simplistic viewpoint, it appears that open source is free, but it’s not free, it’s not free at all. All the power of open source comes with a cost. The cost isn’t money, the cost is in trying to manage your open source. Open source isn’t one thing. Open source is a complex web of pieces that all fit together to solve problems. You could have a few open source components in a project or you could have thousands. We refer to this relationship as the open source supply chain.

What is a supply chain?

A supply chain isn’t a new idea. It’s from the world of manufacturing. In tech we love to think we invented all these new ideas and concepts. While tech has created many things, supply chains aren’t one of them. If you’ve studied any sort of business theory in the last fifty years, you probably heard of a person named William Edwards Deming. He’s most famous for being the creator of modern quality control processes. Studying his work would not be wasted time, but for the purpose of this particular post, the most important thing Deming taught us is the importance of understanding quality as part of the supply chain. You can’t use low quality parts to build a high quality product. Low quality parts result in low quality products. How we build software depends on the quality of our parts, or dependencies. If we use low quality open source to build something, by definition, it will be low quality.

Before we go on, it’s also important to understand the enormity of this open source supply chain problem. Below you will find a picture that shows the supply chain, or dependency graph, of the Signal Desktop application:

Signal is a secure messenger application. As far as many applications go, Signal is pretty simple yet it has a large and complicated dependency graph. It is expected that a dependency will have its own dependencies, which also have dependencies, it’s dependencies all the way down. This is exactly what we see in the Signal graph. Signal is not unique in how it uses dependencies. This is what the dependency graph looks like for most reasonably sized applications. Modern development lends itself to large complex supply chains.

You can’t use low quality parts to build a high quality product. Low quality parts result in low quality products. How we build software depends on the quality of our parts, or dependencies.

Now that we understand what an open source supply chain can look like, it’s important we understand the concept of “if you ship it, you support it”. One possible option for dealing with a complex supply chain is to decide that all the open source packages aren’t your problem and can just be ignored. This isn’t a reasonable position. The history of liability is clear that maintaining a supply chain is the responsibility of the product supplier, or application developer in our case. There is a great presentation that Wendy Knox Everette delivered at BSides Las Vegas about product liability. The supplier is responsible if a part from the supply chain fails. At the end of the day we are responsible for everything we ship, even if we didn’t write it. If you ship it, you support it.

It’s also pretty clear a human can’t possibly keep this problem under control manually. There’s no way we can reasonably keep track of thousands of open source packages shipped in our products. We probably can’t even keep track of a small number of open source packages. Humans make a lot of mistakes, have to sleep, and don’t scale very well. Humans are bad at this sort of work.

Things sound pretty dire right about now. Now that we see the problem what do we do to take control of the quality in our open source supply chain?

Not using open source isn’t an option. You were thinking about it for a second weren’t you.

By using open source you get to focus on the details that are most important to the goal you are trying to accomplish. Open source lets us stand on the shoulders of giants and see farther than we could by ourselves. Not using open source is the same as being stuck down in the mud. You can’t move fast and end up doing a lot of work you don’t want to.

The first challenge is finding which open source you should be using. Remember that bit about quality a few paragraphs ago? Not all open source is created equally. Some is managed by a healthy community, some is managed by one person who is grumpy and doesn’t like it when anyone files bugs. Picking out quality is a great job for humans actually. Humans can easily decide which open source libraries solve the problem they have as well as determining if the community is healthy. Remember open source isn’t free, picking high quality open source is part of the cost.

Once we have our high quality libraries, they need to be taken care of. Remember, humans can’t do this, it’s too big. You need to use automation to solve this.  Robots can be really good at this sort of work. A human can pick the library and integrate it, but after that, robots should be responsible for caring and feeding. If you have thousands of dependencies, it’s very likely one or more of them will be updated every day. Which of the updated dependencies are the most important? Which should you update first? Which dependencies will break your application? It’s not as simple as blindly updating everything.

How to find and pick a robot to help could span a very long series of blog posts, goodness knows this one is already long enough. The simplest advice is to find a partner, not a vendor. We are in the infancy of automating how to care and feed for our open source. There will be many challenges, opportunities, hard discussions, and tears. Vendors sell you stuff, partners help you on your journey. For most of us this is the start of a new and long journey, but luckily we all have the entire open source community to work with. It’s going to be epic!

If you want to learn more, I had a chat about all this with Alyssa Miller (who wrote yesterday’s SecAdvent post) a few months back on the Open Source Security Podcast: Episode 205 – The State of Open Source Security with Alyssa Miller from Snyk