Description
The software supply chain is a vulnerable element of every application and affects all projects
in modern software development with regard to security. Software bills of materials (SBOMs)
have become an increasingly important tool for ensuring transparency and compliance in this
field. The current open-source landscape offers a wide variety of automated SBOM generation
tools for the Java Maven ecosystem. This study aims to analyze five of these tools to assess
their correctness and completeness: CycloneDX Generator (cdxgen), Trivy, Syft, jbom, and
GitHub’s Dependency Graph.
This evaluation is executed by using two main methodologies. In this study, the SBOMs
generated by all five tools are compared for a set of 100 randomly chosen repositories.
Additionally, the components and dependencies for ten repositories of limited size are
examined manually and then compared to the ones declared in the SBOMs generated with
CycloneDx Generator, Trivy, and Syft. This dual approach allows for the formulation of both
quantitative and qualitative statements, thereby offering a comprehensive perspective on
correctness as well as completeness.
The findings indicate that, with respect to the components, cdxgen exhibits considerable
strength, closely followed by the GitHub Dependency Graph. However, an analysis of the
transitive dependencies shows that Trivy demonstrates the most outstanding quantitative
outcomes. A qualitative analysis reveals that cdxgen exhibits superior performance in the scope
of the tested repositories, followed by Trivy and Syft.
The results of this study demonstrate that each of the tools evaluated in the domain of SBOM
generation exhibits specific strengths regarding different metrics. At this stage, none of these
tools can be regarded as the perfect and optimal approach. To guarantee a 100% secure software
supply chain, it has been demonstrated that further development of these tools is necessary.
However, it should be noted that the scope of this study does not encompass a comprehensive
evaluation of all available solutions. Nevertheless, for this scope, cdxgen and Trivy can be
regarded as the most promising stable tools.
|
