Not so long ago, most systems had defined usage cycles and a deterministic set of finite inputs and outputs. Human intervention was then earmarked for situations that fell outside these categories or for situations that were too ambiguous to program. For these systems, a few end-to-end tests were sufficient because most of the complexity was delegated to the operator.Â
Today, the requirements for system verification and validation have significantly changed. Many systems are now operating in the real world without human intelligence to fall back on. For example, functionality such as cruise control and automatic lane keeping are now in every new vehicle. Technologies like these show significant promise and are already changing the world for the better. However, fatal accidents such as the one in 2018 involving a Tesla and a truck when the car’s perception system in autopilot mode failed to detect a white truck with a cloudy sky background showed that we still have a long way to go before widespread acceptance and adoption of autonomous vehicles becomes a reality. In fact, according to a 2019 AAA survey, a worrying 71% of people are afraid to drive in fully driverless vehicles. Therefore, the stakes for system level verification and validation have never been higher than they are today.
kilometers of test driving are required to demonstrate a failure rate significantly better than humans -Â Rand Corporation
System verification and validation can be a complex task, especially in regulated environments. Systems such as autonomous vehicles are constantly exposed to non-linear inputs and quickly changing conditions including weather, people, bicycles, projectiles, etc. Engineers can't simulate all of this in thousands or even millions of runs. To minimize the chances of catastrophic failure and maximize chances of meeting the quality standards and expectations of all stakeholders - including the regulators, engineers need to have:
Collimator is the only tool that allows you to import big data directly via API, model test cases using Python or a graphical UI, validate system performance in real-time, and create reports to share with your team so you’re all on the same page - literally!
.png){kind=small}
.png){kind=small}
.png){kind=small}
.png){kind=small}
.png){kind=small}
