📋 Tender Overview
Vigilant Algorithmic Monitoring Of Software
When Software Supervises Software
At A Time Of Cloud Computing And Cyber-Physical Systems Like Computer-Assisted Driving, Software Complexity Is Growing Faster Than The Rate Of Improvement In Related Quality Assurance Techniques. The Erc-Funded Vamos Project Will Develop Monitoring Software To Identify Potential Vulnerabilities, Errors, And Unfair Decisions At Runtime. Specifically, The Project Will Develop A Quantitative Resource And Approximation Theory For Software Monitoring Which Allows Cost-Precision Trade-Offs. For Instance, Its Monitors Will Be Able To Track The Safety Of Systems As Well As The Fairness Of Algorithms, Security Policies, And Statistical Properties Of Software. Particular Focus Will Be On The Automated Monitoring Of Software That Is Difficult To Verify, Such As Computations Performed By Neural Networks.
Funded Under: Excellent Science - European Research Council (Erc)
Total Cost: € 2 451 281,00
Eu Contribution: € 2 451 281,00
Coordinated By: Institute Of Science And Technology Austria
Austria
Objective
We propose a theoretical basis and systems support needed to turn algorithmic monitoring from a runtime tool in the arsenal of formal methods into a pervasive and trusted engineering paradigm for the deployment of software. The ever-growing number of computational resources (many-core processors, data centers) allows software algorithms–decision makers–to be paired up with software monitors–decision checkers–where each monitor watches an algorithm in real time and provides warnings or intervenes when anything undesirable is observed. In order to be trusted, monitors are designed and linked independently of the monitored software. Monitoring is fundamentally a best-effort endeavor: it does not require complete specifications, nor perfect accuracy, but its widespread adoption requires a theory for the analysis of cost-precision trade-offs. Compared to the mature theories of computability and complexity (what can be computed? at what cost?), the theory of monitorability (what can be monitored, at what precision and cost?) is underdeveloped. We develop a quantitative, fine-grained resource and approximation theory for monitoring which supports the synthesis of monitors with desired cost-precision profiles. Our monitors can track the safety of systems as well as the fairness of algorithms (i.e. the absence of bias), security policies, and statistical properties of software. In addition, we facilitate the engineering paradigm of algorithmic monitoring by implementing systems support for the automated monitoring of software that is difficult to verify, such as algorithms that rely on neural networks, computations that happen in the cloud, and electronic systems that interface with the physical world (e.g. software for controlling medical and transportation devices). In all of these domains, the systematic and independent monitoring of critical requirements and sens
