With a budget of €4 million over three years (September 2023 - August 2026), BACQ brings together, around the project coordinator Thales, Eviden, the CEA, the CNRS, Teratec and LNE.

In summary

Image

BACQ is the first R&D project funded by the MetriQs-France programme. With a budget of €4 million, this three-year project, running from September 2023 to August 2026, brings together six key players in the quantum ecosystem: Thales (coordinator), Eviden, CEA, CNRS, Teratec, and LNE. BACQ’s main objective is to establish benchmarks (or reference criteria) to evaluate the performance of any type of quantum computer from the perspective of concrete applications that are meaningful to end users. In practice, BACQ will produce high-level operational indicators to rate the performance of quantum computers. These indicators will be based on the aggregation of computational and energy metrics relating to the resolution of concrete problems covering many areas of application of quantum computing: optimisation, linear systems solving, quantum physics simulation, and prime factorisation. These indicators will then be aggregated into a single scoring model that takes into account user preferences and allows for the comparison of different machines. With the goal of achieving the broadest possible membership, BACQ will ultimately provide open access to a set of reference benchmarks to be implemented on any type of quantum computer.

Context and issues

Quantum computing promises to revolutionise many scientific fields and sectors of activity: logistics, finance, medicine, scientific research, cryptography and cybersecurity, etc. This has been motivated by the search for the quantum computational advantage, namely the ability to calculate faster and to process more complex problems in less time than a conventional computer. This goes as far as the possibility, in the long term, of performing calculations entirely inaccessible to conventional computers.

Public and private announcements and initiatives around quantum computing are multiplying throughout Europe and the rest of the world to the point that some observers are talking about the dawn of a new era. However, even if fundamental research in the field of quantum technologies has made great strides over the last decade, this rapidly developing disruptive technology faces scientific, technological and engineering challenges that, for the time being, are hampering its adoption by industry and society. Therefore, it is necessary not only to indisputably demonstrate the advantages of quantum computers over conventional computers, but also to compare the different types of quantum computers with each other. 

Comparative analysis of quantum computers is more complicated than in other fields for four main reasons:

• Many quantum computing systems are currently being developed, each using different underlying technologies (superconducting qubits, spin qubits, trapped ion qubits, neutral atom qubits, photonic qubits, etc.) and also relying on different paradigms (analog machines including adiabatic machines, gate machines, machines with error-correcting codes, etc.).
• The market is highly dynamic and evolving: therefore, comparing the performance of quantum computers at a given point in time will not necessarily be indicative of future technological developments.
• The application areas of quantum computing are very diverse (quantum physics simulation, optimisation, factorisation, etc.), and each of them requires different machine properties and therefore different forms of evaluation.
• Advances in quantum algorithms are significant: progress continues not only thanks to hardware advances, but also to algorithmic advances that enable more efficient use of this hardware (error-correcting codes, compilers generating quantum circuits, hybrid algorithms, etc.).

These four factors underscore how essential effective means of comparison are for decision-making by end users (industry, laboratories, governments, etc.). Understanding the state of the technology today and tracking its progress over time, identifying the hardware best suited to a given use case, and comparing one quantum computer vendor to another require systematic testing.

Benchmarking the performance of quantum computers is essential to accelerate progress in quantum computing. Rigorously estimating the computational quantum advantage requires, among other things, the definition and testing of application benchmarks. The key is to design benchmarks that are:

• Useful, telling users what they need to know.
• Scalable, being able to expand and adapt to evolving technologies.
• Comprehensive, covering all relevant attributes.

Objectives

Image

With the support of the MetriQs-France programme and a budget of €4 million over three years (September 2023 - August 2026), the BACQ project aims to develop a reliable measurement instrument to evaluate the performance of quantum computers from the perspective of concrete applications that are meaningful to end users. To measure progress towards a practical quantum advantage, the project will consider not only the computational performance of the machines, but also the resource cost associated with the desired performance, in particular energy efficiency.

The performance evaluation of quantum computers will be based on the analysis of benchmark problem solving: optimisation, linear systems solving, quantum physics simulation, and prime factorisation. Over a three-year period, the partners will develop benchmarks (or reference criteria) for each of the selected problems before defining the multi-criteria scoring model that will aggregate these high-level operational indicators. This unique model will be designed to take into account user preferences and will allow for the comparison of different types of quantum processors and highlight the advantages of each technology for a given application.

BACQ also seeks to achieve large-scale adoption of the measurement tools currently being developed with regard to the specifications of quantum machines and their evaluation methods. To this end, BACQ supports standardisation work and promotes benchmarks developed at European and international levels to ultimately ensure the widest possible penetration of quantum technologies among their potential users. For example, within the CEN-CENELEC JTC22 WG3 Quantum computing and simulation technical committee, BACQ actively participates in a working group on application benchmarks directly linked to the project. Within the IEC/ISO JTC3 Quantum technologies technical committee, BACQ is also heavily involved in an international standards project on application benchmarks, with the objective of building consensus around the vision developed in the project so that it becomes an international reference.

Ultimately, the BACQ project will provide users with free access to a set of reference benchmarks to be implemented on any type of quantum computer.

Consortium

The BACQ project brings together six key players in the quantum ecosystem: 

• Thales, through Thales Research & Technology, is the project coordinator and participates in the development of benchmark criteria related to linear systems solving and optimisation, with particular focus on noise and errors, on both NISQ Noisy Intermediate Scale Quantum) and FTQC (Fault-Tolerant Quantum Computing) machines. Furthermore, Thales is involved in the aggregation of the criteria developed in the project based on the multi-criteria decision tool MYRIAD developed for capturing consensus preferences and providing an explicit global rating.
• Eviden participates in the development of technical reference criteria related to optimisation, by adapting the Q-Score/Max-Cut developed for solving the optimisation problem to a wide variety of quantum machines. The team also contributes (with that of CEA IPhT) to the development of a Many-body Quantum Score based on the simulation of the dynamics of an N-body quantum physics problem.
• CEA contributes with three of its laboratories: the Institute of Theoretical Physics (IPhT), the Laboratory for Quantum Photonics, Electronics and Engineering (PHELIQS) and the Laboratory for Integration of Systems and Technology (CEA-List). IPhT and PHELIQS work on benchmarks related to the simulation of many-body quantum physics problems, for gate machines and analog machines. For example, they work on criteria related to the precision of the preparation, evolution and measurement of entangled states with a large number of particles (Many-body Quantum Score, in collaboration with Eviden). The CEA-List team participates in the evaluation of analog machines and gate machines on several classes of optimisation problems, linear systems solving, and prime factorisation.
• CNRS, through the MajuLab laboratory, defines, tests, and promotes the energy efficiency of simple gate-based quantum machines, focusing on the hardware-independent algorithmic level. The contribution includes developing a simulation module and using it to optimise energy efficiency. The promotion and acceptance of the benchmark will directly benefit from the strong involvement of the CNRS team, which is leading the IEEE recently launched Quantum Energy Initiative working group, to propose a standard on the energy efficiency of quantum computing.
• Teratec participates in building relationships with the ecosystem to promote the project. In this capacity, Teratec consults with industry stakeholders, end users and technology providers. Teratec engages in dialogue and promotes cooperation at European and international levels, particularly regarding similar benchmarking initiatives.
• LNE is the coordinator of the MetriQs-France programme, the national programme of measurement methods, evaluation and standardisation of quantum technologies, which is which is part of the French National Strategy on Quantum Technologies. LNE develops relationships with stakeholders and promotes the exploitation of results, particularly in metrology and standardisation. The LNE uses its expertise in the development and implementation of reference measurement methods. Furthermore, thanks to its status as a trusted third party, it guarantees the impartiality and independence of benchmarks.

Results

On the scientific front, the project will help develop French skills and expertise in quantum computing and simulation: knowledge of quantum machines, algorithms, and programming.

• Establishing a classification of quantum machines according to their application utility will allow for maximum impact of the project. The reliable and rigorous evaluation of quantum computers will allow technology providers to improve the performance of the machines and end users (industry, research laboratories, governments) to develop uses on a realistic basis.
• Beyond that, the development of quantum computing and quantum simulation performance will enable scientific progress, for example in materials science, chemistry, and solid-state physics.
• The project will contribute to the credibility of quantum computing and quantum simulation applications and, consequently, to their industrial adoption, thus stimulating the growth of the associated market.
• The French quantum technology industry, including quantum computing and simulation, will be the primary beneficiary, enabling it to maintain its competitive advantage and gain significant market share over the long term.

Ultimately, the BACQ project will provide users with free access to a set of reference benchmarks for implementation on any type of quantum computer.

All scientific publications related to the BACQ project are available on the HAL platform MetriQs-DEV-France.