Skip to content

Algorithm Audit & Benchmarking

Our service to evaluate solutions & release the potential of Trustworthy AI

It is about making your organization more competitive

We identify AI systems pitfalls that risk your operations, increasing algorithm accountability.

We deliver all necessary recommendations to move towards a competitive AI solution.

What we review

Internal Validation & Model Selection

We scrutinize the AI development & validation schema, reviewing aspects of model selection & feature engineering, linked to overfitting issues.

External Validation & Real World Test

We perform stress tests within real-world conditions to obtain accurate assessments of actual AI performance, and its impact on your operations.

Explainability and Transparency

We assess why & how the AI system makes its predictions; reduce the dangerous notion of black box AI & facilitate the treatment of errors.

Reproducibility and Reliability

Pillars of scientific innovation, we enable you to understand the whats & hows of the AI system, identifying its boundaries.

Fairness & Bias

We look for biases that jeopardize AI operation, as well as data drifting issues.

Other Data Considerations

Purpose limitation, proportionality, data minimization, privacy, ownership, regulation. GDPR compliance.

Algorithm evaluation & benchmarking helps you…

Have a safe deployment plan that maximizes AI impact

Respond to regulatory bodies and e.g. get your medical certification

Avoid investing time & money in a flawed solution

Adhere to a responsible use of AI systems

How we do it

  1. Scope
    We establish what is the operational context, use of the AI solution, and focus of the review.
  2. Review
    We perform a thorough evaluation of available documentation on the AI solution development and operation, identifying the aspects that lack proof, hazardous to the AI operation.
  3. Test & Benchmarking
    We design a real-world test to detect pitfalls of the system, we design the experiment to maximize its significance and the information you extract.
  4. Improve
    We distill the findings and provide actionable insights. This includes the evaluation and scoring of the AI solution, recommendations to address review findings, aiming to push towards trustworthy AI.

Success Stories

Development of a self-evaluation tool for the verification of data and AI systems in Healthcare

Over the last few months, we have had the privilege of working with PickleTech’s team to develop a tool for the verification of data and AI systems in Healthcare. The tool has taken into account legal requirements, as well as the most up to date standards and ethical recommendations. Their expertise and experience in algorithmic evaluation, their know-how, and their understanding of several socio-technical aspects around the design and the impact of AI technologies have helped us make huge progress in the coming Health-PIO model.

Albert Sabater Coll, Director of the Catalan Observatory for Ethics in Artificial Intelligence (OEIAC)

Our Partners