Paper Picks #2: Rethinking Research: Why We Need to Keep an Eye on Applied AI

We see an increasing number of research papers being published that claim to outperform the state of the art in AI. The examples in the research look great, but once you run it on some data “in the wild” the results are rather underwhelming. We have and it’s happened more than once here at Passion Lab.

Enter the eye-opening paper by Herrmann et al., "Position: Why We Must Rethink Empirical Research in Machine Learning".

This paper tackles common problems in AI research, such as results that can't be reliably repeated by other researchers, the pressure to outperform existing methods, sometimes leading to unrealistic claims, and unintended biases in how experiments are set up and conducted.

These issues often arise due to the way machine learning research is currently practised and evaluated. If you want to know how we, as a research community, can do better and why we should embrace negative results rather than fear them, this paper is worth a read!

Here’s our summary of the key points:

The Problem: When Hype Meets Reality

Imagine a data scientist working on a cutting-edge project. They come across a paper boasting state-of-the-art results, complete with dazzling examples. Excited, they implement the method, only to find that it performs no better than the existing solutions.

This scenario highlights a critical issue in ML research: the gap between reported results and real-world performance. But why does this happen?

The Culprits

Non-reproducibility: Many published results can't be replicated, even under similar conditions.
State-of-the-art hacking: The pressure to outperform existing methods can lead to overfitting to specific benchmarks.
Biased experiments: Unconscious biases in experimental design can skew results.

The Solution: Embracing a New Research Paradigm

Herrmann and colleagues argue that we need to fundamentally rethink how we approach empirical research in machine learning. Here's what they propose:

Shift from confirmatory to exploratory research: Instead of trying to prove hypotheses, we should focus on discovering patterns and generating new ideas.
Embrace negative results: Failed experiments are valuable! They help us understand the limitations of our methods and prevent others from going down the same unproductive paths.
Improve experimental design: We need more rigorous standards for designing and reporting experiments to ensure reliability and reproducibility.
Foster a culture of openness: Encouraging researchers to share code, data, and detailed methodologies can help the community validate and build upon each other's work.

Why This Matters

By addressing these issues, the AI community can build a more robust and trustworthy field of machine learning where:

Researchers spend less time chasing unrealistic benchmarks and more time solving real problems
Practitioners can confidently implement methods knowing they'll work as reported
The gap between academic research and industry application narrows, accelerating innovation

So, the next time you hear about an AI solution promising miraculous results for your business, remember to approach it with a healthy dose of scepticism. As a business owner, you can contribute to positive change by:

Asking tough questions about how AI solutions perform in real-world scenarios
Demanding transparency from AI vendors about their testing methods
Sharing your experiences with AI implementations, both successes and challenges

After all, true progress in AI isn't just about flashy features—it's about creating reliable, practical tools that genuinely improve how we do business. By being an informed and discerning adopter of AI technology, you're helping to build a solid foundation for its future in the business world.

If you're interested in exploring how AI can benefit your organisation while navigating the challenges discussed in this article, we're here to help. Let's have a conversation about responsible AI adoption and how it can enhance your business processes in meaningful, measurable ways.

Book a discovery call to learn more about our approach to practical and ethical AI implementation.

‍