1 of 8

What Is Holding Up OMR?

Heinz Roggenkemper

Music 254

April 27, 2021

2 of 8

OMR review 2001

3 of 8

OMR review 2019

The C’t magazine reviewed OMR software again in August 2019. The three products that were rated ‘+’ in terms of note recognition were PhotoScore, SharpEye and SmartScore.

In November 2019 I posted the following note accuracy results on omr-research.net:

4 of 8

Obstacles (1)

Byrd/Simonsen: Towards a standard testbed for Optical Music Recognition (2015):

  • ‘OMR is difficult because CWMN is extremely complex.’
    • CWMN: Common Western Music Notation
  • Other factors: typographic styles, image quality
  • Main message: OMR evaluations are key for progress, but unfortunately can not be automated. Proposal: create a standard testbed for OMR.

​

OMR is small: small market (maybe 20 full-time developers for commercial products), not many researchers

5 of 8

Other Obstacles (1)

Architecture: OMR requires complex systems with corresponding maintenance effort

  • Commercial systems have little incentive to build systems that can be extended by others, and the barrier for research to build such a system is high

Lack of transparency: it is difficult to find out what capabilities OMR systems have, and how well they do. (Solution: automated assessments, based on substantial ground truth)

  • Evaluations of complex systems are always difficult, since they are a form of summative feedback that is dependent on the use case.

​

6 of 8

Other Obstacles (2)

  • Divergent use cases and demanding users
  • Over-reliance on deep learning in research
    • Deep learning has almost become the default tool choice
    • There is a class of problems where the resulting models are big and inference is slow
    • Due to the complexity of CWMN and the role of context it is currently unlikely to succeed as an end-to-end solution
    • This could be a temporary problem - we are still at the beginning of deep learning
      • Francois Chollet tweet: ‘I'd guess I'd summarize it as, being a "deep learning expert" in 2021 is like being a "medicine expert" in 1800. You know a lot less than you think, and most of what you think you know is wrong.’
  • Lack of collaboration between industry and research

​

7 of 8

Obstacle Summary

Some obstacle are big unmovable objects (complexity of CWMN, size of market/space)

No other obstacle has an oversized impact.

There is interdependency between obstacles.

Therefore breakthrough progress in the short term (next 3 years) is unlikely.

8 of 8

What Could Help?

Create the testbed, and make it easy to use

Deeper understanding of user behavior

Get more out of OMR results:

  • Better editing tool (see VHV!)
  • Combine the results of multiple OMR engines
  • ‘OMR as a service’