Hacker news

  • Top
  • New
  • Past
  • Ask
  • Show
  • Jobs

Show HN: I made an audio-to-MIDI model 60x more efficient (https://kobi.music)

62 points by hidude562 4 days ago | 15 comments | View on ycombinator

gpugreg 2 minutes ago |

Nice! Worked surprisingly well.

You might want to implement debouncing or deduplication for your website so users do not convert the same mp3 multiple times (at multiple times the price) when they click "run audio to midi".

epiccoleman 29 minutes ago |

That is awesome! I'll definitely have to check it out. I was very impressed with muscriptor when I used it in a recent project (a conference talk I gave). It was an AI talk with a musical component, and the final "scene" in the talk involved my own guitar playing. I used Muscriptor to transcribe that playing for categorization as part of a "guitar solo generator" that recut a new solo out of my stock phrases each time. Really fun little demo for a bigger project.

Anyway, it would be awesome to be able to do those sorts of transcriptions better and faster - so I'll definitely give this a whirl. Maybe I can dig up the pipeline scripts from processing my licks for that talk and see how dettato slots in.

peaseagee about 1 hour ago |

Congrats on training on open and public domain recordings. Thanks for proving this kind of work can be done in a more ethical way.

oever about 1 hour ago |

Audio-to-MIDI sounds very ambitious. How do you verify that the midi fits the audio? Does it result in useful beat detection? E.g. I'd very much like to have a way to map audio offset in a recording to labels in sheet music. But even beat detection in fairly standard jazz recordings fails often.

Does the model know when a piece (whole piece, not a section) is 3/4 or 4/4 th and can it give the onsets of all measures? Or all the midi notes just based on time and not organized rhythmically?

ch_sm 29 minutes ago |

Just tried it with some home-recorded songs (pop/rock) — this works great!

jsphweid about 1 hour ago |

I've been out of the loop since Magenta's MT3 model. This feels a lot better and significantly smaller. Very impressive, dude. And thank you!

gregzem 3 days ago |

Congrats with SOTA model. How good is it in recognition of the instruments which are overlapping by frequencies? Does is provide note offset and velocity in the transcribed MIDI? Thanks!

mycall about 1 hour ago |

I'm still waiting for someone to convert beatboxing to midi in realtime. That would be a game changer for that art.

IAmGraydon 25 minutes ago |

Excellent! I'll try this tonight. The timing couldn't be better as I've been researching audio->MIDI models for some personal projects recently.

pikann22 about 1 hour ago |

crazy how a 20m param model actually beats the 1,3b muscriptor. awesome work!

chahmed312 about 1 hour ago |

Is it fully open source?

JodieBenitez about 1 hour ago |

I need this in Bitwig !

saascostlab_off 13 minutes ago |

[flagged]

staff000 3 days ago |

[dead]