Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 18:17

    achimmihca on more-ui-toolkit

    continue HighscoreScene by clic… fix StaticsScene Canvas (was no… images keep aspect ratio, bette… and 1 more (compare)

  • 16:24
    achimmihca edited #231
  • 16:20
    achimmihca commented #231
  • 16:17
    achimmihca commented #266
  • 16:17
    achimmihca closed #266
  • 16:16
    achimmihca updated the wiki
  • 16:11
    achimmihca updated the wiki
  • 10:14
    achimmihca commented #266
  • 09:41
    Rom888 opened #266
  • Dec 01 19:34

    achimmihca on more-ui-toolkit

    new: implemented HighscoreScene… (compare)

  • Dec 01 19:33

    achimmihca on more-ui-toolkit

    new: implemented HighscoreScene… (compare)

  • Dec 01 19:25
    watho commented #246
  • Dec 01 18:29
    achimmihca edited #231
  • Dec 01 18:28
    achimmihca edited #231
  • Dec 01 18:27
    achimmihca commented #246
  • Dec 01 18:20
    achimmihca commented #246
  • Dec 01 18:13

    achimmihca on ui-toolkit-more-options

    (compare)

  • Dec 01 18:13
    achimmihca closed #264
  • Dec 01 18:13
    achimmihca closed #263
  • Nov 30 16:18
    watho commented #246
Markus Böhning
@bohning
Great, so it probably was a restarting issue - thanks for the hint/help!
Nico Franke
@ZerNico

I am currently diving through the code a bit and I am kinda curious how the Pitch Tracking exactly works

So far I saw that Dynamic wavelet does the audio processing to get the right pitch

the part I am not 100% sure about:
The mic buffer is checked once per frame and if it is above 1024 samples it is processed and the result of the pitch detection is stored
Then I guess whenever a new note needs to be checked if its correctly sung the last available value will be taken?

And for how the latency is accounted for I have no idea

Nico Franke
@ZerNico
also I suppose update() is called once per frame?
achimmihca
@achimmihca
Pitch detection passes different stages.
MicSampleRecorder collects the raw samples of a mic. It uses a buffer of up to 1 second I think. Unity fills this buffer in a separate thread such that no sample is missed.
Then it depends.
The settings for example uses the simple MicPitchTracker. It waits for a certain number of samples to arrive and then analyzes these. I guess this is what you found.
In the game however, the pitchof every beat is analyzed (UltraStar song format kind of defines a grid of beats via the #BPM tag).
PlayerPitchTracker knows which beat has to be analyzed next. It waits until the Mic buffer contains the samples that correspond to that beat. This is where the mic-delay is considered. If the mic-delay is increased, then the PlayerPitchTracker will wait longer for the samples to arrive. If there are enough samples, then these are analyzed and a PitchEvent for the beat is fired.
Yes, dynamic wavelet is the algorithm that takes the samples and outputs a pitch by default. There is also another algorithm available that uses circular magnitude difference.
Yes, update is called once per frame.
Most of the stuff has been implemented with #169 if you want to dive deeper.
achimmihca
@achimmihca
Are there other Mac users here? @bohning got an error message when trying to start the 0.2.1 release. I would like to know if this is a general issue or only affects his setup.
Markus Böhning
@bohning
I just redownloaded the 0.2.1 alpha, unpacked it, and either way I start it (double click on the app, open *.app from terminal, or start from within the Contents/MacOS/ folder, I always get the popup that the app is corrupted and cannot be openend (no further error message, unfortunately)
image.png
image.png
achimmihca
@achimmihca
@bohning Is there already a log file? Please check ~/Library/Logs/Karaoke/UltraStar Play/Logs
I wonder if the app crashes when Unity is doing something, or if the OS is killing it before Unity even writes a log file.
Markus Böhning
@bohning
There is, but not from the alpha, but from when I was able to compile myself (and run it).
Nico Franke
@ZerNico

Pitch detection passes different stages.
MicSampleRecorder collects the raw samples of a mic. It uses a buffer of up to 1 second I think. Unity fills this buffer in a separate thread such that no sample is missed.
Then it depends.
The settings for example uses the simple MicPitchTracker. It waits for a certain number of samples to arrive and then analyzes these. I guess this is what you found.
In the game however, the pitchof every beat is analyzed (UltraStar song format kind of defines a grid of beats via the #BPM tag).
PlayerPitchTracker knows which beat has to be analyzed next. It waits until the Mic buffer contains the samples that correspond to that beat. This is where the mic-delay is considered. If the mic-delay is increased, then the PlayerPitchTracker will wait longer for the samples to arrive. If there are enough samples, then these are analyzed and a PitchEvent for the beat is fired.

Thanks a lot for the detailed information I'll look further into it!

Really appreciated

achimmihca
@achimmihca
does anyone here know what happened to ultrastar-es.org ?
I cannot reach the website
basisbit
@basisbit
can reach it from here. Their domain is blocked by dome dns provider for legal reasons
achimmihca
@achimmihca
Ahh, I can reach it again as well. Sweet!
Nico Franke
@ZerNico
One more quick question I have so far, so unity always gets the new Audio samples in a seperate thread and once per frame you put the new samples into the buffer of the MicSampleRecorder. The buffer is the sampling rate so it is exactly one second of data? And because the buffer has a limited size the new samples are added to the back and if they hit the end they start at 0 again?
(basically looping around whenever needed) Did I get that correctly?
achimmihca
@achimmihca
yes, all correct.
(has nice image e for the buffer)
achimmihca
@achimmihca

because the buffer has a limited size the new samples are added to the back and if they hit the end they start at 0 again

Actually no. I just checked this. The new samples come into the buffer (highest index has newst samples, index 0 has oldest sample). When the buffer is full, the old samples just "fall out" of the buffer.

No circular buffer involved for this part.
achimmihca
@achimmihca
BTW: You can see the mic buffer in the RecordingOptionsScene. The visualization shows a portion of the newest samples in the mic buffer (See RecordingOptionsMicVisualizer.displayedSampleCount). When you make a sound, you can see it moving from right (high index in mic buffer) to left (low index in mic buffer).
Also, when you increase RecordingOptionsMicVisualizer.displayedSampleCount to the full size of the buffer, then it should take 1 second for the wave to move from right to left (because mic buffer has length == sampleRateHz).
Nico Franke
@ZerNico
ahh that is good to know, thank you!
Nico Franke
@ZerNico

I am working on a Karaoke app as a website/webapp for fun atm
Thought you might be interested to see what I have atm
https://streamable.com/84pny0

It's really only visual atm tho, I've tested some stuff and it definitely is possible to do proper pitch recognition in a browser but it's gonna be a lot of work

basisbit
@basisbit
@ZerNico there already exists a karaoke game as website, but because of legal issues and licensing fees being astronomically high if you are not a big Sony Music licensor or similar, no one did continue to put any more time into such games. code: https://github.com/Katharine/ponytone Website with very niche indie-music content: https://ponytone.online/
however, the code of that project is quite easily readable and might be helpful for you
Nico Franke
@ZerNico

@ZerNico there already exists a karaoke game as website, but because of legal issues and licensing fees being astronomically high if you are not a big Sony Music licensor or similar, no one did continue to put any more time into such games. code: https://github.com/Katharine/ponytone Website with very niche indie-music content: https://ponytone.online/

Thanks, it looks pretty interesting!
My plan was to make it for self hosting and not for one big page so legal issues shouldn't be an issue
Also audio processing changed a lot recently, audioworklets and web assembly should improve the pitch detection quite a bit

achimmihca
@achimmihca
ponytone looks awesome :D
I only knew about MyLittleKaraoke
achimmihca
@achimmihca
@basisbit Codacy requires to add UltraStar-Deluxe Organization again (they changed something it seems). I wanted to add PlayCompanion repo to Codacy but couldn't because it is part of legacy USDX organization. And only organization owner can add the organization.
So please, add this organization in Codacy again.
basisbit
@basisbit
@achimmihca uh, sure, let me try this quickly
basisbit
@basisbit
should be done now
achimmihca
@achimmihca
Nice! If you enter ARTIFACTS_KEY and ARTIFACTS_SECRET on https://travis-ci.org/github/UltraStar-Deluxe/PlayCompanion/settings then the build should be ready as well
Michael Chen
@Drominar_twitter
Hey I downloaded the standalone ultrastar play alpha for mac (and I am the sole administrator and have permissions to read and write) but it still tells me 'You do not have permission to open the application “UltraStar Play”.' - I'm wondering why is this the case?
achimmihca
@achimmihca
Other users had also issues with v0.2.1 on MacOS. These issues might be solved with the next pre-release (there is an outstanding pull request).
Michael Chen
@Drominar_twitter
Thank you- I will look forward to the next pre-release
achimmihca
@achimmihca
You can find the new v0.3 pre-release over here
basisbit
@basisbit
Wow, so much progress! Thank you so much for all the effort you put into this!
Markus Böhning
@bohning
Nice! Thanks a ton, @achimmihca !
Just installed the companion app, really excited to try it out…
Markus Böhning
@bohning
Successfully singing through my smartphone, works out of the box. This is AWESOME.
achimmihca
@achimmihca
You're welcome :) Happy that it's working
It should also be possible now to run the main game on a tablet or phone and then use another smartphone as mic.