I wonder if it would be worthwhile treating command with dictation elements and raw dictation elements special in dragonfly parsing. Raw dictation elements would be used a lot more if they were not greedy. While there's nothing we can do to stop them from being greedy per se but maybe we can change the order of when dragonfly evaluates by always evaluating them at the end rather than just in the context of the grammar. So during parsing a the dictation element in a grammar the context and the command dictation element could be extracted in the current Dictation element ignored. Placing both of those at the end to ensure that there are evaluated last. If this is technically possible it would allow for free dictation like we see in DNS for all engines.
However I think this would mean we would have to implement mechanism rather than letting the engine decide what is a command versus what is free dictation. Which can be buggy at least in DNS/DPI. After all dragonfly knows which elements are dictation/commands before loading the grammar into the engine.
Dictation elements are not really greedy. If there is a command that can be spoken afterwards, then, provided the SR engine back-end supports dictation, it should be recognised and parsed just fine.
I'm not really willing to rewrite the parsing code to handle the buggy behaviour with DNS that you mention. The changes I made in PR #268 should have solved that in most cases.
You may be interested in the following dictation-related example files Christo Butcher (t4ngo) wrote at some point:
Dictation elements are not really greedy. If there is a command that can be spoken afterwards, then, provided the SR engine back-end supports dictation, it should be recognised and parsed just fine.
I'm not really willing to rewrite the parsing code to handle the buggy behaviour with DNS that you mention. The changes I made in PR #268 should have solved that in most cases.
You may be interested in the following dictation-related example files Christo Butcher (t4ngo) wrote at some point:
Thanks for the info! I will look into that.
@LexiconCode No worries. If you like, you can experiment with custom recognition parsing/decoding using the DgnImported class in the test_dgn_import.py as a starting point.
@lunixbochs I agree it is a harder problem and that there are better ways to do this. Personally, I think the current lesser solution works well enough.
I am open to an improved solution for differentiating between dictation and command words. This would be useful for the Natlink and text-input engine back-ends.
@daanzu Yes, that is how it is supposed to work and how it does work with SAPI5 and Kaldi. DNS, however, occasionally doesn't report nicely differentiated words, leaving that task to us. The other two engine back-ends ("sphinx" and "text") have limited support for dictation.
The Modifier element was added by mrob95. It is delightful, isn't it? ^^
Thanks, that's good to know. If an engine back-end is to be added that uses wav2letter directly, then I'll revisit this and see what I can do. I'll give some thought to improving the solution added in #268.
There is an open issue for wav2letter engine support: #245. Although, to my knowledge, there hasn't been much activity on that recently.
@quintijn No worries :-)
It is a pity this a DNS problem, rather than a Natlink one we could fix.
is there a specific reason the decorator module is used in dragonfly.rpc? the usage seems equivalent to:
import functools
def _rpc_method_wrapper(method, *args, **kwargs):
...
def _rpc_method(method):
@functools.wraps(method)
def wrapper(*args, **kwargs):
return _rpc_method_wrapper(method, *args, **kwargs)
return wrapperand switching to the above approach would make integration easier, as I ran into an issue with the decorator module
Okay then, make sense. Given that some users set logging levels using the current names, maybe this should be held off until version 1.0.0. I intend to make a few other changes like this in that release, e.g. resolving issue #238 (Modularize dependencies).
Your changes removing the decorator dependency look good to me.
Incidentally, the requirements for dragonfly.rpc will be made optional extras in version 1.0.0. I will probably remove the relevant import statements from dragonfly/__init__.py.
When working with extras it would be nice in some circumstances to remember the last extra recognized for that particular spec. Let's consider the following command select word [<left_or_right>] [<n>] The user says select word left and then again says select word left extra would be remembered.
Now we could set a directional default in extras. However in more complex scenarios it's not that simple. Now wrapping this up in the function would work but it really does pollute grammar sets making them harder to read. Any thoughts on implementation or should I stick to using a function?
If you do go with a function, then you can get at the underlying rule by including a _rule argument:
def function(_rule):
print(_rule)
print(_rule._defaults["left_or_right"])An alternative is a separate command for explicitly changing your default value for left_or_right. This would change it for all mappings using the extra.
I think Quintijn is right here though, repeating the last action is probably easier.
again do command is similar to what Quintijn described. This looks much more complex than it is due to utilizing Caster asynchronous actions. It could be converted utilizing to pure dragonfly. This allows to repeat not only the last command but the last utterance. Useful for CCR chains or commands with dragonfly repetition elements.# migrated this function from elsewhere for completeness of the post.
def get_and_register_history(utterances=10):
history = RecognitionHistory(utterances)
history.register()
return history
_history = get_and_register_history(10)
class Again(MappingRule):
mapping = {
"again (<n> [(times|time)] | do)":
R(Function(lambda n: Again._create_asynchronous(n)), show=False),
}
extras = [ShortIntegerRef("n", 1, 50)]
defaults = {"n": 1}
@staticmethod
def _repeat(utterance):
Playback([(utterance, 0.0)]).execute()
return False
@staticmethod
def _create_asynchronous(n):
last_utterance_index = 2
if len(_history) == 0:
return
# ContextStack adds the word to history before executing it for WSR
if get_current_engine().name in ["sapi5shared", "sapi5", "sapi5inproc"]:
if len(_history) == 1: return
# Calculatees last utterance from recognition history and creates list of str for Dragonfly Playback
utterance = list(map(str, _history[len(_history) - last_utterance_index]))
if utterance[0] == "again": return
# Create AsynchronousAction
forward = [L(S(["cancel"], lambda: Again._repeat(utterance)))]
AsynchronousAction(
forward,
rdescript="Repeat Last Action",
time_in_seconds=0.2,
repetitions=int(n),
blocking=False).execute()