I think putting some math vocabulary in with dictation is very useful so we don't have to pause or use "say <dictation>" all the time.
Here's what I tried so far (including some of Mike's old code). Capturing dictation and then replacing some words with a dictionary (would be in a separate grammar that would be disabled unless you are doing math).
math_replacement_dictionary = {"math gamma": r"$\gamma$",
"math gamma below beta": r"$\gamma_\beta$"}
def math_replace(dictation, dictionary):
input_list = str(dictation).split(" ")
output_list = []
for i in range(len(input_list)):
if input_list[i] == "cap":
input_list[i+1] = input_list[i+1].title()
else:
output_list.append(input_list[i])
pre_math_string = " ".join(output_list)
output_string = copy.copy(pre_math_string)
for k, v in sorted(dictionary.items(), key=lambda pair: len(pair[0]), reverse=True):
output_string = output_string.replace(k, v)
Text(output_string).execute()
"<dictation>": Function(math_replace, dictionary=math_replacement_dictionary),The first part of the function is just to getcapitalization for regular dictation. The reason I did the sorting stuff is because I want to make sure I am replacing the largestthing first. e.g. replacing "math gamma below beta" before "math gamma" since otherwise that would cause problems.
We can generate a dictionary for the math replacement dictionary. Let me know what people think. David zurow said he had some thoughts about a better way to do this sort of thing am hoping he will share.
I still need to get it to put in spaces at the beginning of each utterance unless there already is aspace which I guess is something that Dragon does automatically but is not done by this. A variety of things seem kind of clunky when all of your dictation is passed through this. e.g. scratch that doesn't work.
@annakirkpatrick you can have backslashes in the Dragon vocabulary word if you put them in the printed form which is in preferences under a word in the Vocabulary Editor. However, if you are using the approach above if you pass everything through that command, the printed form will be lost it will just write out the written form; in other words you can't do both approaches at the same time.
I'm not sure it's worth it really. Losing dragon's built-in editing features is annoying and this would be vulnerable to slight misrecognitions of the command words. I don't think pausing before starting to dictate latex wastes too much time in the scheme of things.
Probably an easier way of accomplishing something similar would be to just dictate as normal until you have a document with numerous instances of "math gamma" and then just do a find and replace for all of them.
By "Dragon 's built-in editing features" do you mean the commands like "insert before <dictation>" ? That still works with this but those don't usually work in math -based apps anyways. There are a few small issues regarding spacing that need to be worked out. for now I think I'm just going to have a space at the end of the text action. but spacing with things like "open paren" works though the spacing does not seem to work properly for custom Dragon vocabulary words or for "slash". I don't think the find and replace approach would be suitable if you have a lot of different symbols going around. The issue of misrecognitions is important; ideally this grammar would be given lower priority than the other grammars so that it would have the low priority that dictation usually has. Rather than just replacing words it might be more flexible to have a restricted set of commands available, especially for Lyx / scientific notebook where you need keystrokes not just text. I just posted David's suggestion on how to do this at the end of this issue dictation-toolbox/Caster#623 .
In general, this sort of thing would not be so much for a situation where you are doing heavy duty calculations but more for a situation with like 70-30 English-math symbols ratio or higher. I will be trying out this sort of thing in the wild myself so I will see how it goes. I think it will at least be good to have as an option which the user can easily enable/disable, but I will have a better sense after using for a while.
Let me know if anyone wants more info about that. Keeping the vocabulary in a .txt file is more compact then XML but does not support printed forms and thus does not support backslashes. In principle we can gennerate these XML files programmatically for all the combinations of letters and superscripts and so on.
To address a couple other lingering points/questions:
-when it comes to computer algebra systems, I have mainly used sage (sagemath.org). I interacted with it mainly as plain text, sometimes in a Python-style interactive shell, sometimes writing scripts with a plain text editor and running from system shell or sage shell. There is a browser-based "GUI" but it mainly amounts to just another way to access the interactive shell. The ability to display graphs and typeset mathematics in the browser is nice, but otherwise it behaves like a shell.
-I totally forgot about printed forms! They are pain, but, yes, you can get backslashes that way.
Pretty much agree. I think that the Dragon system of separating utterances into either dictation or commands is a fairly nice way of working, and it is easy to understand if it doesn't do what you expect. I would be hesitant to introduce a lot more complexity for the fairly minor benefit of reducing the number of pauses. I don't know about you but I pause quite a lot while I'm dictating anyway to think about what I need to say next and to make sure the previous section is correct.
One thing which may be of interest. There is a parallel voice control project called Talon which provides an API for defining grammars in the same way that Natlink + dragonfly does. Until now it has only been available for Mac but is currently being ported to Windows. One of the nice things which the Talon API exposes from Dragon is a parameter which controls how aggressive it is at finishing an utterance (how long of a pause it hears before it decides that you've finished talking). Reducing this would make the dictation -> pause -> commands cycle quicker.
@quintijn good to hear :-) sorry about the email, it is redirecting to my old university email which is no longer valid, I'll update it.
Let me know if there is anything I can do, and if you want to make a pull request for the Dutch translations or just post them in here then that would be a good addition. I set it up so that most of the standard operations which a user might want to tweak are in the TOML configuration files, but that's obviously not particularly powerful for anything more than inserting characters or executing keystrokes. If anything more complex is required then that should be easily doable on the dragonfly side.
"new math line | nieuwe math regel" = "escape:10, end, enter, c-m"
"squared| kwadraat" = "caret, 2, right"
"cubed|derde macht" = "caret, 3, right"
"inverse" = "caret, minus, 1, right"
"(parens | parentheses | haakjes)" = "a-m, lparen"
"square brackets| brekkits | vierkante haakjes" = "a-m, lbracket"
"curly brackets | accolades" = "a-m, lbrace"CCR is complicated, as you probably know, and I am more or less just hacking on top of caster's implementation.
There are some details on the mathfly site, but the the way it is all done is to merge everything that can be recognised in the current context into one rule, and then allow repetitions of that rule. The structure is something like
Rule(
Repetition(
Alternative(
Compound("command one"->action)
Compound("command two"->action)
Compound("command three"->action)
)
)
)language setting is equal to nld. English will always be used as a fallback
Major architectural change. Before updating you will need to run
pip install dfly-breatheI have entirely replaced the old caster-based CCR backend with Breathe, a library I've been working on. Should make it much easier to add support for new apps etc, and I have fully fleshed out the NestedRule concept which was originally prototyped here. Please let me know if there are any issues.
Well basically I wanted rules that were like:
Integral from <stuff1> to <stuff2>Where stuff was a sequence of any other ccr commands. Nested rules were the first implementation based on Caster, breathe allows this by allowing the user to add custom top-level rules. The mathfly examples are in e.g. lyx.py at the bottom but may be slightly difficult to understand at a glance, there are clearer examples on the breathe readme which show greedy dictation and optional repeat values using the same feature.
CommandsRef extra which references a sequence of commands and then include an Exec action to execute the sequence