## Where communities thrive

• Join over 1.5M+ people
• Join over 100K+ communities
• Free without limits
##### Activity
• Feb 21 2021 13:41

mrob95 on master

Deprecation notice (compare)

• Feb 21 2021 13:38

mrob95 on master

Deprecation notice (compare)

• Feb 20 2021 20:25

mrob95 on master

• Feb 02 2021 17:15

mrob95 on master

Move logging set up above drago… (compare)

• Jun 02 2020 19:38
mrob95 opened #22
• Jun 02 2020 19:36

mrob95 on texmaker

• Apr 24 2020 07:18

mrob95 on master

Create requirements.txt (compare)

• Apr 24 2020 07:17

mrob95 on master

Create install.bat (compare)

• Jan 31 2020 21:25

mrob95 on master

change to  for inline math,… add equation* and align* update docs and 2 more (compare)

• Jan 31 2020 21:25
mrob95 closed #21
• Jan 31 2020 21:25
mrob95 commented #21
• Jan 31 2020 19:03
annakirkpatrick opened #21
• Jan 26 2020 14:38

mrob95 on master

• Jan 26 2020 14:34

mrob95 on breathe

Transition sublime and template Update dependencies Add install script and 2 more (compare)

• Jan 15 2020 20:50

mrob95 on master

• Nov 17 2019 14:49

mrob95 on master

• Nov 17 2019 14:28

mrob95 on master

• Nov 01 2019 07:33

mrob95 on master

Update dependencies (compare)

• Oct 27 2019 17:56
mrob95 synchronize #19
• Oct 27 2019 17:56

mrob95 on languages

Improve formatting (compare)

Mike Roberts
@mrob95
Okay I have added a new [commandmisc] section in latex.toml which allows arbitrary alternating commands with the "insert" prefix. So we can do:
[commandmisc]
"line end"           = ["\\\\", "enter"]
Hope pulling changes isn't causing too much bother
Anna Kirkpatrick
@annakirkpatrick
@mrob95 the [commandmisc] section should be great. Thanks!
And pulling changes is no biggie. I'm pretty comfortable with git and certainly couldn't suggest an easier way.
Mike Roberts
@mrob95

np, if you need more flexibility for commands then dragonfly is not at all hard to learn see e.g.

https://github.com/mrob95/MR-caster/blob/master/caster/apps/chrome.py

Anna Kirkpatrick
@annakirkpatrick
@mrob95 thanks for the link! Once I get the basics of LaTeX dictation down (both adding commands and getting over most of the learning curve), I am thinking about setting up better integration with vim for easier editing. That looks like a really useful example to work from.
@mrob95 I packaged up my changes so far into a pull request. It's only a handful of commands, but they are significant in terms of usability. I'm sure there will be more as I continue working, but you might as well have these for now.
Mike Roberts
@mrob95
And thanks for the PR, looks good. Regarding the docs I'm going to do a cleanup job at some point, I think I noticed that pipe is formatted as a long dash and thought "that will do" ^^. Might even change them from LaTeX to markdown
I get an absurd amount of value out of dragonfly and use it for pretty much everything - text editor commands, programming language commands, math commands, git commands, web browser commands, etc etc
Anna Kirkpatrick
@annakirkpatrick

@mrob95 I'm hoping maybe you've seen this error message before and can point me in the right direction. Here's the sequence of events:

1. I started up Dragon but natlink didn't start. I did not get the "messages from Natlink" window, and commands like "enable X" were not recognized as commands.
2. Pulled latest version of mathfly from master, just to be sure. (Couldn't run rebuild command.)
3. I ran the natlink configuration and reregistered NatLink. Rebooted computer and restarted Dragon.
Same problem as 1, so I checked my path. Somehow Python 3 had jumped over Python 2 in my path. I fixed that. Rebooted computer. Ran NatLink again being sure I was using Python 2.7. Reboot again.
4. Now Dragon starts with NatLink, but I am getting a bunch of error messages.

Vocola not active
Ignoring rule 'sublime'. Failed to load with:
cannot import name ContextAction
Ignoring rule 'sumatrapdf'. Failed to load with:
cannot import name ContextAction
cannot import name ContextAction
Ignoring rule 'core'. Failed to load with:
cannot import name ContextAction
Ignoring rule 'ScientificNotebook55'. Failed to load with:
cannot import name ContextAction
Ignoring rule 'LyX'. Failed to load with:
cannot import name ContextAction
Ignoring rule 'latex'. Failed to load with:
cannot import name ContextAction
Ignoring rule 'latex_maths'. Failed to load with:
cannot import name ContextAction
Ignoring rule 'alias'. Failed to load with:
cannot import name ContextAction
- Starting Mathfly -
Say "enable <module name>" to begin, or
"configure <module name>" to make changes.
Modules available:
DNS version: 13
Python version: 27
Windows Version: 10
User defined macro's (UserDirectory) enabled: C:\Users\Anna\Documents\NatLink\mathfly



Mike Roberts
@mrob95
Try running
pip2 install -U dragonfly2
ContextAction is something new which was only added recently to dragonfly
Anna Kirkpatrick
@annakirkpatrick
Thanks! That fixes it.
Alex Boche
@alexboche
@mrob95 @esc123 can you guys explain what you like better about scientific notebook than Lyx?
my experience is mostly with Lyx, but I am also learning regular latex now. considering trying scientific notebook
esc123
@esc123
Jeez it's been so long since I used Lyx now that I can hardly remember why I switched! I recall SN being more convenient for voice rec because it was more streamlined to write multiple lines of Math continuously. Just "math mode" and away you go. I believe in Lyx when you begin a new line you must then specify that you will be writing math in that line. Probably an easy issue to overcome through creating specific commands for such but yeah.
Also I remember there being significantly less requirement in SN for moving the insertion point to the correct place before dictating the actual math. With Lyx I remember having to constantly move the insertion point using ross, dunce etc whereas again SN seemed much more streamlined in that regard.
esc123
@esc123
Furthermore for me specifically , I have a touch screen (Surface Pro) and I am now using the pen to highlight math in SN I've already written so I can copy and paste repetitious math content from one line to the next and then just change the values you are calculating, very convenient for Linear Algebra say. This is quicker and puts less strain on the voice. This way I am using voice commands as an adjunct to this copy/paste with pen along with AutoHotKey scripts to further streamline repetitive tasks and common commands. Anyway, using the pen to highlight text in this way thankfully doesnt really do to much to RSI symptoms and SN + touch screen makes it very easy to do so. This wouldn't be the case with Lyx
Mike Roberts
@mrob95

@alexboche pretty much what @esc123 said. In scientific notebook you can go into a permanent math mode where everything you type is assumed to be math, regardless of new lines et cetera, whereas LyX requires you to constantly reset the mode and it is easy to accidentally jump out by going to the end of the line or w/e. Also in SN you can select and manipulate blocks of math easily. Since I mostly used it for bashing out homework assignments or completing exams the ease of getting stuff down in SN easily outweighed its lack of complex formatting options.

I think the main use case for LyX is as an alternative to raw LaTeX for technical writing, but since I know LaTeX pretty well and am comfortable in a text editor I don't really have a need for it.

btw if you are considering trying SN, then I would definitely recommend version 5.5 over version 6 if you can get it. Version 6 is much closer to the behaviour of LyX and I really think they have made it worse.

Anna Kirkpatrick
@annakirkpatrick
@mrob95 I've noticed in dictating LaTeX with mathfly that Greek letters always have a space after them by default while latin letters do not. Was this an intentional choice? While it doesn't cause any real problems, expressions like $\Omega _{k}$ in source are nonstandard from a style standpoint. In case it's not clear, $\Omega _{k}$ and $\Omega_{k}$ compile to the same typeset result, since LaTeX ignores most whitespace characters in math mode.
Alex Boche
@alexboche
in lyx, one puts a space after anything with a backslash in front of it to get it to render the character, so sounds like maybe that just got accidentally carried over to latex.
Mike Roberts
@mrob95
That is kind of deliberate, because an extra space usually doesn't cause compilation problems whereas omitting a space often can, I'm not 100% sure that it's the best solution though
esc123
@esc123
@mrob95 have you looked into Scientific Workplace at all? Seems to have the same capabilities as Scientific notebook but with much more formatting capabilities albeit for a much higher price.
It's disappointing what you said about SN 6.0. I hadn't looked into it but was thinking about the upgrade. Given what you said I'll most likely abandon that idea.
Mike Roberts
@mrob95

I've looked into it but didn't really fancy spending that much for something which I'm not sure I have a use for. The same commands should work though if scientific workplace were added as another context for the scientific notebook commands.

I downloaded a 30 day trial of version 6 and managed to get most of the commands working so if you wanted to give it a try I could make a pull request? I suspect that if you prefer SN to LyX then you will prefer the old version though. It's a shame because I really liked 5.5 and it is pretty much impossible for anyone to get hold of now without buying MathTalk.

esc123
@esc123
No I'll leave the trial of SN 6.0 at least until after my exams but it is a shame the software has developed that way as 5.5 is very useful. Regarding Workplace, I am in the same boat as you only using SN for homework and the like and wouldn't be sure if I would actually use what looks like full LaTeX formatting capabilities. The rough and ready has been adequate up to now.
Interestingly you said you use SN for exams as well. My Uni wouldnt allow that so I'm using a scribe instead. It's going to be a bit weird as I have never used one before and the dictation will be slightly different than the Mathfly lingo I've gotten accustomed to :P
Mike Roberts
@mrob95
Ah that is a shame, my uni were surprisingly blasé about me using my own PC and being in my own room for exams. I was all ready to have a big argument about it but they just said yeah that's fine do it however you want ^^. Do you know if you'd be able to meet the scribe first to lay down a few abbreviations and ways of describing things?
Then again a lot of the scribes I've met have been PhD students or similar so they may be able to interpret pretty well what you mean
Alex Boche
@alexboche
curious of if you guys have you made any use of the computer algebra system mupad? that sounds like it's integrated with scientific notebook/scientific workplace https://www.mackichan.com/index.html?cas.html~mainFrame
I think Anna mentioned she used a computer algebra system in one of her stackexchange posts?
Mike Roberts
@mrob95
Yeah the numerical evaluation is pretty useful for getting answers without having to type or copy stuff into another calculator program. I think it can do ODEs and stuff but that would be cheating ^^
esc123
@esc123
Yeah I use the compute function in SN frequently. It is very useful in saving the calculator grunt work
I found it particularly useful when checking my work in Linear Algebra for instance
Yeah I can meet up with the scribe a few mins before the exam but that's about it. They are hesitant for me to spend too much time with the scribe before the exam I presume so as to reduce likelihood of cheating perhaps. I was told that the scribe will have domain knowledge but other than that I'm just gonna hope for the best :D
Alex Boche
@alexboche

I think putting some math vocabulary in with dictation is very useful so we don't have to pause or use "say <dictation>" all the time.
Here's what I tried so far (including some of Mike's old code). Capturing dictation and then replacing some words with a dictionary (would be in a separate grammar that would be disabled unless you are doing math).

math_replacement_dictionary = {"math gamma": r"$\gamma$",
"math gamma below beta": r"$\gamma_\beta$"}
def math_replace(dictation, dictionary):
input_list = str(dictation).split(" ")
output_list = []
for i in range(len(input_list)):
if input_list[i] == "cap":
input_list[i+1] = input_list[i+1].title()
else:
output_list.append(input_list[i])
pre_math_string = " ".join(output_list)

output_string = copy.copy(pre_math_string)
for k, v in sorted(dictionary.items(), key=lambda pair: len(pair[0]), reverse=True):
output_string = output_string.replace(k, v)
Text(output_string).execute()

"<dictation>": Function(math_replace, dictionary=math_replacement_dictionary),

The first part of the function is just to getcapitalization for regular dictation. The reason I did the sorting stuff is because I want to make sure I am replacing the largestthing first. e.g. replacing "math gamma below beta" before "math gamma" since otherwise that would cause problems.
We can generate a dictionary for the math replacement dictionary. Let me know what people think. David zurow said he had some thoughts about a better way to do this sort of thing am hoping he will share.

I still need to get it to put in spaces at the beginning of each utterance unless there already is aspace which I guess is something that Dragon does automatically but is not done by this. A variety of things seem kind of clunky when all of your dictation is passed through this. e.g. scratch that doesn't work.
@annakirkpatrick you can have backslashes in the Dragon vocabulary word if you put them in the printed form which is in preferences under a word in the Vocabulary Editor. However, if you are using the approach above if you pass everything through that command, the printed form will be lost it will just write out the written form; in other words you can't do both approaches at the same time.

Alex Boche
@alexboche
this command would be non- CCR
Mike Roberts
@mrob95

I'm not sure it's worth it really. Losing dragon's built-in editing features is annoying and this would be vulnerable to slight misrecognitions of the command words. I don't think pausing before starting to dictate latex wastes too much time in the scheme of things.

Probably an easier way of accomplishing something similar would be to just dictate as normal until you have a document with numerous instances of "math gamma" and then just do a find and replace for all of them.

Alex Boche
@alexboche

By "Dragon 's built-in editing features" do you mean the commands like "insert before <dictation>" ? That still works with this but those don't usually work in math -based apps anyways. There are a few small issues regarding spacing that need to be worked out. for now I think I'm just going to have a space at the end of the text action. but spacing with things like "open paren" works though the spacing does not seem to work properly for custom Dragon vocabulary words or for "slash". I don't think the find and replace approach would be suitable if you have a lot of different symbols going around. The issue of misrecognitions is important; ideally this grammar would be given lower priority than the other grammars so that it would have the low priority that dictation usually has. Rather than just replacing words it might be more flexible to have a restricted set of commands available, especially for Lyx / scientific notebook where you need keystrokes not just text. I just posted David's suggestion on how to do this at the end of this issue dictation-toolbox/Caster#623 .

In general, this sort of thing would not be so much for a situation where you are doing heavy duty calculations but more for a situation with like 70-30 English-math symbols ratio or higher. I will be trying out this sort of thing in the wild myself so I will see how it goes. I think it will at least be good to have as an option which the user can easily enable/disable, but I will have a better sense after using for a while.

Alex Boche
@alexboche
adding printed forms many at a time is a little bit tricky. it appearss that the best method is to edit the XML file associated with thee vocabulary you are using. So far it seems like you might have to first export your vocabulary and save it as an XML file then edit the file and then reimport it because I cannot find where the XML file is stored within your user profile. Here is a sample XML file that shows how to add the printed forms for "math alpha" "math Alpha" and "math beta" to get $\alpha$ $\Alpha$ and $\beta$. https://gist.github.com/alexboche/c02af32574a40854d3ea2d4020a269f4
Let me know if anyone wants more info about that. Keeping the vocabulary in a .txt file is more compact then XML but does not support printed forms and thus does not support backslashes. In principle we can gennerate these XML files programmatically for all the combinations of letters and superscripts and so on.
Anna Kirkpatrick
@annakirkpatrick
@alexboche @mrob95 just some thoughts on the conversation above : While I have found it really helpful to add mathematical words and phrases to Dragon's vocabulary, I don't really see the value in trying to put those words into math fly itself (that is, as a new grammar). I understand what has been said above about the need to pause before and after dictating notation. I actually expected this to be really annoying when I first started using math fly, but I haven't found that to be the case. I've now written nine pages of math with Latex using math fly, and I haven't found the pauses to be particularly annoying. Since Latex already distinguishes text mode from math mode, it's pretty natural to do it in speech too. (I will also note that you would have to have an enormous grammar in order to include all of the words you might want to closely mix with math.)
Anna Kirkpatrick
@annakirkpatrick
A find and replace strategy was mentioned above. That is essentially the strategy that my previous system, NatLaTeX, used. It was slightly more complicated in that it did involve an actual parser that could, for example, distinguish math mode from text mode and apply different replacements in each context. So, the find and replace concept absolutely can work (and it eliminates all need to pause) but it also has some significant drawbacks. (The source is harder to read. Errors are harder to find. It's very easily for a slightly misrecognized word to be unnoticed when dictating but then not be replaced properly by the find and replace script. It can also make collaboration harder. It doesn't work well with text editors aimed at LaTeX. Keeping track of spacing/indentation is a nightmare.) I don't think mixing the grammar/command strategy used by math fly with the find and replace strategy is a good idea. I think you would end up with the limitations of both and quite a bit of confusion.
To address a couple other lingering points/questions:
-when it comes to computer algebra systems, I have mainly used sage (sagemath.org). I interacted with it mainly as plain text, sometimes in a Python-style interactive shell, sometimes writing scripts with a plain text editor and running from system shell or sage shell. There is a browser-based "GUI" but it mainly amounts to just another way to access the interactive shell. The ability to display graphs and typeset mathematics in the browser is nice, but otherwise it behaves like a shell.
-I totally forgot about printed forms! They are pain, but, yes, you can get backslashes that way.
I will briefly note that all of my comments above apply just to LaTeX. Using a GUI program like scientific notebook could be totally different. And there might be value in considering different strategies for these different contexts.
Mike Roberts
@mrob95

Pretty much agree. I think that the Dragon system of separating utterances into either dictation or commands is a fairly nice way of working, and it is easy to understand if it doesn't do what you expect. I would be hesitant to introduce a lot more complexity for the fairly minor benefit of reducing the number of pauses. I don't know about you but I pause quite a lot while I'm dictating anyway to think about what I need to say next and to make sure the previous section is correct.

One thing which may be of interest. There is a parallel voice control project called Talon which provides an API for defining grammars in the same way that Natlink + dragonfly does. Until now it has only been available for Mac but is currently being ported to Windows. One of the nice things which the Talon API exposes from Dragon is a parameter which controls how aggressive it is at finishing an utterance (how long of a pause it hears before it decides that you've finished talking). Reducing this would make the dictation -> pause -> commands cycle quicker.

Anna Kirkpatrick
@annakirkpatrick
@alexboche I finally got around to downloading some screen recording software and recording myself writing some code. https://youtu.be/BWwKV0W7d-k
Alex Boche
@alexboche
@annakirkpatrick Nice speed, that's impressive! Mind if I share that video on some other chat rooms?
Alex Boche
@alexboche
Nice you can talk while you're doing it
esc123
@esc123
@annakirkpatrick Awesome video. Great work on your keyboard layout as well!
Mike Roberts
@mrob95
Yeah very nice video. Interesting to see an alternative to dictation, I think I have probably been underestimating how good eye tracking is nowadays
Quintijn Hoogenboom
@quintijn
Hello Mike, I started with mathfly today, because I have to instruct a student who needs speech recognition and formula's. I must start by making my VERY BIG complements for this grammar. I also learned a bit more about dragonfly and caster, for me, stuck to Unimacro and Vocola, is in a lot of ways another way of thinking. Also thanks Alex Boche for putting me in the right direction.
I now have to see how to adapt for a Dutch user profile, as the student will be working in Dutch. To my pleasure the numbers behave well up-to-now! Making lots of translations in the config files should do most of the work. I am planning to use double words in many cases, so the user can also choose for the english variants when appropriate. The radio alphabet is happily the same.
Quintijn Hoogenboom
@quintijn
Mike, I tried to send you an email too, but the mail address on the instruction pages on your mathfly.org site seems to be not valid any more...