## Where communities thrive

• Join over 1.5M+ people
• Join over 100K+ communities
• Free without limits
##### Activity
• Jul 22 14:07
Danesprite synchronize #352
• Jul 22 14:07

Danesprite on cleanup-kb-code

Fix a regression with keyboard … (compare)

• Jul 21 15:45
Danesprite commented #350
• Jul 21 15:38
Danesprite commented #350
• Jul 21 15:36
Danesprite commented #350
• Jul 21 14:26
Danesprite assigned #350
• Jul 21 14:26
Danesprite assigned #352
• Jul 21 14:26
Danesprite labeled #352
• Jul 21 14:26
Danesprite labeled #352
• Jul 21 14:26
Danesprite opened #352
• Jul 21 14:09

Danesprite on cleanup-kb-code

Cleanup keyboard code (compare)

• Jul 11 19:14
wolfmanstout commented #326
• Jul 11 18:09
wolfmanstout commented #326
• Jul 11 15:44
Danesprite commented #310
• Jul 11 13:33
Danesprite commented #326
• Jul 10 20:45
lunixbochs commented #326
• Jul 10 20:25
wolfmanstout commented #326
• Jul 10 20:24
wolfmanstout commented #326
• Jul 10 20:20
lunixbochs commented #326
• Jul 10 19:56
wolfmanstout commented #326
westocl
@westocl
The text engine approach guarantees they cant ignore the microphones :)
LexiconCode
@LexiconCode
:)
Dane Finlay
@Danesprite

@westocl Ah, okay then, interesting use case!

If you haven't already, I would suggest looking at the source code for the python -m dragonfly test command. It shouldn't be too difficult to do this in your Python process instead.

Regarding window contexts for engine.mimic(), you can pass the remote window context to the method (see TextInputEngine.mimic()). This is specific to the text-input back-end. This is not really good enough for this sort of server-client interaction, however. For that, you would need to have Dragonfly use remote platform implementations of classes like Window and Keyboard.

You may also want to have a look at look at Aenea, which does something similar with Dragonfly. This would be easier IMO. I believe it still needs to be ported to Python 3, however.

sean-hut
@sean-hut
This is a test message. I sent using an IRC client and irc.gitter.im. Can someone respond if the message shows up in Gitter's web client. Thanks
Dane Finlay
@Danesprite
Loud and clear, Sean. :-)
sean-hut
@sean-hut
Danesprite: Thanks
westocl
@westocl
Quick question. I've been using WSR. I usually do not use the "on_failure" callback. However, if when I do, notice that, when a successful recognition happens, it seems like the sequence is , "on_before", "on_failure", "on_recognition", "on_post_recognintion". Why would "on_failure" ever run on a successful recognition? Is this a WSR bug?
Dane Finlay
@Danesprite

Hi @westocl,

Yes, this sounds like a SAPI 5/WSR bug; on_failure() shouldn't be called if the recognition was successful. I am not able to replicate this at the moment, however.

@/all Dragonfly2 version 0.31.0 has now been released. The changes and fixes are listed in the changelog here: https://dragonfly2.readthedocs.io/en/stable/changelog.html

You can upgrade by running pip install --upgrade dragonfly2.

Thanks very much to everyone who contributed!

westocl
@westocl
@Danesprite just read the change log
"Change the Choice element class to allow using list/tuple choices."
Thank you guys for putting this in!!
Dane Finlay
@Danesprite
@westocl No worries.
The documentation for Choice has been updated with info on that: https://dragonfly2.readthedocs.io/en/stable/elements.html#dragonfly.grammar.elements_compound.Choice
DanKaplanSES
@DanKaplanSES
I'm using this question as an example. I don't know if my question is due to python ignorance, or an opportunity to improve the documentation. How do I figure out what I need to import to use this action? https://dragonfly2.readthedocs.io/en/latest/actions.html#pause-action
DanKaplanSES
@DanKaplanSES
Another suggestion: maybe this should say what the numbers represent. I'm assuming it's the same as Pause
2 replies
DanKaplanSES
@DanKaplanSES
and another question: I have this command that is going to run for many seconds. Is there a way I can make it interruptible?
this is the command:
        'youtube seek':
R(Key("1/200, 2/200, 3/200, 4/200, 5/200, 6/200, 7/200, 8/200, 9/200")),
LexiconCode
@LexiconCode
With Casters Asynchronous actions you can make it interruptible.
DanKaplanSES
@DanKaplanSES
oops, that was a caster question anyway :)
Dane Finlay
@Danesprite
Cancelling or invalidating recognitions is a little different than asynchronous actions that can be interrupted. I haven't looked into this, but I guess you would do it with threads and signals.
Hawkeye Parker
@haughki
OT: on Windows, looking for a lightweight (not MSWord) Select and Say editor, multiple undo (not Notepad), doesn't always revert to double-spacing (not WordPad). Just want to make sure I'm not missing something obvious.
This is for Dragon.
Alex Boche
@alexboche
I think there was a thing called Dagonpad that came with Dragon. But for prose writing, mostly Dragon native select and say is just worse than a combination of sophisticated navigation commands and eyetracking. Eyetracking is probably best for long jumps then can use commands to move within a line or small number of lines. Imitation select and say (wolfmanstout accessibility api approach/ocr, Caster, or Talon probably has something). Others can elaborate.
Dane Finlay
@Danesprite
@haughki As Alex mentioned, DragonPad might be what you want. You can open it via the Dragon Tools menu or by saying "open DragonPad".
Hawkeye Parker
@haughki

@alexboche @Danesprite Tx: For some reason, I gave up on DragonPad a long time ago. I'll try again and maybe it will be great, or maybe I will remember why I stopped using it before :)

@alexboche I hear you. I don't use eye tracking or ocr, but for coding, I have a ton of navigation commands -- AceJump, gotoline, scanline, etc. But, for prose dictation, some apps are terrible: slack, gmail -- those are the two I can think of off the top of my head -- often insert double letters during dictation (at least for me), and other weirdness. Other than that, I don't have any solid data, but it feels to me like Select-and-Say applications have overall better recognition, even if I don't use any of the "correct" commands. I do find the "insert before/after" commands to be really useful to quickly get the cursor exactly where I want. Hrm. Maybe it's time to try eye tracking and/or ocr.

Ryan Hileman
@lunixbochs

often insert double letters during dictation

that's a dragon bug

they send multiple keydowns but only one key up, I think it's a timer that fires too fast or something
Dane Finlay
@Danesprite
Ah, fair enough. I don't use DragonPad either. I get by just fine with Emacs. It doesn't have Select-and-Say support though.

James Stout contributed some accessibility / Select-and-Say functionality to Dragonfly a little while ago. You can read about it here: https://handsfreecoding.org/2018/12/27/enhanced-text-manipulation-using-accessibility-apis/

He also has a few posts on and code for OCR + eye tracking if that is something you want to look into: https://handsfreecoding.org/2020/07/25/say-what-you-see-efficient-ui-interaction-with-ocr-and-gaze-tracking/

Alex Boche
@alexboche
Yeah I suppose Dragon select and say has some advantages. The fact that it uses the words on the page to recognize is nice. The annoying problem for me is that you have to pause before and after. I poured my soul in a clipboard select and say approach in Caster that works in almost any app; I don't know if anyone ever used it :(
I actually felt making corrections was occasionally worth doing for common phrases (if you can manage to not have your profile randomly deleted!).
I think there was a rumor that notepad++ was select and say but it didn't seem to work for me last time i checked.
When I was on Windows I found the speech productivity dictation box pretty cool. Rudiger had another type of dictation box with seemless auto transfer that seemed cool.
Its always worth seeing what people are up to on talon slack. People have ideas there.
Anatole Matveief
@amatveie
@haughki over on the know-brainer forum, in addition to the Dragon capture which Rudiger built and is mentioned by Alex, there's another individual who has developed an alternative to the dictation box that gets pretty good reviews https://www.knowbrainer.com/forums/forum/messageview.cfm?catid=25&threadid=35489&enterthread=y . It might be worth checking out. Select-and-Say definitely does a better job during dictation. In particular it gets the spacing right and usually gets the capitalization rate as well which is a big help. With regards to Notepad++ there is a hack for the registry hack, also documented in 1 of those forum threads, that allows it to get some Select-and-Say functionality, but I don't think it's perfect. If you are adventurous and able to install natlink you could also try Mark's vortex code which will generally enable Select-and-Say for more apps. https://github.com/mdbridge/Vocola-2/tree/vortex . I'm surprised that you have a hard time with Gmail as it seems to work pretty well with me in both chrome and edge. For Slack what I do is use it inside of the web browser, which allows for Select-and-Say vs the Windows app which does not. (I actually go one step further and have it as an 'App' in my taskbar from MS Edge. (Just click on the ... menu->Apps->install the site as an app). In other news there's an effort underway trying to get better dictation support for other apps and speech engines by using the accessibility APIs. But that's certainly not ready for prime time.
Hawkeye Parker
@haughki

@amatveie @alexboche Thank you. I have tried Rudiger's app (Dragon Capture) and SP Pro. Neither of them did exactly what I wanted. If I remember correctly, I hit some crashing bugs with SP. I can't remember exactly what it was with Dragon Capture, but it was a show stopper for me, at least.

Gmail/slack: I think you're talking about the Dragon browser plugin? As I remember, that was truly horrible: resource hogging, crashing. But, I'm on Chrome. I'm so deep in the Google ecosystem, that I'm not sure I could switch to MS Edge -- but, if the plugin really works that much better there, it might be worth it.

James Stout's accessibility API work looks amazing, but more than I want to tackle right now; similar feelings about Vortex.

Honestly, and Notepad works well for me %99 of the time: it's superfast, super lightweight, super stable. It's quick for me to switch back and forth, and I have a "transfer <app>" command which just copies everything and focuses the target application.

So, DragonPad may be it for me for now. Hilarious.

Hawkeye Parker
@haughki
So, just to wrap this thread up (on my end) I've been using DragonPad for all of 30 minutes, and it's buggy as hell in my environment. It's inserting random characters at the end of sentences. If I have more than a few lines of text with some newlines, and I go back and edit "above" in the text, sometimes it just starts inserting my dictation at seemingly random places later in the document. I'm not kidding. So, nonstarter for me.
Anatole Matveief
@amatveie
@haughki Yes DragonPad was completely non workable the last time I tried it. You can also look at James's solution for using notepad as the dictation box https://handsfreecoding.org/2015/08/30/avoid-the-dictation-box/ also note that if the standard Dragon dictation box takes a long time to load up, you could experiment with the keys to send for copy and paste in the settings. I have found that using the Windows copy and paste commands directly versus simulating ctrl+c tends to bring it up quite a bit faster, but will fail in certain applications. With regards to edge versus chrome, in my experience chrome was becoming such a memory hog/CPU hog that was unworkable. For whatever reason the browser plug-in hasn't really given me many problems although it did years ago, so maybe they've been ironing out the bugs? I'm on Dragon 15.61 and I vaguely remember the release notes talking about the browser especially edge since they made the recent change to chromium as the underlying rendering engine. Also maybe there's a setting to do this in chrome, but I find that edge is pretty adamant about putting background tabs to sleep which really helps.
tripfish
@tripfish
@haughki If nothing helps, you can also create a simple dictation box windows application. The RichTextBox element also supports Select-and-Say. I tried it a long time ago. Couldn't find any problems. But I haven't tested it for long.
James Stout
@wolfmanstout

PSA: UI Automation is backwards-compatible with IAccessible2 as of the May 2019 Windows 10 update:
https://docs.microsoft.com/en-us/windows/win32/accessibility/accessibility-whatsnew

This means that my Dragonfly integration with IAccessible2 could be completely replaced with pywinauto and would have greater compatibility. Without having dug into pywinauto, I'm guessing this would be a change for the better from a code cleanliness standpoint because the library I'm currently relying on (pyia2) is very janky.

8 replies
Quintijn Hoogenboom
@quintijn
What is wrong with DragonPad? (For Dragon users)... There is also still WordPad, a light weight word version around. I shall try to revive my Unimacro grammar "_edit.py", https://qh.antenna.nl/unimacro/grammars/globalgrammars/edit/index.html. So: not working at the moment. The idea was always to switch to DragonPad (or other program with full dictation control), either empty, or with a paragraph copied, or with the whole original document copied. With "edit ready" the text is pasted back in the original window. I don't know why I left this grammar, it worked quite nice.
6 replies
LexiconCode
@LexiconCode
What kind of dictation would cause the following error?INFO:dictation.word_parser_factory: Selected word parser class WordParserDns11 because natlink.getWordInfo('.\\dot') returned None Reviewing the dragonfly code it looks like it's an issue handling Dragon special formatting?
Dane Finlay
@Danesprite

Dragonfly2 version 0.31.2 has been released, as of a few days ago. A number of issues with keyboard input on Windows are fixed in this version.

If anyone has had trouble with the Key or Text actions not working properly on Windows, then the new version will probably fix it. This will be relevant mostly to those using keyboard layouts other than QWERTY.

LexiconCode
@LexiconCode
Thanks Dane! That's great!
MeteorVsBunny
@MeteorVsBunny
Key("apps") does not work for me in 31.2, it does work in 31.1
I'm using Windows, caster and Dragon
Dane Finlay
@Danesprite
@MeteorVsBunny Thanks for reporting this! Clearly I should have tested my changes a bit more! Apparently that key, along with some others, is what is called an extended key. I will fix this in the next version.
Dane Finlay
@Danesprite

@MeteorVsBunny That is fixed on the master branch now if you need it, though I'll release version 0.31.3 in the next day or so.

Hopefully this is the last thing wrong with Dragonfly's Windows keyboard code. 🤞

MeteorVsBunny
@MeteorVsBunny
thanks, that was fast
Dane Finlay
@Danesprite
No worries. Well, it wasn't a difficult fix. :-)
DanKaplanSES
@DanKaplanSES
if I use the dictation box and correct the text, then cancel, does that still help the ai?
LexiconCode
@LexiconCode

if I use the dictation box and correct the text, then cancel, does that still help the ai?

My guess the data for the correction is saved before you say cancel to the closeout box.

Is there a way to pass a parameter through function context?
Dane Finlay
@Danesprite

Is there a way to pass a parameter through function context?

Well, you can pass default values FuncContext(function, text=""). What is your use case?

Dane Finlay
@Danesprite
@LexiconCode Ah, okay then. You could have a get_grid_mode_context() function for this:
def get_grid_mode_context(grid):
return FuncContext(lambda: get_grid_mode(grid))