Danesprite on doc-changes
Danesprite on master
Make mention of AGPL v3 license… Merge pull request #356 from di… (compare)
Danesprite on doc-changes
Make mention of AGPL v3 license… (compare)
Hmmm... yea i have been thinking about setting this up for a couple of months now and finally got some time to do it.
I plan on getting it set up soon even if the first verion is all hacked together.
Right now i have two applications both with the same grammars. On the client side, I stripped away all the mapping functions and process_recogningion functions.
On sapi5 recognition, client it pings the server. The server runs the text engine and the server replies back with a string that the client uses for its next grammar context function.
Its crude but works pretty well actually.
Right now both the client and server are on the same pc, but looks like its gonna work. When im done i will put the server on a website.
I'll let you guys know when i get something up running.
@westocl Ah, okay then, interesting use case!
If you haven't already, I would suggest looking at the source code for the
python -m dragonfly test command. It shouldn't be too difficult to do this in your Python process instead.
Regarding window contexts for
engine.mimic(), you can pass the remote window context to the method (see TextInputEngine.mimic()). This is specific to the text-input back-end. This is not really good enough for this sort of server-client interaction, however. For that, you would need to have Dragonfly use remote platform implementations of classes like
You may also want to have a look at look at Aenea, which does something similar with Dragonfly. This would be easier IMO. I believe it still needs to be ported to Python 3, however.
@/all Dragonfly2 version 0.31.0 has now been released. The changes and fixes are listed in the changelog here: https://dragonfly2.readthedocs.io/en/stable/changelog.html
You can upgrade by running
pip install --upgrade dragonfly2.
Thanks very much to everyone who contributed!
Choicehas been updated with info on that: https://dragonfly2.readthedocs.io/en/stable/elements.html#dragonfly.grammar.elements_compound.Choice
'youtube seek': R(Key("1/200, 2/200, 3/200, 4/200, 5/200, 6/200, 7/200, 8/200, 9/200")),
@alexboche @Danesprite Tx: For some reason, I gave up on DragonPad a long time ago. I'll try again and maybe it will be great, or maybe I will remember why I stopped using it before :)
@alexboche I hear you. I don't use eye tracking or ocr, but for coding, I have a ton of navigation commands -- AceJump, gotoline, scanline, etc. But, for prose dictation, some apps are terrible: slack, gmail -- those are the two I can think of off the top of my head -- often insert double letters during dictation (at least for me), and other weirdness. Other than that, I don't have any solid data, but it feels to me like Select-and-Say applications have overall better recognition, even if I don't use any of the "correct" commands. I do find the "insert before/after" commands to be really useful to quickly get the cursor exactly where I want. Hrm. Maybe it's time to try eye tracking and/or ocr.
James Stout contributed some accessibility / Select-and-Say functionality to Dragonfly a little while ago. You can read about it here: https://handsfreecoding.org/2018/12/27/enhanced-text-manipulation-using-accessibility-apis/
He also has a few posts on and code for OCR + eye tracking if that is something you want to look into: https://handsfreecoding.org/2020/07/25/say-what-you-see-efficient-ui-interaction-with-ocr-and-gaze-tracking/
@amatveie @alexboche Thank you. I have tried Rudiger's app (Dragon Capture) and SP Pro. Neither of them did exactly what I wanted. If I remember correctly, I hit some crashing bugs with SP. I can't remember exactly what it was with Dragon Capture, but it was a show stopper for me, at least.
Gmail/slack: I think you're talking about the Dragon browser plugin? As I remember, that was truly horrible: resource hogging, crashing. But, I'm on Chrome. I'm so deep in the Google ecosystem, that I'm not sure I could switch to MS Edge -- but, if the plugin really works that much better there, it might be worth it.
James Stout's accessibility API work looks amazing, but more than I want to tackle right now; similar feelings about Vortex.
Honestly, and Notepad works well for me %99 of the time: it's superfast, super lightweight, super stable. It's quick for me to switch back and forth, and I have a "transfer <app>" command which just copies everything and focuses the target application.
So, DragonPad may be it for me for now. Hilarious.
PSA: UI Automation is backwards-compatible with IAccessible2 as of the May 2019 Windows 10 update:
This means that my Dragonfly integration with IAccessible2 could be completely replaced with pywinauto and would have greater compatibility. Without having dug into pywinauto, I'm guessing this would be a change for the better from a code cleanliness standpoint because the library I'm currently relying on (pyia2) is very janky.
INFO:dictation.word_parser_factory: Selected word parser class WordParserDns11 because natlink.getWordInfo('.\\dot') returned NoneReviewing the dragonfly code it looks like it's an issue handling Dragon special formatting?