Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Apr 04 04:17
    Danesprite labeled #367
  • Apr 04 04:17
    Danesprite assigned #367
  • Apr 04 04:17
    Danesprite opened #367
  • Apr 04 04:02

    Danesprite on master

    Remove unnecessary install requ… (compare)

  • Apr 04 04:02

    Danesprite on major

    Remove multiple engine options … Remove the dragonfly.util sub-p… Make dragonfly.accessibility su… (compare)

  • Mar 31 05:17

    Danesprite on master

    Clarify sections of the documen… (compare)

  • Mar 31 05:17

    Danesprite on major

    Remove dragonfly.rpc sub-packag… (compare)

  • Mar 23 11:43
    Danesprite closed #354
  • Mar 23 11:43
    Danesprite commented #354
  • Mar 22 10:39
    Danesprite closed #337
  • Mar 22 10:39

    Danesprite on master

    Update sections of the document… (compare)

  • Mar 20 06:09

    Danesprite on master

    Remove unused test suite module (compare)

  • Mar 20 06:02

    Danesprite on minor

    Fix Kaldi marking of whether re… Add Kaldi optional local or onl… Update Kaldi version, to fix er… and 32 more (compare)

  • Mar 20 06:01

    Danesprite on major

    Remove training mode from the S… Remove built-in sleep/wake mode… (compare)

  • Mar 20 05:36

    Danesprite on talon-dev

    (compare)

  • Mar 19 11:42

    Danesprite on master

    Add missing documentation file … (compare)

  • Mar 19 11:36
    Danesprite closed #240
  • Mar 19 11:36
    Danesprite labeled #240
  • Mar 19 11:35

    Danesprite on 0.35.0

    (compare)

  • Mar 19 11:35

    Danesprite on master

    Update changelog Update to version 0.35.0 Pleas… (compare)

Dane Finlay
@Danesprite
I should mention that you can use the DNS dictation formatting classes to save state information between utterances. Dragonfly doesn't do that because it would lead to some commands having inconsistent effects.
Alex Boche
@alexboche
i made an issue about that formatting stuff a while back #110 Not sure if anything there is relevant to what you're talking about. I think there was also a caster issue. Last I checked the talon community seemed to be pretty die hard into just using various "say [dictation]" type commands for everying instead of fancing transitioning or using separate modes. They may be right, especially with super low latency where pausing isn't so bad.
LexiconCode
@lexicon-code:matrix.org
[m]
natlink 5.1.2 has been released in which you can give an environment variable DICTATIONTOOLBOXUSER pointing to a directory in which the folder .natlink can be located. Simply re-register natlink after setting up the variable
Doug Ransom
@dougransom
@lexicon-code:matrix.org can you note that in the readme.md? otherwise it will be a big secret.
LexiconCode
@LexiconCode
@dougransom Sure in Natlink docs?
Quintijn Hoogenboom
@quintijn
I will work on the readthedocs, yes!
LexiconCode
@LexiconCode

@LexiconCode Formatting is done after observer events. To do it yourself, pass the unformatted words list to Dragonfly's DNS dictation word formatter classes:

from dragonfly.engines.backend_natlink.dictation_format import WordFormatter

words = "\\all-caps\\All-Caps hello world".split()
formatter = WordFormatter()
formatted_output = formatter.format_dictation(words)
print(formatted_output)

There are more examples in the doctests: https://dragonfly2.readthedocs.io/en/latest/test_word_formatting_v11_doctest.html

This is not done for integers. IIRC DNS does not provide this information to client applications. You need to pass the words through Dragonfly's integer content classes for that.

This solves formatting the problem of free dictation with DNS and commands are not a problem. However the dictation element remains troublesome. The purpose of displaying things on a HUD would be what you see is what you got for output on the screen. If any formatting has been applied to a dictation element it of course does not show as it's not applied yet. Any thoughts to overcome the issue with dictation elements?

1 reply
Quintijn Hoogenboom
@quintijn
Did you ever try the nsformat module in natlinkcore? This was originally written by Joel Gould, to solve the formatting of the <dgndictation> imported rule
LexiconCode
@LexiconCode

Did you ever try the nsformat module in natlinkcore? This was originally written by Joel Gould, to solve the formatting of the <dgndictation> imported rule

@quintijn That's an interesting thought as I'm already importing DNS the specific code so why not Natlink directly. Thank you.

Quintijn Hoogenboom
@quintijn
@LexiconCode : I checked nsformat.py with unittestNsformat.py. One failure, that was in 2010 already a failure. I did only commit to "makeflitinstall". If you want an extra release, I will make it...
LexiconCode
@LexiconCode

@LexiconCode : I checked nsformat.py with unittestNsformat.py. One failure, that was in 2010 already a failure. I did only commit to "makeflitinstall". If you want an extra release, I will make it...

When you get a chance it would be good. It's a bit more reliable specifically with the 1st letter in a sentence compared to dragonfly's formater

Quintijn Hoogenboom
@quintijn
I will do so beginning of next week. When you have the preceding text (one character) available, you can start with the appropriate option. I will go through this and work on the documentation...
Rebecca Stevenson
@rebeccaFste_twitter

Hello! I'm new to this chat and first off wanted to thank everyone who has contributed to the dragonfly toolbox or shared their command modules – I'm a researcher and after a bad case of RSI would have been pretty screwed without some way to code by voice. Seriously, thank you! I've recently been trying to find/develop more command modules (right now I'm mostly using _vim and _multiedit) but I'm not a developer and am running into a few issues.

For example, I can't get _bringme to pull up a specific folder (https://github.com/simianhacker/code-by-voice/blob/master/macros/_bringme.py). It works fine for websites and for opening documents but not for opening folders. I'm also on the hunt for a command module to use with slack (for whatever reason my set up fails miserably in this app) – if anyone has a lead it would be much appreciated.

Not sure if this is the best place to post these questions – if there is another forum geared towards more basic questions please let me know!

Shervin Emami
@shervinemami
Hi @rebeccaFste_twitter , welcome to our community! Sorry to hear you've got RSI but glad you found us. This is the right place to ask beginner questions, even if we are a mix of beginners and experts and in-between. Are you using Windows, Mac and/or Linux?I haven't tried the _bringme functionality yet but I know some others here have. I do use a grammar that was based off _multiedit but I've customised my one over time.
LexiconCode
@LexiconCode
@rebeccaFste_twitter Welcome! I'd be willing to help you make slack grammar. I will be available for a few days as I work but I can check in a little later.
Dane Finlay
@Danesprite

Hello Rebecca and welcome! Sorry to hear about the RSI. It can be very painful and frustrating. Glad you've found your way here.

For opening folders using the _bringme.py module, I have managed to get it working. As the module's documentation specifies, you need to add folder entries to the configuration. However, the author is slightly wrong about the use of the r in front of the open-quote. This doesn't work for strings ending with backslashes, e.g. "C:\".

Try the following targets.mapping entries in your configuration:

"my local folder":       folder(r"C:"),  # Without a backslash.
"program files":         folder(r"C:\Program Files"),
"macro system":          folder(r"C:\Natlink\Natlink\MacroSystem"),
Rebecca Stevenson
@rebeccaFste_twitter
Thanks for the welcome everyone! Glad I joined.
@Danesprite, that does the trick, thank you! I just started using the _bringme module a few weeks ago and it makes life so much easier. Glad to have the folder functionality working
1 reply
Rebecca Stevenson
@rebeccaFste_twitter
@LexiconCode , oh wow thank you! I found one bare-bones module here (https://github.com/haughki/MyDragonflyModules/blob/master/slack.py) but I suspect my real issue might be that some kind of built-in shortcut in slack is interacting with my set up and messing things up (it always does unexpected things e.g selecting the menu etc). I guess I can deactivate my current grammars one at a time to try to track down the culprit, if that makes sense.
@shervinemami, I use Windows. As I mentioned above, I just started using _bringme and it is pretty magical. Highly recommended
MeteorVsBunny
@MeteorVsBunny

the commands in that grammer do not really look like slack to me
I have a slack grammar but I am using caster rather than dragonfly so if I shared the whole file it would not work for you. And I'm not familiar with setting up dragonfly only files
I replaced the commands in that file with my own, which should work but I have not tested it
control slash in Slack brings up the list of shortcuts so you can map any commands you want

from dragonfly import (Grammar, AppContext, MappingRule, Integer, Key, Text, Dictation, Repeat)
from supporting import utils

class SlackMapping(MappingRule):
mapping = {
"next [message]": Key("as-down"),
"prior [message]": Key("as-up"),
"open channel": Key("c-k"),
"next channel [<n>]": Key("a-down/15") Repeat(extra="n"),
"prior channel [<n>]": Key("a-up/15")
Repeat(extra="n"),
"keyboard": Key("c-slash"),
"info": Key("cs-i"),
"jump": Key("c-j"),
"attach file": Key("c-u"),
"edit last [message]": Key("c-up"),
"code block": Key("cas-c"),
"quit slack": Key("c-q"),

}
extras=[
    Integer("n", 1, 50),
    Integer("number", 1, 9999),
    Dictation("text"),
]
defaults = {
    "n": 1,
}

context = AppContext(executable="slack")
slack_grammar = Grammar("Slack Grammar", context=context)
slack_grammar.add_rule(SlackMapping())
slack_grammar.load()

def unload():
global slack_grammar
slack_grammar= utils.unloadHelper(slack_grammar, name)

Rebecca Stevenson
@rebeccaFste_twitter
@MeteorVsBunny thanks! I now have a functioning grammar
Rebecca Stevenson
@rebeccaFste_twitter
I also think I figured out which of my other grammars was screwing things up – it seems like the "release" function in multiedit somehow selects the menu in the upper left corner. No idea why. Not sure how to get around this. I tried just getting rid of the "release +" part preceding Key() but it seems "release" is maybe built into the grammar (i.e. line 268 here: https://github.com/t4ngo/dragonfly-modules/blob/master/command-modules/_multiedit.py).
Rebecca Stevenson
@rebeccaFste_twitter
Or something else in multiedit is causing the issue… Not sure it is "release"
David Zurow
@daanzu
@rebeccaFste_twitter I think some apps can do weird things when you release a key that wasn't actually being held, especially the modifier keys
samVanDerStoop
@samVanDerStoop
Im building some commands based on Utter Command's Human-Machine Grammar in dragonfly myself, but I could use some help. I want to avoid accidental command use in free dictation by avoiding one word commands (min two words) while also keeping utterances short, so I dont mind saying "one left" instead of just "left" to move a cursor. But I would like to be able to combine commands so that I can say "left backspace" instead of "one left one backspace", anyone have a good idea of how to have "1" be the default optional value in continous combined commands but have it be non-optional and required for single action commands?
LexiconCode
@LexiconCode

I also think I figured out which of my other grammars was screwing things up – it seems like the "release" function in multiedit somehow selects the menu in the upper left corner. No idea why. Not sure how to get around this. I tried just getting rid of the "release +" part preceding Key() but it seems "release" is maybe built into the grammar (i.e. line 268 here: https://github.com/t4ngo/dragonfly-modules/blob/master/command-modules/_multiedit.py).

There are some comments in the code that you might experiment with related to releasing the keys https://github.com/dictation-toolbox/dragonfly/blob/0a0eae5668baa0f5953972d5c9b5692f6c246309/dragonfly/actions/action_key.py. One thing I would recommend experimenting with is adding a pause before In and see it makes any difference sometimes there's timing issues. Rather than releasing X modifier you could really release all modifier keys without issues. Key("shift:up, ctrl:up, alt:up, win:up").execute() makes it a bit simpler instead of keeping track for every key that's been pressed.

Rebecca Stevenson
@rebeccaFste_twitter
Thanks @LexiconCode ! I'll give that a try. It also occurred to me that I could just switch to using slack on my browser which seems to work great :-)
LexiconCode
@LexiconCode

anyone have a good idea of how to have "1" be the default optional value in continous combined commands but have it be non-optional and required for single action commands?

I would also be curious in this as well as I hope to migrate the Caster repository leveraging Kim's method Utter Command's Human-Machine Grammar. However I don't think it's possible to do that dynamically.

timoses
@timoses:matrix.org
[m]
How to use alternative dictation in kaldi with another model?
Apparently one can pass a callback function as the alternative dictation method. How would that callback look using another kaldi model?
Would it be a second KaldiEngine object?
David Zurow
@daanzu
@timoses:matrix.org You would use kaldi-active-grammar directly, using a PlainDictationRecognizer instance
@timoses:matrix.org If you're feeling experimental, try plugging in https://github.com/daanzu/wav2vec2_stt_python
hakan ergin
@hakanErgin
hi, how can i press numpad slash button in a custom rule?
edit: nevermind, its npdiv (didnt see :))
LexiconCode
@LexiconCode
python -m dragonfly load _*.py --engine kaldi --engine-options " \ auto_add_to_user_lexicon=False"
usage: python -m dragonfly load [-h] [-e ENGINE] [-o ENGINE_OPTIONS]
                                [--language LANGUAGE] [-n]
                                [--no-recobs-messages]
                                [-l {DEBUG,INFO,WARNING,ERROR,CRITICAL}] [-q]
                                [file [file ...]]
python -m dragonfly load: error: argument -o/--engine-options: invalid _engine_options_string value: ' \\ auto_add_to_user_lexicon=False'
Does not work either --engine-options "auto_add_to_user_lexicon=False"
I'm assuming there's some difference between Windows and Linux here. If so that does need to be documented.
Dane Finlay
@Danesprite
Is this with Python 2.7?
2 replies
Dane Finlay
@Danesprite
I've released dragonfly2 version 0.32.1, which includes fixes for various keyboard input bugs. You can upgrade with the following command:
pip install --upgrade dragonfly2
tripfish
@tripfish
:+1:
samVanDerStoop
@samVanDerStoop

Im trying to write a dragonfly command using a function with the clipboard toolkit but Im having some trouble. I want to 1) backup whatever is already on the clipboard 2) copy a new line of text that my cursor is in to the clipboard to read and manipulate it (print for now) 3) then return original backed up content to system clipboard. I tried to use the examples in the docs, but these wont work:

    original = Clipboard(from_system=True)
    print(original.get_system_text())

    Key("home, s-down, c-c").execute() # fails override system clipboard
    clipboard = Clipboard(from_system=True)  # Retrieve from system clipboard.
    current_line = clipboard.get_system_text()
    print(current_line)

    original.copy_to_system()

Weirdly enough, this seems to work every other time, it fails, works, then fails again. Or so it seems to me:

    original = Clipboard(from_system=True)
    print(original.get_system_text())

    clipboard = Clipboard()        # Create empty instance.
    Key("home, s-down, c-c").execute() # mostly fails to override system clipboard
    clipboard.copy_from_system()   # Retrieve from system clipboard.
    current_line = clipboard.get_system_text()
    print(current_line)

    original.copy_to_system()
Alex Boche
@alexboche
@samVanDerStoop I don't have a direct answer to your question atm, but this is the sort of functionality that was developed a lot in Caster docs, code. You may want to build on some of that.
samVanDerStoop
@samVanDerStoop
@alexboche Interesting, Im definitely going to dive into Caster at some point. For now I would like to build a good understanding of the more basic dragonfly buildingblocks. I think having a solid grasp of those will help in the proces of tweeking such (Caster) grammars and building my own later on
LexiconCode
@LexiconCode
@samVanDerStoop this is tricky. Understand that the clipboard was never meant to be a buffer. However outside of accessibility APIs it's the most accessible solution. There are number different types of timing issues working with the clipboard. Primarily applications can have different timings depending on how they interact with clipboard. In addition only one application can access the clipboard at once. Experiment adding pauses before and after different parts of your code.
there's been some enhancements made to dragonfly's clipboard class I have not take advantage in the following but it does work. https://github.com/dictation-toolbox/Caster/blob/d03ccd49ef3843a35bf2d2b60a732cd9a9521a40/castervoice/lib/context.py#L107
samVanDerStoop
@samVanDerStoop
@LexiconCode I see. Ive been looking through the read_selected_without_altering_clipboard function in your link. Can you explain why you use Clipboard directly when retrieving system_text in the first line of loop (prior_content = Clipboard.get_system_text()) instead of the specific instance cb with from_system=True (cb = Clipboard(from_system=True)) created earlier? Also should I always set the system_text to an empty string (Clipboard.set_system_text("")) before trying to overwrite clipboard contents? What does that do? Thanks for the help!
LexiconCode
@LexiconCode
@samVanDerStoop I didn't originally write those classes. I'm not sure why the original author didn't use the single instance of cb. I believe get_system_text is utilized because it's used as a comparison checking against the temporary contents which is also text in line 144 prior_content == temporary. Some enhancements in the last couple months to dragonfly's clipboard class now allows comparing clipboard instances.
Dane Finlay
@Danesprite
@samVanDerStoop You may find the synchronized_changes() and wait_for_change() methods added in version 0.32.0 helpful for what you're trying to do. They are documented on this page and are the enhancements Aaron alluded to above.
samVanDerStoop
@samVanDerStoop
@Danesprite @LexiconCode Awesome, Ill look into those!
Dane Finlay
@Danesprite
:+1:
samVanDerStoop
@samVanDerStoop
@Danesprite Ive had success using the methods. Thanks.
Dane Finlay
@Danesprite
@samVanDerStoop Great!