KnowBrainer Speech Recognition
Decrease font size
Increase font size
Topic Title: MyKBCommands_withAmendedCommands.xml
Topic Summary: Why/when is MyKBCommands_withAmendedCommands.xml created?
Created On: 11/05/2020 07:24 PM
Status: Post and Reply
Linear : Threading : Single : Branch
 MyKBCommands_withAmendedCommands.xml   - Ag - 11/05/2020 07:24 PM  
 MyKBCommands_withAmendedCommands.xml   - Edgar - 11/06/2020 11:12 AM  
 MyKBCommands_withAmendedCommands.xml   - Ag - 11/06/2020 12:06 PM  
 MyKBCommands_withAmendedCommands.xml   - Lunis Orcutt - 11/06/2020 04:15 PM  
 MyKBCommands_withAmendedCommands.xml   - monkey8 - 11/08/2020 08:42 AM  
 MyKBCommands_withAmendedCommands.xml   - Ag - 12/15/2020 11:54 PM  
 MyKBCommands_withAmendedCommands.xml   - monkey8 - 12/17/2020 04:02 AM  
 MyKBCommands_withAmendedCommands.xml   - Edgar - 12/17/2020 12:17 PM  
 MyKBCommands_withAmendedCommands.xml   - Ag - 12/17/2020 04:18 PM  
Keyword
 11/05/2020 07:24 PM
User is offline View Users Profile Print this message

Author Icon
Ag
Top-Tier Member

Posts: 518
Joined: 07/08/2019

I do quite a bit of coding of KnowBrainer commands.  Mostly adding and editing new commands of my own.

 

As is standard software engineering practice, I keep track of changes to my KnowBrainer commands using a version control system.  In this case, I copy MyKBCommands.xml from where KnowBrainer keeps it , and then check it in using git.

 

This has worked fine for more than a year.  Changes I make to KnowBrainer commands using KBCE (the KnowBrainer Command Editor) can be seen in MyKBCommands.xml, git diffed, etc.

 

However, once in a while, and recently the last 3 or 4 times in a row, my changes have NOT been reflected in MyKBCommands.xml.  

 

Instead, I find my changes in MyKBCommands_withAmendedCommands.xml

 

I often can recover any such changes from the AmendedCommands file, recreate the missing KnowBrainer commands, and then take care to find MyKBCommands.xml.  (often by doing a complertely fresh PC restart.)

 

But... this wastes time. It's better than starting from scratch, but I would still prefer not to have to do it.

 

---+ So, my questions are:

 

Q: what is happening that changes are reflected in MyKBCommands_withAmendedCommands.xml but not MyKBCommands.xml?

 

Q: am I doing something, like "Close Dragon and KnowBrainer" via SpeechStart+, that is causing this problem?

 

Q: when is it safe for me to copy MyKBCommands.xml?   I.e. when is it consistent on diskI  always try to wait until I can run KniowBrainer commands again, and I often do "SpeechStart+ Close Dragon and KnowBrainer" -- or, when SpeechStart+ has failed, I kill Dragon and KnowBrainer, and/or restarty my PC.

 

Q: is KnowBrainer saving to MyKBCommands.xml whenever a command is changing? Or is there some buffering/delay? Over and above the usual windows buffering.

 

Q: when I restore MyKBCommands.xml from git/backup, is it sufficient to overwrite just MyKNCommands.xml, or both that and MyKBCommands_withAmendedCommands.xml?

 

Q: isz MyKBCommands_withAmendedCommands.xml well formed?  I.e. could I just copy it to overwrite MyKBCommands.xml?  Or is occasionally in some transient state so that would not be safe?

 

BTW, I nearly always manually inspect the MyKBCommands.xml, and have never observed any corruption.  Either my new commands and edits are all there, or not at all. Never partway.  I also run MyKBCommands.xml through xmllint with no problems.

 

---

 

Perhaps relevant: recently I have been seeing even more KnowBrainer related stalls than usual.  I am fairly sure that KnowBrainer is involved - I can go a day withiout any stalls, using Dragon and my existing KnowBrainer commands, but frequently when I open the KnowBrainer Command Editor just to see what a command does, and then close without saving - i.e. I have just used KBCE to view the command, not edit - I get a stall.  Plus the usual painful stall after using KBCE to edit a command.  If just the latter I would suspect problems rewriting the XML file and/or uninstalling and reinstalling a grammar (command vocabulary) in the running Dragon instance.  That should not be happening if I exit KBCE without making any changes, but perhaps it is.  (I suppose I could start looking at file timestamps...)_

---+ Details
The full paths of the files involved are

C:\Users\XXX\AppData\Roaming\KnowBrainer\KnowBrainerCommands\MyKBCommands.xml

C:\Users\XXX\AppData\Roaming\KnowBrainer\KnowBrainerCommands\StatusChanges\MyKBCommands_withAmendedCommands.xml



-------------------------

DPG15.6 (also DPI 15.3) + KB, Sennheiser MB Pro 1 UC ML, BTD 800 dongle, Windows 10 Pro, MS Surface Book 3, Intel Core i7-1065G7 CPU @ 1.3/1.5GHz (4 cores, 8 logical, GPU=NVIDIA Quadro RTX 3000 with Max-Q Design.



 11/06/2020 11:12 AM
User is offline View Users Profile Print this message

Author Icon
Edgar
Top-Tier Member

Posts: 1041
Joined: 04/03/2009

Originally posted by: Ag 

Q: what is happening that changes are reflected in MyKBCommands_withAmendedCommands.xml but not MyKBCommands.xml?

Q: is KnowBrainer saving to MyKBCommands.xml whenever a command is changing? Or is there some buffering/delay? Over and above the usual windows buffering.

Q: when I restore MyKBCommands.xml from git/backup, is it sufficient to overwrite just MyKNCommands.xml, or both that and MyKBCommands_withAmendedCommands.xml?

Q: isz MyKBCommands_withAmendedCommands.xml well formed?  I.e. could I just copy it to overwrite MyKBCommands.xml?  Or is occasionally in some transient state so that would not be safe?

Look at this thread, second post:
http://www.knowbrainer.com/forums/forum/messageview.cfm?catid=3&threadid=23621&highlight_key=y&keyword1=StatusChanges

And maybe this thread:
http://www.knowbrainer.com/forums/forum/messageview.cfm?catid=3&threadid=33637&highlight_key=y&keyword1=StatusChanges

And possibly this one:
http://www.knowbrainer.com/forums/forum/messageview.cfm?catid=3&threadid=34539&highlight_key=y&keyword1=StatusChanges

I have a suspicion that Windows might be buffering the primary (and maybe the StatusChanges one) file in RAM and Windows and KnowBrainer have a dual (with pistols at fifty paces) which results in the old file being written to disk somehow.

It is imperative to overwrite both and to do so while KnowBrainer is not running.

Obviously, there's no documentation on this duplication of command files and neither Lunis nor monkey8 (Lindsay, his programmer) have responded when I have asked this kind of question in the past.



-------------------------

-Edgar
DPI 15.3, 64-bit Windows 10 Pro, OpenOffice & Office 365, Norton Security, Shure X2U XLR to USB mic adapter with Audio Technica DB135 vocal mic, Asus X299-Deluxe Prime, Intel Core i9-7940X (14 core, 4.3 GHz overclocked to 4.9 GHz), G.SKILL TridentZ Series 64GB (4 x 16GB) DDR4 3333 (PC4 26600) F4-3333C16Q-64GTZ, NVIDIA GIGABYTE GeForce GTX 1060 GV-N1060G1 GAMING-6GD REV 2.0 6GB graphics card with 3 1920x1080 monitors



 11/06/2020 12:06 PM
User is offline View Users Profile Print this message

Author Icon
Ag
Top-Tier Member

Posts: 518
Joined: 07/08/2019

Thanks Edgar.


-------------------------

DPG15.6 (also DPI 15.3) + KB, Sennheiser MB Pro 1 UC ML, BTD 800 dongle, Windows 10 Pro, MS Surface Book 3, Intel Core i7-1065G7 CPU @ 1.3/1.5GHz (4 cores, 8 logical, GPU=NVIDIA Quadro RTX 3000 with Max-Q Design.

 11/06/2020 04:15 PM
User is offline View Users Profile Print this message

Author Icon
Lunis Orcutt
Top-Tier Member

Posts: 38502
Joined: 10/01/2006

Since DPI 15.6 has been released (sort of), we recommend re-creating your personal commands in Dragon; until KnowBrainer 2021 is released.

-------------------------

Forum Mission Statement
Trial Downloads
Dragon/Sales@KnowBrainer.com 
(615) 884-4558 ext 1

 11/08/2020 08:42 AM
User is offline View Users Profile Print this message

Author Icon
monkey8
Top-Tier Member

Posts: 3695
Joined: 01/14/2008

Ag,

 

1) The next time you lose a command, in that it does not appear in your MyKBCommands.xml file,  please can you send me a copy of the following file and taking note of the approximate time and date when it happened. Please also tell me the name of the command you have added.

 

"C:\Users\YOUR WINDOWS USER NAME\AppData\Roaming\KnowBrainer\KnowBrainerLogs\KnowBrainer Professional.log"

 

along with a copy of your MyKBCommands.xml in the state it was in at the time the command apparently disappeared. For some reason your new command is not being added to the commands file (at all or incorrectly) for what could be a number of reasons which is why I need to see the log file above.

 

2) when you first add the command can you tell me If It appears on the KnowBrainer Sidebar and if the command works when called?

 

3) if you lose a command and it does not appear in the main command file does it always appear in the amended commands file?

 

Just to answer a couple of your questions, when you add a command the MyKBCommands.xml file should be updated immediately, there is no buffering. There is an issue with the latest versions of Dragon using WWB scripting where it can take a long time to add a new command or edit an existing command BUT mainly if you already have excessive numbers of commands. I am looking at workaround for this. Not only KnowBrainer but Dragon scripting has the same issue.

 

If you lose some commands and you find them in the amended commands file then it is fine to recreate them from the details in the amended file but the amended file is not complete copy of your commands file so should not be used as a replacement to answer your question.

 

The amended commands file does not need to be copied at any time when restoring commands files from git or wherever, the file should be left alone and not touched and certainly not copied over. If users wishing to fiddle about without knowing what they are doing start replacing and overwriting such files they are causing their own problems.

 

 



-------------------------



 12/15/2020 11:54 PM
User is offline View Users Profile Print this message

Author Icon
Ag
Top-Tier Member

Posts: 518
Joined: 07/08/2019

Thanks @monkey8 (Lindsay?)

Originally posted by: monkey8 Ag,

1) The next time you lose a command, in that it does not appear in your MyKBCommands.xml file,  please can you send me a copy of the following file and taking note of the approximate time and date when it happened. Please also tell me the name of the command you have added.

 

"C:\Users\YOUR WINDOWS USER NAME\AppData\Roaming\KnowBrainer\KnowBrainerLogs\KnowBrainer Professional.log" 

 

along with a copy of your MyKBCommands.xml in the state it was in at the time the command apparently disappeared. For some reason your new command is not being added to the commands file (at all or incorrectly) for what could be a number of reasons which is why I need to see the log file above.

 

Thanks, will do the next time I see this.  At the moment I am trying to avoid editing in KnowBrainer at all, because of the long delays you mention below.  Most of my new code just calls out to AutoHotKey.  I mainly edit KnowBrainer (a) to replace the bodies of KB commands I wrote with callouts to AutoHotKey, and (b) to create stubs for new commands that call out to AHL - the latter because it hekps recognition accuracy when Dragon can see the command names in more detail than "PUFF ".

 

 

 

2) when you first add the command can you tell me If It appears on the KnowBrainer Sidebar and if the command works when called?

 

This I can answer right away: the new command appeared in the KnowBrainer Sidebar, and I used it for several minutes. IIRC I cleanly exited via "Close Dragion and KnowBrainer".  This is why I was particularly surprised that it did not survive.

 

(This happened more than once, although in some cases Dragon and/or KnowBrainer may have hung, so I was not surprised to lose a command then.) 

 

 

 

3) if you lose a command and it does not appear in the main command file does it always appear in the amended commands file?

I will check.  I can't recall this happening - losing a command that does not appear in the main command file - although I recall cases where it may have happened, I have usually blamed it on driver error, e.g. forgetting to change command name when cloning from an old.

 

 

Just to answer a couple of your questions, when you add a command the MyKBCommands.xml file should be updated immediately, there is no buffering.

 

Thanks, that is good to know.

 

There is an issue with the latest versions of Dragon using WWB scripting where it can take a long time to add a new command or edit an existing command BUT mainly if you already have excessive numbers of commands. I am looking at workaround for this. Not only KnowBrainer but Dragon scripting has the same issue.

Amen! and Godspeed to your efforts to find a fix!

 

I have seen those excessive delays in spades. Frequently > 7 minutes - IIRC I once let it run just to see what happened, and it terminated after >40 minutes! BTW, I have been writing scripts to investigate timings from Dragon.log.  I will start looking at  the KnowBrainer log as well.

 

BTW, I have not added that many of my own commands, and I started noticing the problem before I had added many at all.  I.e. the problem arose with an official KnowBrainer distribution.

 

I *did*, however, do a global edit (in the XML file, by emacs and recdntly xmlstarlet) adding my "PUFF" prefix to almost all commands.  But I was seeing the latecy even before that.

 

I am heading towards eliminating many, if not most, of the offcal KnowBrainer commands.  Certainly for the apps I do not use.  Also for the commands I dislike the syntax of - wigth my "PUFF" prefix, I can say things in a more natural way, with less risk of colliding with dictation.

 

 

If you lose some commands and you find them in the amended commands file then it is fine to recreate them from the details in the amended file but the amended file is not complete copy of your commands file so should not be used as a replacement to answer your question.

 

The amended commands file does not need to be copied at any time when restoring commands files from git or wherever, the file should be left alone and not touched and certainly not copied over. If users wishing to fiddle about without knowing what they are doing start replacing and overwriting such files they are causing their own problems.

 

Again, good to know.

 

So far, I have only really recovered individual commands from the amended file.  Even when MyKBCommands.xml has disappeared, I always have my git backups.



-------------------------

DPG15.6 (also DPI 15.3) + KB, Sennheiser MB Pro 1 UC ML, BTD 800 dongle, Windows 10 Pro, MS Surface Book 3, Intel Core i7-1065G7 CPU @ 1.3/1.5GHz (4 cores, 8 logical, GPU=NVIDIA Quadro RTX 3000 with Max-Q Design.



 12/17/2020 04:02 AM
User is offline View Users Profile Print this message

Author Icon
monkey8
Top-Tier Member

Posts: 3695
Joined: 01/14/2008

Hi Ag,

 

Yes Lindsay. When you remove the large number of KB commands that you don't need you will find that the editing and addition of commands will speed up anyway. Meanwhile we will be releasing an update shortly which will overcome this speed issue when adding or editing commands.

 

Lindsay



-------------------------

 12/17/2020 12:17 PM
User is offline View Users Profile Print this message

Author Icon
Edgar
Top-Tier Member

Posts: 1041
Joined: 04/03/2009

While Lindsay (Monkey8) is responsible for (may not have written, but now maintains <shrug>) the part of the KnowBrainer code which deals with "MyKBCommands_withAmendedCommands.xml", and is certainly the "go-to" for these kinds of questions…
I agree that while KnowBrainer is running one should absolutely not mess with the "…withAmendedCommands" file, I don't think their categorical "never" mess with it is valid. After KnowBrainer has successfully terminated (and all Windows disk caching is complete), it is my belief that both files "should" be identical and thus any exterior editing of the base MyKBCommands file can and should be duplicated by overwriting the "…withAmendedCommands" file with the base file (obviously the filenames must remain appropriately consistent). I have done extensive testing on this and find this to be REQUIRED if one wants to remove an unneeded List from KnowBrainer. I wrote some extremely detailed posts about this quite a few years ago.



-------------------------

-Edgar
DPI 15.3, 64-bit Windows 10 Pro, OpenOffice & Office 365, Norton Security, Shure X2U XLR to USB mic adapter with Audio Technica DB135 vocal mic, Asus X299-Deluxe Prime, Intel Core i9-7940X (14 core, 4.3 GHz overclocked to 4.9 GHz), G.SKILL TridentZ Series 64GB (4 x 16GB) DDR4 3333 (PC4 26600) F4-3333C16Q-64GTZ, NVIDIA GIGABYTE GeForce GTX 1060 GV-N1060G1 GAMING-6GD REV 2.0 6GB graphics card with 3 1920x1080 monitors

 12/17/2020 04:18 PM
User is offline View Users Profile Print this message

Author Icon
Ag
Top-Tier Member

Posts: 518
Joined: 07/08/2019

The Makefile in the directory where I manage MyKBCommands.xml diffs it and the amended file. I have not automated what to do if there is a difference, but at least it warns me.

 If they are different, I have usually found that  restarting my PC, editing a command KnowBrainer, exiting cleanly, usually results in both MyKBCommands.xml and the amended file being identical. Of course, before I do this, I make copies of both MyKBCommands.xml and the amended file, in case something I really want is missing.

 I don't really have a good XML tree diff  that understands how trees and ssubtrees can move around, but at least I can diffed the list of commands, and then  patch commands that have been added or subtracted. I suppose I should automate extracting all commands, with a path, sorting that and diffing that.

 

 



-------------------------

DPG15.6 (also DPI 15.3) + KB, Sennheiser MB Pro 1 UC ML, BTD 800 dongle, Windows 10 Pro, MS Surface Book 3, Intel Core i7-1065G7 CPU @ 1.3/1.5GHz (4 cores, 8 logical, GPU=NVIDIA Quadro RTX 3000 with Max-Q Design.

Statistics
31938 users are registered to the KnowBrainer Speech Recognition forum.
There are currently 1 users logged in.
The most users ever online was 12124 on 09/09/2020 at 04:59 AM.
There are currently 563 guests browsing this forum, which makes a total of 564 users using this forum.

FuseTalk Standard Edition v4.0 - © 1999-2021 FuseTalk™ Inc. All rights reserved.