Structural Informatics Group (SIG) logo
Home | Projects | Demos | Downloads | Publications | Local Info | About Us | New site
Go to the first, previous, next, last section, table of contents.

Knowledge Manager Notes

Not very intelligible, but verbatim note I made talking to Kraig, for what they are worth:

Worklist: Onard: Duplicate subtree must replicate all isa links for node. CR: Display all "semantic types" (immediate parents with NLM type set) in "Concept Inspector".

From: Kraig Eno <> To: Jeffrey Prothero <> Subject: patella account Message-Id: <> Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII

Well, you already had an account on patella, but with no user directory. So I set all that up, copied the files over, and created a database on tarsus that you can use. The password is as we discussed, so if it doesn't work send me a note or just go in as me, su to root, and force a change with "passwd jsp".

The application is in /usr/people/jsp/KnowledgeManager; there's a subdirectory called "contract" which contains all the static data source files and millions of utilities to do various jobs with them. A subdirectory of that, "sql", has the SQL table definition files I showed you.

- to compile the app, either use "make" (it does work) or else the following: double-click the file "PB.project", whose icon appears in the top pane of the File Viewer, then click the "Builder" button, then "BUILD".

- to run the app from the project builder, click the "Run" button. Or cd to /usr/people/jsp/KnowledgeManager/ and run the "KnowledgeManager" file.

- to modify the interface from the project builder, click the "Files" button, select "Interfaces", and double-click the KnowledgeManager.nib file. Or open that file directly from the File Viewer, inside the "English.lproj" directory.

- I've put the "GeneralRef" directory into the top pane of the File Viewer, that's sort of near the top of the online documentation hierarchy for the NeXT development system. Within that, 02_ApplicationKit has complete documentation on objects like windows, buttons, menus, and what have you. The "Sybase" documentation, also in that top pane, contains database documentation; the COMREF (chapters 2 & 3) and DBLIB directories will probably be the most interesting.

I created a new database called "kbjsp", owned by you, on tarsus. I've set up your path and an alias in .cshrc/.login so that you can use it interactively; just say "isql" in a shell and it will let you type commands at tarsus, the first of which should be "use kbjsp" and then "go" to access your database. I've defined the tables, by adding the "use kbjsp" command at the top of several files and then executing them, as in: cd /usr/people/jsp/KnowledgeManager/contract/sql isql <db-def.sql (well, actually only the last half of that) isql <terms-def.isql isql <links-def.isql isql <hierarchies.isql cd /usr/people/jsp/KnowledgeManager isql <procedures.sql

You should now have an active copy of both the source and the database. When you run the application, either from your source tree or the one in /NetApps, type "kbjsp" where it normally defaults to "kbtest" and you'll be off and running.


------ Server x database config is in next:/usr/sybase/interfaces ------ db-def.isql: Basic create script for kbjsp db, ------ /usr/sybase/bin/isql <-=-=- the actual executable

# to do admin stuff, instead of just isql (see alias) do: /usr/sybase/bin/isql -S TARSUS -Usa -P<pw>u Example: To list all databases: sp_helpdb <- Show list of dbs on this server. go sp_help <- show all object names in current db go use kbjsp <- switch to given db go sp_who <- list active users and dbs which they are using go quit <- the only command that doesn't need 'go' --- Finding docs: /NextLibrary/Documentation/Sybase/COMREF/Chapter[23] has the good ISQL stuff :) Also: DBLIB/... has c-callable stuff.

--- Using SQL for REal :) sp_help tablename <- also very good! select * from terms where UWDAID=9574 <- Showing term in table delete from terms where Name=""

select * from terms where Name like "%node%" <- find all name with substring

In practice, usually do select UWDAID,substring(Name,1,50) from terms where UWDAID < 1000 and UWDAID > 566 or such. Even better (shhh, secret!): select count(*) from ... <- just count number of matches.

update links set Authority="NeuroNames" where Authority="NN" select count(*) from links where ChildID in (select UWDAID from terms) ------ Removing stuff: "drop" is the operator for just about everything: tables, triggers...

Copying between dbs: in isql 1> use kbjsp 2> go 1> delete terms <- means "empty" -- don't do DROP ! 2> go 1> insert terms select * from kb.dbo.terms ( "where... " if desired) 2> go 1> delete links <- Can add 'where" clause if desired. 2> go 1> insert links select * from kb.dbo.links 2> go 1> delete nextid 2> go qui1> insert nextid select * from kb.dbo.nextid 2> go May want to do below "dump..." after each of above ------ When logsegment overflows: dump transaction kbjsp with no_log ------ Cut and paste on NeXT (and Mac!!): Highlight, Command-C to copy, select dest, Command-V to insert ------ Dumping contents of a table: patella> bcp <- lists usage patella> bcp terms out jeff.sav -S TARSUS Notes: Bcp always uses default db for current user. To list a table from another db, use isql to change the default first Other problems resolved during dry run: My pw is null. Need to specify newline (\n) for final field separator. Save format as terms.fmt or such. ----- 4m ----- Adding user to allowed list for a db: isql 1> use kb 2> go 1> sp_adduser jsp 2> go

------ Setting default db: patella> /usr/sybase/bin/isql -S TARSUS -Usa 1> sp_defaultdb jsp, kbjsp <- set user to given default db 2> go <- yup (If you're doing much, save the format out and re-use) Char is short strings, Text is massive strings (up to 2G.) Don't say int unless you want binary! -- "Char is what you'll most likely use for everything!"`v ------- Note: doing sp_helpdb shows "STATUS" flags on each table: Must have "select into/bulkcopy" set for above to work. This must be set on new databases: isql <db-def.sql gives an example: (sp_dboption kbjsp,"select",true)

Also, if you don't set "trunc" flag, logfiles will accumulate until the fixed-size db file overflows... good to set :)

---- Next pragmatics: command-N to create New shell (on existing shell) ---- procedures.sql (All the sybase stored procedures that implement knowledgemanager database functions/operations are here.) ---- 95Dec05 Interface builder notes: Startup:

Double-click .nib file, under "Interfaces" in Knowledge Manager dir...

ProjectBuilder==maintain Makefile InterfaceBuilder==Window&Buttons Within Interfacebuilder, Tools -> Inspector brings up Window Inspector, the fundamental layout editor.

Making a Connection: Hold CTRL down, drag from widget to... well, something, such as typically: Window title bar: to make window appear/etc: Actions will then display all messages for that. Object. Can also drag from object to widget to build a connection for displaying output. -------- How to compile: Click the hammer icon Select project via "Project" and then "Open" (PB.project) then BUILDER pane. Click build. -------- Installing a 'fixed' (i.e., changed) copy of the app:

Method I:

Bring up file browser, click down jsp/KnowledgeManager/, which is a directory tree containing the complete compiled app. To install, just drag the .app icon to file viewer bar (a pasteboard), go down to /NetApps, and drag the icon onto the NetApps icon.

Method II:

Just use cp -R ...../ /NetApps

Later: su cd ~jsp/KnowledgeManager cp -R /NetApps

Method III:

In ProjectBuilder, select "install" as the target to make (via <BUILD> click), and in "options" set the correct target directory. ------- To run the global (installed) version, just go to /NetApps and double-click it. "Run" in ProjectBuilder is a good way to run the local one, or you can also just double-click the local copy. OR! You can run it unix style -- as, the actual executable.

--------directory tree overview (Kraig) ---------- KnowledgeManager/.exrc (yes, I use vi. I can't stand "Edit" & don't know emacs)

KnowledgeManager/attrib.h KnowledgeManager/attrib.m (Objective-C source for the "attrib" object, a window containing a title and a scrolling text box for displaying free-form text associated with a term.)

KnowledgeManager/centry.h KnowledgeManager/centry.m (Objective-C source for the "centry" panel, used by an author to add a new term along with its synonyms and NLM semantic type.)

KnowledgeManager/cexport.c KnowledgeManager/cexport.h (The C source for a table-driven report generator, called when writing Sybase data out to static files; the generator is used for exporting hierarchies with a format defined in a separate file, "formats.kman". This is a mechanism for the programmer to easily adapt the output formats to a particular need without having to write C code. I could have rolled in XLisp and called a Lisp function for each output row, but I had just read a book and wanted to try my hand at writing a table-driven data generator. Sorry. [Well, I'm not too sorry -- I was quite pleased with the result. KRE])

KnowledgeManager/cinspector.h KnowledgeManager/cinspector.m (Objective-C source for the "cinspector" panel, showing a the preferred name of a particular concept and an NXBrowser listing its synonyms.)

KnowledgeManager/db.c KnowledgeManager/db.h (The C source for a set of glue routines used to make accessing the Sybase DB-Library routines easier. Routines in this file handle the tests for flags and conditions and provide error recovery within the application context, and a few routines for doing common "data lookup" tasks like determining if a given term is present in the database.)

KnowledgeManager/English.lproj/... (this whole directory contains the Interface Builder application definition)

KnowledgeManager/export.h KnowledgeManager/export.m (Objective-C source for the data export control module; this object supplies action methods that can be called when a button or menu item is selected, and defines the panel that lets a user select an output format by name.)

KnowledgeManager/formats.kman (this is a text file specifying the report formats that KMan can use for generating static files out of Sybase data; it is copied with the application, but if a user happens to have a "formats.kman" file in his home directory then it overrides the one that comes with the app. So people can use a variety of predefined formats (notably, the "NLM" format for creating files to send to NLM for the contract) or make their own formats. The format of this file is, however, non-trivial; none of our users will ever change it.)

KnowledgeManager/hand.tiff (the program icon as it shows in the file browser; this is copied into the directory, which is the package that's installed as the binary form of the executable program in /NetApps.)

KnowledgeManager/hwindow.h KnowledgeManager/hwindow.m (Objective-C source for the messages that deal with the hierarchy NXBrowser as an interface object; that is, responding to user selections and doing common jobs like manipulating subtrees in the display.)

KnowledgeManager/kmgr.c KnowledgeManager/kmgr.h (The C source for some application-specific utility routines and definitions that are used throughout the other modules. The best example of this is the km_dbterm2collapsed() and km_matrixentry2dbterm() functions which convert between plain terms and the displayable browser matrix row entries which have tabs on the front and perhaps a special character on the end to indicate a collapsed subtree. Unfortunately you can't just change these routines to get different behavior in the display, because there are places in the code that do things like build up their own tab prefixes for subtree entries in a misguided wish to make everything re-entrant while avoiding lots of copying between buffers. By understanding the code in kmgr.c, though, you'd at least understand )

KnowledgeManager/ (this directory contains the executable code plus the runtim interface definition) ./ (executable object module) ./ (see formats.kman above) ./ (see hand.tiff above) ./ (see termlist.fmt below)

KnowledgeManager/KnowledgeManager_main.m (the _main module is generated automatically, and shouldn't be changed)

KnowledgeManager/lisp/... (a test directory to see how easy it would be to use the Sybase stored procedures from Lisp. It would of course be trivial, just a matter of writing the glue routines.)

KnowledgeManager/m68k_obj/... (this directory contains the separate compiled .o files)

KnowledgeManager/Makefile (a Makefile that's automatically generated and maintained by the Project Builder program; adding and removing files inside of project builder causes corresponding changes to the Makefile to happen invisibly.)

KnowledgeManager/Makefile.preamble (the part of the Makefile process that the programmer gets to control and modify for installation-specific items; in this case, it specifies the library and includes for accessing the Sybase DB-Library routines.)

KnowledgeManager/maketermlist (derived from the ./contract/maketermlist shell script, to help understand how the KMan program would generate a term list as a static file. This script is not called, instead its guts are replicated inside the "export.m" file.)

KnowledgeManager/mkbak (shell script for taking a snapshot of the Knowledge Manager source)

KnowledgeManager/old/.. (this directory contains tar files of the source code from past versions)

KnowledgeManager/panhandler.h KnowledgeManager/panhandler.m (Objective-C source for the "panhandler" object, which is my usual starting point for developing a NeXTStep application. It handles all the messages that don't make sense to handle anywhere else, for example the startup action called "appDidInit". In this application, it also implements all the hierarchy manipulation methods that interact with the database because the use of an NXBrowser in the hierarchy window confused the issue early on and I didn't understand how to factor it all out into a separate "hwindow" object. I've started that process but haven't gone too far -- the only reason it will be necessary is if we ever want to have multiple hierarchy windows open at once.) [The name, "panhandler", comes from the Greek "pan", meaning "all", and "handler", meaning to do what has to be done. It's the all-purpose message handler, and the name also evokes the image of a person "stopping people on the street and asking for food or money", in the words of Webster. I see this module as an ever-present beggar traipsing around the message hierarchy asking for handouts, a button-click here, a sackful of terms there.]

KnowledgeManager/PB.gdbinit and /PB.project (Project Builder files -- open PB.project to open the project in Project Builder)

KnowledgeManager/procedures.sql (SQL code defining the stored procedures which are called from inside the Knowledge Manager. There are over 20 of these, and they are all very specific to the particular table/field defininitions used in this database.)

KnowledgeManager/servers.sql (some Sybase SQL code related to getting a list of available servers by name; this was a test to see if we could let users select from a list instead of typing in the name of the server by hand at program startup)

KnowledgeManager/specs.txt (my early encapsulation of what the Knowledge Manager program should do. Somewhat out of date).

KnowledgeManager/sym/ KnowledgeManager/termlist.fmt KnowledgeManager/test.h KnowledgeManager/test.m KnowledgeManager/tinspector.h KnowledgeManager/tinspector.m

KnowledgeManager/contract/... (the "contract" directory contains the original on-line source files for CR's work on the NLM terminology contract, which is ongoing. The original data was typed in by Shelley & CR in WordPerfect, then converted to straight ASCII by me and run through several filters to get the data into the Sybase tables. There are still unresolved errors; the largest are in the "tributary of" hierarchy, which has many examples of lines starting with blanks instead of tabs which caused much confusion to my filters. I changed my programs to try to deal with that case, but I think it will still result in lots of errors in the database; I am currently waiting for CR to learn enough about the system so I can explain what the problem is and try to arrive at a solution).

KnowledgeManager/contract/bak/... (tar files containing snapshots of source code & data in the "contract" directory)

KnowledgeManager/contract/branchof.out (file containing the "branch of" hierarchy, in the format specified by NLM for delivery for the contract. This is generated by the "makebranchof" script.)

KnowledgeManager/contract/branchof.rosse KnowledgeManager/contract/cr.lsp KnowledgeManager/contract/crlinks.out KnowledgeManager/contract/crterms.out KnowledgeManager/contract/data.lsp KnowledgeManager/contract/data/... (the original source documents, in WordPerfect format, plus their ASCII translations in corresponding ".dat" files)

KnowledgeManager/contract/do-weekly (an explanation of a protocol that was never followed. We originally planned to import new data and generate full output files every week for proofing, but nobody ever got around to it and Knowledge Manager is supposed to supplant this process)

KnowledgeManager/contract/fmt/... (various .fmt files used by the Sybase "bcp" (bulk copy) utility, for exchanging data in and out of flat files)

KnowledgeManager/contract/id-tree KnowledgeManager/contract/ignorelines KnowledgeManager/contract/ih-parse KnowledgeManager/contract/ihpb KnowledgeManager/contract/isa-parse KnowledgeManager/contract/isa.out KnowledgeManager/contract/makebranchof KnowledgeManager/contract/makepartof KnowledgeManager/contract/makestypes KnowledgeManager/contract/maketermlist

KnowledgeManager/contract/mkbak (shell script for taking a source/data snapshot)

KnowledgeManager/contract/mkdeliv (shell script for generating NLM contract deliverables)

KnowledgeManager/contract/nlm/... (directory of files that were sent to NLM on 4/11/95. This included the "branch of", "part of", and "isa" hierarchies.)

KnowledgeManager/contract/partof.out KnowledgeManager/contract/partof.rosse KnowledgeManager/contract/partof.tabs KnowledgeManager/contract/partof.wp KnowledgeManager/contract/problems KnowledgeManager/contract/protocol KnowledgeManager/contract/reverse.awk KnowledgeManager/contract/silisp KnowledgeManager/contract/siserver.bin KnowledgeManager/contract/sql/ KnowledgeManager/contract/sql/db-def.isql KnowledgeManager/contract/sql/hierarchies.isql KnowledgeManager/contract/sql/links-def.isql KnowledgeManager/contract/sql/terms-def.isql KnowledgeManager/contract/sql/treegen.isql KnowledgeManager/contract/src/ KnowledgeManager/contract/src/id-tree.c KnowledgeManager/contract/src/ih-parse-basic.c KnowledgeManager/contract/src/ih-parse.c KnowledgeManager/contract/src/isa-parse.c KnowledgeManager/contract/src/kbfuncs.lsp KnowledgeManager/contract/src/tonumtree.c KnowledgeManager/contract/src/torosse.c KnowledgeManager/contract/src/totabs.c KnowledgeManager/contract/stypes.out KnowledgeManager/contract/synonyms.out KnowledgeManager/contract/t.out KnowledgeManager/contract/t2.out KnowledgeManager/contract/termlist.mb KnowledgeManager/contract/termlist.out KnowledgeManager/contract/tmp/ KnowledgeManager/contract/tmp/additions.log KnowledgeManager/contract/tonumtree KnowledgeManager/contract/torosse KnowledgeManager/contract/totabs KnowledgeManager/contract/tree.out

----------------------------------------------------- Useful things to cover: Dump/edit/restore of a db. Copy production db to test db. Debug: Running with debugger, and/or putting printfs in code and observing them during a run. ----------------------------------------------------- Practical connection hacking: 1) Edit cinspector.h to add new message; To get InterfaceBuilder to notice the change: Get into IB, if necessary by clicking screwdriver icon (just clicking a relevant window will usually do), then in the Knowledgemanager.nib file editor window (lowerleft) find the relevant object, double-click it, then on "Operations" button on that window, pick "Parse". Whee! :) 2) Get back to "Objects" directory(?) in .nib file editor window, HOLD DOWN CONTROL KEY, and drag line from source (button) to destination (in .nib window). This brings up Button Inspector with list of messages which can be sent to that object; 3) Double-click desired message (highlight & select action). 4) (If desired): Save .nib file via Document/Save option on main InterfaceBuilder menu. ----------------------------------------------------- Canonical source is now /Net/betz/usr/local/src ----------------------------------------------------- To extending a db, just do "alter database <dbname>". Example: jsp@scala Unix/bin> isql -STARSUS -Usa -Ptagore 1> sp_helpdb 2> go name db_size owner dbid created status ------------------------ -------- ------------------------ ------ ----------- ------------------------------------------------------------------------------------------------------ kb 6 MB kraig 5 Jan 21 1993 select into/bulkcopy, trunc. log on chkpt. kbjsp 4 MB jsp 7 Nov 9 1995 select into/bulkcopy, trunc. log on chkpt. kbtest 6 MB kraig 6 Jan 21 1993 select into/bulkcopy, trunc. log on chkpt. master 4 MB sa 1 Jan 1 1900 no options set model 2 MB sa 3 Jan 1 1900 no options set pubs 2 MB sa 4 Jan 21 1993 no options set tempdb 2 MB sa 2 Dec 13 1995 select into/bulkcopy

(return status = 0) 1 xact: Clock Time (ms.): total = 1000 avg = 1000 (1.00 xacts per sec.) 1> alter database kbjsp 2> go Extending database by 1024 pages on disk master 1 xact: Clock Time (ms.): total = 11000 avg = 11000 (0.09 xacts per sec.) 1> sp_helpdb 2> go name db_size owner dbid created status ------------------------ -------- ------------------------ ------ ----------- ------------------------------------------------------------------------------------------------------ kb 6 MB kraig 5 Jan 21 1993 select into/bulkcopy, trunc. log on chkpt. kbjsp 6 MB jsp 7 Nov 9 1995 select into/bulkcopy, trunc. log on chkpt. kbtest 6 MB kraig 6 Jan 21 1993 select into/bulkcopy, trunc. log on chkpt. master 4 MB sa 1 Jan 1 1900 no options set model 2 MB sa 3 Jan 1 1900 no options set pubs 2 MB sa 4 Jan 21 1993 no options set tempdb 2 MB sa 2 Dec 13 1995 select into/bulkcopy

(return status = 0) 1 xact: Clock Time (ms.): total = 1000 avg = 1000 (1.00 xacts per sec.) 1> quit jsp@scala Unix/bin> history


Go to the first, previous, next, last section, table of contents.