(MOST RECENT WORK AT THE TOP)
Introduced JSP to the KnowledgeManager app and its associated database data structures, and copied the source tree to his account on patella. Also created a new Sybase database for him to work with.
Finished up the modifications to FrameBuilder, called the result version 1.61, and sent Dave and John a note that it's ready to use.
Did the first round of modifications to the FrameBuilder "Author's Guide". document. It feels like I'm having to apologize a lot less this time around, now that it isn't so eccentric.
Revamped the file saving operations in FrameBuilder, so it works more like a regular program. Corrected a few bugs, like references to an old menu item name that got changed, and the fact that "New Frame" didn't used to check for unsaved modifications before blowing everything away. It's all much cleaner now.
Changed the way names are assigned and edited in FrameBuilder, with simple checks for misplaced space characters and improvements to the way the dialogs are linked. Also included support for Cut/Copy/Paste in the edit menu, which works also on contour shapes themselves. Next, will work on orthogonalizing everything and improving file handling.
Re-arranged the files on the image server to work with the newest version of the browser, so that a single split frame can refer to multiple (3 right now) data areas generated by different authors. Uploaded v2.4b3 of the browser demo, but it's 68K only right now; I discovered a defect that stops it cold when certain frames are loaded.
Fought with TCP/IP networking on Bob's PB5300, and gave up for the moment while Apple investigates why they shipped Open Transport on his machine instead of MacTCP.
Added textual-attribute-display window to KMan.
Made some extensive mods to the browser's pointing quiz module (pquiz.c). "Show Answer" now uses a generic routine from the image-interaction module to draw outlines, instead of having the code in two places; it's smaller, more maintainable, and follows the outline-on time delay mechanism too. The quiz questions themselves come in random order instead of alphabetically, and are chosen by looking at the Contours menu instead of directly manipulating the linked list of contours. The code is lots simpler now. And finally, at the end of a quiz, the scoring results are shown.
Spent a lot of time going through FrameBuilder (my barebones SuperCard image-outlining utility) and writing up a spec for possible future use if someone decideds to do the job for real or for a non-Mac environment.
Started learning the basics of perl, trying to make a filter to munge through my source files inserting logging statements to implement a "coverage monitor" -- a way to find out what parts of a program have been exercised in a particular run or series of runs. Perl is good because it lets me do global substitutions to a group of files, and I had thought I'd be able to use the same script to do the same thing on both Unix and the Mac; but MacPerl doesn't work the way I thought it would, so it'll take some thinking. (The coverage monitor idea works like a charm, though).
Awoke to an extremely loud and heavy thunderstorm and drenching rain, up in the North Cascades about 15 miles from Mount Baker. Everyone was in a rush to go back home all of a sudden...wonder why?
Improved the file & disk sharing setup in the Farr lab, removing dependence on the in-house LocalTalk cable plant; a hard-disk problem manifested itself halfway through the job, but Norton seemed to cure it. Hopefully they'll get better capability and throughput and fewer headaches -- time will tell.
Checked out http://www.uwtc.washington.edu/computing/www/mac/directory.html to get an idea of what kinds of HTML editors are out there for the Mac, and was quite pleased. There were lots of HyperCard/SuperCard-based tools, which I'm not interested in at all, but I did load and test Webtor, HTML Editor, and HTML Pro. The last seemed to fit best with my idea of how such an editor should work, and also had the cleanest interface and was the most streamlined in its performance and memory usage. I did manage to fool it into generating erroneous HTML, so it's not completely solid, but at least it's there so I don't have to write it myself.
Finally, I figured out some more vi commands to help me edit text (a mapping to add the "> " to each line of an attributed text block, and a macro to break up a long line with multiple hard returns).
Changed KMan's login dialog to allow a database name to be specified.
Created a "kbtest" database on synapse with enough stuff in it so that it's a usable testbed.
Enabled remote access between synapse (as a server) and tarsus (as a client) and was able to do a sp_helpdb directly on both machines without having to run isql twice. This means that, if we set it up properly, a single process can log in to its normal dataserver host and query ALL databases on ALL hosts, which ought to make it faster and easier to figure out what's available (by reading the sysservers table to find out what servers are there, and doing sp_helpdb on each of them).
Reinstated the restriction against duplicate term names; the database will work fine, I think, but the interface depends on terms being unique because so much of what it does is name-based instead of ID-based. Allowing multiple duplicate terms will take some thinking and design work.
Implemented the "Add Synonym" command in Knowledge Manager. It's very simple, presenting a one-field dialog;it infers the author field from the value in the regular concept entry dialog's "modified by" field, and leaves most of the other fields (Authority, UMLSID, etc.) blank. The term is immediately visible for editing in the term inspector after a successful addition into the database, so those fields can be set if desired.
Removed the restriction on duplicate term names; they're now allowed, but if the concept-entry procedure sees that one of your new names is already present, it allows you to back out and think about it or cancel the whole operation (or you can forge ahead and create duplicates if you wish). I think this may be a source of trouble eventually, but we have to do it if we want an author to be able to create two separate concepts called "ventricle" with different concept ID's and definitions. (It just occurs to me that many operations, like locating a term in the hierarchy, work by name and not by ID so they'll display behavior that some authors may not expect when collisions in the name space are present).
KMan bugfix: the term insertion was allowing the entry of a blank preferred name, which interfered with everything down the road. I put a restriction in the stored procedure to enforce this, and a similar check in the centry module.
KMan bugfix: find-in-hierarchy was opening trees up properly, but leaving them marked as "collapsed", so that double-clicking an ancestor in the display caused double entries to crop up. Fixed in panhandler.
Got Dennis's IIcx to start up again after last night's power shutdown (sitting cold all night resulted in the infamous hard drive "stiction" problem).
Wrote up some HTML describing the Knowledge Manager design and implementation for Dr. Rosse/NLM/whoever else wants to know. They don't yet approach the level of actual "documentation", but they do answer a lot of questions about what it is and how it's put together. It would also be a good place to talk about design decisions, tradeoffs, and limitations, and that's what'll be going into it in the near future.
The project has been going on since mid-April, and in these four months has grown to a little over 2700 lines of code. It's at the point now where all the major functionality is in place and works, but it has not been extensively tested and there are some things it should do that still aren't there (like editing text attributes). It'll be mostly new features going in now, though, rather than basic design.
Added a new operation to KMan, the "find-in-hierarchy" command functionality. It's just an option in the general find dialog, when checked the program will attempt to locate the target term in context in the hierarchy tree (opening up the intervening subtrees if necessary). It uses a recursive method call which I'm not too happy with, but reasonable nesting levels shouldn't use up too much stack.
I've arrived at a development cycle that seems to work very well: I determine the next two or three features or hurdles that need doing, then spend time in the office reading any necessary information in the NextLibrary so I will know how to go about it; then I lay out the basic structure and do the initial coding at home from Das Mac. I can come back the next day after having thought through it again and double-checked my initial implementation ideas, and put it all through the compiler and put the finishing touches on it. Not being able to compile and test immediately after the initial coding means I have more time to think through the job without getting bogged down in details. Also, using the Mac at work as the code editing station means I do a lot less window-swapping and can concentrate on the task at hand more easily.
Fixed a nasty bug in the browser's new NetGetDescription() routine, which was adding 1 to the returned address of a newly-allocated handle instead of adding 1 to the requested size, as was intended. Changed contour-menu action so all information is retrieved for a structure by name, instead of just highlighting the shape in the image window.
Got the browser's videodisc driver to work on the PB540c; it works on the modem port when the modem is set to "Normal", but doesn't work on the printer port (which tests out busy all the time). I don't know why. The 8100 in F509 is exactly the opposite.
Spent all afternoon making extensive modifications to the routines which implement the tree insertion/deletion/movement operations. It was a while before I realized why the addition of a "root" node was requiring such large changes; what I arrived at was the fact that it violates one of the design assumptions that made the original program so clean. Basically, it was clear from the outset that all the relevant information about a tree member was to be contained visually in the display; for any term that you could see in a given hierarchy, you could always count on seeing the following things: the name of the hierarchy, the preferred name of the term, its parent, any and all siblings, and whether there were any children yet to be retrieved. But artificially hiding the "root" item means that practically all the routines that manipulate the data must also take the special case into account, using different methods to infer the required information instead of accessing it directly from the display.
The code isn't any uglier -- I'm doing some modularization as I go along, pulling common code sequences into their own utility routines, so in some ways it's better than it was. But I don't like the special cases much, nor the implicit knowledge the code has to have about the contents of the database. Ah well, if the world were perfect we wouldn't need custom programs at all. This is a good reminder, though, to think more about the scope of a modification at the design stage before jumping into implementation.
This is an important consideration for the next design change: that of using a special database field to indicate semantic types instead of the "Role" field. To all appearances this should clean things up a lot, but it might have a rippling effect in the code too. Especially, it will be necessary to think about how that information is to be displayed; we have to do more than just supply a checkbox to show the status of the flag. If tree-traversal queries are to use the value of the flag to allow/disallow links from appearing in the display, we have to make the user choose at the time they enter a given hierarchy (switching the flag on and off during traversal would give a hopelessly confusing display, IMHO). And right now, it only makes sense when accessing the "isa" hierarchy -- should we special case that? Another coded assumption about the contents of the database...
Removed the "root" level from the hierarchy displays in KMan; but didn't deal with all the cases of inserting/moving children of root yet.
KMan presents a dialog at startup asking for a host/user combination, in case the user wants to override the defaults.
Checked out MifMucker, a FrameMaker-to-HTML converter. It failed immediately on John's example file, not too surprising; there are many tools for doing this but I doubt if any of them is really robust yet. In any case, there's no way the Web is going to show annotated images with lines and contours and such (we already know this!) so I think the real solution is going to be to change the labels so they're readable on the screen and then do screenshots. A lot of work, and inelegant, but fairly uncomplicated.
Spent most of the afternoon (before Dave's going-away party) trying to figure out AH's Mailstrom problems. After trying many things, I sent to help@cac and Mark Mcnair suggested the inbox might be corrupted in a way that would confuse the client side; it appears that may be the case along about the 5th message down, we'll try fixing that tomorrow and see if it has any effect. If it is, I guess that'll be another mark in favor of using Pine.
Designed and implemented an export module for the Knowledge Navigator. This
is a set of routines and dialog boxes that allows the user to select from
among different possible output formats, and have the program write data
directly to static files for printing or sending to NLM. The formats are
specified in a report-wrting syntax that the user can change or override,
rather than being fixed in C or Lisp code.
This module will be a lot better than having a collection of command line
scripts to use in preparing data to go to NLM, because the user can basically
decide what they want to see and then create the files directly. It also
means that the job of building hierarchies in the correct order with the
correct indentations only has to be done in one place. One point to make about
this module was the fact that I did the design and writing as a sort of
reaction after reading parts of Steve McConnell's book "Code Complete". As
a result of actually performing advance planning and specification, and using
some of McConnell's advice concerning coding practices, the whole thing went
together far faster and with fewer conundrums than is usual with me. It was
a pleasure to piece it all together and have it working so quickly. The book
doesn't really prescribe any particular Method, instead it points out several
methods that people use for various stages of real-world programming jobs
and urges the reader to think about each stage carefully before proceeding
to the next. I find it helps a lot, by reducing the amount of time I spend
moving back and forth between decision-making and problem-solving. My only
point of departure from the book method was that I neglected to write up my
design notes into a form that could be used by others to figure out the
operation of the module, something I should do but there's no place for it
anywhere yet. Maybe I'll put it in a technical documentation section of
some web pages on the Knowledge Manager.
Fixed an interesting problem on the PowerMac 8100 in the microscope lab. Photoshop and Netscape were groaning under the difficult task of opening a new view window for some reason -- they'd do everything normally (including the appearance of dialog boxes and splash pages, managing menus, that stuff) but if you tried to open a new document window the watch cursor would freeze for anywhere between 1 and 3 MINUTES and then you'd get the window and be able to continue working. Very weird. It happened with barebones extensions and even with them all turned off. I finally decided that I'd try installing QuickDraw GX (which wasn't there) and that fixed the problem; then I used the Extensions Manager to turn GX off, and the problem stayed fixed. I conjecture that some other part of the system code thought GX was installed (maybe it had been, and was removed in some messy way).
Over the last couple of work days, I've finally got around to patching up the Atlas navigation code. I don't say "re-wrote", because it was not the complete overhaul that I'd like to do -- but I got rid of the very worst part, which was the "hidden" windows. The original Atlas showed every window on the screen all the time, and then we changed it so some navigation windows would disappear but stay in memory for later...a method I never liked at all. Now the needed information concerning navigation is preserved but the contents of the window (and the window structure itself) is thrown away when the window is not showing.
This means that navigation proceeds correctly (well, I think so, but haven't tested it completely yet) regardless of how the navigation frames are marked. Most importantly, the user can go back to previous levels of the frame list instead of going all the way to the top and then down again. Also, frames CAN be marked as static (either by the author, or by a user during a given session) so they won't go away. This implements the "multiple frames" idea much more cleanly than it has in the past and saves memory unless it's directly needed for viewing a frame.
There's still the difficulty of falling off the end of a frame list onto the orthogonal view, but I haven't resolved whether I want to code this as a specific case yet. In general the basic operation is much cleaner and less wasteful but the code really begs to be re-thought and re-written, which I doubt will ever happen. There are many more important things to do than rebuilding code that already does what it's supposed to. (I think I could make it faster, but other than that the user would see no difference and I have yet to hear any complaints about this part of the program's performance).
Added the necessary logic to the Atlas software to allow it to play movies either directly, in its internal player windows, or through an external player utility via Apple Events.
Improved error handling in the serial I/O code in hopes of making it work on the PowerPC. For some reason the OS always reports that the port is already in use, so the videodisc driver feature doesn't work on the newer machines. But then, neither does Kermit, so I don't feel too bad.
Finished entering the "tributary of" data.
Implemented the "Expand All Subtrees" button, and put in a check so that a term can't be added to a subtree more than once. Also entered a couple more pages of "tributary of" hierarchy data.
Added code to allow the user to change term names and other fields in the Term inspector, and improved validation of the data entry (to disallow special characters and report to the user what's required of them).
Deleted 3 extra copies of TeachText from the Mac in the postdoc office, so now text files will open without the -39 error; and consumed most of the afternoon with other departmental support duties.
(sorry -- it's been a while since I reported in)
Knowledge Manager now has a Term Inspector window (distinct from the Concept inspector) to show database fields for individual terms. It also has controls for re-ordering child links in relation to their siblings.
Knowledge Manager's "Insert Term as Child" operation working.
Fixed an old bug in the image server that would allow the use of ".." in a file specifier, thus letting anyone in the world download any file on the server. Now it's restricted to browser-related files only.
Filled out all the departmental Mac support web pages, as a sort of outline of what help is available and where to get it. ( which see)
Knowledge Manager now lets the user interactively add terms to the database, with improved error-checking and messaging when (for instance) a potential new term is already present.
Implemented operation tracking mechanism in the Knowledge Manager, so I can find out which commands people are using to do their work.
Found and implemented a way for the Atlas to switch out and bring Netscape to the fore when being used as an external viewer.
Knowledge Manager operations in working order:
- pasting a subtree as a root node
- maintaining the order of child nodes
Also, made miscellaneous improvements in the way the Find dialog works
(term entry field automatically selected for you, double-clicking a term
closes the dialog, etc.) and made a great improvement to the way child
nodes are fetched when a subtree is expanded...the old way stored a list
of ID's in a temporary array, the new way uses a stored procedure which
uses a join to replace all the intermediate processing.
Put the finishing touches on the browser-navigation documents, put them on the web, and sent out notices.
Knowledge Manager can now delete whole subtrees, thanks to a nifty little procedure which iteratively marks all of a given parent's descendants. I'm really liking those stored procedures more and more.
Knowledge Manager's term search dialog now fills the concept-inspector information when you double-click a term, correctly resolving synonyms. Also, you can insert a term from the concept inspector as a root node in the currently-displayed hierarchy.
Finished the bulk of the documents outlining navigational strategies and questions, and placed them on the Web in my home page.
Cut & Paste subtrees working in the Knowledge Manager;it does it by moving the selected subtree in or out of a "pasteboard" which is just another term in the tree structure.
Knowledge Manager works entirely from the active Sybase database on TARSUS, no static files are needed. Navigation works cleanly. The next step is to start work on the Cut, Paste, and Delete operations on sub-trees.
Knowledge Manager now retrieves a synonym list in the concept inspector panel, and updates the Role popup properly; and I built a "Find" panel which lets you search the term list in the database for a wildcarded string, with a radio button to tell whether hits should be displayed in the inspector panel or in the hierarchy itself (but neither of those options actually works yet, right now it just retrieves the proper list of names from the database and lets you click on one of them).
Spent quite a bit of time constructing the document that Jim and Scott and I discussed which will afford discussion on a specification for navigation within the Atlas. I didn't actually write the document, but I did set out all of the sections and points that I think are germaine to the problem.
Got the Knowledge Manager's database access routines working; now when you click on a term in a hierarchy, the concept inspector fills in all the info from the "terms" table correctly.
Installed LaserWriter Bridge on the Sage lab computer to allow their Macs to share a localtalk-only printer. Seems to work OK, time will tell.
Showed the newest Atlas software to the DigAnat group with JWS. As a result of some discussion over presentation of text information, I changed the name of the "Glossary" window to "Attributes".
Discussed deliverables and more specific data handling protocols with Jim, Shelley, and CR. Plan is to make the output match his input and try it out with Shelley and I actually proofreading the output.
Showed working indented-hierarchy tree browser to Jim, discussed implementation options for editing functions.
Learned about Exabyte-tape handling and figured out how to pipe the output of MacDump to it.
Installed MacDump on Cortex and Shelley's PowerMac, tested various setups and operations.
3:20 PM -- Knowledge Manager can now load and navigate a group of hierarchies, opening and closing subtrees with a double-click.
Helped Ellen, a new staff member, wade through the process of creating an accounton MAX.
Spent practically all day figuring out how to make the knowledge manager start up and open a file when you double-click it in Workspace Manager.
Got the Knowledge Manager to the point where it will show a list of hierarchies in a browser, and load the proper indented list into the browser when you click one of them. Just works with static files.
I still haven't thought of a good way to let the user drag terms around the hierarchies. I'm thinking of making it purely a noun-verb operation, selecting a single line in the hierarchy and allowing a variety of operations: (1) double-click for opening/closing the subtree below the selected node (2) menu command to add empty line before/after the selection (3) changing the hierarchical level of the selected node (L/R arrow keys?) (4) either "cutting" the term for later pasting in a different position, perhaps with a multiple-object pasteboard, or two different browsers (yuk), or some other mechanism
Added details of networking strategies and protocols to the Atlas technical documentation web page.
Also fixed a small problem with Sandi's PURCH93 database. Bill says that a new system based on Sybase is planned for the 95-97 biennium starting this summer. That'll be nice.
Added a couple of fixes to v2.3 of the Atlas software, to make the handling of multiple movies kinder to the user.
Sent the first iteration of the UMLS data extensions off to the NLM.
Assimilated the UMLS/SNOMED identifiers sent by Mohamed into the database and took care of another small problem or two; e.g. about 100 terms containing two space characters in a row, 6 of which were identical to other terms when the spaces were removed.
Found the missing library that was preventing Metrowerks C from generating a PowerPC native Browser with QuickTime support (see 3/31/95). Spoke with Chuck from HSCER about what needs to be done before sending this fix out. There are some logic issues about when certain dialogs should appear and what they should say, and some more important questions of design and integration (the QuickTime movies, if viewed as just another Browser datatype, show up in a window which is conceptually almost exactly like an image window, so we're going to have to solve the problem of how "multiple images" fits in to the navigation/window-management strategy. But probably none of this needs to be pursued just in order to get a fix shipped in the next couple of weeks.)
Gathered log files from the last year's Browser usage and set up a working directory with some tools for analyzing them. It's all on neuron, in /usr/people/skandha/logs.
Set up a source & data backup procedure to automate the process of taking weekly snapshots of the UMLS contract data sources.
Altered the rule list for CR's terms table to make the "Preferred Term" role be "Preferred Name" instead, which is more in keeping with UMLS usage.
Finished alpha-test version of Atlas program with direct QuickTime player windows. Had to convert to MetroWerks C for the 68K version, because THINK C v5.0.4 headers are too incompatible with the new ones to work without lots of changes. However, the MW PowerPC compiler somehow would not resolve the QuickTime library calls so for now the fix will be 68K-only, running under emulation on PowerMacs. No big deal, running native doesn't really buy much anyway because we spend most of our time in mixed-mode MacTCP or in QuickDraw routines.
Finished first draft of data-file generation utilities for writing the machine-readable sources to send to NLM containing names and links from our database. Had to roll my own tools to generate indented hierarchies, pushing the data this way and that, because XLisp dies with bus errors/other assorted anomalous behavior when the size of a tree exceeds about 100 nodes. No big deal, it works for now and in the future we won't be doing it this way (I hope).
Met with HSCER programmer to talk about possible strategies for dealing with problems the IBA has when installed under At Ease. We'll probably rush a bare-bones QuickTime player mechanism into the existing code and ship it that way, in two to three weeks if all goes well.
Instituted a new window in the Atlas program, called the "Info" window, which displays persistent (non-dialog) messages separately from the Glossary mechanism. Compiled latest version of the program as 2.2b1.
Extended the capabilities of the C and Lisp programs for parsing and loading UMLS contract data, to deal with certain anomalies in the input stream and to make provisional test output files.
Modified the FrameBuilder program, from 1.4 to 1.5, to allow the author to paste text from the clipboard into the FRAME-INFO field in the options dialog; JWS will use this for showing descriptive text about his histological sections.
Wrote the Lisp code for taking parsed terminology data files and entering the data to the Sybase tables. (Functions CR-PREFERRED-TERM, CR-SYNONYM, and CR-ADD-LINK.)
Wrote "ih-parse", a C program to take the "Generic" formatted files from WordPerfect and get them into a form that a Lisp reader can deal with easily. It figures out what hierarchy is being built, locates preferred terms, synonyms, and semantic types, and puts out well regulated files without pesky stray characters.