Notes re. the tv.1 interface, working toward tv.2

Jeff and Scott, thanks for your notes.
I made a categorized, "scroll-like" list of outstanding interface issues,
All "decisions" are pending Roger's response when he gets back.
Let's move some of these along...

== CURRENT STATUS ========================================

2nd generation HTML prototypes are on

   http://origin2.daylight.com/~dave/tv2/

and downloadable from

  origin2:/usr/people/dave/tars/tv2.tgz

The main changes are:
  o  a cleaner look per Jeff's and Scott's suggestions
  o  navigation bar on each page
  o  server identification in the title (!)
  o  a documents page, news and release notes pages
  o  the home page is index.html

Things not done include:
  o  new performance charts (Jeff and Roger should talk about these)
  o  logo change (Scott, will you look at this?)
  o  limitations to time, bandwidth, whatever (Roger, please comment)


== OUTSTANDING ISSUES ========================================

[ ]----- SERVER IDENTIFICATION

DW> Are we ever going to have more than one Terravivo server per network?
    Or for that matter, per host?  Assuming it's possible, we need to
    identify the one we're talking about.  I've used the IP number throughout
    (e.g., 207.225.60.9) but a string indicating host:service name would be
    better (e.g., "origin:terravivo2").
JB> I can't think of why we'd need more than one TV server per host or
    network, but it seems like a good idea to specifically name it and
    reference that name on the interface pages.
DW> Ah-ha!  I moved the server identification to the HEAD/TITLE bar which
    seems to work out great -- it's always available yet out of the way.
    I'm using the string "origin2:terravivo" but "207.225.60.9:terravivo"
    would be better yet.


[ ]----- NAVIGATION

SD> [ ... likes frames or sidebars, gave example ]
JB> [ ... doesn't like frames ]
JB> The Heidelberg SRS database search system has an interesting ...
DW> Ach du lieber!  That's one way to obfuscate a simple idea.
    This is a good example of the the problem with the current Daylight
    website opening page (and why we're going to change it soon!)
DW> I don't like frames either.  And form-based popups seem heavy-handed.
    Something that I've seen work well is a header "navigation bar" on all
    pages: it can be both convenient and unobtrusive (as long as it doesn't
    include obnoxious graphics!) I've implemented a nav bar on the 2nd
    generation set of HTML pages.  I think this is plenty strong and
    addresses Jeff's concerns about losing simplicity.  Will this work?


[ ]----- PAGE TITLES

JB> I suggest we just call the TV home page "TerraVivo - main menu" and
    let home page refer to our Santa Fe site.
DW> Well, I don't like "main menu" much.  For one thing, currently it
    doesn't look much like a menu (even less so with the navigation bar).
    Plus, we'll probably put other stuff on there as time goes on which
    might be even less menu-like.  My current feeling is that the main
    page should be called "Terravivo home page", which corresponds to
    the navbar "Home" link.  That being said, something other than "home"
    or "menu" might be better yet, if we could think of it.


[ ]----- STATUS PAGE

JB> For the status page, the first block and manager's message are good.
    I don't find the status table useful.  I think the most important
    data to communicate is: am I up to date and reliable?  That's why I
    envisioned leaving most of this (redundant) data accessible to users
    through the config menu.
DW> In the current scheme, only the manager has access to the configuration
    page.  IMO, the table on the status page is doing just what you intended,
    i.e., what is available, is it current, access warning messages.
    I'm not sure users ever need to see the "primary server" column, but
    it's good for orientation.  I particularly like the idea of showing
    the users all possible resources (instead of just the available ones).

[ ]----- PERFORMANCE PAGE
    
JB> ... envisioned a status menu with several bar graphs and a XY-plot.
    The graphs should convey a much clearer picture of the system's
    status than a table of numbers.  Why show numbers if you can show a
    graph - unless the invidual values are important. These aren't.
    To summarize again, the graphs are:
    * bar: %up-to-date for each database as of now.
    * bar: 2 bars per database, size in Mb and # sequences for each
	   database as of now.
    * bar: #consecutive failed updates for each database as of now.
    * bar: 2 bars per database. avg # user accesses per day for this month,
	   and since startup, for each database
    * XY plot of free disk space (Y) vs time (X), sampled daily.
      Probably don't need regression curve.
DW> I agree that we need information of these types.  Graphs are OK
    (though they don't replace data tables which can be linked).
    Most of these don't make sense to me:
    * bar: %up-to-date .... what does this mean?
    * bar: size in Mb ... in what apparent format or internal size?
    * bar: #consecutive failed updates ... I- II- or total failure only?
    * bar: user accesses .. MB delivered or number of times touched or what?
    * XY plot of free disk space ... note that this can change invisibly
      and dramatically depending on the current internal format and
      compression policy for a database.  Should this be replaced by a
      warning if they're running out of room or something like that?


[ ]----- CONFIGURATION PAGE
    
JB> For the configure menu, how do you reach it if you're not the manager
    - or is this page reserved for only the manager?
DW> It's for managers only.

JB> I'd replace "incept date" with "startup date".  Incept is an unfamiliar,
    rarely used word; why use a fancy word when you can use a simple one?
DW> Just having a little fun ... it's a reference to Blade Runner ... one
    of the "bugs" in Nexus-6 replicants is that for some reason they don't
    know their incept dates.  Our robots will know better, so they don't
    have to come back and destroy us ...

JB> The proxy server port # is missing.
DW> No it's not ... see the configuration page, bottom line first section.

JB> The menu only lists the installed version of tv; I'd like to see it
    also list the latest version that's available.
DW> It's self-installing, so why would there ever be a difference?
    As I understand it, the installation will come from the same place as
    a site would get the information that an update is availale.  Not doing
    the update is not a choice, it's automatic.  If an update fails for some
    reason, then the server status will be "degraded" and there will be
    warning/error messages.

JB> I don't see any need for a 6 hrs update choice.
DW> How frequently do you think the factory terravivo server should be
    polled? In any event, these choices will be configurable on-the-fly
    for each resource, independently.

JB> Is 'once only' equivalent to 'now'?  We should provide an immediate
    update feature in case somebody needs the data RIGHT NOW.
DW> No, it means "get it once then don't update it".  It might be gotten
    now, or part of it now (perhaps over several days, as normal scheduling
    permits).  But once it's complete, it's not updated.

    The idea of "do it RIGHT NOW" is interesting.  That would be a special
    action rather than a scheduling option and I guess it would allow
    terravivo to over-ride the limitations (just go for it, use as much
    bandwidth as possible?).

JB> we provide a default update choice: 'let TerraVivo decide', which
    would poll our Santa Fe site, compare each database's status, and
    determine whether an update is necessary.
DW> That's the "poll" option, used in the software (which isn't updated
    very often).  My gut feeling is that "poll" is not ideal for everything.
    It would be nice to have these servers operate independently ... e.g.,
    if they can't connect to the terravivo server, they keep running OK.
    In the config menu, the choices marked with a '*' are the factory
    recommendations based on how frequently the resource is updated.
JB> This probably won't be practical until we have biothor, though.
DW> I don't see what biothor has to do with this issue.

JB> I'd like another column in the configure menu to select email
    addresses of users who should receive email status updates ...
DW> Yes, that's the "agent feature", which I assume will not be part of
    the initial release.  When we implement agents, I think it should be
    oriented per user (or per e-mail address) *not* per database due to
    interface scaling issues.  Do you really want to scroll through the
    list of all other users interested in a particular resource?  Even
    if so, should you be able to?  My "knee jerk" approach to an agent
    interface would be to do something similar to the configuration page
    used by the manager.  Another approach would be to have the agent
    operate completely by e-mail (subscribe, unsubscribe, get status, 
    get data) and limit the website to mailto: shortcuts, management
    and perhaps another chart on the performance page.

JB> We also need to add a limit on the total amount of time allowed for
    each tv update (not per database).
DW> Or something like that.  Bandwidth utilization is perhaps more
    relevant than time (e.g., MB/day).  Need to work on this.

JB> We should provide both "Restore" and "Factory defaults" at the
    submit changes section.
DW> Or perhaps an explanation that Shift-Reload is Restore.

[ ]----- INITIAL CONFIGURATION

JB> Roger and I discussed shipping systems preconfigured with data already
    loaded.  This gets impractical really fast ...  we should ship empty
    systems ...

DW> Why?  If we could build a purely self-installing system which is
    practical, I'm all for it.  But I don't see why preconfigured data
    would be "impractical".  Contrariwise, it seems that there are lots
    of good reasons to do so: terravivo will bring itself up to data from a
    given state, so why not ship the latest convenient state for commonly-
    requested databases such as PDB and GenBank?  It seems that this would
    get people rolling quickly, allow near-instant installation testing,
    and save a lot of bandwidth.  In the extreme case of a slow connection
    (the 3K/S we're familiar with) downloading 100 GB would require 414
    full-time days!  If there are copyright problems with some databases,
    we can deal with them or if that's not possible, they wouldn't get
    preloaded.  But what other types of impracticalities am I missing?


[ ]---- CONTROL LANGUAGE

DW> I think we need to develop a control lanugage for terravivo downloading.
    I'm kind of a tree-head, so I was thinking of a control file structure
    which looks something like this:

      /*** terravivo update 42 ***/

      terravivo { "Terravivo Release 42" }
      {
        version   { 42 }
        released  { "15-Dec-1998" }
        copyright { "Copyright 1998 (c) Metaphorics LLC" }
        ... various resources ...
        resource { "h_pylori" }
	{
          name   { "Helicobacter pylori complete genome" }
          type   { "Complete genome" }
	  freqs  { "daily", "weekly", "monthly*", "once only" }
          primo  { "ncbi.nlm.nih.gov" }
          secundo{ "ftp.tigr.org" }
          source { "ncbi.nlm.nih.gov" }
	  {
	    name  { "National Center for Biotechnology Information" }
	    domain{ "ncbi.nlm.nih.gov" }
	    method{ "ftp" }
	    file  { "/tv/genomes/h_pylori/h_pylori.fasta" }
	    {
	      src  { "/genbank/genomes/bacteria/Hpyl/hpyl.faa" }
	    }
	    file  { "/tv/genomes/h_pylori/h_pylori.genebank" }
	    {
	      src  { "/genbank/genomes/bacteria/Hpyl/hpyl.gbk" }
	    }
          }
          source { "ftp.tigr.org" }
	  {
	    name  { "The Institute for Genomic Research" }
	    domain{ "tigr.org" }
	    method{ "ftp" }
	    file  { "/tv/genomes/h_pylori/h_pylori.fasta" }
	    {
	      src  { "/pub/h_pylori/GHP.seq.gz" }
	      conv { "gunzip -c % | seq2fasta"  }
	    }
          }
	  ... other sources ...
        }
        ... other resources ...
        resource { "terravivo" }
	{
          name   { "Terravivo updates" }
          type   { "Software" }
	  freqs  { "6 hours", "daily", "weekly" }
          primo  { "terravivo.com" }
          secundo{ "metaphorics.com" }
          source { "ftp.terravivo.com" }
	  {
	    name  { "Terravivo update mirror in Los Angeles" }
	    domain{ "terravivo.com" }
	    method{ "ftp" }
	    file  { "/tv/htdocs/news/news19981025.html" }
	    {
	      src  { "/updates/42/news/news19981025.html" }
	    }
	    file  { "/tv/bin/swsearch" }
	    {
	      src  { "/updates/42/bin/swsearch"  }
	      post { "chmod 755 /tv/bin/swsearch" }
	      sum  { "50226 31 /tv/bin/swsearch"  }
	    }
	    file  { "/tv/bin/rasmol" }
	    {
	      src  { "/updates/42/bin/rasmol43"    }
	      post { "chmod 755 /tv/bin/rasmol4.3" }
	      sum  { "42901 31 /tv/bin/rasmol43"   }
	      post { "ln -s /tv/bin/rasmol43 /tv/bin/rasmol" }
	    }
	    ... other software, scripts, etc ...
	  }
          source { "ftp.metaphorics.com" }
	  {
	    name  { "Metaphorics, LLC." }
	    domain{ "metaphorics.com" }
	    method{ "ftp" }
	    ... file...
	  }
	  ... other sources ...
	}
        ... other resources ...
      }
      /*** end of terravivo update 42 ***/

    You'll note that the above is very EZ/CEX-oriented, essentially an
    object-property description (the first, unnamed property is the
    stringvalue).  It's not hard to see how the HTML/CGI interfaces can
    be written from such a description, the same goes for the makefiles
    but perhaps that's a bit trickier (Roger?!).  Note that "our" file
    /tv/genomes/h_pylori/h_pylori.fasta can be generated from tigr's
    GHP.seq.gz with the specified conv(ersion) method using gunzip and
    seq2fasta.
    
    Outstanding issues include figuring out whether such a specification
    should dictate the method exactly (e.g., using ftp, gunzip, etc.)
    or would we be better off using a jake-like system with macros to
    do things like FTP(src,dst,hints), checksums, post-processing, etc.
    I like jake a lot, but if we're going to be on a stable platform,
    we might just as well download the actual makefiles (though that
    makes me nervous, too).


== ISSUES NEAR CONCENSUS ========================================

[x]----- OVERALL APPEARENCE

SD> [ prefer transparent background cells to white cells in tables ]
DW> OK.  The 2nd generation tables have transparent backgrounds.
    However, editable configuration fields should still have a white
    background, to make them distinctive, right?


[x]----- THE LOGO

SD> I think the logo is way to green/cyan
JB> the Metaphorics logo looks OK on my machines (perhaps it could
    be a bit darker).
SD> Does the attached gif look any better?
JB> Yes, your gif is better.  But the light part of the shadows still
    shows up as green/cyan when I think that the shadows should fade to
    transparent or neutral instead.  I've attached the one I used in the
    webpage I put together months ago.
DW> Jeff's right: Scott's logo looks better but needs to fade to background.
    BTW, on my machine, Jeff's logo isn't a transparent GIF.

    Scott, could you make an un-cyanized "200-size" logo which fades to the
    "sandy.gif" background?  This would be exclusively for use on the sandy
    background, call it something like "MetaSandyLogo200.gif"?

== SETTLED ISSUES ========================================

[x]----- THE NAME "Terravivo"

DW> "terravivo" is fine with me
JB> TerraVivo is as good a name as any I've heard yet.
YT> We got the domain name "terravivo.com" from InterNIC:
    > REFERENCE No.: 55256
    > DOMAIN NAME:  TERRAVIVO.COM
    > REGISTRANT (Owner): Metaphorics LLC


[x]----- THE BACKGROUND

DW> The background that we previously used (sand.gif) was a bit dark,
    making it hard to read black text, so I lightened it up and used
    that (sandy.gif).
JB> The lighter background looks good
ALL> We'll use "sandy.gif" as our web background.


[x]---- DOWNLOAD PROTOCOL

JB> Does it matter whether we use http or ftp for downloads?
DW> No.  As I indicated, users/manager see the server domain, but the
    details (host, directory, protocol, etc.) are terravivo's problem,
    not theirs.  Terravivo can do anything it wants to get the job done.


== BOTTOM LINES ========================================

[ ] "Terravivo" will be the product name.  We'll probably make the
    domain terravivo.com do terravivo-specific things (like updating).
    We'll recommend the mount point /tv (and document things that way).

[ ] We'll use "sandy.gif" as our web background, "sand.gif" for accent.