Summary of IRC Meeting 2007-02-27:

Sorting API:

store.find({sort:[{attribute:'last-name', descending:true}, {attribute:'first-name', descending:false}, ...]});
store.find({sort:[{attribute:'last-name'}]}); // descending defaults to false

Paging API:

var result = store.find({query:'hobbit', count:20}); // count is *optional* hint for num items on initial result
     // count is passed on as property on result
result.fetch({onItem:callback, start:0, count:20});            // get items 1 to 20
result.fetch({onItem:callback, start:20, count:20});            // get items 21 to 39

// or more simply...
(new dojo.data.CsvStore?({url:"movies.csv"})).find().fetch({onItem: callback}); //for all items
(new dojo.data.CsvStore?({url:"movies.csv"})).find().fetch({onItem:callback,/*std callbacks*/ ,start:50, count:40}); //for items 50-89
// standard callback functions are onBegin, onItem, onCompleted, onError
// onItem is passed an item handle as param to func.  Intended use is for being notified upon arrival of each item.  onCompleted is notification that all items in requested range have been received.

Caching
We also have agreement that result caching should be built into the result/datastore implementations (optional feature).  Cache params can be specified as additional datastore-specific caching options when instantiating the datasource.

---

ISSUES DEFERRED
The following issues were not discussed due to time constraints.  They will be discussed via contributor list and finalized on next Tuesday's IRC (or on this Wed's weekly meeting agenda, time permitting)

• string-matching for Select widget (Combobox):
  • find only the items where name begins with "ca" // or where name includes "ca"
  • possible syntax alternatives:
  • 1) Simple wildcard (single character, and multi-character patterned search)
    • store.find({query:{name:"ca%"}}); // or "%ca%"
      • maps direct to db LIKE, requires simple mapping onto regexp
        
    • store.find({query:{name:"ca.*"}}); // or ".*ca.*"
      • easy to map onto db LIKE, easy regexp mapping
        
    • store.find({query:{name:"?alifornia"}}); // match California or california or Xalifornia...
      • works with both db and regexp
        
  • 2) Full regexp
    
    • store.find({query:{name:/^ca/}}); // or /ca/
      • difficult or impossible to implement on db side
    3) Complex abstract query
    
    • store.find({query:{name:{beginsWith:"ca"}}}); // or {includes:"ca"}
      • difficult to implement on client side
        
  • problem: How do we layer this on top of a datastore that already has its own notion of what a query looks like? For example, here are some dojo.data.DeliciousStore queries:
    • store.find({query:"gumption"}); // "gumption" is a user name
    • store.find({query:"gumption/art+history"}); // find anything for gumption matching art history
    • store.find({query:"tags/gumption"});  // find all entries where tags=["gumption"]
      

CM: This example is both a complex query and full text search...
eg. 'Where user is gumption and text contains "art history"'
Seems very similar to (3) Complex abstract query above
but also requires a CONTAINS function to do full text search on a text field.
Skinner: actually, in the case of del.icio.us, the "art+history" part is not a full text search -- it's a search for items with the tags "art" and "history"

Should we support complex queries and full text search in the client at all?  This can get extremely complicated trying to implement in the client side...even if we could come up with an abstract syntax to map down on the server side.

Maybe this is where a hook for allowing a user-defined client or server search functions could be used.
Another option is to support a "passthrough" to the datastore implementation.  In otherwords, always treat the string version of find() as a direct passthrough to the datastore implementation

store.find("gumption/art+history");
store.find("SELECT * FROM FOO WHERE BLAH IN ('A','B') AND GLOM CONTAINS 'SPLAT' ");

Keese:
store.find({query: {user: "gumption", search: "art history"} });
       --> find all of gumption's art history bookmarks
store.find({query: {tag: "gumption"} });
       --> find all bookmarks tagged as "gumption"

Data Binding Discussion

• Common accessor path notation for general XMLStore scenario vs. JS stores is no longer an issue.  See details here.
  

Other questions

1. what does "widget" mean
   • widget == a UI widget, which you see on screen
   • widget == either a UI widget or an "invisible widget" (a widget is anything that may appear in HTML declarative markup, including binders, transform widgets, etc.)
   CM: We need a parser for declarative markup for things that are non-visual.  Call it declarative parsing into objects or components if we want to make the distinction...  I think we should call UI components widgets, and non-visual components objects if widgets can inherit from component to get the markup behavior, that's even better.  Widgets have a different set of events in their lifecycle than the simple wrappers that sit on top of js data structures (ex. ).  Why can't we just use the same parser & manager util from dijit that is dojo.query based to provide the markup parsing for dojo.data use cases?  Maybe changing the name of digit's util package to something like 'markup' and putting it in core, so that dojox.data doesn't need to have a dependency on dijit to get access to the markup processing functions?
   
2. What should the core dojo.data.core.Notification API look like (independent of how it appears in declarative bindings)  Here's some thoughts.
   

---

Issues Discussed
Data access api's:

• sorting:
  • Option 1:
    1. store.find({sort:'last-name'});
    2. store.find({sort:['last-name', 'first-name', ...]});
    3. store.find({sort:{descending:'last-name''}});
    4. store.find({sort:[{descending:'last-name'}, {ascending:'first-name'}, ...]});
  • Option 2:
    1. store.find({sort:[{attr:'last-name', direction:0}, {attr:'first-name', direction:1}, ...]});
    2. store.find({sort:[{attr:12, direction:0}, {attr:13, direction:1}, ...]});
    3. store.find({sort:[
           { attr: value, direction:1||0 },
           { attr: value, direction:1||0 }
       ]});
    4. // where attr is essentially the column index (attribute name), and direction is asc (1) or desc (0).
    5. Filtering table does something like this today and 0|1 is more efficient than string processing... it would be familiar to users
  • Option 3:
    1. store.find({sort:['last-name']});
    2. store.find({sort:['last-name', 'first-name', ...]});
    3. store.find({sort:['-last-name']});
    4. store.find( { sort:['-last-name', 'first-name', ...] } );
    5. Skinner: this option requires that all the attribute parameters must be strings. Most of the rest of the dojo.data API allows an attribute parameter to be either a string or an object that represents the attribute.
    Option 4: WINNING OPTION!
    
    • store.find({sort:[{attribute:'last-name', descending:true}, {attribute:'first-name', descending:false}, ...]});
    • store.find({sort:[{attribute:'last-name'}]}); // descending defaults to false
      

• paging:
  
  • Option 1:
    • store.find({start: 31, limit: 10}); // limit is maximum num items to return, start is initial offset
      
    • result will have two standard optional properties: start, limit
    • User can change the start (or limit) values on a previous result and pass result back in to find() to get another chunk of items.
      
    • // 1A: to get just the first 20 items
        store.find({query:'hobbit', limit:20}); // get items 0 to 19
      
        // 1B: to get 20 items, and then another 20, then another 20
        var result = store.find({query:'hobbit', limit:20});
        result.start = 20;
        var result = store.find(result); // get items 20 to 39
        result.start = 40;
        var result = store.find(result); // get items 40 to 59
      
        // 1C: to get the first item
        // (every dojo.data datastore is required to have the first item at 0,
        // even if the underlying database thinks the first number is 1)
        store.find({query:'hobbit', start:0, limit:1}); // 0 is always first
      
        // 1D: to get the second item
        store.find({query:'hobbit', start:1, limit:1}); // 1 is always second
      
  • Option 2:
    • var result = store.find({query:'hobbit', limit:20});
    • result = store.findNext(result)       // get items 20 to 39
    • result = store.findNext(result)       // get items 40 to 59
    • Setting the result.start again seems like a PITA when paging like this is so common. 
    • (Is the name "findNext" potentially confusing, since we already have an "onnext" argument in the find() method, where "onnext" is talking about the next item but "findNext" is talking about the next chunk of items?)
      
  • Option 3
    • var result = store.find({query:'hobbit', limit:20});
      
    • result = store.findPage(result, 1)            // get items 20 to 39
    • result = store.findPage(result, 2)            // get items 40 to 59
    • (where the first page is #0, second is #1, etc) and the findPage call uses the "limit' on the original result set to figure out how many items to get.  This supports the Gooooooogle style paging--just pass in which "o" they clicked on. 
      
  • Option 4
    
    • var result = store.find({query:'hobbit', limit:20}); // limit is max items on initial results
      result = store.findRange(result,20,39)            // get items 20 to 39
      result = store.findRange(result,40,59)            // get items 40 to 59
      
    • [not part of proposal...for clarification only]Paging api (Option 3) can be implemented on top of this as follows...
      var result = store.find({query:'hobbit', pageSize: 10 limit:20}); // pagesize is num items per page, pageSize defaults to limit if unspecified
      result = store.findPage(result,1)            // get page 1 (items 1-10)
      result = store.findPage(result,2)            // get page 2 (items 11-20)
  • Option 5 (separate fetchRange functoin)
    
    • var result = store.find({query:'hobbit', limit:20}); // limit is max items on initial results
    •      // limit is passed on as property on result
      result = store.fetchRange(result,20,20)            // get items 20 to 39
      result = store.fetchRange(result,40,20)            // get items 40 to 59
  • Option 6 (find option)
    
    • var result = store.find({query:'hobbit', limit:20}); // limit is max items on initial results
    •      // limit is passed on as property on result
      result = store.find(result,20,20)            // get items 20 to 39
      result = store.find(result,40,20)            // get items 40 to 59
  • Option 7 (find/fetch split on store/result) - WINNING OPTION!
    
    • var result = store.find({query:'hobbit', count:20}); // count is *optional* hint for num items on initial result
    •      // limit is passed on as property on result
      result.fetch({onnext:callback, start:0, count:20});            // get items 1 to 20
      result.fetch({onnext:callback, start:20, count:20});            // get items 21 to 39
    • // or more simply...
    • (new dojo.data.CsvStore?({url:"movies.csv"})).find().fetch({onItem: callback}); //for all items
      
    • (new dojo.data.CsvStore?({url:"movies.csv"})).find().fetch({onItem:callback, start:50, count:40}); //for items 50-89
    • // standard callback functions are onBegin, onItem, onCompleted, onError
    • // onItem is passed an item handle as param to func.  Intended use is for being notified upon arrival of each item.  onCompleted is notification that all items in requested range have been received.
      
• caching:  AGREED THAT CACHING SHOULD BE BUILT INTO DATASTORE IMPLS (Optionally)
  
  • We don't want the widget to have to do caching...can be built into datastore impls
    
    • eg. (Note: This example is for illustrative purposes...the actual find api is Option 7 above)
      
    • var result = store.find(... start: 1, size: 10);
      ...
      result.range = {start: 11, size: 10};
      result = store.find(result);
      ...
      result.range = {start: 21, size: 10};
      result = store.find(result);
    • without going to the server every 10 rows.  
      
    
    

    ---

Transcript:

   <ttrenka>    i'm sorry, I know I haven't been paying super attention
    <brian_skinner>    wildbill: even though find() does fetching work?
    <ttrenka>    but having two different methods seems wrong
    <ttrenka>    at least of the same basic name
    <ttrenka>    if find is the initial query then the ...Range methods are essentially cursor functions, right?
    <ttrenka>    (making sure I understand this correctly)
    <wildbill>    right.
    <brian_skinner>    maybe
    <wildbill>    normally, like with databases, the so-called find() method doesn't return data but the dojo.data folks were trying to simplify the simple case where you do one query and get all your data in a single function call.
    <ttrenka>    ok.
    <brian_skinner>    i think when the actual query execution happens is an implementation detail -- it could happen in the find() method or it could happen in each fetchRange() method
    <ttrenka>    Something about this is really bothering me but I don't have the bandwidth to think about it hard right now :(
    <slightlyoff>    brian_skinner: yep
    <slightlyoff>    ok, so where are we?
    <wildbill>    brian, do you have more questions?
    <ttrenka>    we're at "tom shuts up and gets back to writing slides about async programming"
    <slightlyoff>    ttrenka: heh
    <brian_skinner>    if we go with fetchRange() and find(), then I think it's weird that find() actually does fetching, just like fetchRange(), yet we don't call it fetch
    <slightlyoff>    brian_skinner: yeah, agreed
    <slightlyoff>    brian_skinner: what's the fundamental difference between what they do?
    <brian_skinner>    (i think i actually voted to call if fetch back in October :-)
    <brian_skinner>    slightlyoff: in my opinion, there is no fundamental difference :-)
    <brian_skinner>    store.findRange({query:'hobbit'}, 20, 39); == store.find({query:'hobbit', start:20, limit:10});
    <jared_j>    Find() executes the intial query and fetch interates/slides along views into the query is how I see it. Sematic difference.
    <wildbill>    right
    <wildbill>    just like on google, hitting next-page button is different than doing a new query.
    <slightlyoff>    jared_j: why is that not an internal detail of the store?
    <ttrenka>    slightlyoff: right
    <slightlyoff>    wildbill: actally, it might be
    <jared_j>    Could be internal to the store.
    <slightlyoff>    wildbill: it's not ours to know = )
    <wildbill>    i'm not talking about the implementation
    <slightlyoff>    so we can just elide this away with a fetch() method?
    <wildbill>    i'm talking about the "api"
    <slightlyoff>    wildbill: understood
    <slightlyoff>    wildbill: but all it gives you is offset and length
    <peller>    it suggests just that. You're not necessarily doing the same thing with the two calls (though you might be)
    <peller>    as long as it's two different function names, I don't care.
    <jared_j>    I think part pf it also may be you could use the result itself to hold part of the 'cache' being used for the initial query.
    <slightlyoff>    peller: why should they be different?
    <slightlyoff>    peller: how does it affect your usage?
    <jared_j>    Otherwise a single datastore could have iddues caching if multiple widgets are executing completely different find() calls.
    <ccmitchell>    store.findRange({query:'hobbit'}, 0, 39); == store.find({query:'hobbit', limit:39});
    <peller>    it implies that there's some state retained by the store
    <ttrenka>    (slightlyoff: *this* is what I meant by "good you're here, you can take over)
    <peller>    it probably doesn't matter to usage, but gives some insight on what it's doing
    <jared_j>    peller: Right
    <jared_j>    And how would state maintaining work if the same DS instance has multiple widget uses in terms of caching?
    <slightlyoff>    peller: but that's the point of the data APIs....that's not something you should ever be presented with
    <slightlyoff>    jared_j: provider specific
    <brian_skinner>    i think we're going to have a lot of simple datastores that don't ever "execute an initial query" -- they just read data from an HTML table or an XML data island or a CSV file or a web-service
    <slightlyoff>    jared_j: LRU, perhaps
    <peller>    and also, it's two different signatures. I didn't like overloading being in the mix
    <jared_j>    slightlyoff: Okay. Leave it up to the DS. :)
    <slightlyoff>    peller: no, I get what you're saying, but what's *different* about what they do?
    <brian_skinner>    peller: we can avoid having two different signatures
    <peller>    you're passing it back state. That's about it
    <peller>    brian_skinner: are you talking about the original proposal where the query was passed in each time?
    <brian_skinner>    yup
    <wildbill>    would we ever pass back the state for a new query? i can see that happening for combobox
    <peller>    brian_skinner: I suppose the data store could figure out if the query is unique, but that seems awkward
    <peller>    brian_skinner: then the datastore has to worry about more than it should, IMO
    <ccmitchell>    Can we take a vote and move on for now? OPTION1: store.findRange({query:'hobbit'}, 20, 39); OPTION2: store.find({query:'hobbit', start:20, limit:10});
    <brian_skinner>    peller: the result object always includes the original query anyway -- so in either case we're actually always passing back the query
    <jared_j>    But the result object can host other things the datastore may use, like the cache details for a particular query.
    <slightlyoff>    ccmitchell: #2 +1
    <ccmitchell>    #2 +1
    <wildbill>    huh? you need to pass back the result set
    <ttrenka>    #2
    <brian_skinner>    ccmitchell: i'm not sure those are the only two options -- neither of those options has the features peller want
    <ccmitchell>    oops, i copied wrong entry
    <peller>    I'm not sure what I want :) It just seemed like we might need to preserve state with the result set. Do we?
    <peller>    and if so, do we want to push that responsibility back on the datastore?
    <peller>    am I making any sense?
    <brian_skinner>    peller: i'm not sure i understand
    <wildbill>    i'm not sure what "preserve state" means
    <peller>    a cursor
    <peller>    perhaps I'm thinking too much about the backend.
    <peller>    it may not matter
    <brian_skinner>    the datastore can hang cursor info on the result object if it wants to
    <peller>    and where's that cursor in #2?
    <brian_skinner>    and use that cursor if the same result object is passed back in
    <jared_j>    bran_skinner: But if you never pass back the result set, the cursor on it would be meaningless, yes?
    <slightlyoff>    wait, peller, do you *want* a cursor?
    <brian_skinner>    jared_j: right -- so for any UI code that cares about performance, you'll pass the same result object back in
    <brian_skinner>    but you don't have to
    <jared_j>    Ah, so you're saying the first parameter can be the result set *or* the query.
    <peller>    slightlyoff: no, I probably don't. I'm probably looking for a concept that doesn't exist on the client.
    <brian_skinner>    jared_j: yup
    <slightlyoff>    peller: and shouldn't have to!
    <peller>    I guess it depends on the protocol
    <slightlyoff>    peller: a good data store should cache-and-forward to a server-side cache
    <peller>    is a protocol likely to require such context?
    <slightlyoff>    peller: probably not
    <peller>    if we can make that assumption, then sure, life just got simpler
    <slightlyoff>    peller: the server-side interface should do that if it's required
    <wildbill>    hmm
    <peller>    slightlyoff: that's not going to scale well. We just put a requirement on the server it doesn't need.
    <wildbill>    right
    <slightlyoff>    peller: ?
    <peller>    without actual protocols, I'm just making all this up
    <slightlyoff>    peller: the client can't do it better!
    <wildbill>    cursors
    <peller>    we're talking about 'cookies' here
    <slightlyoff>    peller: and even if you want the client to manage it, that should be transparent o the data store user
    <peller>    precisely. which is why we were passing around resultset objects
    <peller>    or so I thought
    <ttrenka>    once again I'm not paying super attention but it really sounds like you guys are trying to recreate ADO.NET
    <peller>    ugh.
    <peller>    ok, don't pay too much attention to me then
    <ttrenka>    disconnected data sets that don't rely on server implementations? that's what it is.
    <brian_skinner>    ttrenka: is that good or bad?
    <ttrenka>    depends on your point of view
    <ttrenka>    :)
    <slightlyoff>    peller: don't we get a result object?
    <peller>    define 'we'
    <slightlyoff>    peller: can't/shouldn't that hold state for your data store?
    <peller>    of course
    <peller>    I think the question is, who holds that state
    <peller>    if it's the client-side datastore object, I think that's kind of ugly
    <jared_j>    The DS or the result object?
    <peller>    but it could work
    <slightlyoff>    peller: I guess I don't care so long as it's not the API user
    <ccmitchell>    brian, can you take a look at option 5 & 6 on wiki and clarify for vote?
    <peller>    if we pass a cookie back and forth on the fetches, it's really not that awkward and we preserve state
    <brian_skinner>    ccmitchell: will do
    <slightlyoff>    peller: that's up to the store
    <peller>    but I don't want to "overengineer" this if it's not a requirement
    <slightlyoff>    peller: not the API's problem
    <peller>    that's where you lost me. I think this all hinges on our paging API
    <peller>    how else would the store know? It would have to compare the query and maintain a cache of results?
    <slightlyoff>    peller: either the client or server would, yes
    <slightlyoff>    peller: or at a minimum, the last state
    <wildbill>    that's why are passing the result object back into fetchRange(). to avoid maintaining a cache of results.
    <peller>    or, worse case, you have state maintained in multiple places, which I think is what I'm afraid of
    <peller>    what wildbill said.
    <jared_j>    I think the idea with passing back the result set is that the DS can use the resultset to stash data on so it dpesn't have to maintain its own internal tables, etc.
    <slightlyoff>    wait, then what good is the data store?
    <peller>    it knows the protocol, and it can cache
    <slightlyoff>    I thought it was supposed to proxy caching and state?
    <brian_skinner>    ccmitchell: in either option 5 or option 6, i think the args should be (start and count), not (start and end) -- so, (result,20,10) rather than (result,20,39)
    <slightlyoff>    and then you pass around it's internal state in the result object?
    <slightlyoff>    ugg
    <ccmitchell>    bk: ok
    <peller>    slightlyoff: I think we're saying you *could* pass around state in the result object
    <peller>    in case it's needed for server interactions
    <wildbill>    there's another reason too
    <jared_j>    No one says you have to, only that the capability would be there.
    <slightlyoff>    peller: ...and now we have to expalain that too users
    <wildbill>    slightlyoff: this is a pattern that's been around for decades
    <peller>    slightlyoff: why? just because you pass the resultset object around? Seems intuitive to me
    <peller>    the user doesn't have to know what's in it
    <slightlyoff>    peller: the user, then, shouldn't have to be bothered!
    <peller>    I did a search. This represents the result. Get me items 100-200
    <slightlyoff>    peller: yep
    <ccmitchell>    skinner: final review wiki to prepare for vote
    <slightlyoff>    peller: then your "get me items 100-200" should be a method on your result object
    <peller>    well...
    <wildbill>    yeah, that's how all major apis work
    <peller>    brian_skinner had reasons why he didn't want methods on the result object
    * ttrenka    wishes slightlyoff was there in October...
    <slightlyoff>    peller: if that passes state back and forth to the store, it would be a dumb design, but you could do it
    <peller>    I forget the details
    <slightlyoff>    brian_skinner??
    <brian_skinner>    ccmitchell: my vote is for option 6
    <slightlyoff>    c'mon people, don you actually use these things?
    <brian_skinner>    or option 1
    <slightlyoff>    doesn't passing and tracking all this shit by hand seem brittle to you?
    <wildbill>    no, it's a standard pattern
    <slightlyoff>    that doesn't mean it's not crap
    <wildbill>    just look at jdbc, oracle, etc
    <ttrenka>    wildbill: not really.
    <ttrenka>    and those are great examples *why*
    <wildbill>    because they are the standard
    <ttrenka>    for SQL maybe
    <slightlyoff>    that others have self-flagelated is no excuse for us to do it
    <brian_skinner>    peller: i wanted to keep the requirements for the result object as simple as possible, so that it would be very, very easy to implement a simple datastore that just read data out of an HTML table -- simple things simple
    <slightlyoff>    brian_skinner: ok, so how would the simplest case look? you get a response objet, then you you need to "get all" , right?
    <slightlyoff>    brian_skinner: you still need some fetch method
    <slightlyoff>    brian_skinner: or would you do a magical "pass a string not a response" kinda thing?
    <brian_skinner>    slightlyoff: simplest case looks like this
    <brian_skinner>    store.find({onnext:callback});
    <brian_skinner>    unless we decide to continue supporting sync access
    <brian_skinner>    in which case you don't even need the callback
    <slightlyoff>    brian_skinner: and that doesn't change in the return-objects-have-methods world, does it?
    <brian_skinner>    slightlyoff: right
    <slightlyoff>    so where's the beef?
    <brian_skinner>    i'm worried about three things
    <brian_skinner>    (1) how easy is it do simple things -- how many lines of code to just read some items from a table
    <slightlyoff>    (as an aside, this ETel talk I'm in sucks)
    <slightlyoff>    brian_skinner: yes, and we've covered that
    <brian_skinner>    (2) how easy is it to learn to do simple things -- how many manual pages are available that you could sift through -- is there separate documentation for the result object as well as the datastore itself
    <brian_skinner>    (3) how easy is it to implement a new datastore -- can you make a new datastore that talks to some random web service by just subclassing a single class, or do you have to worry about some interaction between a datastore class and a result object
    <slightlyoff>    ok, a good consideration
    <slightlyoff>    don't see how that's harder if we return bog-stock response objects that do the "right thing" where "right thing" assumes some dumb internal data structure
    <slightlyoff>    and doesn't have the store's state wrapped up in it
    <brian_skinner>    slightlyoff: so what's the response object that you've got in mind?
    <slightlyoff>    brian_skinner: well, it should have some sugar to fetch()
    <brian_skinner>    the response object?
    <slightlyoff>    brian_skinner: w/o needing another handle to the store
    <slightlyoff>    brian_skinner: yeah, returns another response obj, right?
    <brian_skinner>    okay so let me try an example...
    <slightlyoff>    brian_skinner: otherwise, why return it? why not force folks to register a callback on the store itself?
    <brian_skinner>    slightlyoff: well, right now we return it, but you can safely ignore it
    <brian_skinner>    in the simple cases
    <slightlyoff>    so this is the conflict
    <slightlyoff>    we assume that we want the response object for the complex cases to ease the use
    <slightlyoff>    but don't want to add syntactic sugar to it to enable that?
    <slightlyoff>    presumeably because it perhaps requires explaining things twice
    <slightlyoff>    so lets solve that
    <brian_skinner>    doesn't option 6 provide the needed syntactic sugar?
    <slightlyoff>    brian_skinner: /me checks
    <brian_skinner>    it may look unconventional, but what does it actually fail to do?
    <wildbill>    it works. it's much harder to understand than something simple like JDBC
    <slightlyoff>    brian_skinner: why do I have to keep that handle around?
    <jared_j>    Isn't it also consistent with the getValue() api? Where you pass in the item to get an attribute off it? This makes the resultset an opaque thing.
    <slightlyoff>    if I'm gonna do that, why do I need to get back at the store object too?
    <slightlyoff>    you're making me keep 2 things around which, onstensibly, know about each other
    <brian_skinner>    slightlyoff: so are you suggesting this:
    <brian_skinner>    var store = new dojo.data.CsvStore({url:"movies.csv"});
    <brian_skinner>    var result = store.find();
    <brian_skinner>    result.fetch();
    <brian_skinner>    ?
    <slightlyoff>    yes
    <wildbill>    heh, that's exactly the database api that you said was so terrible
    <ccmitchell>    +1
    <slightlyoff>    wildbill: uh, it doesn't require me to pass in a cursor
    <jared_j>    That does make more general sense.
    <wildbill>    result is the cursor.
    <slightlyoff>    wildbill: I think I was objecting to the idea that it would be a "cursor" in the DB sense or be managing server-side state
    <slightlyoff>    wildbill: that API makes tons of sense
    <wildbill>    let's clarify the api.
    <slightlyoff>    great
    <brian_skinner>    okay, if we do that, what args does find() take, and what does fetch() take
    <wildbill>    i'm assuming that you would do result.fetch(callback, 300, 100)
    <brian_skinner>    ?
    <slightlyoff>    wildbill: correct
    <slightlyoff>    brian_skinner: find takes a query
    <brian_skinner>    willbill: when is callback called
    <ccmitchell>    i'm updating options now on wiki
    <slightlyoff>    brian_skinner: fetch takes a range (or returns all if none is passed)
    <slightlyoff>    brian_skinner: so simplest use is:
    <slightlyoff>    (new dojo.data.CsvStore({url:"movies.csv"})).find("*").fetch(callback);
    <peller>    ok, so this is pretty much what we had before, except we're back to putting methods on the result set, yes?
    <peller>    we've added a bit of complexity to the store implementation in return for a more O-O model for the user
    <wildbill>    right
    <slightlyoff>    peller: yes
    <peller>    fine with me
    <slightlyoff>    wildbill: can the table be implemented with this API?
    <peller>    slightlyoff: then you and I weren't that far apart, perhaps? the concepts remain the same
    <slightlyoff>    peller: yes, you're right
    <wildbill>    slightlyoff: sure
    <slightlyoff>    peller: sorry I got so grumpy
    <brian_skinner>    and we've added one extra step for anyone who's using a simple datastore
    <slightlyoff>    (did I mention this ETel talk blows chunks?)
    <brian_skinner>    i'm okay with that
    <slightlyoff>    brian_skinner: fair enough = )
    <wildbill>    i think it's the best compromise.
    <peller>    slightlyoff: heh
    <brian_skinner>    but i think fetch() needs a variety of args
    <slightlyoff>    brian_skinner: at least we don't have the "introduce it twice" problem
    <slightlyoff>    brian_skinner: yeah, I'm OK w/ that
    <slightlyoff>    brian_skinner: I trust you will know what those should be
    <brian_skinner>    onnerror callback, onnext, oncomplete, scope, etc.
    <slightlyoff>    great
    <brian_skinner>    okay, is this resolved?
    <wildbill>    yeah, i think so.
    <slightlyoff>    *rad*
    <ccmitchell>    tried to keep up with wiki edit
    <slightlyoff>    ccmitchell: thanks
    <ccmitchell>    take a look at option 7
    <slightlyoff>    ccmitchell: is this looking sane?
    <slightlyoff>    ccmitchell: looks good
    <ccmitchell>    i missed the object with multiple handlers issue...
    <slightlyoff>    gotta drop off for a couple of minutes
    <ccmitchell>    one secv
    <brian_skinner>    ccmitchell: thanks for keeping the wiki in sync
    <slightlyoff>    vote w/o me = )
    <jared_j>    Except the params of fetch(), should that be an object with the various args brian_skinner noted?
    |<--    slightlyoff has left irc.freenode.net ()
    <wildbill>    yeah #7 is fine w/me.
    <jared_j>    (new CsvStore?({url:"movies.csv"})).find("*").fetch((callback: callback, start: 50, count:40, ... etc?});
    <ccmitchell>    ok, check # 7 again
    <peller>    is 'hobbit' some sort of named query concept, or is it a placeholder?
    <brian_skinner>    can I modify #7 slightly -- to look more like this;
    <brian_skinner>    (new dojo.data.CsvStore?({url:"movies.csv"})).find().fetch({onnext: callback, start: 50, limit:40, ... etc?});
    <ccmitchell>    y
    <wildbill>    y
    <brian_skinner>    peller: hobbit is just a simple example query -- not a named query
    <ccmitchell>    the object with funcs is the anon obj param
    <ccmitchell>    makes sense
    <wildbill>    is everyone else ok with #7?
    <ttrenka>    +1
    <ccmitchell>    brian, you editing it?
    <brian_skinner>    ccmitchell: yup, just finished
    <brian_skinner>    didn't mean to get in the way -- sorry
    <ccmitchell>    another minor tweak...
    <brian_skinner>    wildbill: +1
    <ccmitchell>    use count rather than limit
    <ccmitchell>    since we moved this onto Result aap
    <ccmitchell>    api
    <brian_skinner>    ccmitchell: i'm fine with count -- I think limit is what SQL uses
    <ccmitchell>    any objections to count rather than limit?
    <wildbill>    ok, go with count
    <peller>    any reason to specify at all on the find()?
    <wildbill>    it's more like a hint
    <brian_skinner>    peller: i didn't understand that question?
    <jared_j>    Count makes more sense to me
    [ERROR]    Connection to irc://irc.freenode.net/ (irc://irc.freenode.net/) reset.
    [INFO]    Connecting to irc://irc.freenode.net/ (irc://irc.freenode.net/)...
    [INFO]    Connecting to irc://irc.freenode.net/ (irc://irc.freenode.net/)...
    ===    *** Looking up your hostname...
    ===    *** Checking ident
    ===    *** Found your hostname
    ===    *** No identd (auth) response
    ===    The nickname “jared_j” is already in use, trying “jared_j_”.
    ===    *** Spoofing your IP. congrats.
    ===    *** You are exempt from K/D/G lines. congrats.
    ===    *** Your host is anthony.freenode.net[anthony.freenode.net/6667], running version hyperion-1.0.2b
    ===    idoru set your hostname to "nat/ibm/x-b481f50d92ed9988"
    =-=    User mode for jared_j_ is now +i
    -->|    YOU (jared_j_) have joined #dojo-meeting
    =-=    Topic for #dojo-meeting is “http://dojo.jot.com/2007-02-27”
    =-=    Topic for #dojo-meeting was set by wildbill on Tuesday, February 27, 2007 4:07:40 PM
    [ERROR]    Connection to irc://irc.freenode.net/ (irc://irc.freenode.net/) reset.
    [INFO]    Connecting to irc://irc.freenode.net/ (irc://irc.freenode.net/)...
    [INFO]    Connecting to irc://irc.freenode.net/ (irc://irc.freenode.net/)...
    [INFO]    Connecting to irc://irc.freenode.net/ (irc://irc.freenode.net/)...
    [INFO]    Connecting to irc://irc.freenode.net/ (irc://irc.freenode.net/)...
    ===    *** Looking up your hostname...
    ===    *** Checking ident
    ===    *** Found your hostname
    ===    *** No identd (auth) response
    ===    The nickname “jared_j_” is already in use, trying “jared_j__”.
    ===    *** Spoofing your IP. congrats.
    ===    *** You are exempt from K/D/G lines. congrats.
    ===    *** Your host is calvino.freenode.net[calvino.freenode.net/6667], running version hyperion-1.0.2b
    ===    idoru set your hostname to "nat/ibm/x-a887493c28b4bfd1"
    =-=    User mode for jared_j__ is now +i
    -->|    YOU (jared_j__) have joined #dojo-meeting
    =-=    Topic for #dojo-meeting is “http://dojo.jot.com/2007-02-27”
    =-=    Topic for #dojo-meeting was set by wildbill on Tuesday, February 27, 2007 4:07:40 PM
    <wildbill>    yeah
    <brian_skinner>    or are there better names entirely
    -->|    slightlyoff (n=alex@dsl093-079-130.sfo1.dsl.speakeasy.net) has joined #dojo-meeting
    =-=    Mode #dojo-meeting +o slightlyoff by ChanServ
    <wildbill>    i don't understand onnext
    <wildbill>    it seems like it should be onCompleted
    <brian_skinner>    wildbill: that's for setting a callback that will be called once for each item that gets returned
    <wildbill>    oh
    <wildbill>    ok
    <brian_skinner>    wildbill: is there a better name for onnext?
    <peller>    onReady?
    <ccmitchell>    onNextItem
    <wildbill>    or onItem. i don't care
    <brian_skinner>    peller: is onReady a replacement for onBegin?
    <peller>    next doesn't apply, really, in random access?
    <wildbill>    right :-)
    <ttrenka>    onItem
    * peller    looks for onBegin
    <peller>    whatever
    <slightlyoff>    ttrenka: good
    <slightlyoff>    ttrenka: I like that
    <wildbill>    ok, onItem it is.
    <ttrenka>    slightlyoff: thank Bil
    <slightlyoff>    ttrenka: heh
    <brian_skinner>    okay, so then we have: onBegin, onItem, onCompleted, onError ?
    <wildbill>    yup
    <peller>    what's onBegin?
    <slightlyoff>    "here comes the data" ?
    <peller>    oh, it passes them back one at a time? duh.
    <brian_skinner>    yup, onBegin is for setting a callback that will called if there was not an error and the data is about to start pouring in
    <peller>    I figured they came down in one chunk. Sorry
    <peller>    ok
    <brian_skinner>    peller: we wanted to handle streaming results -- like an RSS feed
    <ccmitchell>    Ok, take another look at #7
    <peller>    disregard my comment about next then :) unless the items are not guaranteed to come down in order within a particular fetch?
    <peller>    I'm fine with what you have.
    <ccmitchell>    lets take a vote on #7
    <wildbill>    well, i think the rest of us just voted to change the names to onBegin, onItem, onCompleted, onError
    <peller>    so onItem would include an index arg, I suppose?
    <wildbill>    yeah
    <slightlyoff>    peller: you could also imagine an big-chunk onData()
    <ccmitchell>    no, just the item handle
    <slightlyoff>    peller: don't know what's better
    <jared_j__>    slightlyoff: Isn't onCpmpleted == onData then?
    <peller>    big chunk is simpler.
    <peller>    if we have paging, do we need this granularity?
    <ttrenka>    to be honest, supporting both wouldn't be a bad thing
    <jared_j__>    Er, onCompleted
    <brian_skinner>    slightlyoff: you can use onCompleted if you want to just get a big chunk
    <ttrenka>    I had to add onAddDataRange to Filtering because of inefficiencies with doing onAddData only
    <peller>    brian_skinner: cool
    <ccmitchell>    eg. all items are finished streaming ntw
    <ccmitchell>    now
    <slightlyoff>    great
    <slightlyoff>    sounds good
    <slightlyoff>    thanks
    <brian_skinner>    peachy
    <ccmitchell>    Vote on #7
    <ccmitchell>    +1
    <jared_j__>    +1
    <brian_skinner>    +1
    <wildbill>    +1
    <peller>    +1
    <brian_skinner>    okay, do we have a volunteer to go through all the existing code and unit tests and convert it to the new spec
    <slightlyoff>    +10e6
    <brian_skinner>    i'll do all the OpenRecord code!
    <slightlyoff>    don't everyone jump at once = )
    <jared_j__>    I'm willing to make a pass at applying the changes noted here and mailing a patch.
    <slightlyoff>    jared_j__: much appreciated
    <brian_skinner>    jared_j__: that would be great!
    <slightlyoff>    4 minutes to hard cutoff
    <slightlyoff>    jared_j__: I'll mail you then when we're going to port dojo.data in to 0.9 Core
    <slightlyoff>    and try to sync up then
    <jared_j__>    Ok.
    ===    Private messages from unregistered users are currently blocked due to spam problems, but you can always message a staffer. Please register! ( http://freenode.net/faq.shtml#privmsg )
    <slightlyoff>    wildbill: is the new outline good enough for the table?
    <wildbill>    the only problem is highlighting a row and then sorting
    <slightlyoff>    ccmitchell: can it be made to work w/ xml data stores?
    <slightlyoff>    wildbill: understood
    <slightlyoff>    wildbill: I think we might be able to elide that away w/ a rowid of some sort
    <wildbill>    i think it's more complicated but i'm willing to defer it for now.
    <ccmitchell>    selection is something that the widget needs to keep track of
    <peller>    slightlyoff: you mean by adding something special to the schema?
    <ttrenka>    yes it does
    <ccmitchell>    eg selection manager
    <slightlyoff>    peller: well, to the data store
    <slightlyoff>    ccmitchell: yes
    <wildbill>    ccmitchell: that's not the part i'm worried about.
    <brian_skinner>    wildbill: what's the part you are worried about?
    <wildbill>    basically, for a given row (with primaryKey=abcdef)
    <wildbill>    after i sort the table by a given column, then what record # does that row become?
    <ttrenka>    wildbill: the record number itself really shouldn't change
    <wildbill>    in other words, by resorting the table, the 5th row can become the 500th row
    <slightlyoff>    ttrenka: yeah, but he wants to know the new index
    <jared_j__>    Should the datastore/resultset even be reordering its results based on a sort occuring in aother widget?
    <ttrenka>    slightlyoff: I know
    <wildbill>    and a typical app like thunderbird will position at row 500 for you, so you don't lose your place
    <ccmitchell>    your selection state needs to handle the map of indexrow in wdiget to item handles
    |<--    jared_j has left irc.freenode.net (Connection timed out)
    <slightlyoff>    ccmitchell: I think bill's asking for a special query type
    <brian_skinner>    the item handle will typically not be an integer anyway
    <ttrenka>    but I understand what wildbill is saying: if you are using the paging facilities, how do you know what page you're on if a sort happens?
    <slightlyoff>    ttrenka: yep, stable sort around a single item
    <brian_skinner>    i see
    <jared_j__>    Ah.
    <wildbill>    right. how do you know what page to go to.
    <ccmitchell>    item handles dont change across finds/sorts
    |<--    jared_j_ has left irc.freenode.net (Connection timed out)
    <wildbill>    select rownum from .... where primaryKey=12345 order by xyz
    <wildbill>    or in english (if i sort this table by column xyz, what will be the row number for the 12345 item?)
    <slightlyoff>    ccmitchell: yes, we understand ,but there's no provision for getting the new location in the sorted data
    <ccmitchell>    ah, ok
    <wildbill>    (w/out downloading the entire data set)
    <slightlyoff>    (and then iterating)
    <wildbill>    yeah, linear search
    <ccmitchell>    or loosing the selections, unless sort is done client side...
    <brian_skinner>    so punt on that feature?
    <slightlyoff>    brian_skinner: for now, I think we have to
    <wildbill>    maybe . or at least punt for now.
    <brian_skinner>    sounds good
    <brian_skinner>    okay, so same time next week -- pick up with the string-matching for Select widget (Combobox) questions?
    <slightlyoff>    brian_skinner: might imply a special sort/search type for some special providers
    <slightlyoff>    brian_skinner: great
    <ttrenka>    we'll need to make it clear that a sort resets the dataset
    <ccmitchell>    so for now, client side sort, can retain selection state, server side sort requires refresh
    <ccmitchell>    and loses selection state
    <slightlyoff>    ccmitchell: yes
    <wildbill>    huh?
    <slightlyoff>    ccmitchell: we'll need a flag to indicate as much somehow
    <wildbill>    client side sort in the FilteringTable code, or inside the dojo.data code?
    <ccmitchell>    if sort is done inside the widget, selection state is tracked there...
    <slightlyoff>    wildbill: inside the dojo.data code for fully-cached data sets
    <ccmitchell>    but
    <wildbill>    well, then we need an api
    <ccmitchell>    we've recently been discussing doing client side sort directly in datastore api
    <slightlyoff>    wildbill: we'll need a way to negotiate it w/ the data store
    <ccmitchell>    let's discuss this via contrib list and revisit monday
    <slightlyoff>    wildbill: but we can punt on the feature until we get the API spec'd
    <ccmitchell>    sorry, tues
    <slightlyoff>    ccmitchell: yes
    <slightlyoff>    great
    <slightlyoff>    we're at hardcutoff anyway = )
    <slightlyoff>    good meeting everyone
    <slightlyoff>    good progress
    <slightlyoff>    and I think the new API is very sane
    <peller>    sane is good :)
    <brian_skinner>    thanks for coming -- good to have more eyes on this
    <slightlyoff>    later!
    <brian_skinner>    bye
    <wildbill>    bye
    <ccmitchell>    ttyl
    |<--    brian_skinner has left irc.freenode.net ("Chatzilla 0.9.77 [Firefox 1.5.0.10/2007021601]")
    <jared_j__>    Definitely. Bye.