Some thoughts on the find() and fetch() methods in the dojo.data Read API...

Option 1: store.find() and result.fetch()

example

 1 |  var store = new dojo.data.CsvStore({url:"movies.csv"});
 2 |  var result = store.find();
 3 |  result.fetch({onItem:displayItem});

parameters

• find() takes:
  • query
  • sort
  • count
  • onError
  • scope
  • but not, I think:
    • onBegin?
    • onItem?
    • onComplete?

• fetch() takes:
  • start
  • count
  • onError
  • onBegin
  • onItem
  • onComplete
  • scope
  • but not, I think:
    • query?
    • sort?

pros

1. For people who are used to database access APIs, with the notions of queries and cursors, the semantics are familiar and make sense.

cons

1. If we have a goal to "make simple things simple, and make complex things possible", then adding we're failing to keep the HtmlTableStore simple if we require a user to call both find() and fetch() just to the get the contents of a table that's already on the page. In the rest of the dojo.data API we've been careful to layer the API. For example, the Write API is layered on top of the Read API. You can have a read/write datastore, but if you have a read-only datastore the read-only datastore is not in any way complicated by the fact that a Write API exists. Likewise for sorting. Any find() method can take a sort parameter, but if you don't want to sort then your code is not in any way complicated by the fact that sort was an option -- you don't ever have to say "sort:null". It seems like a shame to complicate all the datastores that don't need queries and paging in order to allow some datastores to have queries and paging.
2. If we allow count on find(), then it's not clear why start and count aren't both allowed on find().
3. If the user already provided scope and onError parameters when they called find(), then why do we require them to provide scope and onError parameters again when they call fetch()? But, on the other hand, if do have the fetch() method use the scope object provided in the find() call, then there's a potentially confusing action-at-a-distance.
4. We end up with a division of code between the datastore object and the result object, with an unclear division of responsibilities (who does caching, who does update notification, etc.) There's no way to write a general-purpose, off-the-shelf dojo.data.Result class that can easily be re-used by anyone who is implementing a new datastore -- in order to be performant, the result.fetch() method needs to be able to get just a range of items items from the datastore, but the datastore provides no method to get just a range of items. Which leads to the next option...

opinions

• brian doesn't like this option
• alex says he doesn't mind this option
• jared's reviewer says: "As the comments already note, (relational) database people are comfortable and understand option 1. We agree!"

Option 2: store.find(), store.fetch(), and result.fetch()

example

 1 |  var store = new dojo.data.CsvStore({url:"movies.csv"});
 2 |  var result = store.find();
 3 |  result.fetch({onItem:displayItem});
    OR
 3 |  store.fetch({onItem:displayItem, from:result});

pros

1. Offers the exact same API as option 1, if people want to use that API.
2. Makes it possible to write a general-purpose, off-the-shelf dojo.data.Result class that can easily be re-used by anyone who is implementing a new datastore. The standard result.fetch() implementation will just be two lines of code: "keywordArgs.from = this; return store.fetch(keywordArgs);"
3. Allows the guts of the fetch() method to be implemented in the datastore object rather than the result object, since the datastore object has all the smarts and knows how to do efficient caching, range queries, etc.

cons

1. same as Option 1, con #1 -- the store.find().fetch() idiom is needlessly complicated for any store that isn't doing paging
2. same as Option 1, con #2 -- unclear which args should be allowed on which methods
3. same as Option 1, con #3 -- unclear whether fetch() should "inherit" scope and onError from find()
4. creates a confusing duplication -- why do we have both store.fetch() and result.fetch()?

opinions

• brian doesn't like this option
• alex says he doesn't mind this option
• jared's reviewer says: "seems odd, given that you already have the Result, what's the benefit. Pretending that the "smarts" are on the store is confusing since it's the Result that actually has the state. We're not sure what a general-purpose Result implementation would look like since there are so many flavors of datastore, and would suggest that this could be refactored later if it makes sense when there are more implementations to analyze."
• jared's reviewer says: "For both Option 2 and Option 3, how functions get factored between the store and the result should be an implementor's decision and should not be reflected into the API."

Option 3: store.find(), store._fetch(), and result.fetch()

example

 1 |  var store = new dojo.data.CsvStore({url:"movies.csv"});
 2 |  var result = store.find();
 3 |  result.fetch({onItem:displayItem});

pros

• all the same pros as Option 2

cons

• all the same cons as Option 2, except that by making the store._fetch() method private we avoid con #4

opinions

• jared's reviewer says: "For both Option 2 and Option 3, how functions get factored between the store and the result should be an implementor's decision and should not be reflected into the API."

Option 4: store.find() and store.fetch()

example

 1 |  var store = new dojo.data.CsvStore({url:"movies.csv"});
 2 |  var result = store.find();
 3 |  store.fetch({onItem:displayItem, from:result});

pros

• same pros as Option 2, except the now you call fetch() on store instead of result

cons

• all the same cons as Option 2, except we avoid con #4
• but we've also now made it impossible to chain the find() and fetch() calls, which Chris said he thinks Alex wanted: store.find().fetch()

opinions

• alex hates it (because of passing the result handle)
• jared's reviewer says: "For Options 4-6, we don't like having the fetch on the store since logically this is an operation that's associated with query result rather than the store itself to which multiple queries can be directed."

Option 5: store.find() and store.fetch(), with optional find()

example

 1 |  var store = new dojo.data.CsvStore({url:"movies.csv"});
 2 |  store.fetch({onItem:displayItem});
   OR
 1 |  var store = new dojo.data.LargeRdbmsStore({url:"jdbc:odbc:foobar"});
 2 |  var result = store.find({
 3 |      query: {type:"employees", name:"Hillary *"}, // string matching
 4 |      sort: [{attribute:"department", descending:true}],
 5 |      onError: handleFindError
 6 |  });
 7 |  var fetchArgs = {
 8 |      start: 0, count: 20,  // get the first 20 items
 9 |      scope: displayer,
10 |      onBegin: showThrobber,
11 |      onItem: displayItem,
12 |      onComplete: stopThrobber,
13 |      onError: handleFetchError,
14 |      from: result
15 |  };
16 |  store.fetch(fetchArgs);
...|    ...
40 |  // and then when the user presses the "Next Page" button...
41 |  fetchArgs.start += 20;
42 |  store.fetch(fetchArgs);  // get the next 20 items

pros

1. "simple things simple, complex things possible" -- Queries and cursors and paging are possible, but the simple case is not complicated by the need for queries and paging.
2. Makes it possible to write a general-purpose, off-the-shelf dojo.data.Result class that can easily be re-used by anyone who is implementing a new datastore.
3. Allows the guts of the fetch() method to be implemented in the datastore object rather than the result object, since the datastore object has all the smarts and knows how to do efficient caching, range queries, etc.

cons

1. can't chain the find() and fetch() calls: store.find().fetch()
2. the semantics are now not consistent across all use cases -- the find() is now optional, so sometimes the user code will include it and sometimes the user code won't

opinions

• alex says: "feels like an API we would have ratified in Dojo 0.3.x days, which means it's got things to recommend it, but will be hard to introduce"
• jared's reviewer says: "For Options 4-6, we don't like having the fetch on the store since logically this is an operation that's associated with query result rather than the store itself to which multiple queries can be directed."

Option 6: store.fetch()

example

Simple case (streaming, no paging):
 1 |  var store = new dojo.data.CsvStore({url:"movies.csv"});
 2 |  store.fetch({onItem:displayItem});

Simple case (all items as one chunk):
 1 |  var store = new dojo.data.CsvStore({url:"movies.csv"});
 2 |  store.fetch({onComplete:displayAll});

Simple case (Paged chunks):
 1 |  var store = new dojo.data.CsvStore({url:"movies.csv"});
 2 |  store.fetch({count:20, onComplete:displayPage});  // Return the first 20 items.  If start not defined, assumed started at 0.
...
40 |  store.fetch({start:20, count:20, onComplete:displayPage});  // Grab the next twenty items.

Semi-complex case (paging, with query, sorting, etc, specified.):
 1 |  var store = new dojo.data.LargeRdbmsStore({url:"jdbc:odbc:foobar"});
 2 |  var fetchArgs = {
 3 |      query: {type:"employees", name:"Hillary *"}, // string matching
 4 |      sort: [{attribute:"department", descending:true}],
 5 |      start: 0,
 6 |      count: 20,
 7 |      scope: displayer,
 8 |      onBegin: showThrobber,
 9 |      onItem: displayItem,
10 |      onComplete: stopThrobber,
11 |      onError: handleFetchError,
12 |  };
13 |  store.fetch(fetchArgs);
...|    ...
40 |  // and then when the user presses the "Next Page" button...
41 |  fetchArgs.start += 20;
42 |  store.fetch(fetchArgs);  // get the next 20 items

pros

1. "simple things simple, and complex things possible"
2. Creates fewer method to document, fewer methods to test, and fewer methods for a user to learn, because there's now no store.find() or result.fetch().
3. Under the covers, a datastore can still use database cursors, and does not need to run the same query twice. This option is just as performant as other options, but now that performance improvement is an implementation detail, not something exposed in the API.
4. The semantics are consistent across use cases.
5. Makes it possible to write a general-purpose, off-the-shelf dojo.data.Result class that can easily be re-used by anyone who is implementing a new datastore.
6. Allows the guts of the fetch() method to be implemented in the datastore object rather than the result object, since the datastore object has all the smarts and knows how to do efficient caching, range queries, etc.

cons

1. does not feel familiar to people who are used to working with databases

opinions

• brian likes it
• alex at first said "gives me the willies because then you've got users juggling location in the result set", but now says he's okay with it
• jared's reviewer says: "For Options 4-6, we don't like having the fetch on the store since logically this is an operation that's associated with query result rather than the store itself to which multiple queries can be directed."
• jared has said he's okay with it, and i think has said that chris is game to sign off on it as well