Outcome:

Agenda Item 1:

    JaredJ? to work with Adam Peller, Brian Skinner, and Alex to get current changes committed in.

Agenda Item 2: Changes to the Identity API

   Change the name to fetchByIdentity, leave it sync for now.  There are other sync issues to address in the future, if it turns out we actually need to.  So, the overall issue is put on hold for dealing with at a later date.

Agenda Item 3:  Other misc API tightening.

1. Leave name as unsetAttribute.
2. fetch() should not throw exceptions generally on bad args.
3. For documentation, use '||' to indicate an api can take two types for the argument.
   

Agenda Item 4:  How is cache cleared?

  Add an API to the read interface of store:  store.close().   It can take the form of two possibilities:

• store.close().   Flush entire cache for all queries and close all 'open' connections to the backend, if any.
• store.close(requestObj);   Flush the cache for a particular request object (as per returned from a store.fetch() and close any 'open' connections to the back-end server, if any.

---

Agenda Item 1:  Status from last meeting (if any)

JaredJ:  All API descisions from the previous meetings have been implemented.  Will be working with Adam Peller and Brian Skinner on checking in what has currently been done.

Agenda Item 2: Changes to the Identity API. 

• Change findByIdentity() to fetchByIdentity()?
  • Since fetch in Read is async only by definition, should the Identity functions also by async by definition?  If so, should it take callbacks like fetch() in read does?
  • skinner: Now that we've entirely gotten rid of sync access in the fetch() API, I'm worried about having sync access in the findByIdentity method.  For datastores than fetch data via JSON or JSONP in a script tag transport, I think async is the only option, and sync isn't available.  If the findByIdentity request can't be satisfied by a client-side cache lookup, and the datastore has to go to the server, then findByIdentity won't be able to return a sync result.
  • skinner: In the Read API we've gotten rid of the old find() method and replaced it with the new fetch() method.  In the Identity API, we originally named the findByIdentity() to match the find() method in Read, but now "findByIdentity" is an anachronism.  Maybe we should change findByIdentity() to something else, like fetchByIdentity()?

Agenda Item 3:  Other misc API tightening.

1. change the name of unsetAttribute() in the Read API?
   • skinner: The unsetAttribute method is a peer to all "value" methods, like setValue, setValues, getValue, getValues, and containsValue -- but the name "unsetAttribute" doesn't make that obvious.  And, to make matters worse, the "unsetAttribute" seems to suggest that the method is a peer to the "attribute" methods, hasAttribute and getAttributes.  I suggest we change the name to "unsetValue" or "unsetValues", or better yet, to "clearValue".
     
2. Should fetch() throw exceptions on obviously invalid parameters, such as start being non-integer or negative.  Same for count.
3. Should we use | or || in documentation to indicate a parameter that can be of one of the two types?  what is the dojo stanadard here?

Agenda Item 4:  How is cache cleared?

1. For stores that cache, how should we recommend that stores handle clearing? 
   
   1. One idea:  Should we add a close() api to Request that is peer to abort().  The purpose of close being that it instructs the store to clear any cached data linked to a particular request in some manner?

---

Minutes:

[20:03] <jared_j> Afternoon (Morning for Bill). Are there other folks we're waiting for? ccmitchell, are you here?
[20:03] <tk> another meeting?
[20:03] <tk> Data?
[20:03] <jared_j> dojo.data, 4-6 EST.
[20:03] <jared_j> 1-3 PST.
[20:03] <tk> 2-4 MST
[20:03] <tk> :P
[20:03] <jared_j> Some ugly hour in Japan.
[20:04] <brian_skinner> agenda?
[20:04] <brian_skinner> http://dojo.jot.com/2007-03-27
[20:05] <jared_j> http://dojo.jot.com/WikiHome/dojo.data/Dojo%20Data%20Meetings/2007-03-27
[20:05] <jared_j> Shall we begin? Who here is actually awake/active? :)
[20:05] <brian_skinner> aye
[20:06] <peller> me, but only for a bit
[20:06] <slightlyoff> 4am in Japan
[20:06] <brian_skinner> slightlyoff: are you in Japan?
[20:06] <slightlyoff> brian_skinner: no, but wildbill is = )
[20:06] <peller> are we killing/postponing the cursor discussion?
[20:06] <wildbill> hi. it's 5am here. you guys jumped an hour ahead a few weeks ago.
[20:06] <peller> it sounds like a problem we cannot solve now, and I'm willing to accept that
[20:07] <jared_j> Okay, quick status as per usual. I'm mainly just been updating things Brian noted in review, as well as what slightlyoff noted on style corrections. I pretty much have that done. Also looking at perf improvements and talking to Brian about em.
[20:07] <jared_j> As per the cursor issue...
[20:07] <slightlyoff> wildbill: *much* better ;-)
[20:07] <slightlyoff> jared_j: I had a LONG conversation w/ wildbill last night
[20:07] <jared_j> Chris and I are working woth folks here to protoype out actual RDB backed stores to see what works, what doesn't.
[20:07] <slightlyoff> jared_j: are there any results from that work yet?
[20:08] <ccmitchell> hey guys, i'm here (kind of)
[20:08] <jared_j> slightlyoff: Yes, I gleened that from the e-mail. Is there anything you want to share from it?
[20:08] <brian_skinner> hey ccmitchell
[20:08] <ccmitchell> getting pulled into meetings left and right :(
[20:08] <peller> I'm willing to say that we're not going to get this right first pass and that we need to move on
[20:08] <slightlyoff> jared_j: well, I think we're at the point where we agree that there are a couple of substantive technical issues and at least one style issue
[20:09] <slightlyoff> peller: "any decision we make here today is going to be wrong, so lets just make one and move on"
[20:09] <peller> sort of :(
[20:09] <slightlyoff> peller: a favorite quote from an old job of mine = )
[20:09] <peller> but the point I'm getting to
[20:09] <peller> things like whether we reuse the request object... we need to take a stand on that
[20:09] <peller> if not now, definitely by 0.9
[20:09] <jared_j> Not yet on results. But Chris and I also contacted a co-worker who's huge in the data access space in other fronts and asked him to look over the client apis as well. We also discussed a bit with him on how to deal with the cursor.
[20:09] <peller> we can't just say you "could" do that
[20:09] <jared_j> I'm thinking we should say you should.
[20:10] <peller> it's a reasonable position to take for now
[20:10] <jared_j> IT provides a point datastores can put other internals that it could use for performance.
[20:10] <jared_j> But not necessarily stuff that need s to be client viisble.
[20:10] <peller> ...which we could end up breaking someday, but whatever :)
[20:10] <peller> we've discussed the pros/cons to death
[20:10] <slightlyoff> peller: well, the thing here is that your concern is that stores need to do this work
[20:10] <peller> and we can document these as known issues and come back to them, I guess
[20:11] <slightlyoff> peller: from the perspective of the users, that's a *good thing*
[20:11] <ccmitchell> we are going to need a close() api
[20:11] <peller> slightlyoff: no, that was the *other* approach
[20:11] <peller> reusing fetch is putting the burden on the users
[20:11] <slightlyoff> peller: that you can have poorly performing stores is OK, so long as you can also have *great* stores which do the right thing
[20:11] <peller> slightlyoff: um... maybe
[20:11] <peller> slightlyoff: as long as it's possible to write a great store for a given situation
[20:11] <slightlyoff> peller: yes
[20:12] <wildbill> so, from our conversation yesterday, my biggest fear was that we were backing ourselves into a corner
[20:12] <peller> wildbill: that assumes we'll never change the APIs
[20:12] <peller> see favorite quote above :)
[20:13] <wildbill> right, so it's not so bad. we always can change them, or add two apis like brian sugested.
[20:13] * peller shrugs
[20:13] <slightlyoff> so if we pass back a result object now, does everyone feel that it could be augmented in the future to support the styles we're discussing?
[20:13] <wildbill> i don't understand the Q
[20:13] <slightlyoff> wildbill: peller is very worried that we don't have an accurate way to hint things
[20:14] <peller> If I understand what you're saying, slightlyoff, yes, I think that would help
[20:14] <slightlyoff> wildbill: the result object is the obvious place to stuff cross-request hinting info if that's what's required
[20:14] <peller> are you talking about the dreaded 'handle' object?
[20:14] <peller> as part of the API contract, rather than just passing back the same object to fetch?
[20:14] <slightlyoff> peller: yes, but I'm asking if there's a progressive path to that *if it's required*
[20:15] <wildbill> ah, i think peller suggested this to me.
[20:15] <wildbill> init() and close() calls w/the same parameter as fetch()
[20:15] <peller> there are several possible ways to do this, I think
[20:16] <peller> ...but that was the least "OOP" I could think of
[20:16] <slightlyoff> what's init? and how is it different than creating a store instance?
[20:17] <jared_j> Are you suggesting to have a store.init() that takes the params to construct a 'request' object, that is then either handed to a fetch(), or to a close().
[20:17] <jared_j> ?
[20:17] <peller> yes
[20:17] <peller> which was basically a scheme to force the user to pass the same object to fetch each time
[20:17] <peller> or an exception would be thrown, or something like that
[20:17] <peller> init would be a no-op much of the time
[20:17] <peller> and synchronous, I think?
[20:17] <slightlyoff> ...at which point there's no diff between init()/fetch() and find()/fetch()
[20:18] <peller> basically. yeah
[20:18] <peller> :)
[20:18] <brian_skinner> is that a feature we could add to the API later, when we actually have a store that is able to take advantage of it?
[20:18] <peller> I suppose
[20:18] <brian_skinner> and for now, allow people to call fetch() without having first called init()?
[20:18] <peller> sounds reasonable. I don't want to get in the way anymore :-|
[20:18] <slightlyoff> we *aren't even sure this is a problem*
[20:18] <jared_j> Not sure. If you require fetch() that an 'initted' object, then what do you do when we add init and fetch no longer takes non'initted' objects?
[20:19] <jared_j> Hm, I phrased that bad.
[20:19] <jared_j> Basically, how do you tighten down fetch acceptable args after the fatc without breaking people?
[20:19] <slightlyoff> I'd like to cordially suggest that we all wait to see what the big-data teams determine about the proposed API
[20:19] <jared_j> slightlyoff: I think that is wise.
[20:19] <brian_skinner> slightlyoff: i'm fine with that
[20:20] <slightlyoff> peller? wildbill?
[20:20] <peller> jared_j: you'd break people. it wouldn't be the first time.
[20:20] <jared_j> But, I do think we need to add a close(), regardless.
[20:20] <peller> slightlyoff: +1
[20:20] <wildbill> jared_j: what would close look like?
[20:20] <jared_j> Because of the issues such as cusro closing, or say, a way to tell a store to clear any cache related to a request.
[20:20] <jared_j> request = store.fetch(whatever);
[20:20] <slightlyoff> jared_j: so there's some ambiguity around close(), I think
[20:20] <jared_j> request.close()
[20:20] <peller> jared_j: if we add a close, I would be tempted to add an init and remove it later... that migration is easier, no?
[20:21] <jared_j> Like reuqest.abort().
[20:21] <peller> but waiting a bit is a reasonable answer too, if we have people looking at this
[20:21] <slightlyoff> jared_j: mainly because if we've got page unload, we may not be able to send anyone close() information
[20:21] <slightlyoff> jared_j: so I don't think servers can or should count on getting a close()
[20:21] <peller> that is, checking in code and moving forward while we wait
[20:21] <jared_j> Servers shouldn't count on it.
[20:21] <slightlyoff> jared_j: with that caveat, let's proceed = )
[20:21] <peller> it would be an optimization, at best
[20:22] <brian_skinner> even if servers can't count on it, it might be a useful hint
[20:22] <jared_j> A proper backend service that's going to hold a cursor/connection open should be rigged to autoclose it after a timeout anyway.
[20:22] <brian_skinner> and it's a pretty painless addition
[20:22] <brian_skinner> it could even be optional
[20:22] <jared_j> But yeah, being able to close it earlier is a nice hint.
[20:22] <wildbill> so, think about forwards vs. backwards scrolling too.
[20:22] <brian_skinner> if (request.close) { request.close() }
[20:22] <slightlyoff> jared_j: so most servers already handle this, right?
[20:22] <wildbill> some DB's, if you want to go backwards in the data you need to rerun the query.
[20:23] <slightlyoff> jared_j: every JDBC provider to a web environment does connection pooling for exactly this reason
[20:23] <jared_j> I would hope so, otherwise you'd run out of handles/whatever in the DB eventually.
[20:23] <slightlyoff> jared_j: well, what I'm getting at is that the web tier *already does this*
[20:24] <slightlyoff> jared_j: so I'm really not sure what we're adding by sweating bullets over it, unless we think we're not going to be talking to these data stores through the traditional web tier
[20:24] <wildbill> it's more of a safeguard. let's say that there's a 10M query result that is cached on the server.
[20:24] <wildbill> you want to free it as soon as possible.
[20:25] <brian_skinner> slightlyoff: potentially could have a datastore connected directly to a database -- like when dojo is running on a server instead of in a browser -- but i think we should ignore that use case
[20:25] <jared_j> slightlyoff: It's mainyl for efficiency on the backend if you want to close connections early. In reality from what I've been told, you don't want to keep cursors open anyway and backends should close them quickly. And what WildBill said. For a store that wants to close it early.
[20:25] <peller> chances are, a good server would free it anyway. we'd have to be prepared for the event that some handle expired and we need to reissue the query?
[20:25] <wildbill> yeah, it's just efficiency.
[20:26] <wildbill> well, and the fact that holding a database cursor open for 10 minutes rather than 0.5 seconds could be really bad
[20:26] <wildbill> if someone is trying to modify said data
[20:26] <jared_j> And other types of stores would just have it as a no-op anyway. It's mainly just a hook for particular stores like DB.
[20:26] <jared_j> Or at least that's the current thinking.
[20:27] <slightlyoff> ccmitchell: can you shed some light here?
[20:27] <jared_j> I'd like to see what the folks here in DBland come back with on API issues.
[20:27] <peller> got to go offline. I'll try to be back in a bit
[20:27] <jared_j> And if they think it's a big deal or not, ultimately.
[20:27] <brian_skinner> jared_j: any idea when we'll hear back from the folks in DBland?
[20:27] * peller (n=peller@pool-151-203-123-17.bos.east.verizon.net) Quit ()
[20:27] <brian_skinner> i'm sure we will need to change the APIs, as we learn more, but we definitely should not *plan* to change the APIs -- we've already got 6 or 7 datastores that we're trying to maintain, as well as a bunch of unit tests, and code like OpenRecord that's using the API -- even small changes to the API are a pain -- additions are not a pain
[20:28] <jared_j> agreed on changes are a pain. :) Adds, not so bad.
[20:28] <slightlyoff> so I'm dubious about the use case for an explicit close()
[20:28] <wildbill> brian_skinner: so adding a future required init() and close() call would not be so bad?
[20:28] <slightlyoff> wildbill: I don't htink so
[20:28] <jared_j> brian_skinner: Well, We're hoping either later today or tomorrow at least from the expert on various data access apis and patterns.
[20:29] <slightlyoff> wildbill: I think getting the fetch() done is the big thing
[20:29] <jared_j> Hopefully by next week sometime on actually having a testcase that uses a backend DB with lotsa data in it.
[20:29] <brian_skinner> wildbill: a future require init would be a pain -- a future optional init would be much less of a pain
[20:29] <brian_skinner> required
[20:30] <brian_skinner> s/require/required
[20:30] <slightlyoff> brian_skinner: I can't really see how it would be requried....the semantic MUST be that it's implicit on first fetch()
[20:30] <slightlyoff> brian_skinner: anything else is lunacy
[20:30] <wildbill> yeah, that's possible
[20:30] <brian_skinner> okay
[20:31] <wildbill> so, part of the frustration for me and us server guys is that to us the find() fetc() api is much more intuitive, as users.
[20:31] <slightlyoff> it's like Python's read()
[20:31] <brian_skinner> jared_j: cool -- glad to hear we'll have feedback so fast -- that's great
[20:31] <slightlyoff> jared_j: yeah, that's awesome
[20:31] <wildbill> but i understand that some people hate it
[20:31] <slightlyoff> wildbill: so that's the stylistic thing
[20:31] <slightlyoff> wildbill: and it's valid
[20:31] <slightlyoff> wildbill: I don't want to minimize it
[20:31] <wildbill> right
[20:31] <brian_skinner> wildbill: that intuitive thing is what i was trying to address with the proposed mixin
[20:31] <slightlyoff> wildbill: but I want to talk about it for what it is
[20:31] <wildbill> sure
[20:32] <brian_skinner> with a 10-line mix-in, you can have that API if you want it, without requiring that everybody use it
[20:32] <wildbill> right... the other thing is fear about either leaving open cursors or leaving around a giant cache of data for longer than you should.
[20:32] <brian_skinner> request.close()
[20:33] <jared_j> What does a mixin of find actually do for you at that point, though?
[20:33] <wildbill> right. it's store.close(), right? not request.close()?
[20:33] <jared_j> Why would you close the store?
[20:33] <jared_j> The individual request data is what you want to clear out.
[20:34] <wildbill> same reason you fetch the store. what's the object that you call fetch on?
[20:34] <wildbill> isn't it store.fetch()?
[20:34] <slightlyoff> yep
[20:34] <jared_j> Then you'd need it as store.close(request);
[20:34] <slightlyoff> that's fine w/ me
[20:34] <jared_j> To clear a particular request
[20:34] <jared_j> And store.close(); th thwack them all, maybe.
[20:34] <wildbill> request is the object with query, sort, start, end, right?
[20:34] <jared_j> Bill: Right
[20:34] <slightlyoff> wildbill: yep
[20:35] <slightlyoff> wildbill: although if we define "uniqueness" as just the query and the sort, we can make it even easier
[20:35] <wildbill> so, it sounds pretty safe, except for corner cases where you have 2 cursors open on the same query
[20:36] <jared_j> Well, it might be better to have it as the whole requets object, to allow stores that use optional properties, have those as part of whatever it key/hashes/caches.
[20:36] <jared_j> Or a identifier property for the case wildbill denotes too.
[20:36] <jared_j> As part of the overall 'request' object.
[20:37] <slightlyoff> yep
[20:37] <slightlyoff> and the store figures it out
[20:37] <slightlyoff> which is fine by m
[20:37] <slightlyoff> me
[20:37] <slightlyoff> if it wants to be fine-grained and put stuff in requests for folks ot hand back, OK
[20:37] <slightlyoff> if not, no one is burdened by it
[20:38] <jared_j> Yeah. It also lets a store tag these things onto the request object when it's built, unknown to the caller. You don't see or care, it's store meta data for itself. Or I might be overthinking it a bit. :)
[20:38] <wildbill> that's good, it allows future expansion
[20:38] <slightlyoff> yep
[20:38] <wildbill> brian_skinner, slightlyoff : do you see the corner case problem w/no init() call?
[20:39] <brian_skinner> wildbill: i think no init call will be okay
[20:39] <wildbill> there is a corner case that isn't addressed by no init() call
[20:39] <slightlyoff> wildbill: yes, but that's a data-provider issue, and not a general-purpose case
[20:39] <brian_skinner> although i'm open to hearing otherwise, when we get feedback from people who actually are writing DB-based datastores
[20:40] <slightlyoff> wildbill: if you're going to want people to handle that to work w/ your store, document it
[20:40] <wildbill> i just want to make sure you guys understand the case.
[20:40] <brian_skinner> wildbill: can you summarize it?
[20:41] <wildbill> if there's no init() call, then you can't open two cursors for the same query at the same time.
[20:41] <brian_skinner> the 2 cursors open on the same query case?
[20:41] <brian_skinner> okay, got it
[20:41] <wildbill> yeah. hmm, maybe i'm wrong.
[20:41] <brian_skinner> do you ever actually *want* two cursors open on the same query?
[20:41] <wildbill> it seems pretty unusual
[20:41] <brian_skinner> isn't it fine if the two callers share the query?
[20:42] <jared_j> Uh, two scrolling tables side by side with different ganges in view on the same dataset? Seems a bit odd.
[20:42] <wildbill> i guess if you have two charts on the screen showing the same data
[20:42] <wildbill> yeah, two tables seems odd, but maybe a table and a chart
[20:42] <slightlyoff> wildbill: but even then, why have two cursors? and can't the server better optimize whether or not it should leave multiple cursors open?
[20:42] <wildbill> because most databases, you can't move the cursor backwards
[20:43] <brian_skinner> right, isn't it okay for them to share a cursor, since they're displaying the same result set?
[20:43] <slightlyoff> wildbill: server-side resource pooling seems like a bad reason to define a client API
[20:43] <slightlyoff> wildbill: right, but the server knows a lot more about that constraint
[20:43] <slightlyoff> wildbill: and can more efficiently deal w/ it
[20:43] <jared_j> And isn't it really up to the datastore impl whetherit shared or not anyway?
[20:43] <slightlyoff> jared_j: I think it should be, yes
[20:43] <slightlyoff> wildbill: I'm not saying it's not a problem
[20:44] <slightlyoff> wildbill: I'm just saying that I'm not sure it's *our* problem from the Dojo side = )
[20:44] <jared_j> That's not really user-need-to-know... I'm just not sure we have to expose anything for it.
[20:44] <wildbill> the problem is that the server can't be smart enough to handle this case unless it can figure out that there are two simultaneous queries
[20:44] <jared_j> Why wouldn't it be able to?
[20:44] <jared_j> What have we done to limit that?
[20:45] <wildbill> well, for databases, you need two open cursors on the same query.
[20:45] <jared_j> What have we done to limit that? :)
[20:45] <jared_j> If you reuse a 'request' object, part of the metadata a store may have stashed on it was a cursor id, for example.
[20:46] <jared_j> So you just create two request objects with the same query and sort ... and you could get two cursors. Or maybe I'm missing something.
[20:46] <brian_skinner> wildbill: in a datastore where you can't move the cursor backwards, then we may have problems regardless of whether the two queries share a cursor or not -- if the user scrolls backward in the UI, then that's a problem even with a single query, with it's very own cursor
[20:46] <wildbill> jared_j: yeah, i guess you are right.
[20:47] <slightlyoff> again, it's a problem, but I don't think it's the clients problem
[20:47] <wildbill> brian_skinner: right. so maybe connecting a Grid object to a database isn't reasonable; you need to cache the whole table on the app server.
[20:47] <slightlyoff> wildbill: well, at least not w/o a big query cache
[20:48] <slightlyoff> wildbill: if we're saying "we're gonna serve up huge data out of RDBMSes w/ no smarts or caching on the server", we're fucked no matter what we do at this API level
[20:48] <brian_skinner> wildbill: i didn't follow that last part -- i think we do want to connect a grid to a database
[20:48] <wildbill> i meant, *directly*
[20:48] <jared_j> How realistic is it that you would have a store connected directly to a DB, or through some sort of service ontop of the DB?
[20:48] <brian_skinner> got it
[20:48] * slightlyoff puts the odds at nearly zero
[20:49] <wildbill> well, there's always a proxy through the app server.
[20:49] <brian_skinner> nearly zero
[20:49] <jared_j> I think you'd be on crack to not be going though a limited service, but that's me.
[20:49] <wildbill> it's just a question of what the app server does to mitigate the fact that the database cursor can't go backwards
[20:50] <jared_j> Like slightlyoff says, not a client issue, really.
[20:50] <wildbill> yeah, just as long as the server knows what are the currently open grid widgets on the client
[20:50] <wildbill> so the server knows when a scroll backwards request is likely to come
[20:51] <slightlyoff> wildbill: well, the server knowing is different than the DB knowing, right?
[20:51] <wildbill> sure
[20:51] <wildbill> i'm not sure what you are getting at
[20:52] <jared_j> If the server service exposes paging services, then it needs to have a forward/back. And deal with it in whatever manner the db it uses does it. To be sane, anyway, I would think.
[20:52] <wildbill> sure
[20:52] <slightlyoff> so we can say that the client API doesn't need to carry hints for this stuff to pass to the server, right?
[20:52] <wildbill> huh? i thought we were saying the opposite.
[20:53] <jared_j> The client APi doesn't need to expose anything. I would think if the client needed anything, it's internal to the store impl.
[20:53] <jared_j> At least that's my current take on it.
[20:53] <wildbill> oh right, but it might hide something in the request object (the arg to store.fetch())
[20:54] <jared_j> Yes, but not client API visible. :)
[20:54] <wildbill> fair enough
[20:55] <wildbill> yeah, so i think i can live with option #6 (from http://dojo.jot.com/DojoDataFindAndFetch) augmented maybe with a store.close(fetchArgs) method.
[20:55] <jared_j> I do agree we need the store.close() hook for some store types.
[20:55] <jared_j> Things like JsonItemStore ... it would just be a no-op.
[20:56] <wildbill> right
[20:58] <jared_j> Okay, are we then in general agreement for the moment to add th store.close() and leave fetch as is, pending any replies from DBland folks?
[20:58] <wildbill> yeah, i guess so
[20:58] <slightlyoff> +1
[20:59] <brian_skinner> +1
[20:59] <wildbill> can you tell us what they said when they reply (on dojo-contributors), so brian and alex can see it?
[20:59] <slightlyoff> dmachi? robkinyon?
[20:59] <jared_j> Should be able to.
[20:59] <slightlyoff> awesome
[20:59] <slightlyoff> thank you!
[21:00] <jared_j> So ... other agenda items (I think these will go quick. I hope anyway).
[21:00] * slightlyoff crosses fingers
[21:00] <jared_j> A small change to the identity api.
[21:00] <jared_j> findByIdentity -> fetchByIdentity() To be consistent.
[21:00] <jared_j> And I think it needs to be made async.
[21:00] * wildbill changes topic to 'http://dojo.jot.com/2007-03-27'
[21:00] <jared_j> It's currently defined and synchronous.
[21:00] <jared_j> Er, defined to be
[21:02] <jared_j> My take is it would be: findByIdentity( {identity: <whatever>, onComplete: doSomething, onError: doSomethingElse). Where onComplete just takes param of the item located, or null if none. onError would take the error object.
[21:02] <jared_j> Er, fetchByIdentity.
[21:03] <brian_skinner> sorry to complicate this discussion, but ... i'm concerned either way...
[21:03] <jared_j> Hm?
[21:03] <brian_skinner> if we keep it sync, that causes potential problems down the road
[21:03] <slightlyoff> I like both of those changes
[21:03] <slightlyoff> they make it consistent
[21:03] <brian_skinner> but if we switch it to async, i think it may be pretty much unusable -- becomes a pain in the butt for the user
[21:03] <brian_skinner> i don't have a solution
[21:04] <brian_skinner> just wanted to raise a red flag
[21:04] <brian_skinner> one option is to just make it async -- i'm not opposed to that
[21:04] <brian_skinner> another option is to leave it sync, but make no promise that it will work for all items -- may only work for cached items
[21:05] <brian_skinner> another option is to have two separate methods, one sync and the other async
[21:05] <jared_j> And as you saw with JsonItemStore required some poking ahead of time for it to work.
[21:05] <jared_j> Two apis == ugh, IMHO.
[21:05] <brian_skinner> i have no preference among those options -- i have no clue what the right thing to do is
[21:06] <slightlyoff> well, two APIs is at least clear, but sync sucks and no other part of the data fetching APIs are sync
[21:06] <jared_j> Well, what were you visualizing the original use case as?
[21:06] <dmachi> i heard my name, but am not sure what i was being asked, sorry
[21:06] <brian_skinner> right, and that poking ahead of time with the JsonItemStore required the availability of a sync transport, which we may not always have
[21:07] <brian_skinner> slightlyoff: actually, a lot of the other parts of the API are sync, like getValue()
[21:07] <slightlyoff> brian_skinner: hrm, OK
[21:07] <jared_j> There's also an issue with JsonItemStore sand getFeatures ... but I can talk with you in mail about that one.
[21:07] <jared_j> The item locating apis are async, though
[21:07] <jared_j> Item maniptulation apis are sync.
[21:08] <brian_skinner> slightlyoff: (and those other sync methods are going to come back to bite us someday too, once we have datastores dealing with large distributed object graphs, like LinkedIn)
[21:08] <brian_skinner> but we can't make everything async, or the API is unusable in the real world
[21:08] <jared_j> Very.
[21:08] <slightlyoff> brian_skinner: well, it sounds like we need a general-purpoes "out"
[21:08] <slightlyoff> purpose, rather
[21:08] <slightlyoff> here's one option
[21:09] <slightlyoff> all of these API should take as an optional last parameter a Deferred object
[21:09] <slightlyoff> if they receive a deferred, they *can* work async
[21:09] <slightlyoff> (not required)
[21:09] <slightlyoff> but it's then up to the user to determine what level of effort they want to go to
[21:09] <jared_j> These apis meaning the getValue, setValue, etc apis?
[21:10] <slightlyoff> jared_j: yep
[21:10] <slightlyoff> I know you guys are trying to keep the dep chain down
[21:10] <slightlyoff> so we could scope that in broadest terms
[21:10] <brian_skinner> slightlyoff: that's an appealing idea, except for the dependency thing
[21:10] <slightlyoff> (we get the last param and it's got a "callback" and "errback" methods)
[21:10] <brian_skinner> that seems reasonable
[21:10] <slightlyoff> well, I don't think it's really any more complicated than that
[21:11] <brian_skinner> and we don't need to make that part of the basic Read api
[21:11] <brian_skinner> we can layer that on top
[21:11] <slightlyoff> the last wrinkle is passing back an ordinal to say "this is operating async"
[21:11] <brian_skinner> if a store offered that feature, it could advertise it with a separate getFeatures() value
[21:12] <slightlyoff> sure
[21:12] <slightlyoff> but I don't think it needs to be that optional
[21:12] <slightlyoff> consider the case where it's actually sync under the covers no matter what
[21:12] <brian_skinner> yup, point taken
[21:12] <slightlyoff> if you get the optional arg, you just d.callback(value); immediately
[21:12] <slightlyoff> it's like 5 lines of code, I think
[21:13] <slightlyoff> and making supporting that style required eliminates problems later on
[21:14] <brian_skinner> it's only a few lines of code, but that code has to appear in a whole bunch of methods, right? -- getValue(), containsValue(), getAttributes(), etc.
[21:14] <jared_j> Yes.
[21:14] <slightlyoff> brian_skinner: yes, but it gives us an out to a really thorny problem and future-proof's us
[21:15] <brian_skinner> i think it's very important to make it easy to implement simple datastores -- and easy to understand the basic documentation about the read-only API
[21:15] <slightlyoff> I agree
[21:16] <brian_skinner> we need to get Eugene on board, so that he's using dojo.data for the Flickr access in his demo, and get Brad on board, so that he's making dojo.data the default for the offline toolkit -- and then, wonderously, the Flickr demo suddenly *just works* with the dojo offline toolkit
[21:16] <brian_skinner> but for that kind of adoption to happen, everything needs to be really, really painless
[21:17] <slightlyoff> brian_skinner: so we have one complication w/ this API
[21:17] <slightlyoff> brian_skinner: we will need to document that if the store only supports async and you don't pass a Deferred, you will get an exception
[21:18] <brian_skinner> well, and we need to document the fact that you can pass a deferred in the first place
[21:19] <slightlyoff> yes
[21:19] <brian_skinner> which makes the basic documentation more complicated, even for people who will never use the feature
[21:19] <slightlyoff> so our other option are optional async APIs for everything we're discussing
[21:19] <slightlyoff> and frankly, I don't much care which way we go
[21:19] <slightlyoff> I think they both have the same overhead for explaining what to do and when
[21:20] <jared_j> A Read.js and a ReadAsync.js?
[21:20] <slightlyoff> Deferreds require users to create them
[21:20] <brian_skinner> can we maybe punt for now?
[21:20] <brian_skinner> and only have sync versions of getValue, containsValue, etc.
[21:20] <slightlyoff> brian_skinner: so long as we're comfortable that we have a path to getting there that's not going to cause code changes to existing stuff
[21:21] <slightlyoff> (and by stuff, I mean user code)
[21:21] <jared_j> brain_skinner: Well, what about fetchByValue? Leave it defined as sync or go ahead and make it async?
[21:22] <brian_skinner> slightlyoff: well, zero code changes for existing apps that continue to use datastores that actually are able to provide sync answers
[21:23] <slightlyoff> brian_skinner: for providers that don't, I don't see how you can get away w/o a system that generates/returns deferred or has async functions
[21:23] <slightlyoff> brian_skinner: and give that's the price of entry, you just have to decide what you want
[21:23] <brian_skinner> jared_j: if we punt on the other methods for now, i'm not sure what that suggests we should do with fetchByIdentity()
[21:24] <brian_skinner> jared_j: what do you think is the right thing to do?
[21:24] <brian_skinner> slightlyoff: yup, agreed
[21:25] <slightlyoff> so sync is a special case of async
[21:25] <jared_j> brian_skinner: Well, if we're treating it like a special version of fetch, I would say make it async with callbacks to line up with generic fetch(). But that's just a gut feeling.
[21:25] <slightlyoff> or can be treated that way later on
[21:26] <brian_skinner> slightlyoff: i didn't follow that last bit -- "so sync is a special case of async, or can be treated that way later on" -- what API are you saying we should use?
[21:27] <slightlyoff> brian_skinner: I think you're conflicted about being bitten by sync stuff at some point
[21:27] <slightlyoff> but in all of the async API cases, it could be sync under the covers
[21:27] <brian_skinner> slightlyoff: i just wish that there was some way to wrap an underlying async call and make it appear sync to the user, even if that hangs the app for a while
[21:28] <slightlyoff> so I'd say that if you can get away w/ a sync API now and it's easy to explain, that might be good
[21:28] <slightlyoff> brian_skinner: sorry, we don't have that = |
[21:28] <jared_j> I wish JavaScript had a wait() API :P
[21:28] <slightlyoff> brian_skinner: so you've gotta pick
[21:28] <brian_skinner> yup, sync for now, for all the getValue() type methods
[21:29] <brian_skinner> as for fetchByIdentity(), i have no vote one way or the other
[21:29] <slightlyoff> and we have at least 2 workable approaches for dealing w/ it later
[21:29] <jared_j> If we end up needing to.
[21:29] <brian_skinner> yup
[21:29] <jared_j> What are other folks feeling on fetchByIdentity, then?
[21:30] <slightlyoff> I daresay make it sync if you can get away w/ it
[21:30] <slightlyoff> that lets us deferr the async discussion entirely, I think
[21:30] <jared_j> Ok, I'll leave it defined as sync for the moment.
[21:31] <jared_j> To deal with later if we have to.
[21:31] <brian_skinner> and, sorry to do this, but...
[21:31] <brian_skinner> if we leave it sync, then we shouldn't use "fetch" in the naem
[21:31] <brian_skinner> name
[21:31] <jared_j> getByIdentity, then?
[21:31] <jared_j> byIdentity()?
[21:31] <brian_skinner> getItemByIdentity()
[21:32] <jared_j> Sure.
[21:32] <jared_j> I'm okay with that.
[21:32] <brian_skinner> okay
[21:32] <jared_j> Any other opinions?
[21:32] <brian_skinner> anyone else still here ;-)
[21:32] <slightlyoff> no opinion
[21:33] <jared_j> I'll go with getItem, then so it doesn't get associated with store.fetch()
[21:33] <brian_skinner> peachy, thanks
[21:33] <brian_skinner> next agenda item?
[21:33] <jared_j> The unsetAttribute one.
[21:34] <jared_j> I'm fine with any of the suggested name changes, rea;;y/
[21:34] <jared_j> really.
[21:34] <jared_j> That seems just sorta nitpicky to me. :)
[21:35] <jared_j> Options were: unsetValue, unsetValues, or clearValue
[21:35] <brian_skinner> okay, if you don't mind making the change, i'd like to go with clearValue
[21:36] <jared_j> IS another option deleteAttribute, perhaps? Because isn't it intended to remove an attribute totally from the item?
[21:36] <jared_j> Or is that confusing?
[21:36] <brian_skinner> i want to try to keep the word Attribute out of the name
[21:36] <brian_skinner> "The unsetAttribute method is a peer to all "value" methods, like setValue, setValues, getValue, getValues, and containsValue -- but the name "unsetAttribute" doesn't make that obvious. And, to make matters worse, the "unsetAttribute" seems to suggest that the method is a peer to the "attribute" methods, hasAttribute and getAttributes."
[21:37] <jared_j> Does ir remove the value completelt and the attribute? Effectively sets it to undefined?
[21:37] <brian_skinner> okay, now i'm losing my confidence about this
[21:37] <jared_j> Eh I'm thinking too much on it, probably.
[21:37] <jared_j> What is the general purpose of the API in your mind?
[21:38] <brian_skinner> i guess setting the value to undefined is effectively the same thing as removing the attribute entirely
[21:38] <brian_skinner> the Read API actually says that...
[21:38] <brian_skinner> "Saying that an "item x does not have a value for an attribute y" is identical to saying that an "item x does not have attribute y""
[21:38] <slightlyoff> then that's solved?
[21:39] <brian_skinner> okay, nevermind
[21:39] <slightlyoff> what's next?
[21:39] <brian_skinner> i apologize for bringing it up
[21:39] <jared_j> I'm fine with clearValue, really. Since the read api states the above. I'm fine with leaving it alone too. No real opinion either way. :)
[21:40] <jared_j> Leave it for now, then, unless we get a real pressing need to change it.
[21:40] <brian_skinner> let's leave it for now -- no point making random changes unless we have at least one person who all afire abou them
[21:40] <brian_skinner> about
[21:41] <brian_skinner> ...
[21:41] <jared_j> Okay, #2. Should fetch() be doing parameter validation and throwing exceptions on obviously wrong input? Like, oh, a negative start, or a non-integer start?
[21:41] <brian_skinner> fetch() exceptions?
[21:41] <brian_skinner> i can see arguments either way
[21:41] <jared_j> So can I.
[21:41] <slightlyoff> is negative start an offset from the back?
[21:41] <brian_skinner> slightlyoff: potentially
[21:42] <jared_j> That would be a strange interpretation, I think.
[21:42] <brian_skinner> but a Date would clearly be an invalid start
[21:42] <jared_j> Right
[21:42] <jared_j> As would say, a boolean
[21:42] <brian_skinner> if this were Java, I would say we should require conforming datastores to throw an exception
[21:42] <slightlyoff> even for time-series data?
[21:43] <jared_j> So, should we state it should do basic validation and throw errors on bad params.
[21:43] <slightlyoff> I really don't even like enforcing this
[21:43] <brian_skinner> slightlyoff: point taken
[21:43] <jared_j> start is the index into the full list of items.
[21:43] <slightlyoff> people should be allowed to pass good stuff and enforce what they like in their stores
[21:43] <jared_j> Okay.
[21:43] <brian_skinner> okay
[21:43] <jared_j> Next topic!
[21:43] <jared_j> Style thing
[21:43] <brian_skinner> slightlyoff: this one's for you!
[21:44] <slightlyoff> heh
[21:44] <jared_j> In docs, sould we use | or || to denote what types a param can be when it can be multiple.
[21:44] <slightlyoff> lets go w/ "||"
[21:44] <brian_skinner> done
[21:44] <jared_j> Sold!
[21:44] <slightlyoff> rad
[21:44] <slightlyoff> next!
[21:44] <jared_j> And that's generally it for the agenda unless folks have anything more?
[21:44] <jared_j> We already talked about agenda item 4, really.
[21:45] <slightlyoff> huzzah!!
[21:45] <jared_j> With the close() api, it could be alsi used for cache clearing
[21:45] <jared_j> How I see it, anyway.
[21:45] <jared_j> You close the daya, you dump anything the store cached.
[21:45] <brian_skinner> yup, sounds good
[21:45] <slightlyoff> I think there's gonna be some magic in here too where some providers to the Right Thing (TM) and other's done
[21:45] <slightlyoff> s/done/don't/
[21:45] <slightlyoff> and that's OK
[21:46] <slightlyoff> the one point that wildbill made was that we don't have a way to show that request is unique
[21:46] <slightlyoff> some sort of "bypass the cache" or "invalidate the cache" flag
[21:46] <jared_j> Sure we do, if you're reusing the request object.
[21:46] <slightlyoff> w/o having refs to previous fetch() results
[21:47] <jared_j> Right.
[21:47] <slightlyoff> some optional "bypassCache" flag might be good, but I"m fine w/o it for now
[21:47] <brian_skinner> slightlyoff: i'd like to think that you would never want to bypass the cache
[21:47] <slightlyoff> it's a no-op for stores that don't cache
[21:47] <brian_skinner> a good datastore should be keeping the cache up to date
[21:47] <wildbill> so, there's a larger issue, because usually with a cache, even new queries use the cached data from old queries.
[21:48] <brian_skinner> keeping the cache up to date is what comet is for
[21:48] <wildbill> yeah, i guess hibernate will keep the cache up to date, right?
[21:48] <slightlyoff> brian_skinner: hehe
[21:48] <jared_j> The store could take optional init params of how often to flush items form its cache
[21:48] <jared_j> Such as, any tiem of X minutes, punt it and get it again.
[21:49] <jared_j> Or, yeah, as noted a store could define an optional param on its args to force a reload.
[21:49] <jared_j> In fetch
[21:50] <wildbill> yeah, that's easy to add later then, i guess.
[21:50] <jared_j> I'm not sure we need t require anything here in the client api.
[21:50] <slightlyoff> passthrough stores get out of this problem no matter what
[21:50] <slightlyoff> yep, easy to add later
[21:50] <slightlyoff> cool
[21:50] <jared_j> Yeah, we can add later.
[21:50] <slightlyoff> I think we can punt on it
[21:50] <jared_j> If it becomes an issue.
[21:50] <slightlyoff> huzzah!
[21:50] <wildbill> i don't understand the write api though
[21:50] <slightlyoff> wildbill: get some sleep! ;-)
[21:50] <brian_skinner> wildbill: what part of the write api?
[21:51] <jared_j> wildbill: Can you write up what parts you don't understand? We could cover it in the mailing list or next meeting. We're 10 min to stop.
[21:51] <wildbill> oh, just the synchronization between reads and writes.
[21:51] <wildbill> i don't understand it because i haven't read the docs thoroughtly; it's not your fault.
[21:52] <jared_j> Ok, want to review the latest iterations of them and write up your thoughts when you're more rested?
[21:52] <wildbill> sure
[21:52] <jared_j> You may also want to look at the simple Notification API, as it would likely come into play on some things too.
[21:53] <wildbill> ok
[21:53] <jared_j> Well, is that a wrap, then?
[21:53] <brian_skinner> so, one last topic -- do we need to talk about getting the latest code checked in?
[21:54] <slightlyoff> jared_j: are you gonna work w/ peller? or do you need another committer's help?
[21:54] <jared_j> I thought it was just a matter of iinishing up style issues then working through you or Peller?
[21:54] <slightlyoff> yep
[21:54] <slightlyoff> that's all I know of
[21:54] <jared_j> I'm fine with working through peller.
[21:54] <slightlyoff> I can review/merge too
[21:54] <brian_skinner> cool
[21:54] <slightlyoff> actually, that might be preferable
[21:54] <brian_skinner> thanks
[21:54] <slightlyoff> just let me know what you want to do
[21:55] <slightlyoff> and know that I'm high-latency = )
[21:55] <jared_j> Ok
[21:55] <slightlyoff> whereas peller might not be
[21:55] <brian_skinner> okay, i'm off to go do packing
[21:56] <jared_j> I'm off to deal with visiting relatives. Thanks, everyone!
[21:56] <brian_skinner> yup, thanks!
[21:56] * jared_j (n=chatzill@cpe-071-077-064-047.nc.res.rr.com) Quit ("Chatzilla 0.9.77 [Firefox 1.5.0.11/2007031202]")
[21:56] * wildbill (n=bill@FLH1Acz201.hkd.mesh.ad.jp) has left #dojo-meeting
[21:57] <slightlyoff> good meeting all
[21:57] <slightlyoff> later!
[21:57] * slightlyoff (n=alex@204.14.154.182) has left #dojo-meeting
[21:58] * brian_skinner (n=brian_sk@adsl-67-127-58-153.dsl.pltn13.pacbell.net) Quit ("Chatzilla 0.9.77 [Firefox 1.5.0.11/2007031202]")