So long CouchApp, we hardly knew ye

Apr 3, 2014 at 3:18 PM
When I started on Thali I did some prototyping in CouchDB, using the couchapp model [https://github.com/couchapp/couchapp]. This enables you to push a tree of files into CouchDB and serve up your JS app from there.

But that's a CouchDB thing, and I'm now realizing we won't be doing that in Thali. Which, I've concluded, is no great loss. The couchapp.org website was sparse when I last checked, and doesn't even load at the moment.

Currently I'm running out of the filesystem with JS making PouchDB calls into Thali via the Chrome extension. As I understand it, we are aiming for a model that entails building a Java jar that launches a WebView driven by some tree of local assets (HTML, JS, CSS). I'm not yet clear what will be our equivalent to couchapp, by which I mean not the appdev model but the tool that pushes that tree of assets into the database.

Or not the database, if we keep that stuff in the jar. I know you've though about that stuff, it'll become clear when we get there, but it occurred to me to ask: are there cases where we'll need CORS support in Thali?
Apr 3, 2014 at 5:07 PM
There are actually a few different questions here so let's tease them apart.

Short Term

In the short term we want to let people build apps using the web. We want to do this by creating a Java container that runs a WebView that loads the app's HTML. The WebView container will make available the ability to speak PouchDB over Thali. The HTML/JS/CSS that define the app can live inside the Jar file or they can live in the file system. Depending on the environment there are arguments for both. But Thali won't require one over the other. We will easily support both.

Evergreening a.k.a. Auto Update

One challenge we do need to think about is how a program keeps itself automatically up to date. That is having to install 'new versions' is generally a bad idea. I suspect in most cases people don't update not because the update is broken but because they just can't be bothered. As we saw with browsers this leads to really huge security issues. So for most people you really want your programs to update themselves.

Now depending on the distribution mechanism this update might occur via the platform (think of Android as an example if you got the app via a store) but we should probably support auto update as a core part of Thali. Synching to a feed to get updates seems pretty obvious. And if your app is running out of the feed then updates are auto-magical. Now we know that in practice that is really dangerous. It is perfectly possible to only get part of an update and this could break your app. In other words the CouchDB instance might contain foo.html and bar.html and an update was supposed to update both but the web being the web only foo.html got updated but you didn't quite get the bar.html update in time. Now what? Your app is in a half state and likely to break.

So I suspect in practice the app wouldn't run out of the feed directly. Instead the feed would just have update information that the app would detect and apply as appropriate. If the app is running off the file system then these updates are pretty straight forward. The feed would contain diffs or full updates (aka checkpoints for you database folks) that would be applied in order. The CouchDB database effectively becomes a transaction log and we 'run it'. Each step in the log is self contained so at worst we might not update to the very latest version (because we don't have that part of the log yet) but we are guaranteed to update to a consistent point in time.

But if they are running from files in the Jar then I'm not sure how they update themselves since typically one can't (easily) update the Jar.

This probably argues that even on Android we should encourage people to perhaps package their code in the apk (the Android equivalent of an executable) but to then unpack it into local storage and run from there to make updates easy. I don't know, maybe Android has magic here. We should check.

Thali as the real web

Now we know that one of our big longer term goals is that Thali is the web which means Thali enabling the browser. This brings up the question of how do you download a Thali app into the browser? The default answer will be - it's the web stupid, make HTTP requests. But for privacy and performance reasons it would be beyond awesome if we could just download an entire Thali app as a single file and run it locally. This is where the CouchApp model was kind of cool since it meant you could just sync the database, confirm you had a full sync and then start running right out of the feed.

But given the sync issues already mentioned in the previous thread this will probably be a two step as above. E.g. download the content in a 'read' feed and then copy it out someplace to run. Updates would come down the 'read' feed but only applied once confirmed to be complete.

Conclusion

So it does look like we probably won't run using Couch Apps but I think we might be being a little short sighted here. Might it make sense to fix the update consistency issue with Couch Apps inside of CouchDB rather than doing the two step described above? I'm not sure but it's something to think about. I registered this issue here
Apr 3, 2014 at 5:31 PM
In regards to CORS (the previous mail was long enough) I think the answer is a very definite maybe.

Why CORS?

The reasoning behind CORS was a security argument.

The browser security model (and I use that term very loosely) is based on domain security. The idea is that if you load a page from domain X then that page is assumed to be trusted by domain X and should be allowed to do anything it wants with domain X.

But what if the code from domain X wants to do something with domain Y?

Now today we already allow that but only for safe methods(e.g. GET) and we don't let the requester see the response. E.g. if a page loaded from domain X makes a request to domain Y and the request is 'safe' then the browser will allow it but it won't let the code from domain X see the results of the request.

Now right away you should realize that the previous statement is a bald faced lie. And this lie has led to a lot of security holes. The browser doesn't just allow safe methods. It also allows domain X to make a request to domain Y using a very unsafe method, namely, POST.

Now, in theory, that cross-domain POST call is only allowed if it's a HTML FORM POST which in theory is detectable by the server and therefore servers should just know not to trust POST requests containing HTML FORMs unless there is an additional security mechanism (no, not cookies). But in theory this is considered a 'solved' problem and so is o.k.

But as XML HTTP Request (XHR) came on the scene this opened up a new channel of attack. Initially XHR handled this by just using the same domain policy. But this proved too restrictive in practice. There were lots of good reasons why a web page from domain X would want to send an arbitrary HTTP request to domain Y and see the result (think mashups).

So this eventually led to CORS. It essentially lets the server say "I am o.k. with websites loaded from domains other than mine sending me requests". It actually allows finer grained security than that but the previous gives you the idea.

In theory so long as a server in domain Y says "HIt Me!" then domain X can send it whatever requests it wants. Presumably the server in domain Y has put in place the right protections to keep the endless list of browser attacks from working against them.

How does this affect Thali?

Short Term

Oy. What a mess!

O.k. so first of all to the extent that we are running inside of a WebView we have to decide if we activate cross domain security. Most Web Views let you just turn it off. Since the Web View model is NOT a browser model but is an app model my strong preference is that we just turn off cross domain security in the web view. I believe at that point XHR should 'just work' without any need for CORS.

Long Term

But obviously this situation gets much more complicated when we think about a Thali Browser. One thing I am sure of - the domain model makes zero sense for Thali. We are a peer to peer system. Who you got an app from means little or nothing. I suspect an intent/capabilities security model is what we need. Not a domain model. A Domain model is inherently based on a centralized system. Thali is inherently decentralized.

My suspicion is that we will refuse to let cookies be used with Thali connections and from there we will let Thali requests skip the domain based security model.

But honestly this is a big complex area and I expect that eventually we'll have to spend a lot of time on it.

But not today.