Where are we?
So right now our strategy for having PouchDB talk to the Thali Device Hub (TDH) is very generic. We write an adapter for PouchDB that bridges to Java code that then speaks mutual SSL auth and in all ways walks and talks like a normal Thali client.
This is nice because it makes PouchDB look just like a Java Thali client or a .net Thali client.
And better yet, it really does work.
But there is a price and I'm deeply concerned this price is quite likely to go off the deep end.
The price is that every request and every response has to be turned into a string in order to be dragged over the Java bridge.
But there is an even bigger price that has me really worried. Eventually we will want to handle attachments. More to the point we'll want attachments that are pictures and movies that we will want to display in the browser.
The idea of dragging a movie over a string interface isn't my idea of a good time. And don't even ask me how we get it to the movie handler. We might be able to do something with the binary data type on modern enough browsers but now we are making things really
I strongly suspect it's a requirement that someone can dump in, say, an img tag and have that properly work.
With our current setup the img tag wouldn't work because the webview doesn't know what a httpkey url is.
So then we start walking down the weird path. How do we make img tags work? Or video tags?
Stick with the strings
I think for the PPNet demo we will need image support. So we have this problem now. But images aren't videos and in the short term we could potentially just live with taking the attachments, turning them into base 64 strings, moving them over the bridge and
then making them appear as blobs from the PouchDB API. Note, btw, that our xmlhttprequest adapter doesn't currently support blob types. But that shouldn't be brain surgery to fix.
But we have to decide how hard we want to make this work. For example, if there are big pictures then their full value will have to be serialized into memory before being moved over. In theory we could fix this by introducing a streaming interface but man what
a complete waste of time!
So if we are going to stick with strings for now we better stick to small pictures!
Setting up a second listener for getting attachments
Right now the TDH listens for mutual SSL auth connections typically on port 9898.
Imagine we set up a second listener on localhost on say port 8989 (or whatever). Because it's on localhost it can't accept requests from the Internet, only from local apps with access to the localhost handler.
But not all local apps are authorized to get everything in the TDH and the second localhost listener can't tell who sent an incoming request.
But there is a pretty well known pattern for solving this, a bearer token. We make the client get a bearer token and then use that bearer token in its requests. We need the token to be in the URL because the img and video tags aren't going to support authorization
arguments. Normally putting bearer tokens in a URL is a really bad idea. But in our case I believe it's o.k. because we are talking about stand alone applications that don't share caches.
So the idea is that the app would get a list of documents and see it has an attachment. It would take the attachment ID and create a URL of the form:
http://127.0.0.1:8989/[bearer token string]/[database name]/[document id]/[attachment name]
This would go to our localhost Couchbase lite listener who would validate the bearer token and then let the request through.
So the workflow would be:
- App gets its bearer token via some java bridge voodoo
- App talks via PouchDB using our adapter and string bridge nonsense
- App gets URLs for attachments with images and video it wants to display
- App mangles attachment URLs into form described above
- App uses DOM to submit mangled URL
- WebView (in Java or Android) does a HTTP GET to the mangled URL
- The localhost listener receives the URL, validates the bearer token and completes the request
To make things even simpler we can almost certainly implement this proxy server by taking a Couchbase Listener and screwing around a little bit with its URL processing logic (basically enough to validate the bearer token and then pull it out). So this shouldn't
be too hard to implement. The bearer token itself can be something trivial like an expiration date plus a cryptographically secure random number. Not brain surgery really.
Double down on the localhost listener
But if we are going down this path then it begs the question - why not go further? Why not just get rid of the PouchDB adapter all together? What if we just use the mangled URL syntax for PouchDB? In that case PouchDB just sees and plays with a perfectly normal
http URL (the database name is moved one path segment farther to the right but I believe PouchDB can handle that) and now we can use off the shelf PouchDB.
I think this works. I'm not 100% sure. I don't think CouchDB's API ever puts fully qualified URIs in the request or response. Instead URIs have to be constructed. So in theory as long as PouchDB knows the 'base' (e.g. including the bearer token) then things
should 'just work' without the Couchbase core have any clue what we've done. I think.
But wait, why didn't we do #2 from the start?
The reason was that I was afraid of having to write and maintain two different listeners with two different security architectures.
Honestly, I'm still afraid of that. As the only dev every feature I add takes us farther away from ever finishing this thing!
TL;DR - This section describes really fragile tricks for intercepting new URL types in webviews that aren't going to work in the real world, so feel free to skip this section.
Another trick wouldn't require the proxy server at all. That trick is to have httpkey URLs stuck into the img and video tags. In theory this might, possibly, potentially, theoretically, work.
In the case of android there is an interface on the WebViewClient (which handles callbacks from the webview) called shouldInterceptRequest that in theory should get called and give the app the chance to either let the request go or to put in its own response.
In theory this should work for random URL types. So in theory we should be able to use it, detect httpkey, handle the request using a nice streaming library (built into Java) and make things work sensibly. Unless img or video or embed or whatever doesn't properly
use souldInterceptRequest. :(
In the case of Java things get uglier. The only mechanism I can see that might, theoretically, work here is the URL factory handler. You can register a URL factory and the system will check with it when resolving URLs. Now, this assumes that the WebEngine in
JavaFX actually pays attention to this handler. Which it might not. But another fun fact is that per JVM instance only a single call to register a factory handler can be made and Couchbase is calling it!!!! Now in theory this might not matter. The reason is
that the TDH shouldn't be running in the same process (and hence on the same JVM) as the client application. But that means (wait for it....) that we can't bundle our TDH management UX in HTML with the TDH itself because we would run into this issue.
There are two possible workarounds.
- We could wrap Couchbase's factory with our own. Heck, we have the code, we can do what we want.
- There is another approach that seems exactly like the kind of fragile scary thing one debugs forever.
So with one of the two above choices we could get using HTML as our UX for the TDH's own UX.
But to complicate things further we want to eventually use Chromium and I'm not even sure the intercept APIs there work in the same way and if they do we would have to use JNI to expose them in Java since I don't think they are in the existing CEF. But I couldn't
be completely wrong there since I haven't used the Java CEFs yet.
So now what?
So I think we can write off intercepting URLs. It behaves differently in each environment (e.g. Android, JavaFX and Chromium) and it's not even clear how robustly it would work.
So this leaves us with either trying to move large binary objects over strings or setting up the localhost listener.
Now to complicate things further there might be real value in keeping around the PouchDB adapter. It means that a HTML client can talk to ANY TDH, not just its local one. That's kinda cool. So it might be worth at least doing the work to support the blob type
for getAttachment in pouchDB, even if we do it in a horribly inefficient way.
Alternatively we can stop putting good money after bad and just switch to the second listener approach whole hog.
My guess is that we'll do the blob work because I'm too scared to mess with our demos for HTML 5 and London. But if we do get another dev resource then we probably should prioritize the second listener so we have a solid story for video and images.