Growth for Findka and simplified query subscriptions for Crux

Findka

Lots of growth last week. The article I posted last week was much more popular than I anticipated. It’s had about 2.5K page views so far, and our number of signups jumped from 45 to 106. One person added over a hundred items to Findka. I thought maybe it was spam when I first saw the aggregate number, but it’s not—just an enthusiastic user.

The cool thing is that Findka now has a pretty good critical mass of users and data for me to work with. I have a database of almost 800 items to pull recommendations from. Rather than thinking mostly about how to get new users to sign up, I can focus on improving the service for existing users.

I’m starting to feel more pressure from that actually. For a few weeks, Findka was mainly coasting in terms of feature development while I focused on marketing. I started porting Findka from Firebase to Biff in order to make future development easier, and now I’m racing to get that done. Speaking of which…

Biff

Progress has been coming nicely. I’ve finished the rules system for write operations, so you can submit arbitrary transactions from the frontend and they’ll be authorized and then submitted to Crux (or rejected). I’m working on reading data now. I made a subscription interface, so all you have to do to provide a subscription is write three handlers on the backend:

  • One for handling new subscription requests

  • One for handling unsubscribe requests

  • One for notifying existing subscribers when relevant data changes

I’m in the middle of writing these handlers for Crux. Of course the tough part is (efficiently) figuring out which subscribers need to be notified (and with what data), but I’ve finished a rough draft for that. I’m working today on cleaning up the code and testing it. That’s actually the last major part of Biff left; after that it’ll be a lot of tying up loose ends and making everything neat. Authentication is already done thankfully. Biff provides authentication through email links, the method that Findka uses currently. Password and SSO authentication will likely come eventually.

Usually I’d give more details or some code examples; however I am frankly exhausted right now from working on Crux subscriptions. I’ve shifted my sleeping schedule so I usually stay up till 2 or 3 AM coding. I’ve always found it’s easier to focus at night. Though sometimes I wake up tired, like today.

Fun story: when I was 18, on my last Friday night before leaving home to be an LDS missionary for two years, there were some updates I really wanted to get done for an Android app I had been developing. I stayed up all night coding (literally; I didn’t sleep at all) and didn’t feel tired. I even went roller skating the next day. It was one of the weirdest experiences I’ve ever had. I’ve tried to replicate it a couple times but have been unsuccessful.

Back to Biff—here’s one code sample actually: query-contains?. This takes a simplified, subscribable Crux query (maybe I should call it a “Biff query”?) and tells us if a given document would be returned by that query:

(query-contains?
{:where [[:age age]
[(>= age 17)]]}
{:crux.db/id :some-document-id
:age 66})
=> true

The difference between Biff queries and normal Crux queries is that Biff queries do not allow cross-document joins. Notice in the [:age age] clause that we omit a variable for the document ID. It’s unnecessary since all clauses are for the same document.

This makes Biff queries much more restrictive than Datalog queries, but the upside is that this restriction means we can efficiently:

  1. Take a Crux transaction result.

  2. Get a list of the documents affected by that transaction.

  3. Go through our list of Biff query subscriptions and figure out which ones were affected by the changed documents.

And, as I’ve demonstrated with the Mystery Cows app, joins can be handled declaratively on the frontend instead.

In any case, I’m very excited to get Biff released soon :).