[megathread] Waku Platform & Developer Adoption Metrics

Waku Connect’s mission is to push and facilitate the adoption of Waku, focusing on platforms and developers.

To ensure that we move forward, then forward must be defined. We have selected a set of metrics to measure the level of adoption of Waku.

The metrics are focused on platforms and developers. A separate set of metric will focus on actual network usage.

We selected metrics that are easily available:

  • GitHub stars convey how popular a project/library is among developers.
  • npmjs.com downloads convey how often a library is used as part of the development of another library

We then split the library we are tracking in 2 categories:

  • Waku Core Projects: libraries that implements the Waku protocol.
  • Waku Platforms: users of the core projects, platform that have adopted Waku.

For Waku Core Projects, we measure the success in a linear manner: 1 star/ means one more developer/platform interesting in/using Waku.

For Waku Platforms, we measure the success in a quadratic manner (square root) as we want to maximize the number of platforms and the size of the platforms using Waku.
We rather have several medium size platform using Waku than one big platform.

1 Like

8 Feb 2022 Update

Graphs: Waku Metrics - Google Drive

Note: go-status and chat sdk will be added as platforms next week.

Week-on-Week Summary

Spike of js-waku npm downloads from 379 to 523. Likely due to:

  • Railgun trying out Waku for their relayer network
  • Hashconnect testing Waku for imminent launch

GitHub Stars Waku Platforms

Quadratic sum of stars on GitHub repos of platforms using Waku.

Comment

Currently 2 projects are published and using Waku. Wallet Connect uses nim-waku, Hash Connect uses js-waku.

GitHub Stars Waku Core Projects

Linear sum of stars on GitHub repos of Waku core libraries and projects.

Comment

Steady growth, no evident un-starring :+1:

NPM Downloads Waku Platforms

Quadratic sum of downloads on npmjs.com of libraries of platforms using Waku.

Comment

Currently 2 projects are published and using Waku. Wallet Connect uses nim-waku, Hash Connect uses js-waku.

NPM Downloads Waku Core Projects

Linear sum of downloads on npmjs.com of Waku core libraries.

Comment

Steady increase since mid-january. Likely due to Railgun trying out Waku for their relayer infrastructure.

21 Feb 2022 Update

Graphs: Waku Metrics - Google Drive

Note: go-status has been added.
chat-sdk does not have downloads at the moment so it’s not added but we are monitoring.

Week-on-Week Summary

Trent is flat.

GitHub Stars Waku Platforms

Quadratic sum of stars on GitHub repos of platforms using Waku.

GitHub Stars Waku Core Projects

Linear sum of stars on GitHub repos of Waku core libraries and projects.

NPM Downloads Waku Platforms

Quadratic sum of downloads on npmjs.com of libraries of platforms using Waku.

NPM Downloads Waku Core Projects

Linear sum of downloads on npmjs.com of Waku core libraries.

I think it’s a good idea to track interest-related metrics and network usage metrics separately. Both show adoption from a different perspective, or rather at a different stage of the adoption funnel. On that note, I would argue that there are a few other metrics that could be useful.

GitHub

  1. Repository Page views - As an indication of the general interest. Page views alone show how popular the repository is.

  2. Repository Stars - As a measure of the “first impression” this repository makes. Developers will star it if it seems interesting enough. It might be interesting to watch Stars to Page views ratio’s change over time. Optimizing this repository (aiming at the mentioned ratio’s increase) will help make it seem more attractive to developers.

Ok, developers may have heard about Waku (Page views) or even liked it (Stars), but that’s just an early stage of the “adoption funnel”. What we could also measure are:

  1. Repository Downloads

  2. Repository Clones

  3. Repository Forks

  4. Sum of above three metrics will give a good understanding of how many people decided to use Waku. Added to (or compared with) npm downloads this could be a neat adoption metrics. Also, this sum (only for GitHub) divided by Page views could become our GitHub Adoption Conversion Rate - which we can use in a myriad of experiments (and optimizations).

Brand and Community

Adoption requires community and brand recognition because:

a) your goal is to get people (= community) to use Waku

but in order to create this community you need

b) people to hear about Waku (= brand recognition) in the first place

People who haven’t heard about Waku will not join Waku’s Community.

That’s why, on the path to adoption, long before any download, there are some useful brand and community related metrics like:

  1. Discourse / Daily Engaged Users
  2. Twitter / Followers and Mentions
  3. Google Searches (for branded keywords)
  4. Website Page views

and so on.

It may seem like a lot, but it is not - especially if you look at your adoption strategy as a marketing funnel - with the conversion volume as your main KPI, and analyze specific metrics for specific stages of the funnel.

For example (not to make this post excessively long):

  1. People need to somehow discover Waku. Either way, they need to find out that there is such a thing as Waku. This is branding - to which you apply brand related metrics.

Once they hear about Waku, they might want to know more about it.

  1. Website and GitHub Page views will reflect rising interest.

Some will find Waku so attractive that they will decide to use it.

  1. That’s what the downloads/forks/clones sums are for.

Yet, the overall success is a sum of small victories along the way (conversion journey). The more metrics you’ve got (or rather the more granular the reporting) the more effective you can be at optimizing this journey (and it’s particular stages) and speeding up the adoption.

Thank you @AK47 for your input. Today, I learned.

I think that indeed, page views makes absolute sense to track interest generated. In terms of platform, I agree that we should have the following metrics:

I am not quite convinced of the value of clone and forks. js-waku is an npm library, the npm downloads metric will give more value at this stage.
Clone and forks are interesting if we want people to contribute to js-waku, which is not the focus at the moment.
Maybe it can be something we track without looking too much into and hopefully at some point it spikes to confirm we have reached a certain level of adoption, I just don’t want to add trackign work with low ROI.

In terms of conversion rate it goes this way, let me know if you disagree:

  1. Campaign (Twitter, hackathon, Talk, business dev effort)
  2. https://wakuconnect.dev/ is the entry point we are focusing on
  3. https://docs.wakuconnect.dev/ interested has been generated, audience is looking at the docs
  4. npm downloads conversion has happened, audience has downloaded the npm package to integrate it in a project

Then, in term of retention, that is where we either:

  • Manually track platform actually integration js-waku
  • Switch to network metrics and measure # of connections, # of different content topics, etc

Yep, I agree on the brand and community related metrics. I think I am getting it and keen to have your help to set it up. I understand it’s related to point (1) above (Campaign).

Excellent!

Understood!

Yes, this makes absolute sense.

I completely agree.

Tracking retention is going to be an interesting challenge.

Do you think it would make sense to keep track of open source projects on platforms like GitHub, looking for js-waku code? I mean, I can imagine such a bot - I would even enjoy creating one.

Nice, I like this one a lot.

Excellent, I will add this to my TODO list :slight_smile:

5 Apr 2022 Update

Graphs: Waku Metrics - Google Drive

Note: Chat SDK is now “Status Web” and is handled by a new Status team. The package has been relocated to the "@status-im` npmjs.com organization.

Summary Update

No major changes since last update.

We are aware of several parties interested in integrating go-waku but haven’t seen an uptake in go-waku gh stars (yet).

Note that as per the discussion above, metrics could be improved.
Some initiatives have started (web analytics for websites).
Other options need to be studied.

GitHub Stars Waku Platforms

Quadratic sum of stars on GitHub repos of platforms using Waku.

GitHub Stars Waku Core Projects

Linear sum of stars on GitHub repos of Waku core libraries and projects.

NPM Downloads Waku Platforms

Quadratic sum of downloads on npmjs.com of libraries of platforms using Waku.

NPM Downloads Waku Core Projects

Linear sum of downloads on npmjs.com of Waku core libraries.