blog: cohort analysis

[sixhobbits]

Addresses #2990 and FusionAuth/fusionauth-issues#2303

• Needs https://github.com/ritza-co/fusionauth-user-charts moved to the fusionauth repo and links updated
• Needs an author
• We could refocus this more on showing how to build the code and endpoints used rather than the usage side if that's preferable.

[nathan-contino]

Lots of feedback on minor style issues. The incorrect chart is the biggest issue, but hopefully you'll incorporate most or all of this feedback.

[nathan-contino]

Let's update this to the current year, at least :)

[nathan-contino]

Let's use a real person. Would you like to use your own name here?

[RichardJECooke]

I've added Dan's name as the author as I'm not comfortable recommending that readers install modules without the safety of containers

[nathan-contino]

Suggested change

	- [Introduction](#introduction)
	- [FusionAuth Native Reports](#fusionauth-native-reports)
	- [An Overview Of Cohort Analysis](#an-overview-of-cohort-analysis)
	- [Understanding Additional User Charts](#understanding-additional-user-charts)
	- [Total Users Charts](#total-users-charts)
	- [User Acquisition Charts](#user-acquisition-charts)
	- [User Age Chart](#user-age-chart)
	- [Logins Per Year Charts](#logins-per-year-charts)
	- [Percent Logins Per Year Charts](#percent-logins-per-year-charts)
	- [Abandonment Charts](#abandonment-charts)
	- [Activity Cohort Chart](#activity-cohort-chart)
	- [Returning Users Chart](#returning-users-chart)
	- [Friction Chart](#friction-chart)
	- [Login Frequency Chart](#login-frequency-chart)
	- [Cohort Analysis Chart](#cohort-analysis-chart)
	- [How To Run The Chart Code For Your FusionAuth Instance](#how-to-run-the-chart-code-for-your-fusionauth-instance)
	- [Extract Customer Data](#extract-customer-data)
	- [Calculate And Display The Charts](#calculate-and-display-the-charts)
	- [Summary](#summary)
	- [Appendix — How To Create Fake Customer Data In FusionAuth](#appendix--how-to-create-fake-customer-data-in-fusionauth)

I don't think a TOC at the top of the page is especially helpful, particularly for blog posts. Can we remove this?

[mooreds]

Agreed. Please remove.

[nathan-contino]

Suggested change

## Introduction

I prefer to omit the Introduction header for docs content. It takes up a lot of vertical space at no benefit to the user.

[nathan-contino]

Suggested change

	If you use an authentication gateway like FusionAuth and want to improve your business's customer acquisition and retention, you need to perform user cohort analysis. Without analyzing your users' behavior, you have no information to guide your product development. Cohort analysis allows you to answer questions like:
	- How frequently do customers use your app?
	- How many people register but quickly lose interest in your service?
	- Did new features or advertising campaigns entice new users or irritate existing users?
	This article demonstrates how to use FusionAuth to track customer statistics: retention rates, user age analyses, and customer cohorts (groups). While you need to investigate your own application database for the reasons *why* customers stay or leave, having a base of FusionAuth login statistics to work from lets you know *whom* to analyze.
	Understanding users' behavior can help guide product development and improve customer retention. This article demonstrates how to use FusionAuth to track customer statistics. Understanding *why* customers stay or leave might require data from your application database. But FusionAuth login statistics can help you understand *who* your customers are, and help you sort them into cohorts (groups). Cohort analysis allows you to answer questions like:
	- How frequently do customers use your app?
	- How many people register but quickly lose interest in your service?
	- Did new features or advertising campaigns entice new users or irritate existing users?

The first paragraph here is a shade too wordy for my liking; I think we can remove all of the auth gateway/fusionauth mentions, as well as the rather unwieldy phrase 'business' customer acquisition and retention' -- after all, cohort analysis is useful for just about any software! Even better, if we rephrase 'analyzing your user's behavior' to 'understanding', we sound more like humans and less like aliens.

Additionally, this section puts the cart before the horse by defining 'cohort analysis' before explaining what a cohort is (or how it relates to FusionAuth). My suggestion above moves the bullet points defining cohort analysis to the bottom of this section, so users get a chance to understand our context about before we start introducing new terms.

[nathan-contino]

Suggested change

	The app reads all data from `users.json` in the `main()` function, calculates the charts, then inserts them into `5page.html`, and returns that page when requested. It calculates each chart in parallel, using the `runParallel()` function in the `getChartData()` function. The chart calculations themselves are just loops through the users and login dates.
	The web server reads all data from `users.json` in the `main()` function, calculates the charts, then inserts them into `5page.html`, and returns that page when requested. Each chart calculates in parallel, using the `runParallel()` function in the `getChartData()` function.

Suggestion: let's not start using the term 'app' here, if we haven't used it before. Slight rephrases to active voice.

[nathan-contino]

Suggested change

	You can also edit `4app.go` and `5page.html` to add any new charts you need. LLMs handle Go and HTML code very well due to the languages' simplicity. If you provide both files to any chatbot, it should instantly be able to add any chart you want.
	If you want even deeper analysis on a regular basis, you need to dive into the world of business intelligence (BI): data extraction, denormalization, and dashboards.

We already told users how to add and edit charts. Let's not repeat ourselves and oversell.

[nathan-contino]

Same feedback as this same copy/pasted snippet above on splitting these commands up, removing 'download and unzip it', and marking the snippets as console.

[nathan-contino]

Same feedback as this same copy/pasted snippet above on the placeholder etc.

[nathan-contino]

Suggested change

	```sh
	```console

[nathan-contino]

Lots of feedback on minor style issues. The incorrect chart is the biggest issue, but hopefully you'll incorporate most or all of this feedback.

[mooreds]

@nathan-contino sorry, I should have mentioned I was reviewing this as well and will have feedback for the author too.

[mooreds]

Suggested change

	title: "Analyzing Why Customers Leave — Cohort Analysis in FusionAuth"
	title: Cohort Analysis

[mooreds]

Agreed. Please remove.

[mooreds]

In the same vein, add another example with an app that has only 1000 users but 95% of them returning. That would be a healthier app, even if it has less users.

[mooreds]

Suggested change

	Notice that this chart is the mirror image of the new users per year chart.
	This chart is a mirror image of the new users per year chart. If a user registered in 2020, then in 2026, they will have been registered for six years.

[mooreds]

Don't mention this unless we show how to do deduplication.

[mooreds]

See above about 999999 not working. In particular, I got this error message when I printed out the body:

body: {"fieldErrors":{"numberOfResults":[{"code":"[invalid]numberOfResults","message":"The [numberOfResults] is invalid because it has exceeded the maximum res
ult window size. The sum of [startRow] and [numberOfResults] must be less than or equal to [10000]."}]},"generalErrors":[]}
httpError! status: 400

[mooreds]

Again, any reason not to assume the reader has golang installed locally?

[mooreds]

as noted above, if the user is only registered to one application, the first loginDate will always match teh registered date.

Also why are these being held as strings? why not uint64?

[mooreds]

Use a .env, please.

[mooreds]

let's assume the user has golang installed locally.

[jamesdanielwhitford]

Hi @nathan-contino. I just ran a QA pass on this one. Some small things I've touched up:

• Fixed absolute URLs.
• Added a header image.

The publish_date in the frontmatter is still set to the draft date. Will need to be updated with the actual publish date when ready to merge.

Let me know if anything needs adjusting.

[nathan-contino]

Pinging @mooreds so we don't step on each other's feet: is this ready for a final copy edit and merge?

[mooreds]

No, there were some issues with the code before so I wanted to run it locally.

[mooreds]

Hi folks,

There's still an issue with pagination and running the golang code on a system with > 10,000 users. (Worked great when I had 8 users.)

Here's the error message I got when I tried to do this:

% go mod tidy && go run extractUsers.go
Extracted 1000 users (1000 total)
Extracted 1000 users (2000 total)
Extracted 1000 users (3000 total)
Extracted 1000 users (4000 total)
Extracted 1000 users (5000 total)
Extracted 1000 users (6000 total)
Extracted 1000 users (7000 total)
Extracted 1000 users (8000 total)
Extracted 1000 users (9000 total)
Extracted 1000 users (10000 total)
httpError: numberOfResults: The [numberOfResults] is invalid because it has exceeded the maximum result window size. The sum of [startRow] and [numberOfResults] must be less than or equal to [10000].

Here's our pagination docs but basically make sure you use the nextResults token.

Please make sure this code works, as that is a prerequisite to delivering the blog post.

[RichardJECooke]

There's still an issue with pagination and running the golang code on a system with > 10,000 users

@mooreds

We could not reproduce your error. I tested FusionAuth versions 1.64.1, 1.66.0, and even 1.47.1 where the 10000 limitation is supposed to occur successfully with 20 000 users. We've tested on Linux and Mac. With and without OpenSearch.

Are you using the latest code from https://github.com/ritza-co/fusionauth-user-charts?

If so, please zip your entire FusionAuth folder, with .env files, kickstart files, docker-compose file, and mail it to us so we can try reproduce the problem locally.

[mooreds]

@RichardJECooke I'm running this against a live server in the cloud. I am using the latest code, and am running 1.66.0. Can't share access to this particular server, but let me stand up a new one and verify I see the same behavior. Will share details in slack.

[mooreds]

@RichardJECooke slack thread with details on the server that I used to replicate the issue: https://inversoft.slack.com/archives/C0363LCJ267/p1778712502330409?thread_ts=1776801470.181799&cid=C0363LCJ267

[RichardJECooke]

@mooreds Fixed, thanks.

[mooreds]

@RichardJECooke thanks! the search retrieves all the users now (took about 13 minutes to process from the fake server on my machine).

However, there's no code that actually writes the charts.html file. It appears to be checked in, and when I removed it and ran go mod tidy && go run charts.go no new chart.html file was created. I looked at the golang file and I'm no expert, but I didn't see anything that wrote to chart.html or wrote any file at all.

This is problematic because we want anyone downloading this repo to be able to build the charts for themselves, based on their instance.

[RichardJECooke]

@mooreds The charts.html file is a template file that charts.go serves and populates with data. If you delete the HTML file the app will fail. The HTML file handles the chart display and the Go file handles the chart data calculations. The data is not hardcoded though, if that's your question. The data comes from users.json after extraction, which will differ for every reader of the guide.

Or if you're asking how readers can add new custom charts, they need to edit the .html and .go file.

[mooreds]

Thanks @RichardJECooke . When I run the command above with the charts.html file as named in the repo, I don't see it change at all. Nor do I see any writing of any data to that file in the golang code. (Again, I'm not a golang expert, so could be missing something.)

[RichardJECooke]

@mooreds Yes, no files changed. The Go file reads the user.json file, calculates the chart statistics (in memory), loads the html file, injects the chart data, and serves the file to the user. If you'd like us to explain more in the guide, please let me now. I was trying to keep it from getting verbose. The guide currently says

The web server reads all data from users.json in the main() function, calculates the charts, then inserts them into charts.html, and returns that page when requested. Each chart calculates in parallel, using the runParallel() function in the getChartData() function.

[mooreds]

Ah, my bad! I wasn't following the guide, just trying to work from the repo. No changes needed at this time, but appreciate the explanation.

[nathan-contino]

@mooreds is that an LGTM i hear?

[mooreds]

nope, not yet @nathan-contino . Still working through my errors on the chart side, then need to review the blog post :) . But progress!