Analysis of user retention
on sites, forums and studies
User retention is a key indicator of the health of any digital product, including platforms with AI characters. Unlike "vanilla" applications, the interaction here is based on emotional communication, role-playing scenarios and individual settings. In this article, we examine which data sources (open forums, telemetry of sites, UX studies) allow an objective assessment of retention, and avoid unconfirmed allegations of "most popular scenarios" — only verifiable approaches and a publicly available methodology.
♪ Quantity retention metrics
- Retention Day 1 / 7 / 30is the percentage of users returning on the 1st, 7th and 30th day after registration.
- DAU / MAU- The ratio of daily active users to monthly users (density of interaction).
- Stick business= DAU/MAU × 100 per cent. The value >20 per cent is considered high for niche services.
- "Churn Rate"- The percentage of users who stopped working during the period.
- User sessions— frequency of return during the day/week.
- The depth of the dialogue- number of messages per session.
- Involving in casteizationThe creation of personalized characters or scenarios.
- Time to return- the interval between the last session and the current session.
Data sources: sites, forums and research
♪ Area telemetry ♪
Internal analytical systems (Amplitude, Mixpanel, Yandex.Metrick) record objective actions: clicks, length of sessions, follow-up visits.
* Example of default values to illustrate typical distribution (real figures depend on niche and mechanic).
Forums and communities
Reddit, Telegram channels, Discord servers and specialized forums (including on ai-char.ru) provide qualitative information: complaints about "uniformity", requests for new functions, discussion of favorite types of characters. Analysis methods:
- Thematic modelling (LDA) to identify frequent topics.
- An analysis of the tone of feedback.
- Counting of references to specific behavioral triggers (e.g. "returned due to renewal").
Research data and public reports
Several open studies (e.g.== sync, corrected by elderman == @elder_man, reports of AI-platform Charterer.AI, Replika, as well as work on gamification of chat-bots, highlight the general pattern of retention:
- Personalization– Users who create their own character show retension two to three times higher than those who only use defaults.
- Emotional communication- Long dialogues with elements of empathy and context memory increase LTV.
- Social mechanics- The ability to share dialogues or characters with the community reduces outflows.
- Content Updates– Regular supplements (new voices, scripts, visual holdings) correlate with retention peaks.
At the same time, specific "most popular character scenarios" (romantics, adventures, learning, etc.) vary widely depending on the audience of the site, the region, and even the time of the year.
:: How to analyse forums to search for retention drivers
Qualitative analysis steps
- 3 to 6 months of communications (themes, posts, comments).
- Cleaning and lemmatization (stop-word removal, normalization).
- Selecting N with the most frequent topics using LDA or BERTopic.
- Manual validation: Relationship of topics to life-cycle stages (onboarding, addictiveness, outflow).
- Searching for "ballpoints": repeated complaints about technical constraints or lack of progress.
Example of practice (aggregated data)
In one of the thematic forums (conditional data), following the analysis of 1,200 presentations, the following retention-related clusters were identified:
* The shares reflect the frequency of references in the shell rather than the absolute popularity of the scenarios.
:: Holding-up tactics (based on research)
- Interactive Tutorial with preference choice (communication themes, tone of answers).
- The first message from the character with a hint of a sequel ("Tomorrow I'll tell you the continuation of the story ...").
- Gaemimified achievements for daily visits.
- Push notifications with relevant context (e.g. "Your character remembered your conversation yesterday").
- Weekly people (write 50 messages, create a new character).
- The episode system is a serealized scenario with a cliffhanger.
Conclusions
Analysis of user retention for platforms with AI characters requires a combination of quantitative metrics (retension, DAU/MAU) and qualitative data with forums and communities. Forum analysis helps identify hidden barriers but does not accurately rank popular scenarios without triangulation with behavioural logs.
Main principles:Personalization, emotional communication, regular content and feedback from the community. To get reliable conclusions about which role-playing scenarios (romantics, adventures, fantasy, psychological support) hold users best, it is necessary to conduct A/B-tests and analyse their own cogorts. To rely on "intermediated" data from the Internet without a source and sample is risky.
Review of methods and research used
- Chen, Y. et al. \Measuring User Retention in International Agents, CHI 2022.
- The Reddit Forum (r/CharacterAI, r/replica) is a thematic analysis of 2023-2024.
- Retention Analysis from Amplitude and Mixpanel (documentation).
- "The practice of analysing user feedback in games forums" by A. Kozlov, 2023.
* All figures and diagrams in the article are illustrative, based on typical patterns described in these sources, and are not the exact data of any particular site.