Hey yall - the topic of ‘cash grab rooms’ and ‘low effort rooms’ has come up a lot lately, and it came up a lot yesterday in the Creator Hub Discord while we were chatting about the Maker AI news. This started a discussion internally at Rec Room and Nick (CEO) wrote something up that I pasted here so that creators could read it too.
If you want to discuss the post hop into the Creator Hub Discord and something like the General channel or the Maker AI channel (if its related) would be a good place to discuss this if you have thoughts: Creator Hub
Here’s the post from our Slack:
Nick [12:00 PM]
Let’s talk about Ranking and Discovery
I know a common complaint you’ve been hearing from the community is that “low effort cash grabs” are dominating our ranking. Let me outline what our data is telling us and how we want the discovery system to work. First, let me just share some stats so we’re all on the same page.
- The top 100 rooms generate 50% of all time spent (excluding the rooms we make)
- The top 100 rooms generate 73% of all UGC room revenue
- 85% of the top 100 rooms have more than 24 hours of aggregate creation time
- 74% of the top 100 that have been updated in the last 90 days
So I think we have two takeaways. First, if your room is not in the top 100, it’s probably not getting much attention. Second, the rooms in the top 100 are generally the result of a lot of work. Creators have put a lot of time into constructing them and they’ve done so recently. Ok, so why are people concerned about cashgrabs? Well I believe there are a few things:
-
New Room Promotion - First, there are 10s of thousands of rooms published every week. There is no way for us to manually look at them, which means we need to approach the problem algorithmically. A lot of these rooms get momentary promotion in the watch so we can see how they perform. Still… people see these rooms in their carousels and assume it’s because we “like” the room. When they ignore certain rooms and don’t click on it, we use that signal to demote it and move on to testing other rooms. On the other hand, if people click on the room and spend time there, we promote it to more people.
-
Misconceptions about what’s actually making money - The most common complaint that I hear is that there are “tons of costume rooms in the park and RR lets them persist bc they rake in the money”. Well I looked at how much revenue we’re getting from any room that takes place in the park. It’s less than 1% of our revenue. There are no rooms based on the park in the top 50. These rooms exist for sure, but they are not making us or the creator much money. A certain group of players does seem to like them, but it’s not a group that spends money. Just time. This is not a revenue source for us.
-
Misconceptions about how the ranking algorithm works - There seems to be a belief that our algo heavily weighs monetization. It doesn’t. It primarily is looking for engagement (play minutes). If you have some monetization in your room it will help you vs a room with similar engagement but no monetization. Still… predicted engagement minutes per thumbnail impression is the primary.
-
My room (or one I like) isn’t getting popular - This is probably the biggest factor. 10s of thousands of new rooms launch each week but… only about new 10 - 20 rooms each week break into the top 100. That means 10s of thousands of potentially disappointed creators. And maybe a dozen who are happy with the way ranking works
-
Creators build a room or they see a room that they like, and it’s not getting popular. They assume that it must be because we’re prioritizing “cashgrabs.” Hopefully you can see above… we’re not. I think the issue is that “perceived quality” and engagement / retention are not the same thing. We see a lot of truly beautiful rooms with very little in the way of interaction. They dont generate many play minutes or return visits or their thumbnail doesn’t bring people in, thus they dont rank well.
-
For a room to rank, it needs a great thumbnail to convert people into visitors. It needs strong engagement once you’re in the room. It needs to be appealing to a wide audience (niche content starts to see its per user metrics degrade with more promotion). If the room has been materially updated recently or newly published - that helps. If the room effectively monetizes – that also helps. Still, it’s really about appeal and engagement of a broad audience.
-
What do we want the top 100 rooms to look like? We want the players to pick the winners, and so our rankings are primarily based on how well the Rooms engage the community. Here are our principles:
-
We want it to be fresh content - that means it’s new or it’s being actively worked on. We don’t want stale content stuck in the top 100. This is the biggest thing we’re pushing on. A few months ago, it was a lot harder to break into the top 100. We want to give new rooms a better shot at success.
-
We want it to be engaging - This is an empirical measurement not a subjective quality assessment. If we put it in a ranking surface area, do people spend time there? Note: click through rate on the thumbnail image is a factor here. If no one clicks your thumbnail, that’s bad. If people are misled by your thumbnail and quickly leave after arriving, that will also penalize you. Players need to click through and stay for us to reward the room with more traffic.
-
All things being equal, we want it to be monetizing - Engagement is the primary ranking metric, but if two rooms are about equal, the one with better monetization will eclipse a room with no monetization. At the end of the day, we’re a business. Everything we do has a real cost - from server infra to moderation. We need to select the content that is going to help us pay those bills.
So what should we do to help the community understand what we’re doing?
-
First we need to be better explaining to the community what’s actually happening. I think a lot of people hear the complaints from the community and either don’t know what’s actually happening or dont know what they’re allowed to say. I’m going to try and fix that.
-
Second, I think it would be good if someone who primarily focuses on Creator Tools & Outcomes committed to being a strong stakeholder in the discovery workstream. We are failing creators by leaving a vacuum of information out there. They are not succeeding. They dont know why. They’re generating their own theories for why. Lets fix this. Discovery should be able to clearly articulate for all creators: “this metric is why your room isn’t ranking”. People in the UGC org should be able to say “and here’s what you should do about it”.
