Thinking of removing other website ratings from TARDIS Guide

Hey everyone,

When TARDIS Guide first started out (almost 2 years ago!) it was just me, nobody knew about the site, and I wanted to get a feeling for what were the most highly rated stories, so I scraped ratings from other review sites and incorporated them.

I currently get ratings, where available, from:

  • Goodreads
  • Trakt.tv
  • Letterboxd
  • The Time Scales

And now even though on many stories the ratings from TARDIS Guide outnumber the ratings from other sites, we still include them and they are all mixed together.

This has the following benefits:

  • More ratings generally means a better consensus
  • Some stories have no ratings on TARDIS Guide, so the other sites fill the gaps
  • Big numbers make my brain go brrrr

But it also has the following downsides:

  • I’m sure some of my members also rate stories on those other sites, so their opinion gets unfairly boosted by being included more than once
  • Some stories are have vastly more ratings than others. For example The Day of the Doctor gets over 28,000 ratings on Letterboxd and 1,300 on Trakt, so the consensus from TARDIS Guide users is drowned out.
  • Those other sites are a more general audience, so when their stats drown us out they actually hide the consensus - TDotD is higher rated on average by us TARDIS Guide members than by the majority of Letterboxd users, but the score is lowered because of the mix.
  • It feels icky to use The Time Scales, but they are a big source of audio ratings
  • This all uses website resources - with over 5000 stories now the site is constantly checking other sites for ratings.
  • I am using valid API credentials where possible, but on sites that don’t have them I am having to scrape, which they might not approve of.

So I am thinking of removing the ratings from the other sites completely. I will still include links, which can be visited to see the ratings.

OR a hybrid approach - show TARDIS Guide ratings on stories that reach a threshold enough to make an opinion, and continue to scrape for stories that do not have enough votes.

What do you think I should do? Thanks!

18 Likes

Personally I like being able to compare the ratings on TARDIS Guide to other sites and seeing which stories do better where. Maybe just keep them as links and percentages on the page of each story, without including them into the score? Not sure if that saves any resources, though.

17 Likes

I think it’s a good idea to remove them after a certain threshold is met. Not sure what that one would be though, my first thought was 100 but that could be a bit high? Something like 50 or even 20 might work better

13 Likes

Is it possible/not too difficult to allow users to choose which ratings are shown? Maybe just TARDIS guide as a default, but then you can check a box on an individual story or on the search filters to allow you to search by/see all the ratings put together? Because it seems like there’s times when both would be useful - A lot of times, people will want to see specifically which stories are popular with this userbase, but other times people might want to see what stories are most popular within the overall fandom.

10 Likes

I think it’d be interesting to see both numbers, showing us what our ratings look like next to the combined rating.

9 Likes

As someone responsible for making that worse in several ways, oops…

11 Likes

I’m for removing them but then ratings have never really interested me that much (except my own which are obviously the baseline everybody should judge their own wrong opinions by :wink: )

15 Likes

It seems like people like being able to see & compare the ratings but not necessarily have other sites control the overall ratings.

My thoughts so far:

  • I will make only TARDIS Guide members’ ratings count where I have a minimum number of votes. Maybe 20?
  • I’ll still show other ratings for comparison
  • I will keep the ratings up to date but perhaps only scrape every 3 months something that is a year old, every month for everything a month old, and every week for something between a month and a week, and every day for something less than a week old. What do you think? That should help reduce the resources. I’ll add a little note that says when it was last checked.

Does that sound like a good compromise?

23 Likes

Sounds good to me. Definitely think TARDIS Guide ratings should only show once you hit a certain amount.

10 Likes

Seems fair. Classic Who ratings aren’t going to change as often as new Who, and definitely as much as modern on other sites, so having how often it scrapes ratings be different for different episodes makes sense.

I’d also make sure it shows how many ratings are in there for Guide members vs. the overall ones, or at least the percent of the combined rating that’s TARDIS members.

11 Likes

Oh, also, it’d be a good idea to scrape ratings for things featured in one of the clubs when they are posted, so the numbers are up to date when everyone’s looking at them…

8 Likes

This is something I’ve talked about a little in other parts of the site.

Having our ratings take over when there are enough sounds like a great idea. 20 seems like a sweet spot. Any less and it’s too small, and larger could be a while before we get there.

Not abandoning the other sites ratings to help seed our own is good. While we have a decent size community, there are still stories that we only have a couple of ratings on. I’m currently going through the BBV stories and there are some with only 1, 2 or 3 ratings overall.

Being able to still compare our ratings to other communities on the ‘Reviews’ section seems fun.

We are a specialist ratings site, so I don’t mind our ‘elitism/entrenched’ ratings having more weight than general audience ratings.

I’d still like a profile switch to turn off all offsite ratings completely, but I think I’m in the minority.

12 Likes

I’m in that minority too :slight_smile:

10 Likes

It gets pretty messy to show all the different offsite stuff. I would just remove it. Tried to make a mock-up of it.

8 Likes

Yeah a bit of information overload :exploding_head:

We will declare our independence from other review sites. It’s TARDIS Grexit :grimacing:

19 Likes

TARDIS jobs for the TARDIS’. Pump money back into the…

Sorry. Threw up in my mouth a little.

7 Likes

It is often easy to say that you want all the information but it is often unnecessary and hard to use.

7 Likes

I will be surfacing information about how many favourites a story gets. That’s fun information. But it just isn’t so important that it needs to go there. Will probably be something you scroll down to.

My designer is working on this page now.

I want to have everything visible as you scroll, and clicking to load in more info.

So excited for a redesign!

15 Likes

Might have added that because I wanted it to be added :joy:

9 Likes

I do love a good redesign :star_struck:

8 Likes