When TARDIS Guide first started out (almost 2 years ago!) it was just me, nobody knew about the site, and I wanted to get a feeling for what were the most highly rated stories, so I scraped ratings from other review sites and incorporated them.
And now even though on many stories the ratings from TARDIS Guide outnumber the ratings from other sites, we still include them and they are all mixed together.
This has the following benefits:
More ratings generally means a better consensus
Some stories have no ratings on TARDIS Guide, so the other sites fill the gaps
Big numbers make my brain go brrrr
But it also has the following downsides:
I’m sure some of my members also rate stories on those other sites, so their opinion gets unfairly boosted by being included more than once
Some stories are have vastly more ratings than others. For example The Day of the Doctor gets over 28,000 ratings on Letterboxd and 1,300 on Trakt, so the consensus from TARDIS Guide users is drowned out.
Those other sites are a more general audience, so when their stats drown us out they actually hide the consensus - TDotD is higher rated on average by us TARDIS Guide members than by the majority of Letterboxd users, but the score is lowered because of the mix.
It feels icky to use The Time Scales, but they are a big source of audio ratings
This all uses website resources - with over 5000 stories now the site is constantly checking other sites for ratings.
I am using valid API credentials where possible, but on sites that don’t have them I am having to scrape, which they might not approve of.
So I am thinking of removing the ratings from the other sites completely. I will still include links, which can be visited to see the ratings.
OR a hybrid approach - show TARDIS Guide ratings on stories that reach a threshold enough to make an opinion, and continue to scrape for stories that do not have enough votes.
Personally I like being able to compare the ratings on TARDIS Guide to other sites and seeing which stories do better where. Maybe just keep them as links and percentages on the page of each story, without including them into the score? Not sure if that saves any resources, though.
I think it’s a good idea to remove them after a certain threshold is met. Not sure what that one would be though, my first thought was 100 but that could be a bit high? Something like 50 or even 20 might work better
Is it possible/not too difficult to allow users to choose which ratings are shown? Maybe just TARDIS guide as a default, but then you can check a box on an individual story or on the search filters to allow you to search by/see all the ratings put together? Because it seems like there’s times when both would be useful - A lot of times, people will want to see specifically which stories are popular with this userbase, but other times people might want to see what stories are most popular within the overall fandom.
I’m for removing them but then ratings have never really interested me that much (except my own which are obviously the baseline everybody should judge their own wrong opinions by )
It seems like people like being able to see & compare the ratings but not necessarily have other sites control the overall ratings.
My thoughts so far:
I will make only TARDIS Guide members’ ratings count where I have a minimum number of votes. Maybe 20?
I’ll still show other ratings for comparison
I will keep the ratings up to date but perhaps only scrape every 3 months something that is a year old, every month for everything a month old, and every week for something between a month and a week, and every day for something less than a week old. What do you think? That should help reduce the resources. I’ll add a little note that says when it was last checked.
Seems fair. Classic Who ratings aren’t going to change as often as new Who, and definitely as much as modern on other sites, so having how often it scrapes ratings be different for different episodes makes sense.
I’d also make sure it shows how many ratings are in there for Guide members vs. the overall ones, or at least the percent of the combined rating that’s TARDIS members.
Oh, also, it’d be a good idea to scrape ratings for things featured in one of the clubs when they are posted, so the numbers are up to date when everyone’s looking at them…
This is something I’ve talked about a little in other parts of the site.
Having our ratings take over when there are enough sounds like a great idea. 20 seems like a sweet spot. Any less and it’s too small, and larger could be a while before we get there.
Not abandoning the other sites ratings to help seed our own is good. While we have a decent size community, there are still stories that we only have a couple of ratings on. I’m currently going through the BBV stories and there are some with only 1, 2 or 3 ratings overall.
Being able to still compare our ratings to other communities on the ‘Reviews’ section seems fun.
We are a specialist ratings site, so I don’t mind our ‘elitism/entrenched’ ratings having more weight than general audience ratings.
I’d still like a profile switch to turn off all offsite ratings completely, but I think I’m in the minority.
I will be surfacing information about how many favourites a story gets. That’s fun information. But it just isn’t so important that it needs to go there. Will probably be something you scroll down to.
My designer is working on this page now.
I want to have everything visible as you scroll, and clicking to load in more info.
)
