I Analyzed The Data: Is FM17 the best FM yet?

June is usually the time of year when you start to see a lot of changes.

The weather is nicer, there is no more football on tv and you spend less time playing FM.

And it’s the time of year, where I see more photos from the beach than FM screenshots in my Twitter feed.

More specifically, it’s usually that way but this year it’s different.

So why this change?

My own theory is simple: Football Manager 2017 is one of the the best and most polished games in the series for a long time.

You might be saying:

You’re right, Steve, but how do you really figure out if that’s true?

As I’m a geek that loves numbers, I’ve decided to look at data to prove (or disprove) my theory.

And in today’s post, I’m going to show you that despite worse reviews and fewer players at its peak, FM17 is based on the data the best in the series for quite a while.

1

Reviews for FM 2017 were good, but with lower overall scores than other games in the series

The first and obvious place to start in terms of answering the question “is FM 17 the best in the series” is to look at the reviews.

For this, I’ve gathered data from two review aggregator sites, MetaCritic and GameRankings.com. The data is presented in the graph below.

(you might be saying reviews don't matter, but stay with me...)​

The first thing to note, is that reviewers agree FM 2013 was the best and since then, the relative quality of the game has declined.

FM 2015 is deemed to be the worst in the series, while FM 2017 is the second worst reviewed.

However, there are quite a couple of issues using the review scores as an indicator:

  • Most reviewers will only have played the game for 20 hours. This is next to nothing in FM terms. For me, all FM’s have really started to show their worth after putting more than 100 hours into the game.
  • The game is reviewed pre-patches. Usually, these patches correct all smaller errors and have a huge long-term gameplay impact

What's the bottom line?

​FM2017 was not hyped in reviews.

​While reviews are a good indicator of the quality of the game at release, they are likely not a good indicator of the long-term playability of the game.

As such, reviews of FM are poor at predicting how good the game is long term.​

2

FM 2017 has the lowest number of peak players in the series

On Steam, peak players is defined as “the highest number of players, playing a game at the same time”.

As peak plays will usually happen pretty close to release of a game, it is a good way to measure the hype of a game.

A famous example of this is the much hyped, less loved game, No Man’s Sky.

As can be seen from the graph, it was extremely hyped upon release only to have almost all players disappear one week after release.

So, here's the deal:​

Peak players tells how hyped a game is.​

For FM 2017, the “hype” data is quite clear: FM17 is the instalment with the least number of maximum players, which indicates less hype around the game at release as evidenced by the graph below.

72,608 players at the peak vs. 87,420 players at the peak for FM 2013, or a decrease of 17% is quite a drop. Compared to FM 2016, the drop is 12%.

The reason for this?​

This goes well in line with favouring, if unflattering reviews. Here is an excerpt from The Guardian giving it 3/5 stars, while still calling it “Football Manager 2017 review – the best in the series, but only for dedicated fans”.

The Guardian

“Over the past four years, Football Manager editions have shaken up the way players create tactics, introduce in game manager stats or provide new game modes to play. FM 2017 offers none of that. As a result, it is difficult to recommend beyond the dedicated fanbase. For those who love the series and have dedicated hundreds of hours to it, purchasing the game is an unavoidable ritual. It’s more of what you want, with a lick of paint and up-to-date player stats.”

Here is another example from Gamespot’s review:

Gamespot

“Sure, I still have gripes with the tactical interface, and there isn’t anything new there worth writing about. But if your rear end has ever been entrenched in the virtual dugout or you're just a fresh-faced hopeful looking to begin your journey, Football Manager 2017 is easy to recommend to the budding manager.”

While both reviews are praising the game, they are not exactly giving people who are in doubt about buying a reason to push anyone out of their way as they rush to get the game.

From the World of FM bloggers here is also quite an interesting take on what FM17 offered from FM_Samo.

FM_Samo

I think I'm always inclined to be biased towards the most recent FM because it's the one that sticks out most in my memory. There's two ways I like to look at it though, the actual game versus the saves you do on the game.

In terms of an actual game, I think FM17 is probably the most polished version we've ever had. I've hugely enjoyed playing it, and I think there's been some great tweaks and ironing out of previous issues that've just added to the overall quality factor of the game.

That's not to say it's still the ideal game though! In terms of my personal enjoyment, that always comes down to the saves I have on the game, and FM17 is up there competing with my favourite saves ever! I've loved taking on managing a club that's so hated in their country and one that has been such a popular team for one reason or another across the 2016/17 football season, RB Leipzig.

I think it was maybe DoctorBenjy that said FM17 feels like FM16.5 and I'd agree with him there. It's FM16's more polished older cousin.

Follow @FM_Samo on Twitter or visit his website here.

What's the bottom line?

The hype around FM17 was relatively low as there were no new significant features.

As a result, fewer than usual went out and bought the game, which resulted in the lowest amount of peak players on Steam since FM2013.

FM17 is a very polished version of FM, but in terms of features, it's FM16.5.

3

FM13 to FM16 all peaked in January with a steady decline in players afterwards

We have already touched upon the concept of peak players on Steam.

Another data point is the number of average players a game has. This is defined as the average number of players playing the game throughout the day.

For example, if 10 players played the game at noon and 30 players played the game at midnight, the average would be 20. Steam of course gathers the data at many more intervals, but that is the basic concept.

Where am I going with this?

The reason for looking at this data, is that it tells you how good a game is long-term. Simplistically you could say that peak players tell something about the hype of a game, while average players over a longer period tells you how popular the game is long term.

To look at time series of data you should always have a comparable time period to benchmark against.

Thus, we are first going to look at the data for FM16 back to FM13 (the oldest game with full data I could find).

We start by looking at FM16 as that is the most recent game and a good starting point. Here the data is quite clear: January had the highest number of peak players with a ~10% decrease in players come June.

The data for FM15 is almost a mirror image of FM16, although with slightly less of a drop come June.

The data for FM14 is far more erratic. Average number of players in January is 4k lower compared to its two younger brothers and the same goes for the number of players in June.

FM13 is the game with the by far most pronounced trend graph. For the first four months, the number of players was a mirror image of FM15 and FM16, but after that, it dropped significantly. By June, almost 1/3 of the players were gone.

What's the bottom line?

FM14 to FM16 had very similar trend curves and had lost ~10% of its average log-ins by June.

FM13 was a mirror image of the other FM’s in the first four months, but had lost 1/3 of its average players by June.

​There is usually a slight increase in players in June, most likely due to the Steam Summer Sale.

4

FM17 is the only Football Manager game in the series to increase the number of people to play it month-by-month and peak in June!

We now have a pretty solid data foundation from FM13-FM16 with a very clear trend: People buy the game at release and some get it over Christmas (great tradition!) after which they play the heck out of it in the dark & cold Winter month of January.

Following that, most continue to play the game, however, less often than they used to.

We will now dive into the data for FM17 and so if there are any differences

Hint: There are!

As you can see from the graph above, the first four months was business as usual.

People acquired the game in November/December and by January, the number of players topped with almost 38k on average per day.

This by the way, is ~5k fewer than was the case for FM16 in January.

As usual, the number of players declined in February and March.

But here’s the kicker:

After the patch in March, average number of players started increasing without looking back and June is by far the month with the largest number of players.

In fact, with ~41,965 average players, it gets pretty close to the normal January level for other editions of FM.

Also, it’s the first time FM has had its peak number of players in June!

The exact time when it happened was at 5 PM on Sunday the 11th of June. At this time, it was a lovely day in London, 21 degrees Celsius and a partly cloudy sky.

Two of the only guys who didn’t play FM that day

To me, this data is quite clear:

FM17 has an incredible longevity and only gets better with time. At this time of the year, previous FM editions had lost ~10% of their players, while FM17 has gained ~10% more players – that’s a 20% swing!

FM17 might be the one with the highest stamina, but that doesn't mean it's the best FM.

Data is great and I love data. However, the feelings you have when you play a game are not necessarily mirrored in a game.

I absolutely love ​FM17 and have felt fully immersed in it, but this is partly due to an amazing save, I've had on the game - and it's a far better save than I've had in FM16.

This ​was something that both Keysi Rensie and John McIntosh covered when I asked for their oppinion.

Keysi
Rensie

If I would have an option I would go back to FM 2011 because I’m sure I loved this version very much.

But… Personally, FM17 is in my good game but not in my TOP 3 version from FM 2012.

I liked FM16 because of my long-term save in Denmark.

Maybe I’m too much nostalgic but the previous versions were better in my eyes.

Follow @Keysirensie on Twitter or visit his website here.

John McIntosh

In terms of my own enjoyment of the game since 2012 anyway, this years version of the game probably comes a second to FM 2014 but it is the most realistic. There is much more focus on tactics on this series and you can implement a lot of different styles but there are still issues with the match engine and how certain roles act that needs some work.

Keepers still make some ridiculous errors and the Inverted Wing back role is not quite right yet although there has been some work on that, I still feel that crossing is overpowered and there aren't quite enough variations of goals although it has clearly been worked on.

The game just runs a lot smoother on this version, coaching reports are a lot easier on the eye with more information and the team report tab is interesting. I feel the analysis with prozone of the match is a lot easier to understand as well.

In general I feel Sports Interactive have done well in terms of just fine tuning the game, whilst this series was a lot more about simply making what they had already a bit simpler, faster and easier on the eye. Of course it is enjoyable, addictive, infuriating and creates special moments.

Follow @JohnMcIntosh on Twitter here. 

What's the bottom line?

FM17 is the only FM edition to gain players as time goes on.

The longevity of FM17 is unprecedented in recent years.

Compared to the norm from previous editions, 20% more people play the game now than expected.

However, like with all FM versions, you need to have a save you get fully immersed in (which can be hard) to fully enjoy it!​

If FM17 was a player in FM17, it would have 20 in Stamina! 

Click to Share on Twitter
5

The game is finely balanced, and SI should focus on a more realistic transfer market

So far, we’ve established that FM17 is a very polished game.

While this is all well and good, we all think there are some improvements that can be made in FM18.

To figure out what people want from FM18 I combined my own experiences with a good ol’ Google Search to figure out, what people were most frustrated with. I then ran a Twitter poll spending €20 boosting it to get as many answers as possible.

Before we get to that tough, I’ll run through the 4 most commonly cited areas of improvement for FM18:

The match engine: This is a common item on the list and in my opinion, something SI should improve on every year. Basically, at this time each year the higher-level players have figured out how to beat the match engine and you can find “fit all” tactics on various forums and sites. While people will always figure out how to exploit a computer system, SI has done good work making this harder and harder every year – and we expect them to continue to do so.

Player interactions: While the match engine is a finely tuned engine, general sentiment is that the same cannot be said for player interactions with the most common point of critique being erratic player reactions. While it is okay for players to have a negative reaction, it is hard to figure out why they have said reaction and more information on this would be very welcome.

Dynamic Player Potential: When a game starts and regens are brought into the game, they have a fixed potential. Even without the use of editors, it is relatively easy to figure out what this potential is with the use of scouting. In long games, it becomes relatively easy to hoard talent. With the addition of dynamic player potential instead, this would be significantly harder to do. After all, we have seen plenty of examples of highly talented players not becoming world beaters in real life.

More Realistic Transfers: General sentiment in the community seems to be that it’s too hard to sell players you want to get rid of and that the AI makes unrealistically low bids for your good players. While this will require an improved AI, it hardly seems like the hardest problem in the World to fix.

So, what do the FM players on Twitter want from FM18?

I ran the Twitter poll for 24 hours. Thanks to a couple of retweets and the afore mentioned advertising of the post, 399 answers were given to the poll.

This is quite a significant number and as a reference, many public opinion polls have data from 1,000 voters, so the result of the poll should be quite robust.

What's the bottom line?

The most requested improvement in FM18 is to the transfer system. Given the amount of screenshots I have seen with frustratingly low bids from the AI, I was not surprised with this result.

The game is finaly balanced as is, which can be seen in the narrow result from the poll. 3 of the 4 options are improvements / refinements to the game / AI, while only dynamic player potential would be a new feature in the game.

Here you are, SI. Twitter users want more realistic transfers in #fm18

Click to Share on Twitter
6

FM bloggers are looking for an overhaul of training and set pieces while the new features need some polish in FM18

The poll used for point 5 only had 4 options (due to Twitter limits) and I asked people to reply me, if they wanted something else from the game.

The first to do so was BoodFM (Twitter, YouTube).​

This was not a point I had thought about, but I completely agree with him. This was also something that was highlighted by FM_Samo:

FM_Samo

“The big ones for me now are training and set pieces. Both need to be looked at in much greater detail for the full fat game experience.

I imagine set pieces will be more difficult for them to look into getting right, but there must be more they can do with training. It's so basic right now.

If FM18 came out tomorrow and was exactly the same as FM17 in every way apart from with new training and new set piece options, SI can take my £30.”

Follow @FM_Samo on Twitter or visit his website here.

Training is also highlighted by Keysi Rensie, who first and foremost would love to see changes to the set piece creator.

Keysi
Rensie

When I had an option to see FM17 for the first time I said to myself something like „Oh, that’s FM16, where is something new?“.

My probably favourite new thing is „Show linked events“ in Analysis so you can see the passes which led to the goal. There are surely some small new things I like as well but that’s not so important.

In terms of FM18, I would really like to see improvements around Set Piece Creator for example. Like I said many times before we are still very limited by options in the creator – exact player roles, no freedom during creation etc. I know it’s not easy to make it perfect in all ways but I would welcome more freedom in some areas.

We can also mention improvements in training, match engine and many many things like, for example, custom screen views.

Follow @Keysirensie on Twitter or visit his website here.

John McIntosh

I'd also say that more effort needs put into certain ideas like the Social Media tab, I actually think it is a decent idea just executed terribly. You get one positive, one negative and one on the fence response to most events which isn't good enough.

The new roles of Sport scientists and data analysts are pointless as the attributes count for nothing. Also I know it's so small and petty but can we get shirts on the tactic screen over squares?

Finally, I'd like to see SI give more attention to the great bloggers out there. They have given websites such as Higher Tempo Press and Keysi Rensie promotion, but their lack of promotion for Merry Guido for the FBA awards was terrible.

Follow @JohnMcIntosh on Twitter here. 

What's the bottom line?

There is high demand for an overhaul of the training and tactics system...

… while the new features from FM17 can work - they just need some polish to get there.

Here's a summary of what I've discovered

1. FM 2017 is the second worst reviewed game since 2013, only “outdone” by FM 2015

2. The hype of FM17 has likely been the lowest in many years

3. FM13 to FM16 all peaked in January with a steady decline in players afterwards…

4. FM17 on the other hand, is the only Football Manager game in the series to increase the number of people to play it month-by-month and peak in June!

5. To improve on FM17, SI Games should focus on polishing the current game and overhaul training.

6. FM bloggers are looking for an overhaul of training and set pieces while the new features need some polish in FM18.

#FM17 is a very polished game and SI should focus on overhauling training and set pieces for FM18.

Click to Share on Twitter

Thank you!

A big thank you goes out to all who voted in the Twitter poll and a special thanks goes out to Keysi Rensie, FM_Samo, John McIntosh and BoodFM.

What do you think?​

Leave a comment to let me know if you agree or think I'm out in the deep end?

Or maybe there's an aspect that you think I've missed?

Either way, leave a quick comment below right now.

Sources

Steam data from ​Steam Charts, Steam Spy and SteamDB.

Click here to add a comment

Leave a comment:


Set Yourself Up For Succes With Our 50 Page Free eBook

x