Hello fellow aggregators, and welcome to a celebration of teamwork with data. Rob Mitchum is one such aggregator, and for the past 5 years he has been putting together a spreadsheet per year of all the top 50 AOTY lists released by reputable publications. For this post I did something I don’t normally do, and that’s ask permission to use data, specifically that spreadsheet.

More information can be found on his page, but, essentially, Rob finds all the lists he deems worthy and handscores their ranks into a google sheet. From there, he calculates the average rank for each album (the AVG column), the average rank divided by the number of appearances (WT-AVG column), and a consensus score (CONS SCORE column) which is the average rank of the lists if every non-appearance was given a rank of 75 (an album is thus penalized for every publication ranking it doesn’t appear on). The albums are sorted by that final consensus score and as of January 11, 2018 (which is when I downloaded the data) Kendrick’s Damn. currently stands at number 1.

This very site right here released its own set of end of year album lists so I thought it would be cool to compare our own lists to the individual publications and consensus score rankings. To accomplish this I grabbed the staff top 50 list, the user voted top 50 list, and the rating-based user-usage adjusted top 50 from my year-end rankings post. It wasn’t entirely seamless, I had to change some album titles and band names to fit with discrepancies between the sputnikmusic database (as well as any artifacts from my own scraping of the database) and the AOTY spreadsheet. Another quirk, since there is effectively a loudness war over which publication can release their AOTY list first, I also had to scrape one album from late 2016 (Run the Jewels 3) since it appeared on a few lists for 2017.

(For posterity’s sake, here is the hyperlinked column headers of the AOTY spreadsheet corresponding to each publication’s top 50 album list website link (and the sputnik ones I added): ALTERN, CMPLX, CONSENSUS, CoS, CRACK, DRWND, FACT, GvsB, LINE, MOJO, NME, NOISEY, NPR, PASTE, PFORK, POPMAT, Q, QUIETUS, RSTONE, RTRADE, SKINNY, SPIN, SPUTRATINGS, SPUTUSERS, STAFF, STRGUM, TMT, UNCUT, UPROXX, VNLFCT, WIRE)

So right off the bat, given the consensus scoring, how would the consensus rankings have differed if the sputnik staff list, the user list, and the sputnik user-usage adjusted rankings were included?

Adding the three sputnik based rankings would have shot up Fleet Foxes and Julien Baker by more than 10 ranks, put The National and Mount Eerie into the top 10, and even brought Run the Jewels and Brockhampton into the top 50 (assuming RTJ even counts).

Another question I had regarding this data was how similar were the rankings for each publication to the others. The following is a plot of the matrix of similarities between each of the publications’ lists as measured by cosine similarity (the similarity metric I used for the musical neighbors post).

In this plot, the stronger the blue, the more similar the publications’ lists are. Note that the top left blocks are greyed-out because they are redundant with with comparisons in the bottom half of the plot. The plot is pretty, but also busy, so what are the top 20 most similar lists in a much more palatable format?

The consensus ranking (CONSENSUS) comes up quite a bit in this top 20 which makes sense given that it is aggregated from all the other lists found on the AOTY spreadsheet. The Pitchfork (PFORK) list is the most similar publication to the consensus score, it and Spin Magazine (SPIN) are the most similar publications, unsurprisingly the sputnik user list (SPUTUSERS) is most similar to the sputnik user-usage adjusted rankings (SPUTRATINGS), and (though not represented above) the sputnik staff list (STAFF) is most similar to SPUTUSERS and SPUTRATINGS lists, respectively, followed by The Skinny (SKINNY) and the Consequence of Sound (CoS) lists.

Code for this post, including data, can be found here. I may do another post on this data, and/or the data of previous years. Comment if you have any ideas.