Monday, November 29, 2010

A Two Year Analysis: What Have Gamers’ Been Tweeting About?

While many corpuses have been analyzed to create frequency tables of words for use in lexical and content analyses, there has been little done in the realm of user generated content (UGC) due to the significant variation in prose. However, to create more accurate processes to determine contextual sentiment in UGC, we believe that one must spend the time in understanding and creating a UGC corpus. Moreover, to apply accurate analyses to UGC that is limited in content length, such as found among Twitter users, one must begin with the Twitter lexicon.

Having collected more than 30 million Twitter statuses related to the video gaming market, we decided to analyze a segment representing nearly two-thirds of our corpus. Namely, those tweets dealing with game titles as their primary topic.

Since UGC, in general, is characterized as having enormous lexical variation and micro bloggers’ are communicating in 140 character bursts with a proclivity to attach URLs and multiple hash tags, we analyzed several thousand individual statuses before proceeding with any data cleansing.

The first step was to allow for the use of single- and double-quotes in the escaped raw data, which we found were used quite frequently by our population. This effected 3.4 million of the statuses.

We ran several processes that targeted specific norms found in our user base by extracting all hash tags, at symbols (“@”), and urls. This allowed us to segregate the conversational content from the normal clutter while providing valuable insight into how much of this behavior is utilized in the given population.

We then created a word candidate frequency hash set, applying several filters to further clean the data. This allowed us to eliminate many lengthy word combinations, as we found most of these were of little contextual value. These process steps reduced our working dataset form 18 GB to 608 MB.

Having created a raw frequency dataset of 4.8 million word candidates representing, more than 259 million occurrences, we then removed all remaining non-alphanumeric characters, resulting in many duplicate words being exposed, as they may have been surrounded by any number of non-alphabetic characters. Upon inspection of the data, and running numerous elimination samples, we also decided to eliminate all numeric data at this time, as we found their continued inclusion not statistically meaningful. This resulted in reducing our word candidate frequency data to around 1.5 million.

We then manually inspected and processed the ~3600 candidates that had a frequency greater than 3000, combining like words, removing nonsensical strings (eg: “abababa”), and combining obvious slang to non-slang equivalents (eg. “willin” with “willing”). These combinations were only done for a handful of obvious words which typically had ratios of proper spelling to slang in excess of 4:1. This process was completed in three steps from f > 12,500, 5,500 < f < =12,500, and 3,000 < f < =5,500.

These manually processed datasets represented more than 110 million of the 136 million occurrences of our word candidates. As expected, the remaining 26 million occurrences resided in more than 2.1 million remaining word candidates.

All of the manually pre-processed frequency candidates were then combined forming a unique word set with a bit more than 80% of the total being represented by these ~3600 words. A final process that accumulated all of the remaining word candidate frequencies into their respective unique words yielded our final word count of 73,006.

We now have a very specific word frequency of our corpus for use in our sentiment analysis. We were very pleased to find that our initial run against a common adjectives dataset yielded a 94.6% hit rate, showing that our user base is more verbose, than not.

Partial Word Frequency Table

Some Simple Validation of Expected Values
In looking at the partial word frequency table above, we can walk through some examples that you would expect that data to support.

As all gamers know, Microsoft’s Halo franchise was and is a big hit. So, the word “halo” shows up 1,570,066 times. Well, is everyone talking about the original title still, or are they discussing Halo ODST, or Halo Reach? If we search our table for both the words “reach” and “odst”, we find 851,967 and 519,133 occurrences, respectively. Therefore, it is pretty safe to conclude that nearly 1.4 million of the 1.57 million times “halo” was mentioned (87.3%) they were talking about one or the other. In addition, it would appear that Halo reach was significantly more popular than Halo ODST.

Well, we did it, and finally got around to publishing it here. We think it's pretty cool. Have fun in drawing your own conclusions.

Game on, my fellow gamesters!

Thursday, November 25, 2010

Gran Turismo 5 to Challenge Halo: Reach?

With the much anticipated release by Sony Computer Entertainment of Polyphony Digital’s Gran Turismo 5 finally here, we are seeing some very interesting trends in your collective chatter.

We haven’t seen too many folks wanting to speculate on this title’s sales potential, so we figured we’d throw down the challenge. It appears that Gran Turismo 5 may give Microsoft’s Halo: Reach a run for its’ money this year.

Assuming that the reported numbers for Halo Reach are correct, more than 3 million units sold in 24 hours, generating more than $200 million in sales. and more than 4 million units in the first week, our models say watch out Halo Reach!

We are forecasting that Gran Turismo 5 will sell 2.3 - 2.5 million units worldwide day one, for more than $140 million. We also estimate that first week sales will see between 4.2 - 4.5 million units, for more than $250 million in sales. Will this installment in the series go on to reach the the levels of games past? Like many of you, we don’t think so. Maybe it will top out at 8 - 9 million units, or so, but we’re not so sure this genre has the same staying power as some others. That said, the PS3 has a very loyal fan base that may surprise us all!

Well, did they challenge Halo Reach? If pre-orders exceeded 1.6 million units, then they will match Halo Reach’s first day sales. Our little hedge against a three week delay in release. Regardless, the Gran Turismo franchise will certainly add to its lead over the Halo franchise, and both titles helped make 2010 a great year for gamers.

Let’s end with a mighty THANK YOU! and our best wishes to all of you and your families for a safe and happy holiday season! Enjoy your faves and keep on gaming!

Sunday, November 21, 2010

Brotherhood Blows by Hot Pursuit in the Quarter Mile...and Will Not Be Caught!

We know, we’s not a fair comparison! They don’t really compete with each other, but we needed a title.

Assassin’s Creed: Brotherhood has really challenged our models. This title looks to be a smashing success, as we estimate first five days sales at 2.1-2.3 million units worldwide, generating more than $125 million. Moreover, we forecast that Brotherhood will finish this year selling 7.2-7.4 million units worldwide, raking in more than $432 million. Furthermore, we believe that this title has a very good chance of exceeding 10.2 million units within a year. Congrats to Ubisoft for another wonderful release!

Unfortunately, for EA, Need for Speed: Hot Pursuit may not deliver this Holiday season. We estimate the first five days sales at 1.1-1.3 million units worldwide, generating more than $65 million in sales. However, we believe that overall quarterly sales will accelerate more like an old, blue, Ford Escort, rather than a bright orange Lamborghini, forecasting between 2.8-3.2 million units worldwide, generating more than $160 million by EOY. This is quite a bit less than what Mike Hickey at Janco Partners has recently stated. While the headline reads 4.2 million units for the holiday season, he somehow discounts EA’s 4 million unit estimate by more than 20%, when suggesting $185 million for the quarter. At $60 a pop, his dollar estimate would yield only 3.08 million units. What’s up with that, Mike? EA will need to quickly roll out the marketing brilliance seen with Medal of Honor, to put this title into high gear. We actually hope we are wrong on this one.

While we missed forecasting Fallout: New Vegas’ debut, our model shows they probably did around 1.8 million in the first five days, as opposed to the 1.4 million units VGChartz and others were speculating. Granted, we are all just estimating figures here, but it will be interesting if our models have a 20-25% better accuracy rate than others’ methods. Then again, maybe we’ll be way off on some of these forecasts, and find ourselves taking it out on some unsuspecting n00bs in your favorite multi-player, FPS.

Game on, peoples.

Friday, November 19, 2010

Call of Duty: Black Ops - EOY Projections and More

As we are finishing up our predictions for Assassin's Creed: Brotherhood and Need for Speed: Hot Pursuit, we wanted to share our EOY and Full Year forecasts for Activision's record-breaking Call of Duty: Black Ops release.

We predict that Black Ops will sell between 16.8-17.0 million units by EOY 2010, for more than $1 billion in worldwide sales. We expect Treyarch's blockbuster hit to sell more than 22.8 million units, surpassing $1.3 billion in worldwide sales by November, 2011. And, for you wall street types, don't forget we have been predicting margin expansion, to boot.

For the "and More" part of our title, we thought we would share a little more insight into what we are doing in our Analytics Lab with another chart.

These are just a few of the titles we have analyzed.  We can certainly state without reservation that your collective voices truly represent the gaming industry.  We appreciate your interest and game on!

Wednesday, November 10, 2010

Six out of Ten - A Roast of Jim Sterling

Aggregame is getting a little pissed off about the politics involved in those that rate video games. While we certainly respect everyone's individual opinion, there has to be some common sense knocked into some of these so called critics. We're calling out Jim Sterling from Destructoid who gave a 6/10 for Call of Duty: Black Ops. Really Jim? Oh boy!

We actually agree with most of his assessment of the PC version of the game. However, as anyone in his position should know, the PC SKU represents far less than 10% of the folks that have, or will, buy Black Ops. Moreover, he waits until 42 other outlets that influence Metacritic's scoring, which incidentally yielded a 90 for Black Ops on 11/9, to publish his score. In his review, though, he states "Once the multiplayer is fixed, feel free to pop at least two more points, most likely even three, into this review's score."

As an obvious Modern Warfare 2 fanboy, when Infinity Ward 'essentially crapped all over the PC gamers' when they delivered a mediocre day one experience, he promptly provided it with a 9.5/10. If memory serves us correctly, the PC community was outraged by MW2 PC, IWNet, and the lack of Dedicated Servers. Jim must have evaluated the Xbox 360 version, eh? Or, could it be he just needs to try and drive the Black Ops Metacritic score below a 90 for his own agenda / publicity?

We all know the PC version of any game is the hardest to tune and get right due to the variations in target machines. We also know that Treyarch has a stellar record in "fixing" what isn't good on day one. And, they seem to do so pretty quickly. To presume anything else, or have it reflect your "score", is absurd. Hey Jim, how about giving Black Ops a 9 and dropping it 2 or 3 points if they don't fix anything? Is that too complicated for you?

Just be glad we're letting you off the hook this easily, Jim. We've since read your past COD franchise reviews, and could write a dissertation with all the contradictions and double-speak you portray from one year to the next. You are the truest practitioner of hypocrisy, and the furthest from a journalist we've ever seen.

If Jim's not a fanboy, and he did this for the attention - he got us. We took the bait and even sourced his articles. Doesn't matter; this needed to be said. Whether he's a fanboy, or just playing politics, Jim is what is wrong with the game industry media. He carries himself professionally like a 12 year-old carries himself on message boards. Shame on Destructoid for giving Jim a soapbox he can't handle.

Alright Alright, Sorry Jim.

Roasting Jim was fun, but we're actually calling out a lot more people than just Sterling. Jim just happened to in the wrong place at the wrong time during the most popular entertainment launch of the year (and possibly history). Though, he did put himself in this position. He was "the guy" that had the balls to do it to such a high profile title this year. It happens every year, and actually happened more than once this year with more than one title. There are always "journalists" who want to stand out from the crowd, and do what they think will turn the most heads. They are running a business like any other. Like FOX News and the Obama "Terrorist Fist Bump" stories. They sure got viewers, didn't they?

But it didn't make it right. Neither is what Jim (and others) are doing within this industry. How do you think Obama felt to have such ludicrous accusations brought against him by such an influential media powerhouse? Presidential candidate or not, that had to take a toll on him, his family, and his administration. The fact is, some people actually believed those stories, and cast judgement on Obama because of those silly stories. What Jim did here is the same thing, proportionally. How do you think the hundreds of men and women at Treyarch feel seeing a 6/10? They don't give a crap what Sterling says or does on a personal level, but the audience Sterling commands, on the other hand, is of great concern.

Jim can't defend his review score. He may try, but deep down inside he knows he didn't give Black Ops a fair shake. He's a smart guy, and knows how these things go. He knows there are day-1 and week-1 patches, he knows these issues will be addressed promptly. He knows that the way the PC SKU was developed and tuned, ran perfectly in all of the test machines. He knows that, performance bugs aside, Black Ops is at least a 9/10 (he admits it within his own article). Worst case, he's just a fanboy that will never let go. Best case, he was trying to make some righteous point; A dramatic valiant stand, but in doing so did more damage than it was worth. Most likely case, he knew it would get the most amount of attention and discussion, in which case he should apologize to Gamers, Destructoid, Treyarch, Activision, and the rest of the Industry for his insincerity.

Our ultimate message to Jim, and the other "journalists" in our industry who want to pull this kind of stunt, is to cut it out. If you just can't help yourselves, or really want to be that controversial guy, then remove yourself from the Metacritics and GameRankings of the world and stop polluting the other good standing game critics that are still left in the industry.

Black Ops to Make History

While there has been ample speculation from wall street analysts and gaming industry pundits regarding the level of success Activision’s Call of Duty: Black Ops may realize, we felt compelled to make our own bold predictions based upon your twitter chatter.

However, instead of playing it safe by saying “could be more than 11 million”, or “could exceed 18 million, but not match MW2”, or “estimating 7 million sold on day one”, we figured we would provide some rationale to our predictions and provide our thoughts on dollars and profits.

After all, what good is a wall street analyst if they can’t provide the latter two elements.

So, what we found is some pretty cool correlation between the amount of chatter over very specific time periods, the number of unique twitter ID’s, positive versus negative sentiment within the chatter, and sales.  What?

In digging through a bunch of historical twitter chatter corresponding to previously released titles, with which we can find published sales data, we were able to construct some models that held up pretty well to our testing.

This is our final answer...

Day One Sales:   

5.9 - 6.1 million units (US & UK), 6.6 - 6.8 Worldwide,
Generating more than $350 million.
First Week Sales:    

9.1 - 9.4 million units worldwide, 
generating more than $540 million
Thru EOY 2010:    

We will publish our best estimate around 11/18 

We also anticipate above normal margins being realized from these sales due to the number and mix of platforms supported, a suspected reduced/deferred royalty expense due to ongoing legal matters (could be a wash depending on reserves amount), and significant benefits from various partnerships.

Now, could the lofty 7 million units on day one that someone projected come true? Sure, depending on what and when you are counting.  But, we believe that Activision will probably be rather conservative in their accounting and we base our estimates on the first 24 hours of sales (eg. through midnight ET on 11/10/10) for the Xbox, PS3, PC, and Wii. We actually suspect that the 7 million figure was met around 7pm ET on 11/12/10.

And, just to provide you with some pretty pictures, you can get a sense how your collective launch chatter reflects the relative success of video games. Granted, finding a useful visual representation of our data set always seems to be the biggest challenge!

In addition, we thought we would share with you how many of your voices were heard. Can you say, wow!  While these numbers are very impressive, didn’t it seem like the whole world was talking about Black Ops?

Until next time, game on!

Monday, November 8, 2010

Sorry Mike Hickey... You're Wrong!

Last week, Janco Partners' Mike Hickey predicted that Treyarch Studios' latest shooter, Call of Duty: Black Ops, could sell 18 Million units after launching Tuesday, pegging it just shy of MW2's sales. To put it bluntly... He's wrong.

Hickey has severely miscalculated his prediction. How are we so sure? Well let's start with some traditional methodologies: Black Ops has more pre-orders than Modern Warfare 2. By the numbers, on the 360 + PS3, Black Ops has 2.36 Million pre-orders as of the week ending October 30th. MW2 had 2.24 Million at the same time last year. Considering how much more positive Black Ops' PC outlook is than MW2's, we'll just assume Black Ops has higher pre-sales on that platform as well (although, VG Chartz stopped counting PC Pre-Orders for some reason). Also, on multiple occasions this year, various retailers made very ambitious statements regarding Black Ops setting records in both pace of pre-sales, and overall pre-order numbers. "Record Setting" translates directly to "Better than MW2" - Only more objectively, and less sensational. In other words, more sincere.

In light of the above information, what could Hickey's rationale possibly be? The weaker economy? If the economy really had an impact on a title like this, we'd have seen it reflected in the pre-sales...

Now, we're a Twitter-oriented site here, and much of what we base our predictions on comes from less traditional analysis of consumer behavior from Tweets. Later this week we'll reveal some brand-new, one-of-a-kind, can't-find-anywhere-else Twitter analytics to put the final nail in Mr. Hickey's coffin. We wanted to get it on the record before Black Ops launched, however, that we disagreed firmly with Mike Hickey's prediction.

If you're going to be an analyst, you need to A) Get it right, and B) Not just write what you think the media wants to hear. We all know it's fun to put the Modern Warfare's ahead of the other COD entries, and we all know how much the mainstream gaming press likes to regurgitate those "findings" - But it's just not the case this time around.

Wednesday, September 8, 2010

PAX vs E3... In Tweets!

Some rather interesting data came through the tubes this week. As most gamers know, PAX Prime in Seattle wrapped up over Labor Day Weekend, and thus closed another chapter in 2010's video game convention schedule. What most of you probably don't know, is that PAX yielded a fraction of Twitter chatter compared to E3 2010 earlier this year, despite drawing higher attendance figures than E3.

But how can that be? PAX Prime 2010 saw record attendance figures - over 67,000! Meanwhile, E3 2010 had a mere 45,500 attendees... What does this say about Twitter, gamers, and the respective conventions? That's what we at Aggregame are pondering today.

Let's start with the hard data... In this report, we analyzed the data of E3 2010 tweets during a one week window starting 2 days before, and ending 1 day after E3. Using a similar window of time for PAX Prime, we see the below results:

Now, in PAX's case, we grabbed the most ideal window of time (tried to give them to benefit of the doubt), so we gave them one extra trailing day (which saw more tweets, but ultimately the difference was negligible). Still, their total Tweet count for the week ended just over 78,000 Tweets

E3 2010 Statistics:
45,000 Attendees
605,000 Total Tweets
13.3 Tweets per Attendee

PAX Prime 2010 Statistics:
67,000 Attendees
78,000 Total Tweets
1.2 Tweets per Attendee

Of course not every tweet was literally tweeted by an attendee of these conventions; plenty of onlookers contributed to the chatter. Nevertheless, if you measure the value of an event by social media chatter - "Tweets per Attendee" - PAX doesn't hold a light to E3.

What's The Point
How do you think Publishers and Developers should (and do) look at the value of a trade show? Should they care about the 40k - 60k attendees? Is that what they invest their Millions of marketing dollars towards? Or are they looking for that mass viral chatter that proliferates from such an event...? We suspect it's the latter... In which case, why would any Publisher or Developer want to take part in PAX, when E3 clearly generates way more hype?

Perhaps we're being a bit harsh on PAX. After all, E3 2010 had a far more powerful lineup of games at their show. Certainly that has a direct impact on number of tweets about an event. Of course "Electronic Entertainment Expo" is also better known, has a longer history, and is sort of the de facto trade show to be compared with.

So we're going throw down the gauntlet right now: How well do you anticipate the Tokyo Game Show (TGS 2010) will hold up against E3 and PAX? TGS is THE largest video game trade show in the world, dwarfing all other attendance figures. Is it as well known in the Twitterverse, though.. That is the question.

We'll find out later this month!

A Little Something Extra
We wanted to leave you with this bonus image. During PAX Prime, 2K Games and Gearbox unveiled Duke Nukem Forever (Google it to learn about the lore behind this Goliath). This of course spread like wildfire through the gaming world. Here is the snapshot of Duke Nukem Forever mentions on Twitter over the same period as PAX Prime's Event Stats above:

That's right... Just about half the Total Tweets for the entire PAX Prime event channel, for one game. Just sayin'!

Friday, July 23, 2010

1 Year of Gaming Tweets [Infographic]

We want to let the image speak for itself, so will keep this entry brief. What you see below is approximately one year of Twitter video game-related chatter from the period of July 27 2009 to July 22 2010. It shows the 150 most commonly used words in gamers' tweets. The graph was generated using over 28 Million gaming-related tweets collected by AggreTweet - a live streaming aggregator for gaming conversations happening on Twitter. Platforms, Games, Events - even Communities like Joystiq and Kotaku have real-time feeds on AggreTweet.

As promised, we'll keep this entry short. But I want to leave you with one last thing: we've recently launched our #PartyUp on Twitter feature to help gamers connect via Twitter - please check it out and read more about it. It's one thing for us to make it, but we need your help to spread the word! Now, enjoy this Infographic...

A Year of Gaming Tweets, in an Image:
(click to enlarge)
One Year of Gaming Tweets

Tuesday, July 13, 2010

Introducing #PartyUp on Twitter!

Ever find yourself in need of an extra player or two to round out your Xbox LIVE or PlayStation Network party? We've just launched a new tool for gamers that makes finding players for your favorite games easy, thanks to Twitter - called #PartyUp on Twitter!

How it works:
  1. Put "#PartyUp" in any context in a tweet.
  2. Watch the feed for your tweet to appear.
  3. Wait for fellow gamers to @ reply you!
Hopefully gamers will find this a useful resource for further socializing their gaming experiences!

Wednesday, June 30, 2010

New AggreTweet Feature - Filter Top-10!

It came to our attention that given the overwhelming popularity of the Nintendo Wii, Xbox 360, PS3, and newly announced Kinect, the list of Gaming Trends on the front page of AggreTweet has become a little... crowded. Since those 4 platforms were always occupying the top 4 slots in the list, it only allowed for a maximum of 6 games to trend. Once you toss a few events into the mix (like E3) which are sure to trend high for weeks on end, it starts to get a little out of hand - and functionally useless.

So we've today launched an "Options" link above the Gaming Trends which can be clicked. In it, you'll find 5 convenient check boxes which can be used to specifically filter out which topic types you want displayed (pictured left).

This feature only affects the trend graphs, it does NOT affect the actual results that will be displayed in the Live Feed. You'll still see all the topics flowing in as you'd expect. The proper way to filter down the Live Feed is to go to a specific Topic channel (to see only tweets for that topic).

As the site continues to grow, and Twitter becomes more widely used, we'll continue to design useful features to make tracking your video game conversations and trends that much more useful!

Friday, June 18, 2010

E3 2010 Nets Over 500,000 Tweets

As anticipated, E3 2010 yielded a massive amount of social media chatter, including over half a million tweets in a single week!

E3 2010 ran from June 15 - 17 of this year, with a Microsoft Press Conference on Monday the 14th, along with a ridiculous Activision concert that same evening. Essentially, this entire week was all E3. And according to AggreTweet, E3 was the top trending topic among gaming all 7 days.

To be precise, from Saturday-to-Friday of E3 week, there was a total of 605,000 tweets. Given that I am writing this entry a little before midnight, naturally that number will change by a factor of a few thousand. However, the important take-away from this is just how accurately we can predict, track, and trend Twitter activity based around video games.

You may have remembered we predicted this outcome last week. How did we know? Well that's our little secret... But we have spent a lot of time and energy perfecting our technologies here at Aggregame, and will continue to do so. AggreTweet in particular continues to prove it is the most powerful Twitter analytics software in existence, and we will be growing and expanding this technology even further in the future.

Friday, June 11, 2010

E3 to Net 500,000 Tweets in 1 Week!

Over the last several months, we've learned a lot of valuable Twitter trends and behaviors while watching the AggreTweet graphs. One thing we pride ourselves in is our ability to assess user behavior in real-time. We've gotten so good at it in-fact, we're prepared to make a prediction...

We anticipate E3 2010 chatter to net at least 500,000 Tweets in a 1 week period, Saturday-to-Friday of next week (June 12 - June 18). This will be an unprecedented amount of Twitter game-related conversations, and should set the bar for the industry.

To put this into perspective, currently the Playstation 3 (as a platform) has the highest weekly tweet count, with just over 100,000 tweets this past week. That's a product with a healthy install base, and a large library of games for players to tweet about in relation to their platform. Never has a gaming event surpassed a platform in weekly tweets - not even PAX East earlier this year.

It's shaping up to be a big year for E3 with games like Killzone 3, Halo: Reach, Gears of War 3, Medal of Honor, Call of Duty: Black Ops, SOCOM 4, Rock Band 3, Crysis 2 (to name a few), and of course the highly anticipated deep-dive into Project Natal Microsoft is expected to make at their press conference on Monday. Amidst additional rumors such as Hulu coming to the Xbox 360, and speculation on Valve's big surprise (probably Half-Life 3 / Source Engine 3), there will be plenty of things for gamers to tweet about =).

Monday, May 31, 2010

Graph Maintenance - Complete!

As noted in the previous entry, we recently updated the AggreTweet graphing technology to display more accurate information. Now, graphs will at-most be 3 minutes out of date. Considering that the graphs only plot points in 6 minutes intervals (by design), this should really not be noticeable by anybody looking.

So, in cases like a Halo: Reach BETA launch, and everyone is clamoring for a code via Twitter, you will see the spike in the graphs as soon as the next plot point on the graph is generated. This also means that if a given topic gets a high volume of tweets in rapid succession, it will rise to the Top-10 list much sooner (and accurately) than before.

We'll be monitoring performance as always, and if we notice any other anomalies we'll be sure to address them as quickly as possible. As always, look for updates here!

Friday, May 28, 2010

AggreTweet Graph Maintenance - 5/28/2010

Notice 5/28/2010: The trend graphs on AggreTweet are not displaying accurately at present, affecting overall trends / rankings. We are working to rectify.

So what does that notice atop Aggregame mean? Well in short - the overall rank-ordering of top games, as well as the graphs displaying on game channels, are not actually representative of the most current Twitter activity. This only affects the graphs - actual Tweets are flowing real-time, so that's good.

However, AggreTweet is most useful for tracking trends. It's no secret that reading the front page of AggreTweet is, for all intents and purposes, a waste of time. It's a real-time feed of every tweet for all major video games, so needless to say there's a lot of data constantly streaming in. To try to actually read everything coming in would be like swimming upstream against the current. Then again, if the stream of tweets was broken, it would be an even worse scenario.

Thankfully, we are still tracking all the data. That means once this problem is rectified (hopefully sometime over the weekend), the graphs will suddenly be displaying properly again and you'll be able to filter through that historical data (Past Day, Week, Month) and see how the weekend's trends really played out.

A Deeper Look into the Problem:

Given the volume of tweets and Twitter users AggreTweet tracks (over 20 Million tweets stored, with hundreds of thousands of Gamers contributing) our physical databases are enormous. Working with this volume of data takes time (and CPU cycles).

AggreTweet is architected in such a way that it is extremely efficient at handling high volumes of tweets, rapidly - and more importantly - handling lots of concurrent viewers watching the live feeds. So if there happens to be thousands of visitors chatting simultaneously in various Game Channels on AggreTweet, it can handle it and not crash the website due to that many people all requesting those streams concurrently.

Given this primary charge, some trade-offs must be made. One of those is in how we store and index each tweet, and subsequently how we plot that tweet against one another in the graphs. So given the nature of this particular failure (a failure to scale in quantity of total tweets, not concurrency), working to fix the issue (permanently) is a bit of a chore (and waiting game given how much data we must re-factor). Rather than put effort into a "band-aid" fix up front, we're going to spend all our time and resources fixing it the right way, once.

As such, the maintenance will take most of the weekend. We'll have an update for you when we have some conclusive results. At least we know where the problem is, and how to fix it. If we erased all historical data we could have the problem solved immediately. But that's not a very appealing option... So instead, we must arduously wade through 20 Million tweets (and more importantly, test our results to make sure this fix is not going to cause any other problems).

Apologies for the inconvenience. AggreTweet will remain online (as it is still technically functioning); you'll just have to assume that none of the rankings or graphs are displaying current data. Actually, technically all the graphs are accurate (relatively speaking), but are actually showing the plot points from hours ago. So if you hover the graph and it says that a topic had "30 tweets" 6 minutes ago, it really had 30 tweets at that particular moment - 5 hours and 6 minutes ago (because the graphing engine is so back-logged due to this performance issue).

In any event, we'll have it solved as quickly as possible. Stay tuned.

Thursday, February 25, 2010

The 2010 Shorty Awards!

When we first learned AggreTweet had been receiving nominations in the #apps category for this year's Shorty Awards, we were humbled and excited. Thanks to all you wonderful gamers and Twitter-ers, we're a FINALIST!

The winners for each category will be announced at the Shorty Awards Ceremony in New York City on March 3rd. We'll be in attendance, and keeping our fingers crossed! Nevertheless, we wish the best of luck to the other Twitter applications in our category: @unfollowr, @colourlovers, @TweetDeck, @HootSuite, and @MailChimp - such a great variety of useful apps!

As promised, we did give away our first #PSWii60 prize pack to @joannaonthelake - she selected a PlayStation 3 as her platform of choice, which was promptly shipped, and we're told she and her family are enjoying every minute of entertainment the console is providing.

Remember, follow @Aggregame on Twitter - for every 2,000 followers we receive, we'll be awarding a random follower a free #PSWii60 console of their choice (contest rules here) - spread the word!

Friday, January 15, 2010

NPD Confirms Aggregame's Initial Findings

Well, now that NPD’s numbers are finally out, we are proud (and relieved) to announce that our AggreTweet analysis was spot on in determining how console sales compared and fared for Holiday season 2009. (see Top Console & Gaming Trends, Christmas 2009 - published December 29 2009)

The week's worth of Twitter chatter we analyzed directly correlated to actual sales numbers. Moreover, this data highlights the power of micro-blogging social media as possibly the earliest market indicator, as we reported our findings sixteen days before NPD’s validation. We believe that there is significant value in knowing the outcome of an event weeks before the published results from leading market research firms.

What really stands out in this particular case, was the amount of negative press Nintendo received from May, 2009 all the way through Thanksgiving regarding their sales numbers... Analysts were pontificating that Nintendo’s Wii had plateaued and would suffer during the Holiday season - only to have a huge Christmas surprise!

So, a shout out to all of you Tweeters! Your collective chatter matters, and we’re working hard to make your collective voice heard - in real-time.

As we continue to research and develop a far deeper insight into leveraging Twitter chatter for use in our analytics products and services, we’ll keep you informed of any breakthroughs.

We hope you enjoyed us sharing our data and process in initially testing and validating some of our research. And thanks again for participating, even though most of you had no idea we were watching! ;-)

Wednesday, January 6, 2010

Vote AggreTweet for a Shorty Award!

The 2nd annual Shorty Awards have just kicked-off, and as a Twitter-based web app, AggreTweet has been nominated in 3 categories - #apps, #gaming, and #tech!

We'd really appreciate your support in voting! Official instructions can be found right here, but it's really quite simple: if you have a Twitter account, just login and tweet (one or all) in the following format:
  • I nominate @AggreTweet for a Shorty Award in #apps because it's a powerful Twitter application built on Twitter's Streaming API!
  • I nominate @AggreTweet for a Shorty Award in #gaming because it's a great tool for gamers or industry professionals to track gaming trends!
  • I nominate @AggreTweet for a Shorty Award in #tech because it's a useful application of Web 2.0 technologies and techniques!
Using similar technologies AggreTweet itself uses, the Shorty Awards website will automatically detect your vote and will count it (it may take a little bit of time).

You can vote in as many different categories you want, and of course are encouraged to write your own reason following the "because" - we just provided the above as examples =).

Thanks for your support!