Make (virtual) room Enterprises – Social Media and Virtual Worlds are Teleporting to your organization

About a year ago, after my visit from the VW Expo 2008 (now called the Engage Expo) I wrote about how enterprises were cautiously approaching virtual worlds by deploying small pilot projects around support functions. One of the main concerns that were brought up at the conference was around how to measure Return On Investment and whether organizations would be willing to publicly talk about their success stories. On the other hand, there was widespread belief that the areas where there would be some clear benefits would be around training and collaboration – operational cost savings, efficiency and productivity.

Well we didn’t have to wait too long to see what happened. By April of 2009, 3D  TLC 1.0 happened, a conference around the application of 3D worlds towards Training, Learning and Collaboration. Not only is it commendable that they were able to pull off a whole conference around this, but there were a plethora of enterprises present to talk about how these new technologies helped them. The 3D TLC 2.0 is being held with the Engage Expo in San Jose on Sept 23-24th. I am also encouraged to see that the event continues to attract enterprises from various industries.

So is the case with social media. Facebook, Twitter and Youtube are being increasingly used today by enterprises for viral marketing, collaboration and even advertising by companies such as Home Depot, Best Buy and Wal-Mart. More importantly, they are also able to gauge the effectiveness of using these media. Best Buys’s Twelpforce is an innovative way to provide online customer service with a personal connection; so did HomeDepot. Wal-Mart’s elevenmoms leveraged the power of collaboration to bring together money-saving communities – something central to Wal-Mart’s messaging.

This doesn’t necessarily mean that these immersive environments and social media technologies have become main-stream in enterprises. But it certainly shows that enterprises are finding real applications for these in solving business problems. And it definitely provides a boost to other companies on the fence who have thought about trying it but haven’t made the move yet.

I was part of IBM’s Data In Action Virtual Event back in February and therefore got to experience this first hand from the “supplier” side. There were some clear benefits to this:

  • Obviously it allows for many interested parties to take part in this event who otherwise would not have traveled out to a conference
  • Registration and lead tracking is far more automated
  • “Floating” around the VW via your persona is obviously time-saving and efficient :-)
  • The ability to replay a presentation in the conference was certainly very useful

It also brought to light some things that could enhance the experience, notably, if there was a notification mechanism for the booth staff if someone visited (so I wouldn’t have to poll the screen all the time I was on duty) and also if there was a mechanism to exchange files or e-business cards.  But I am sure these capabilities – and more – will be available soon.

While this wouldn’t necessarily replace a full blown real world event, it does compliment it. A virtual conference before a real world conference would allow me to either better address or keep focus on key priorities. A virtual conference after an event would give me a venue to follow-up on key feedback items from the main event.  And that’s really how I see the value being played  out today.

The technology is only getting better and so are the options for the enterprise. This is exactly what OpenSim offers – an open source choice for customers who would like to deploy virtual worlds inside the enterprise.  With the integration of OpenSim and Informix, you now have a “best in class” OLTP database to manage scalability and growth – enabling companies to start building best of breed solutions – that’s what progress is all about!

Leave a Comment

Informix integrated with OpenSim

Informix is now integrated with OpenSim. This means you now have the option of running Second Life (SL) clients with Informix via the Opensimulator. OpenSim is a 3-D application server. To read more about OpenSim check it out here : http://opensimulator.org/wiki/Main_Page

There is also an OpenSim grid where you can get started quickly: http://www.osgrid.org/elgg/

Leave a Comment

Happy New Year

A very Happy New Year!! Here’s hoping for an exciting year ahead in virtual technologies. Judging from this article on virtual architects, it looks like virtual technologies are slowly but surely finding their place in solving real business problems.

Comments (1)

Power of Informix Extensions – Take 2 – TimeSeries

I want to point out to all of you an interesting video, which I will use as a basis to talk about an Informix extension that could find some application in MMORPGs. Check out this link (it’s a video):

http://news.bbc.co.uk/1/hi/england/7632521.stm

As you would have observed from the video, this is a system designed to monitor real time energy usage inside a home – turn on the dryer and up spikes your energy meter! I would find this really useful at my home – it would tell me exactly how much each energy consuming device in my house is costing me – and then I can make adjustments to reduce my energy bill. Of course that might mean, I may end up limiting the up time of my computer, which in turn would mean I would be blogging less and .. – OK, never mind. I want to bring your attention back to what I think is an important and interesting aspect of this system – time based data capture. This data is what is characterized as time series data. Time series data are identified by timestamps and stored in a database. Many other examples come to mind in terms of time series data – stock prices, seismogram data, network performance monitoring, etc.

MMOGs can capture some interesting time series data. An example would be the possession of virtual assets over time by players. Among other uses, this data is also useful for instance, to validate a player’s claims of ownership of an asset or his current level – where someone calls into the game’s customer support and claims the loss of an asset or disputes that he was actually at a higher level. With time series data, support/service staff can backtrack possessions/levels to specific timestamps within a given day. Another really interesting (or creative ?) application that comes to mind is using time series data for “replays” – imagine being able to replay parts of the game back !!

This is where the power of the Informix TimeSeries extension comes into play. Instead of a row per timestamp, all entries related to the time series are entered into the same row of an Informix table. TimeSeries is what is known as an extended data type in Informix that has its own set of powerful analysis and manipulation routines. You could store time series data as row per timestamped entry – but this leads to poor performance and inefficient storage especially considering the fact that time elapsed data can really grow.

For the sake of simplicity let’s assume that you are tracking the position of a player. Let me state up front that the tables/examples used here are purely for illustrative purposes (meaning I haven’t put any emphasis on the actual organization of the data and I haven’t validated any syntax, etc, etc).

With Informix TimeSeries data type, this could be stored in the a table as shown:

player_id position_info
P1 TimeSeries(pos_chg)
P2 TimeSeries(pos_chg)

down-arrow-blue-small3

t1 t2 t3 t4 …….

down-arrow-red-small3

TimeStamp position place_id
DATETIME YEAR TO FRACTION (5) Point CHAR

As you can see, a TimeSeries data type allows you to keep a single row for every player irrespective of how many times assets change hands or how many times they change their position. This also helps save storage space.

To understand how this works, think of the TimeSeries column as a container for a row subtype. This subtype represents a structure of elements (t1, t2, t3, …). Each element represents a single set of data for a timestamp. In the positioning table example, this sub-type is labeled “pos_chg”. The TimeSeries data type acts as a constructor for this subtype as shown in the table: TimeSeries (pos_chg).

Again each element (t1, t2, t3, …) is another row datatype that begins with the timestamp column and has additional columns. The Timestamp column can be used as an index for the elements.

The creation and loading of time series data is more elaborate and I am not getting into it here – this is well documented in the Informix TimeSeries Guide. My intention here is to illustrate the power and flexibility of theTimeSeries extension.

Once created and loaded, a variety of SQL routines (as well as C and Java API access routines) can be applied to this data – select individual elements (by timestamp, offset, specific position); modify elements or set of elements by specified time range, at a given offset; retrieve subsets of data using aggregate functions; compare and perform arithmetic operations on a pair of timeseries data and much more. These functions demonstrate the real versatility of using the TimeSeries data type extension.

As I mentioned before, this type of information is extremely useful for validating customer claims or for replays of important parts of the game. And although I mentioned that I am not getting into all the details of creating and loading the TimeSeries, I am still going to give you a bonus (heck, it’s the holidays!!) – check out this article on IBM developerworks – it is a nice beginners article on how to create and use the Informix TimeSeries extension and it’s based on game data ! (well sort of , sports stats to be more accurate :-) ).

Leave a Comment

Informix Rated #1

I nearly forgot to post this rating that came out over 3 weeks ago!! Informix was rated the #1in customer satisfaction by VendorRate. Check out the report.

Leave a Comment

Power of Informix Extensions – Take 1

Sorry for the loooooong break! With the election fever and kids homework, things got really wound up :-) . Now that one is out of the way, I have found some lost time! Today I want to touch on a powerful Informix capability that can either help solve some interesting problems for game developers or find new applications! This capability is known as Informix Extensions – a capability that allows users to extend the capabilities of the Informix data engine and thereby open the door to some really innovative solutions.

Informix provides extensions using it’s datablade technology. Extensions in the Informix context can be thought of as domain specific data-types and their associated routines that are treated as a standard data type by the engine. As an example, consider the Informix spatial extension. The Informix spatial extension consists of data types such as Point, Polygon and include manipulation and query functions such as Intersects(), Boundary(), Union() and so on. The power of extensions is obvious when you consider that they are loaded on demand , executed in the data engine’s process space and accessible via regular SQL, thereby providing both flexibility and performance.

Informix provides an API that allows users to define their own extensions. These extensions can then be registered with the Informix engine as user defined extensions. Let’s say you wanted to manage asset dependencies in your game (see visual below) – for instance, let’s assume that changes to a mesh would trigger changes to the compiled model, the texture page file and the final character. Tracking these dependency changes becomes complex and error prone if it’s done using application business logic. However, by creating an Informix Node extension that represents these dependencies and the actions to be performed on them drastically reduces the complexities and makes it so much easier to use by encapsulating the data type and providing functions such as GetParent(), Compare(), Ancestors() and so on.

node_example21

If you are excited about the potential this holds and would like to dive into the details let me know. Also I recommend starting with the following material :

The IBM Informix datablade family overview

Developing Extensions to IDS (from the online Informix Information Center)

In the next post, I am going to lead you into another built-in extension in Informix that can be useful in MMOGs and Virtual Worlds.

Leave a Comment

Blogging from the GDC 2008 event, Austin

The GDC 2008 kicked off Monday Sept 15. The exhibit floor seemed very familiar to what I saw in 2007 in terms of technology platforms and vendors, some of the prominent vendor exhibits being Icarus Studios, BigWorld, Simutronics, Multiverse and of course many of the service providers were present – like Vivox and Aria systems.

In terms of the overall roadmap and business models, many vendors continue to focus on “serious games” along with attracting MMO developers and publishers. Learning and education seemed to be primary areas for serious games. There is also an increased focus on the need to get more out of the data from gameplay – improved metrics and deeper data mining.

Icarus Studios exhibited “Fallen Earth” which is now content complete. I believe Fallen Earth would be the first MMO in production on Informix.

In the overall scheme of things, providing rich content and intriguing gameplay to keep customers challenged, happy and coming back still ranked as the top priority. So was there talk around and interest in database technologies ? Indeed yes if you talk to MMO developers who are serious about managing data efficiently and having flexible options to scale data – which by the way most of them are serious about. I am going to blog the database perceptions and challenges in this industry in a separate post – I invite any experiences from you as an MMO developer or publisher on this topic.

Leave a Comment

The Great database debate –What’s the deal ?

When I look at the online game industry and its dynamics, it reminds me somewhat of earlier innovations – pivoting around data management – that took place in other industries. Consider for example, the automation of reservation systems in airlines with SABRE, or the blinding competitive advantage that Wal*Mart build up (and enjoys even today) with its automated supply chain and real time inventory management. These were among other companies and products with visions about the potential power of data. Capturing and storing data is one thing, realizing the competitive advantage of data is another.

Ok, so what does it mean to the online gaming industry?

I have had many a conversation and I still see a lack of “thinking through the database strategy”. Some in the industry are now realizing that the data is strategic to their business and that in terms of game performance, the database engine needs to perform and scale like their game engines. Yet, many companies out there seem to settle on their decision with open source, specifically, MySQL, too quickly. Here are some of the things I usually get:

  • It’s free
  • It does “most” of what we want it to do
  • Google, Wikipedia use MySQL – it scales for them!

Now I will say upfront that I have nothing personally against MySQL – there are scenarios where it meets the needs of certain business requirements adequately. But there’s a lot of omission of detail in much of these statements (MySQL does advertise that Google uses them) that is actually presenting a lot of misleading conclusions – I think it’s probably time to put things in the right perspective. So , you might want to grab that cup of coffee and a snack to bite on , this could be some long reading.

Let’s put cost in perspective – “It’s free” refers to the licensing cost of acquiring the open source community edition. Most people I know always sign up for some level of support which has to be paid for. You then need to install it, run it, design database schemas, load data and maintain it. That’s investment in time and DBA resources, aka total cost of ownership. Calculate those costs for at least three years out for all the instances that need to be deployed and managed (and the supporting hardware and people needed to manage it). Add to that any additional costs for upcoming requirements such as cluster support or high availability. That’s the price you are going to have to pay. That, my friends, does not spell FREE.

In terms of “it does most of what we want it to do” – the biggest risk you face there is working around capabilities that the database management system doesn’t provide (but ought to). Think about it, this can not only be a huge cost, but also sub-optimal in terms of performance and efficiency. It means:

  • that prior to MySQL 5.0 (which is very recent), code that dbmses were efficient at executing such as stored procedures and triggers had to be written in application logic. This further means that business logic that needs to be shared, cannot be and have to be made redundant across applications
  • that additional support components for functionality such as load balancing and failover have to be acquired or again coded in the application.

If you can see through that, you realize that those are hidden costs that are not accounted for. Another risk that I have seen materialize is when customers find that a critical capability they need is missing, they have to invest in porting a lot of their code  – adding to the cost bucket significantly.

Now on to some interesting stuff – Google and Wikipedia!

First, yes, (a) Google uses MySQL and yes (b) Google scales but concluding (b) happens because of (a) – er, that’s really stretching it and is not factual.

Google has tens of billions of web pages, over 8 billion indexes, Google Earth data and Google Analytics and of course Google’s AdWords application. The only publicly cited application of MySQL at Google is AdWords. To my knowledge, once you sign up for AdWords and fill in your ad and keywords, then it’s pretty much a read-only data model where the ads are looked up and placed in Google pages/results and websites that cater to AdSense. Google’s infrastructure is much more beyond the AdWords application.

So Google does not scale on MySQL. In factt I did some research and have come to learn that Google scalability is built on:

  1. Google File System (distributed file system clusters)
  2. BigTable ( a specific data management system for a specific need)
  3. Chubby – Distributed Lock service
  4. MapReduce

They need the combination of the above four to store and scale large amounts of data – essentially they needed to scale across hundreds of thousands of distributed servers that collectively store petabytes of data.

Without going into too much detail, (there’s lot of info already on the internet) BigTable is a persistent multi-dimensional sorted map – the map is indexed by a row key , a column key and timestamp. In order to perform parallel computations on the data stored in BigTable, Google uses a software framework called MapReduce. All this runs on top of Google’s distributed file system cluster.

As you can see, Google made huge investments in building their own technology to scale their infrastructure – including their own data management system – BigTable. So there you go.

Wikipedia is built on top of the MediaWiki application server. Again Wikipedia does not scale on MySQL alone, in fact, there is a rich set of supporting scaling infrastructure that sits above MySQL:

  1. A complete content delivery and acceleration network front-ended with caching servers – squid servers, load-balancing with Linux Virtual service – this load balances between the CDN, Application (MediaWiki) and Search (Lucene)
  2. PowerDNS for geographical distribution
  3. MediaWiki Application Server
  4. memcached for distributed object cache
  5. A separate Fulltext search component using Lucene – because MySQL fulltext is only available on MySQL ISAM and does not perform or scale as well as Lucene
  6. Separate Media storage and server for static files including images
  7. And finally, MySQL to host the core database – one database per wiki – core data consists of metadata, users, article revision history, article references. 90% of the queries will be single table lookups on primary keys – there are roughly about 12 tables per core database. When an edit is submitted, mediawiki writes it to the database, without deleting previous versions.

So how does Wikipedia scale? Look closely it’s a combination of Squid caching servers + MediaWiki + Lucene+MySQL+memcached.

A lot of hits are serviced by the cache and they “avoid” expensive database queries.

So in summary this works for WikiPedia because:

  1. of lots of cacheable content.
  2. scaling users is almost completely handled at the application level – MediaWiki and powerDNS
  3. database requirements are very simple

The beauty with Informix is that it does not require applications to “help” out and does not need to rely on a host of supporting software to get work done. On the other hand, if the enterprise uses third party solutions as part of their infrastructure, Informix works well and complements many third party software and hardware high availability and scalability solutions. So stay tuned for a look at how built in Informix capabilities work to maximize your applications performance, scaling and availability requirements.

I hope that gives things a little bit more perspective. And hopefully, you made it through without a refill of your coffee cup! At the end of the day, what works for Google works for Google’s business needs, what works for Wikipedia works for Wikipedia’s needs. Your business is your business and the infrastructure needs have to be thought through at every level.

Stay tuned for more !!

Leave a Comment

My musings on the Virtual World Expo 2008 in LA

I had the opportunity to be gorgeous LA attending the VW Expo. This was my first at any of the VW conferences.  I frequent the GDC

more. Anway, the thing about this being my first event was that I didn’t really have any particular expectations set. I was to there to take in the technology, the business models and the trends. And from that standpoint, it was a reasonably valuable experience for me! My colleague Karen Keeter forwarded me a few photos from the expo that I post here!

The details of the location and track can be found at the VWExpo site. There were 5 tracks for sessions and i spent my time in the Enterprise, Futures and Technology tracks. The keynote sessions were interesting in that they were more of a Q & A format with an industry expert interviewing the “keynote” speaker. My main interests were really around the application of VW for business value. From that perspective let me first kind of highlight what my takeaways from the conference were:

  • Virtual Worlds hold a lot a lot of potential, however, enterprises are taking a cautious approach by deploying small pilots around support functions
  • Adoption has several challenges – security of assets, scalability, integration with enterprise applications
  • Nobody thinks or expects there to be a killer app, but many panel experts suggested that if there was one it may come out of IBM
  • A lot of interest in bringing custo mized virtual world hosting inside the firewall

Effective collaboration continues to be the area where VW deployments are providing benefits – examples of sales training, education, online conference rooms were frequently mentioned. IBM incidentally has a large mindshare in this industry. One of the keynote speakers was Colin Parris, the VP of IBM’s Digital Convergence business unit. Stay tuned for some Lotus innovations with integrated 3-D interaction capabilities.

In terms of potential I get pretty excited just thinking about all the avenues this opens up – think of the post Katrina University of New Orleans and their virtual campus as a risk / contigency management investment – there’s a direct application of VW for disaster recovery!!

There was an Informix demo around 3-D commerce. That’s right. Shopping carts in 3-D worlds! If you look today, while there are a lot of companies with islands inside of SL, much of it is targeted towards brand marketing. But what if you could visually model an item in the world itself for fit,review it’s looks and then add it to a shopping cart inside the 3-D world itself? Leveraging Informix’s spatial capabilities , Lance Feagan (an Informix R&D engineer) whipped up a framework on top of IDS to enable these aspects of commerce as well as get some data mining capabilities. Here is an article on this topic.

Then there was the IBM demo of virtualizing the whole data center for enhancing data center monitoring – adding the “situational awareness” aspect to remote monitoring. It also greatly enhances “what if” scenarios and capcity planning. Now if they could capture the live sensors in those data centers, one can get real-time visuals on elements such as energy consumption and temperature!! I told you I could drum up some potential applications :-) .

Yes there were some really neat demos  – I personally liked the Qwaq demo – Qwaq has some impressive capabilities around virtual conference rooms – sharing and editing a variety of document formats inside of the conference room, meeting room and document persistence, collaborative editing, integrated voice, etc.

I also liked the ESC demo (Electric Sheep Company) and their concept of “2.5 D” – personally I didn’t think you could do a whole lot with Flash,but I was impressed with the amount of immersiveness it can produce. So with WebFlock (their flagship product), your virtual world can be inside of a browser – no client downloads. I think the 2.5 comes from the fact that to interact with an asset inside the world such as a document, it has to be done outside of the flash world.

Leave a Comment

For Starters…

Now I flat out will let you know that I am not an expert on all things about virtual worlds and MMOGs. But I do know a thing or two about databases :-) . And I do know about game data and some of the challenges and requirements for their management. For starters about how Informix capabilities can help manage game data, check out this white paper I developed – Fun and Games: The Data Server for Interactive Entertainment.

I look forward to your feedback and comments!

If you are new to Informix, I recommend you visit the IBM site for Informix.

And if you still need some real life validation that database technology matters, read this article on why one company Icarus Studios, switched to Informix.

My colleague George Dolbier, who is a well-respected IBMer in this industry has this valuable article on sizing.

Also, here is a nice article by David Kennerly on aspects of game data mining. If you are an MMO developer, this article gives you good insights into why (and why not to !) , when and what data to mine from your game.

Leave a Comment

Older Posts »