Archive

Posts Tagged ‘cicp’

Glasshouse injects 3D representation of data into a virtual world

March 14th, 2009

Glasshouse by Green Phosphor is a gateway which can take a database query or a spreadsheet and place a 3D representation of it into a virtual world. Users can see data, and drill into it; re-sort it; explore it interactively – all from within a virtual world. Glasshouse produces graphs which are avatars of the data itself. 

glasshouse_sl_scatter

Virtual worlds are environments where avatars can observe and interact with eachother. This is a wonderful advance for human to human interaction; and it is also a wonderful advance for human to program interaction. Imagine a useful program residing somewhere on the Internet… and to interact with it, all you need to do is invite its avatar into a virtual world with you. The avatar of the program is its 3D user interface. 

Imagine the program is architectural software and it can generate into the environment anything you are designing. Suppose you have it put a house into the virtual space, and all the components of the house are interactive – you can click on a door and change the type of door; you can click on the floor and choose a new kind of tile from an actual inventory database out there on the net. Imagine interacting with a molecular model of a protein and when you click on various amino acids and ligands they present menu options for modifying the model or cross-referencing with others.

Imagine a program which searches the web for you. The program’s avatar looks like a librarian and follows you around, and when you say “find me the most authoritative website on bodybuilding” it injects a web page into the virtual environment. Imagine interacting with software as you would with a personal assistant.

All this is possible through the use of protocols such as CICP and MXP, which allow external entities to place interactive content into virtual worlds. All the logic and machinery stays outside the world and just the user interfaces (objects) go inside the world for you to interact with. This is the vision that led me to create CICP – and the particular way Green Phosphor is using CICP is to create interactive representations of data.

Green Phosphor’s product is called Glasshouse, and it is a gateway which can take a database query or a spreadsheet and place a 3D representation of it into a virtual world. Users can see data, and drill into it; re-sort it; explore it interactively – all from within a virtual world. Glasshouse produces graphs which are avatars of the data itself. We’ve tailored the system for the use of biotech companies, specifically for drug discovery and development. Dr. David Resuehr, a molecular biologist, recently joined Green Phosphor as our Chief Scientist. We’ve just now entered our marketing phase – for more information please visit http://greenphosphor.com/?location=Biotech.

glasshouse_oil_net_with_avatars

The power of interacting with data in 3D has to be experienced to be believed. Some day soon many people will have jobs which require SQL skills and virtual world skills; these ”data wizards” will turn information into intelligence within the 3D environment. They’ll facilitate a data-driven decision-making process which will hopefully reduce some of the short-sightedness and stupidity we’ve seen lately in many of our critical institutions.

Jani: What do you refer to when you say: “…will hopefully reduce some of the shortsightedness and stupidity we’ve seen lately in many of our critical institutions.”? Could you give an example?

Ben: I have a couple examples which I’m sure won’t offend anyone and a third which may. First off I believe that visualization of money in/money out could have turned authorities on to the fishy accounting Enron was up to, and caught them earlier. Perhaps better visualization would have revealed Madoff’s ponzi scheme as well. It’s a matter of seeing the big picture; money must come from somewhere and be tied to intrinsic value at some point. My third example is similar, but rather than money in/money out it is about energy in/energy out. I believe proper visualization of the process for producing ethanol fuel, including energy cost of fertilizer, would reveal the uselessness of subsidizing cornbased ethanol as an alternative to fossil fuels. Much of the value of looking at data has to do with getting in touch with history rather than trying to look at realtime feeds with everfiner granularity. There is tremendous value in looking at the past in objective ways – and the best objectivity comes from hard, unadulterated data. Combine data visualization and visible process simulation with massive collaboration – crowdsourcing, masterminds formed by teams – and the human race can be more intelligent.

Jani: I’d like to get a quote from Dr. David Resuehr.

DrDavid: For science, the 3-D environment may really have a very prosperous future. We know that the discovery of many drugs was more or less coincidental like for instance when Fleming discovered Penicillin. Others were specifically and tediously designed, tested and created. Insufficient testing can – as it happened with Thalidomide – have terrible consequences. During the 50’s and early 60’s thousands of children in Africa and Europe were born with severe malformations because their mothers had been given this drug against morning sickness.
What does all this have to do with virtual worlds and 3D environments? The connection lies in the fact that molecules are not flat but have a (sometimes very elaborate) three dimensional structure. Here is the anecdote: Thalidomide exists in two  isomers (enantiomers), which are mirror images of each other, like your right and left hand. The problem was (as was found out later) that one of the versions, the ‘S’ enantiomer, was teratogenic [causes malformations]; but the ‘R’ enantiomer was an effective sedative. I like to refer to the ‘S’ enantiomer as the ”evil twin”. Had the developing scientists been able to display and most of all discuss this and its possible implications with their peers, maybe they would have been more diligent and could have prevented the release of this cocktail and its devastating effects for thousands of people. Clearly it is impossible to recognize and tell apart a “bad” molecule from a good one by just looking at it, but having it dangling right in front of your nose in cyberspace may make you think about it a little more thoroughly and prime discussions about it.

DrDavid: Virtual worlds like Wonderland and OpenSimulator offer never before available possibilities for researchers from all fields to interact with each other and display and scrutinize their data in new and more intuitive ways. This platform also offers a link from the people in the lab to the people at the desks and enables everyone to give their input, from wherever in the world they are – many brains are better than one and sometimes it may take a non-scientist to come up with a great idea; after all it’s about collaboration and working as one strong team.

DrDavid: A long-term goal is, in collaboration with bioinformatics and cell biology specialists, to create a complete virtual cell. If we can accurately display and simulate cellular pathways, we may be able to better understand drugs’ molecular actions, explain adverse effects, predict risks, and discover potential therapeutical targets for the development of new drugs.

Ben: I believe one of the keys to intelligence is the elimination of bias and preconceptions. This elimination opens the door to truth. One of the great features of looking at raw data in a virtual world is you can start without any labels or explanations of what the data is. You simply see the patterns, the outliers… then click in and see what the actual datapoints are, and be prepared to accept what is revealed. I’d like to close by mentioning a pilot project in Second Life which is using the free version of Glasshouse to show the effects of various companies’ customer loyalty efforts on revenue. The data was provided by Leslie Pagel and the graphs are on display at Gronstedt Group’s Train for Success2 region. Here’s a slurl: http://slurl.com/secondlife/Business%201/211/48/33.

Darkstar, OpenSim, realXtend , , , , , , , , , , , ,

Metaverse Exchange Protocol (MXP) – Enabling default bone structures and real time gesturing for OpenSimulator

March 7th, 2009

OpenSimulator community controlled protocol extensions like Open Metaverse Structured Data (OMSD) extension for Metaverse Exchange Protocol (MXP) and custom viewers enable long lived dreams to be realized.  One of these dreams is real time recorded and played gestures. This vision is fast becoming reality as dependencies to 3rd party protocols and viewers are not blocking innovation anymore. Traditional input devices or full blown motion tracking devices can be integrated to viewers for real time recording of gestures.

OpenSimulator is currently in process of adopting Metaverse Exchange Protocol (MXP) as alternative protocol for Client to Server communication. The ideas and design principles for MXP originate from Internet (see: MXP Background Reading) and two protocols specifications: SETP and CICP (Ben Lindquist / Green Phosphor). The modular architecture of OpenSimulator allows for multiple protocols to be supported simultaneously.  In other words SecondLife breed of clients and MXP enabled clients (IdealistViewer) can interact in the same simulation.

Open Metaverse Structured Data (OMSD) schema contains required MXP extension fragments for transmitting poses for default biped bone structures and custom bone structures. Smooth gesturing is achieved by interpolating frames between different poses (key frames) with simple well known algorithms. Tomorrow you can greet you friend with improvised moves and bone driven facial expressions are not far behind.  Open source virtual world systems are now taking a step ahead of their commercial game counterparts.

Figure 1. Illustration of Open Metaverse default biped bone structure. (Nathan Vegdahl's public domain biped.)

Figure 1. Illustration of Open Metaverse default biped bone structure. (Nathan Vegdahl's public domain biped.)

Open Metaverse default skeleton originates from realXtend project which was first OpenSimulator compliant platform to support custom character meshes. RealXtend extensions are available as OpenSimulator module (Modrex) and OpenSimulator extensions to MXP are compliant with realXtend character skeletons.

As OpenSimulator technology advances other areas affecting end user experience gain momentum. Avatar model quality is an immersion factor and modeling humans requires both time and talent. Community effort has been initiated to create creative commons licensed male and female archetype avatars for OpenSimulator and compliant systems. We are inviting all open collaboration spirited modeling artists to join this push towards next level of Metaverse experience.

OpenSim, realXtend , , , , , , , , ,

Green Phosphor gives CICP to Public Domain

February 16th, 2009
Comments Off

The purpose of this article is to publish a provisional patent application I filed back in March 2008. The provisional covers Content Injection and Control Protocol, which I created to allow Green Phosphor’s data visualization application to interface with virtual world platforms (see previous article on CICP here).

I recently asked the Slashdot readership whether I should fork out the additional $10k it would take to properly complete the app, since I am releasing CICP to the public domain anyway. Despite a large number of trolls saying I was stupid to ask such a question on Slashdot, I received a lot of very good advice; and it sums up to this: publish the work, and as prior art it protects the protocol. I think this is important. Not only is CICP useful for business applications to produce 3D user interfaces within virtual worlds; it is a predecessor to MXP (see previous article on MXP here). MXP has the potential to tie together many worlds and programs into one metaverse. The MXP project is seeking developers… go to http://www.bubblecloud.org.

Without further ado, here’s the publishing of the provisional patent application for CICP. CICP is for all.

Darkstar, OpenSim, realXtend , , , , , , , , ,

MXP: a universal Metaverse eXchange Protocol

January 30th, 2009

Tommi S. E. Laukkanen and I have merged the next versions of our protocols into something called Metaverse eXchange Protocol (MXP). This is a combination of the principles of CICP (see Ben’s earlier article about CICP), my protocol for allowing external entities to place interactive content into virtual worlds; and SETP, Tommi’s UDP-based protocol for efficient client->server and server->server interaction.

MXP begins with first principles, which are described below. Following the discussion of first principles is a list of terms which we consider “foundational definitions” for MXP; following that list is a description of the messages comprising MXP.

bubbles1

First Principles

We begin with the principle that a virtual world need only contain things which are perceivable. This is the key to interoperability. Logic, process, and internal object attributes exist outside the world. Just as knowledge and thought exist within the minds of humans and are only evidenced when they cause something perceivable to happen, such as a gesture or movement, or a sound; so the humans and programs participating in a virtual world are using the world only as a medium of expression and perception. Concepts such as internal object state, scripting, and logic belong to the participants, not the world.

Our next principle is one of layering. The communication necessary for a virtual world to provide a shared experience among participants takes place using a transport layer and a content definition layer. The transport layer provides a mechanism by which participants may make each other aware of the presence of objects, and then allow participants to transmit messages among their objects. The content definition layer provides the means for participants to publish any specific details and/or assets which define the appearance of their objects, the capabilities of their objects, the available options for interacting with their objects, and the state of any external attributes of their objects.

The layering principle allows us to limit the scope of MXP. MXP provides the transport layer and only enough of the content definition layer to enable the expression of the states of mutable, public attributes of objects. The content definition layer will specify the format and meaning of messages transmitted from object to object using MXP, and will provide a means for participants to publish and fetch content assets associated with objects. It is expected that the content definition layer will utilize XML for encoding of messages and definitions, and that assets and definitions will be fetched using HTTP. To extend the analogy to HTTP and the World Wide Web: the transport layer (MXP) is equivalent to HTTP, and the content definition layer is equivalent to MIME types, html, and other content.

To continue reading, please download the pdf version of the article.

Uncategorized , , , ,

Content Injection and Control Protocol (CICP): HTTP for Virtual Worlds

January 22nd, 2009
Comments Off

My involvement with virtual world protocol design began with a very practical need.  I had used Second Life to develop a prototype of a gateway for querying a database and producing a three-dimensional representation of the query result within the virtual world.

For business reasons, I wanted my gateway to be able to work not only with Second Life, but also with Sun’s Wonderland, Forterra’s OLIVE, OpenSim, realXtend, and any other virtual world platform.  This cross-platform requirement caused me to design a protocol called Content Injection and Control Protocol, and to implement the protocol in both Second Life and Sun Wonderland.  I call the program which implements the protocol for a given virtual world platform an “adapter” for that platform.

In addition to the cross-platform benefit of the protocol, which enabled me to create one version of my gateway which is able to work simultaneously with both Second Life and Wonderland, I discovered another significant benefit: the ability to shield a proprietary, closed-source program from attachment by Second Life’s Terms of Service, and Wonderland’s Gnu Public License.

By releasing the protocol itself to the public domain, conforming to Second Life’s Terms of Service with the Second Life adapter, and contributing the Wonderland adapter back to the Wonderland project for release under GPL, I created a communication mechanism which my gateway can use to generate temporary artifacts in the worlds. The gateway itself never resides within a virtual world and is never linked to a virtual world at a code or library level.  Rather, it simply communicates with CICP adapters using sockets and a simple text-based protocol much like HTTP. It is of course important that I do not desire to protect the temporary artifacts, which happen to be representations of data, from any intellectual property issues; I am only concerned with protecting the algorithms and processes contained within the gateway program.

port_many

I believe the most useful aspect of my CICP work is a way of looking at virtual worlds from the perspective of an external application which must place content into them for viewing and interaction by other entities (users) who are also external to the virtual worlds.

Looking at a virtual world from this perspective can lead to the development of a universal protocol: an HTTP for virtual worlds.  There are few who would dispute the usefulness of HTTP in the development of the World Wide Web.  The presence of a well-defined protocol has enabled various groups to develop client software (browsers, for example) and server software (web servers, for example) which work together.  Closed-source servers work just fine with open-source clients, and vice-versa; one company’s client software works well with another company’s server software.  One of the aspects of HTTP which enables this interoperability is its simplicity.  HTTP is a very simple protocol and is concerned with the mechanism of transferring commands and content from one place to another, rather than with the meaning or structure within the content.

For more information please visit http://greenphosphor.com/?location=Developers

Darkstar, OpenSim, realXtend, wonderland , , , , , , , , ,