Archive

Posts Tagged ‘protocol’

Real time application sharing for virtual meetings

March 27th, 2009

realXtend platform offers a way to share live applications over the network to realXtend users. Effective virtual world meetings usually require that you can show your slides (here’s how to do that). It may also help to browse web so that others can view the same web page with you. Sometimes you may want to show how to use an application in real time.

In this example we use VNC (Virtual Network Computing – a simple and working desktop & application sharing system) to share the desktop. 

vncrex

Firefox window shared using tightVNC server and viewed in realXtend.

VNC allows also sharing the control between participants, and users can also join using a standard VNC client, if they do not want to use full virtual world to join the meeting.

In this example we use localhost, but I have tested that VNC works also over the network. In a real world situation I would set up another computer to serve as a VNC server – otherwise running the realXtend client on the same computer is a bit awkward as you need to have the shared content visible on screen to share it. Below is a screenshot how sharing the realXtend viewer window, which is looking at the shared content, causes infinite mirror -effect.

Watching your shared desktop getting shared, getting shared, ...

Watching your shared desktop getting shared, getting shared, ...

Follow these simple steps to test VNC application sharing with realXtend.

  • Install and run tightVNC server; Everything should work with other VNC servers as well, but I have tested this only using TightVNC.
  • Adjust the settings as in the following screenshots:

rexvnc_server1

Set the view-only passport, all the connecting users need to know it.

rexvnc_server2

rexvnc_server3

Be careful to have the settings exactly as here, change them after you get them to work for the first time. 

 

  • Bring a new texture to realXtend (file->upload)
  • Create a new cube (right click on ground -> create)
  • Edit cube and set the new texture on it (you can also use the texture on meshes for an interesting effect! Who doesn’t want to play doom and use a virtual sofa for the display?)
  • Edit texture properties (find it from your inventory and right click it) and edit the mediaurl:

 

vncrex_mediaurl

  • Edit the cube again and set it to play media when clicked:

vncrex_mediaplay

  • Now close the edit window and click the cube to get the VNC run, you will get a dialog asking for the password (which you defined for the server earlier):

vncrex_connectTop left dialog is the server telling that someone is connecting. 

Warning! After you hit enter, tightVNC client is launched – it is embedded to the realXtend viewer code – do not close it! Just bring your realxtend viewer to front and you see desktop sharing in action. You can use the tightVNC viewer to get the control over shared application, if the server owner grants you permission to do that.

vncrex_onmesh

Current VNC implementation is a prototype and it has known bugs. User can start one VNC session and after it is finished, the viewer needs to be restarted to be able to start a new VNC session. Users also need to be warned in advance so that they do not close the VNC viewer as it pops up after starting to view VNC.

realXtend , , , , , , , ,

Glasshouse injects 3D representation of data into a virtual world

March 14th, 2009

Glasshouse by Green Phosphor is a gateway which can take a database query or a spreadsheet and place a 3D representation of it into a virtual world. Users can see data, and drill into it; re-sort it; explore it interactively – all from within a virtual world. Glasshouse produces graphs which are avatars of the data itself. 

glasshouse_sl_scatter

Virtual worlds are environments where avatars can observe and interact with eachother. This is a wonderful advance for human to human interaction; and it is also a wonderful advance for human to program interaction. Imagine a useful program residing somewhere on the Internet… and to interact with it, all you need to do is invite its avatar into a virtual world with you. The avatar of the program is its 3D user interface. 

Imagine the program is architectural software and it can generate into the environment anything you are designing. Suppose you have it put a house into the virtual space, and all the components of the house are interactive – you can click on a door and change the type of door; you can click on the floor and choose a new kind of tile from an actual inventory database out there on the net. Imagine interacting with a molecular model of a protein and when you click on various amino acids and ligands they present menu options for modifying the model or cross-referencing with others.

Imagine a program which searches the web for you. The program’s avatar looks like a librarian and follows you around, and when you say “find me the most authoritative website on bodybuilding” it injects a web page into the virtual environment. Imagine interacting with software as you would with a personal assistant.

All this is possible through the use of protocols such as CICP and MXP, which allow external entities to place interactive content into virtual worlds. All the logic and machinery stays outside the world and just the user interfaces (objects) go inside the world for you to interact with. This is the vision that led me to create CICP – and the particular way Green Phosphor is using CICP is to create interactive representations of data.

Green Phosphor’s product is called Glasshouse, and it is a gateway which can take a database query or a spreadsheet and place a 3D representation of it into a virtual world. Users can see data, and drill into it; re-sort it; explore it interactively – all from within a virtual world. Glasshouse produces graphs which are avatars of the data itself. We’ve tailored the system for the use of biotech companies, specifically for drug discovery and development. Dr. David Resuehr, a molecular biologist, recently joined Green Phosphor as our Chief Scientist. We’ve just now entered our marketing phase – for more information please visit http://greenphosphor.com/?location=Biotech.

glasshouse_oil_net_with_avatars

The power of interacting with data in 3D has to be experienced to be believed. Some day soon many people will have jobs which require SQL skills and virtual world skills; these ”data wizards” will turn information into intelligence within the 3D environment. They’ll facilitate a data-driven decision-making process which will hopefully reduce some of the short-sightedness and stupidity we’ve seen lately in many of our critical institutions.

Jani: What do you refer to when you say: “…will hopefully reduce some of the shortsightedness and stupidity we’ve seen lately in many of our critical institutions.”? Could you give an example?

Ben: I have a couple examples which I’m sure won’t offend anyone and a third which may. First off I believe that visualization of money in/money out could have turned authorities on to the fishy accounting Enron was up to, and caught them earlier. Perhaps better visualization would have revealed Madoff’s ponzi scheme as well. It’s a matter of seeing the big picture; money must come from somewhere and be tied to intrinsic value at some point. My third example is similar, but rather than money in/money out it is about energy in/energy out. I believe proper visualization of the process for producing ethanol fuel, including energy cost of fertilizer, would reveal the uselessness of subsidizing cornbased ethanol as an alternative to fossil fuels. Much of the value of looking at data has to do with getting in touch with history rather than trying to look at realtime feeds with everfiner granularity. There is tremendous value in looking at the past in objective ways – and the best objectivity comes from hard, unadulterated data. Combine data visualization and visible process simulation with massive collaboration – crowdsourcing, masterminds formed by teams – and the human race can be more intelligent.

Jani: I’d like to get a quote from Dr. David Resuehr.

DrDavid: For science, the 3-D environment may really have a very prosperous future. We know that the discovery of many drugs was more or less coincidental like for instance when Fleming discovered Penicillin. Others were specifically and tediously designed, tested and created. Insufficient testing can – as it happened with Thalidomide – have terrible consequences. During the 50’s and early 60’s thousands of children in Africa and Europe were born with severe malformations because their mothers had been given this drug against morning sickness.
What does all this have to do with virtual worlds and 3D environments? The connection lies in the fact that molecules are not flat but have a (sometimes very elaborate) three dimensional structure. Here is the anecdote: Thalidomide exists in two  isomers (enantiomers), which are mirror images of each other, like your right and left hand. The problem was (as was found out later) that one of the versions, the ‘S’ enantiomer, was teratogenic [causes malformations]; but the ‘R’ enantiomer was an effective sedative. I like to refer to the ‘S’ enantiomer as the ”evil twin”. Had the developing scientists been able to display and most of all discuss this and its possible implications with their peers, maybe they would have been more diligent and could have prevented the release of this cocktail and its devastating effects for thousands of people. Clearly it is impossible to recognize and tell apart a “bad” molecule from a good one by just looking at it, but having it dangling right in front of your nose in cyberspace may make you think about it a little more thoroughly and prime discussions about it.

DrDavid: Virtual worlds like Wonderland and OpenSimulator offer never before available possibilities for researchers from all fields to interact with each other and display and scrutinize their data in new and more intuitive ways. This platform also offers a link from the people in the lab to the people at the desks and enables everyone to give their input, from wherever in the world they are – many brains are better than one and sometimes it may take a non-scientist to come up with a great idea; after all it’s about collaboration and working as one strong team.

DrDavid: A long-term goal is, in collaboration with bioinformatics and cell biology specialists, to create a complete virtual cell. If we can accurately display and simulate cellular pathways, we may be able to better understand drugs’ molecular actions, explain adverse effects, predict risks, and discover potential therapeutical targets for the development of new drugs.

Ben: I believe one of the keys to intelligence is the elimination of bias and preconceptions. This elimination opens the door to truth. One of the great features of looking at raw data in a virtual world is you can start without any labels or explanations of what the data is. You simply see the patterns, the outliers… then click in and see what the actual datapoints are, and be prepared to accept what is revealed. I’d like to close by mentioning a pilot project in Second Life which is using the free version of Glasshouse to show the effects of various companies’ customer loyalty efforts on revenue. The data was provided by Leslie Pagel and the graphs are on display at Gronstedt Group’s Train for Success2 region. Here’s a slurl: http://slurl.com/secondlife/Business%201/211/48/33.

Darkstar, OpenSim, realXtend , , , , , , , , , , , ,

How ReactionGrid will Expand it’s HyperGalaxy

March 6th, 2009

(via ReactionGrid)

A question I full well knew was coming recently cropped up. ReactionGrid has promised a closer touch with it’s Gridizens, or virtual world inhabitants. How will ReactionGrid both technically handle the load of more Gridizens using the grid and how will you keep that personal touch as you expand in size?

The answer is HyperGrid technology recently introduced to OpenSim, the platform ReactionGrid is based on. HyperGrid is the start of what can be reduced to as being HyperLinks for 3D Virtual Worlds. This means we can keep ReactionGrid as a portal grid, like MSN, Yahoo etc. We can then branch off smaller grids connected to us via HyperGrid portals.

For example we recently had a growth spurt in K-12 educational simulators. As these expand on ReactionGrid we will form a partnership with one or more of the educators who will help us manage the new, completely isolated, spinoff grid focused on K-12 that can allow traffic to and from ReactionGrid via HyperGrid. This allows the grid to maintain its own local OpenSim database of users, scripts, prims, everything.

When this specialized grid wants to connect to ReactionGrid this is available to them at anytime. After the visit they can the de-establish the HyperGrid portal and know they are secure from any traffic other than their local users in their local database. All of this travelling carries clothes, hair, all inventory and requires no login from ReactionGrid at all.

I will be hosting an event soon to explain this technology and it can be experienced on our grid right now by going to the HyperGrid Core sim using our inworld map.

Below is a very early, still formative plan on how ReactionGrid will partner with specialists in each category of sims to expand and in fact bring a smarter personal touch over time as the managers we partner with will know much more than we about their profession/industry. I will translate the image below into a 3D model next that with some new HyperGrid tricks just coming out will make it a clickable model for traversing the Metaverse.

 

hg1sflbashx

Watch soon for an event to show HyperGrid to you personally when the 3D portal translation of the image above is complete.
Kyle “G” aka Dr_Manhattan

OSGrid, OpenSim , , , , ,

Opensim compatibility and MXP

February 22nd, 2009
Comments Off

So, I have a small announcement to make.

OpenSim has had the capability to run multiple different clients connected to the same world since version 0.3. We translate a large number of our internal programming into a somewhat vendor neutral interface; the big catch has been – in the 21 releases since 0.3, we’ve never actually implemented a true second protocol. As of today that has changed, OpenSim now supports two client protocols that can be used simultaneously at the same time to display the same world with the same updates, avatars and interactivity. This means right now you can use either the LLUDP transport used by Second Life™, OpenSim and realXtend, or the new MXP transport developed by the Tommi over at Bubblecloud.org.

We also have some good news from the guys over at VastPark – over the next few days, I will be working with their team to try write another client transport which allows you to use the VastPark viewer to connect to OpenSim. Watch this space for some information about that.
MX Deck and Second Life in a single Region (Click to Enlarge) MX Deck and Second Life connected to the same region (Click to Enlarge)

Why MXP?

The simple answer here is the protocol is reasonably well thought out (it’s very refreshing compared to LLUDP), it’s also pretty simple and easy to understand. There’s a good reference implementation on their website which we were able to utilize extensively. To users, there’s potentially some big advantages in how you can route packets from a region simulator to viewers – possibly eliminating some of the issues people see when trying to host from behind firewalls and routers. This is however conceptual – a decent client will still need to be written to utilize the MXP protocol (right now the only clients using it are for demonstration purposes).

The big reasons were however that it’s appropriately licensed, there’s good reference code out there and the protocol is straight forward. It was also in the right place at the right time — while it gets the crown for first additional protocol, we’re hoping to have quite a few more now the infrastructure is in place and tested to properly support alternate client libraries.

How complete is this implementation?

Not very – we implement at the moment about 5 of 250 “functions” the Second Life viewer can present to us, ontop of the basic connection management. Mostly these are relating to display of primitives, updating their positions and chat. We still need to do a good deal towards displaying avatars correctly (and assigning a ‘ruth’ for each clients display of the other). Given the adapter is however less than 48 hours old – I expect to see some rapid progress here in adding features and capabilities to it. To give some idea of what’s working however below is some screenshots of the same region connected to in each client.

How do I connect a MXP client to OpenSim?

Grab a copy of the Demo Client from Bubblecloud.org (see Downloads -> MXP Demo Client.zip) and a copy of the latest release of OpenSim from the OpenSim Trunk SVN.

In your OpenSim.ini, add the following settings to the end

[MXP]
Enabled = true
Port = 1253

Start up your OpenSim and create a region if you havent already – now go into your OpenSim Region’s directory and open your default.xml – cut and paste the sim UUID (looks like this: b18ddeb1-f3d4-4c05-aa55-f1ca55532220 )

Install the MXP Client and edit the MX Deck.exe.config file – you need to paste your region UUID into the “BubbleID” field, and edit the “ServerAddress” to read “127.0.0.1″

Start MX Deck.exe and you should be connected to your OpenSim region.

Voila!

Note from CTN editor: Check CTN post about MXP, and also about CICP (MXP is based on CICP), and how CICP is now Public Domain

OpenSim , , , , , ,

Green Phosphor gives CICP to Public Domain

February 16th, 2009
Comments Off

The purpose of this article is to publish a provisional patent application I filed back in March 2008. The provisional covers Content Injection and Control Protocol, which I created to allow Green Phosphor’s data visualization application to interface with virtual world platforms (see previous article on CICP here).

I recently asked the Slashdot readership whether I should fork out the additional $10k it would take to properly complete the app, since I am releasing CICP to the public domain anyway. Despite a large number of trolls saying I was stupid to ask such a question on Slashdot, I received a lot of very good advice; and it sums up to this: publish the work, and as prior art it protects the protocol. I think this is important. Not only is CICP useful for business applications to produce 3D user interfaces within virtual worlds; it is a predecessor to MXP (see previous article on MXP here). MXP has the potential to tie together many worlds and programs into one metaverse. The MXP project is seeking developers… go to http://www.bubblecloud.org.

Without further ado, here’s the publishing of the provisional patent application for CICP. CICP is for all.

Darkstar, OpenSim, realXtend , , , , , , , , ,

MXP: a universal Metaverse eXchange Protocol

January 30th, 2009

Tommi S. E. Laukkanen and I have merged the next versions of our protocols into something called Metaverse eXchange Protocol (MXP). This is a combination of the principles of CICP (see Ben’s earlier article about CICP), my protocol for allowing external entities to place interactive content into virtual worlds; and SETP, Tommi’s UDP-based protocol for efficient client->server and server->server interaction.

MXP begins with first principles, which are described below. Following the discussion of first principles is a list of terms which we consider “foundational definitions” for MXP; following that list is a description of the messages comprising MXP.

bubbles1

First Principles

We begin with the principle that a virtual world need only contain things which are perceivable. This is the key to interoperability. Logic, process, and internal object attributes exist outside the world. Just as knowledge and thought exist within the minds of humans and are only evidenced when they cause something perceivable to happen, such as a gesture or movement, or a sound; so the humans and programs participating in a virtual world are using the world only as a medium of expression and perception. Concepts such as internal object state, scripting, and logic belong to the participants, not the world.

Our next principle is one of layering. The communication necessary for a virtual world to provide a shared experience among participants takes place using a transport layer and a content definition layer. The transport layer provides a mechanism by which participants may make each other aware of the presence of objects, and then allow participants to transmit messages among their objects. The content definition layer provides the means for participants to publish any specific details and/or assets which define the appearance of their objects, the capabilities of their objects, the available options for interacting with their objects, and the state of any external attributes of their objects.

The layering principle allows us to limit the scope of MXP. MXP provides the transport layer and only enough of the content definition layer to enable the expression of the states of mutable, public attributes of objects. The content definition layer will specify the format and meaning of messages transmitted from object to object using MXP, and will provide a means for participants to publish and fetch content assets associated with objects. It is expected that the content definition layer will utilize XML for encoding of messages and definitions, and that assets and definitions will be fetched using HTTP. To extend the analogy to HTTP and the World Wide Web: the transport layer (MXP) is equivalent to HTTP, and the content definition layer is equivalent to MIME types, html, and other content.

To continue reading, please download the pdf version of the article.

Uncategorized , , , ,

Content Injection and Control Protocol (CICP): HTTP for Virtual Worlds

January 22nd, 2009
Comments Off

My involvement with virtual world protocol design began with a very practical need.  I had used Second Life to develop a prototype of a gateway for querying a database and producing a three-dimensional representation of the query result within the virtual world.

For business reasons, I wanted my gateway to be able to work not only with Second Life, but also with Sun’s Wonderland, Forterra’s OLIVE, OpenSim, realXtend, and any other virtual world platform.  This cross-platform requirement caused me to design a protocol called Content Injection and Control Protocol, and to implement the protocol in both Second Life and Sun Wonderland.  I call the program which implements the protocol for a given virtual world platform an “adapter” for that platform.

In addition to the cross-platform benefit of the protocol, which enabled me to create one version of my gateway which is able to work simultaneously with both Second Life and Wonderland, I discovered another significant benefit: the ability to shield a proprietary, closed-source program from attachment by Second Life’s Terms of Service, and Wonderland’s Gnu Public License.

By releasing the protocol itself to the public domain, conforming to Second Life’s Terms of Service with the Second Life adapter, and contributing the Wonderland adapter back to the Wonderland project for release under GPL, I created a communication mechanism which my gateway can use to generate temporary artifacts in the worlds. The gateway itself never resides within a virtual world and is never linked to a virtual world at a code or library level.  Rather, it simply communicates with CICP adapters using sockets and a simple text-based protocol much like HTTP. It is of course important that I do not desire to protect the temporary artifacts, which happen to be representations of data, from any intellectual property issues; I am only concerned with protecting the algorithms and processes contained within the gateway program.

port_many

I believe the most useful aspect of my CICP work is a way of looking at virtual worlds from the perspective of an external application which must place content into them for viewing and interaction by other entities (users) who are also external to the virtual worlds.

Looking at a virtual world from this perspective can lead to the development of a universal protocol: an HTTP for virtual worlds.  There are few who would dispute the usefulness of HTTP in the development of the World Wide Web.  The presence of a well-defined protocol has enabled various groups to develop client software (browsers, for example) and server software (web servers, for example) which work together.  Closed-source servers work just fine with open-source clients, and vice-versa; one company’s client software works well with another company’s server software.  One of the aspects of HTTP which enables this interoperability is its simplicity.  HTTP is a very simple protocol and is concerned with the mechanism of transferring commands and content from one place to another, rather than with the meaning or structure within the content.

For more information please visit http://greenphosphor.com/?location=Developers

Darkstar, OpenSim, realXtend, wonderland , , , , , , , , ,