Neural network powered email inbox in virtual world
This is how your email inbox could look like if it was automatically categorized by a SOM – a type of artificial neural network – and fed into realXtend virtual world platform. You can walk or fly, even teleport around a beautiful landscape, knowing that each of the snow covered hills is actually a bunch of emails. Goodbye to the messy and overflowing inbox!
A self-organizing map (SOM) is a type of artificial neural network that is trained using unsupervised learning. One possible use for SOM is to automatically categorize data. The data categorization can be visually interpreted from the SOM U-matrix graph.
Here is a experiment I did together with Tony Manninen (Ludocraft CEO). We fed hundreds of emails into SOM in order to find a way to automatically categorize incoming emails. Now, years later I took the U-matrix and loaded it as a terrain file to realXtend virtual world platform.
Here I am standing on a virtual hill that represents a category for emails from the internal mailing list, the big hill in the front represents a category for MSc studies related emails, and so forth.
I used water level to better visualize areas that are below certain point, and different textures at different heights. As you could guess, the categories are the hills and valleys separate categories from each other. If I was to really present this to someone, I would set up flags to mark different hills and we would fly from hill to hill to get a feeling of the whole landscape.
Below you can see the actual U-matrix that was used to generate the terrain. The screenshot is taken from the up left corner, watching to about south-east.
To get full potential from this kind of visualization, it should be easy to find out examples of emails that fall in a certain point etc. That would require further work, perhaps writing a region module or MRM module to realXtend or Opensim.
The benefit from this kind of visualization is that it is a lot easier to get the feeling of the terrain while walking/flying around rather than looking 2D grayscale image. Also for a research workgroup it is surely easier to focus on some oddities together and discuss about their meaning when they can literally walk around in the data.
Here’s the terrain file, if you want to give it a go. I removed labels and blurred the image a bit to avoid very sharp edges on the result. After loading, I multiplied the height values by 0.3 and adjusted the height to fit the water level nicely. Use “terrain help” at the console to get list of all the commands. Then I changed the terrain textures and heights to emphasize hills visually. Get an empty realXtend server from the files section.
Artificial Intelligence, OpenSim, realXtend
;-O
Apart from being a technical achievement, there is a definite poetic beauty in this development.
I can see in the future the creation of fractal trees and rocks in function of wich type of data is chosen, birds and other animals using other class, interaction through nature ooooohh ¡Olé!
Yes, could have flowers sprouting where new email has been received, dog p**p for spam, etc.
It is interesting to think how landscapes, scenes and places with which we are familiar with from nature .. and thus have the mental machinery for understanding them, could be used for information representation, helping to deal with the complexities in a peaceful manner.
Packet Garden is a few years old interesting experiment that actually kind of has flowers sprouting when email arrives, http://packet-garden.en.softonic.com/ (uses soya3d, a pyrex-written gpl 3d game engine for python) “Packet Garden is an experimental project which allows you to visit your internet usage in a 3D generated world. The application works by capturing packets from your daily internet traffic, from which it creates a 3D garden where websites you visit are represented as plants and mountains. Basically, the more you visit a site the bigger a mountain it becomes”
We also made one demo quality ambient visualization thing using Ogre two years ago, a scenery with real-time water etc. that reacted to local sensor information like light and temperature.
Now have been thinking how opensim/realxtend developers could ‘eat our own dogfood’, i.e. what kind of a virtual place would help us in the global collaboration on platform development .. have been thinking of viewing the developing code trees in a garden, seen forks and branches there perhaps, and bugs and perhaps fruits as releases etc. Have some notes about it but close to nothing yet, http://www.playsign.fi/FirstAndLast/AnGarden .. looking forward to actually working on that at some point.
In the Otherland series of scifi books from Tad Williams, there is a guy who visualizes all the data from the Internet as a virtual garden. Recommended reading!