ESME Blog

We are currently collecting ideas about how to best collaborate with #ubimic.

#ubimic is a group of university researchers interested in microblogging from different points of view (computer science, information systems, organizational science). Together we think that microblogging has great potential when it comes to integrating normal users and things like sensors, machines and software. We are experimenting with different microblogging tools and scenarios.

There are interesting collaboration possibilitites and we’ve already started describing potential uses cases where we might be able to collaborate. Examples of these use cases are:

• Use Messages for Back-end Requests
• Microblogging SAP System
• Task management
• Device-related Use Case (RFID)

The big picture of the collaboration efforts might look like this:

Take a look at our wiki page for more details about our collabortive efforts.

ESME got a nice mention in Mark Yolton’s (Senior Vice President of the SAP Community Network ) keynote at “Open Source Meets Business” (OSMB) conference. Take a look at Mark’s blog about his presentation (slides are here). The slides are interesting, because they describe the changing relationship between open- and closed-source software. This discussion is obviously relevant to the manner in which ESME evolves.

In a previous blog, I described our use of the Stax Cloud to perform load tests for ESME. Well, Daniel Koller has just finished his analysis of the results of this initial test series and has made a few interesting discoveries. For the ESME team, these initial tests were primarily focused on gaining experience with Stax as an environment to perform such tests (including the use of clusters) as well as creating a test bed for future ESME tests.  The tests used both the REST-API and the Web UI.

Now you may be thinking the main focus should be on measuring ESME performance. True. However, the ESME version that we used is based on a older Scala library with a known memory bug. Thus, the test results are not really representative of the current ESME code base.

If you look at other microblogging platforms (including Twitter) – irregardless of whether they are focused on the enterprise or not, you’ll be hard pressed to find any load test results that are published and available to potential users. However, such tests are usually mandatory for IT projects, especially for larger companies with many potential users.  Thus, we would like start publishing our results so that those interested in ESME can make better sizing decisions.

We will soon be starting a new round of load tests based on the Apache code base and a new Scala library, so expect a blog in the future with these results.

For those interested in performing the tests themselves, Daniel has also made the test scripts available

Now that ESME works in the cloud, I was thinking about ways to take this existing installation and going viral with it so that a variety of cloud-based ESME  installations could be easily created.   The following steps would be necessary:

1. Download existing ESME source code from Stax
2. Create new application in Stax via the application console
3. Download the new application into the local installation
4. Create new database in Stax via the application console for the new ESME application
5. Copy the existing ESME source into the deployment directories of the new application
6. Change the source code to meet the basic style configurations of the new ESME installation
7. Change data-source configuration of the new application to point to new database
8. Upload the new application

 One shortcoming with this description is that as owner of the original ESME application, only I have the ability to create copies of the current installation.  At the current time,  just individuals who I invite to join my Stax application team have access to the ESME source deployment. Ideal would be the ability to create public clones of Stax-based deployments.  Then, other Stax users could very easily create new cloud-based ESMEs.  Although ESME is open-source and the source is available to all via the Apache repository, it would be easier if this was possible via Stax itself.

As a ESME supplier (someone who creates, maintains and/or supports cloud-based ESME installations for others), these steps could be patched together to create a process to very rapidly create micro-blogging sites for customers. The only task that requires more effort would be the customer-specific adaptions regarding styles, etc.

Dennis Howlett recently sent the ESME team a link to a long article about VAXNotes which was a collaboration tool that was active in the 1980’s (!) at  Digital Equipment Corporation (DEC). I was reading this article and had a real case of Deja Vue.  The article provides an excellent description of the use cases involved in the use of the tool as well as the corporate culture that was necessary for its widespread usage within DEC.

As I read this article, I realized that many of the use cases that micro-blogging tools are meant to solve are actually problems that have been around a long time – for example, the need for employees in distributed companies to be able to collaborate and discover individuals (those old “weak ties”) who can solve their problems. The high level of employee involvement at DEC regarding VAXNotes (irregardless of job title, location, etc.) shows that the assumption that active enterprise collaboration is just restricted to the generation who have grown up with Web 2.0 technology is wrong. VAXNotes emerged in an environment that was pre-Web 2.0 – indeed it was pre-Web 1.0.  Thus, the assumption that it is just Generation Y employees who will be the primary users of such micro-blogging tools is misleading. If the corporate culture (as evidenced by the DEC article) supports such tools, users from all generations will use them.  

Read it all..

I’m been playing with Stax -  “the fastest way for developers to build, manage and scale Java applications on EC2″ as a means to get ESME into the cloud. I’ve already created an example application (not ESME) based on the standard functionality and am now exploring the environment to get ESME up and running. Currently, the main problem is associated with the fact that ESME uses a local Apache Derby database for test purposes. One requirement on Stax is that local files can’t be created, so I have to use a JDBC data-source to a DB hosted in the Amazon cloud or somewhere else. I’ve created a database in the Amazon cloud (via Stax – real easy by the way -) and am now looking at configuring ESME to use this database. Once this step is done, the next step/scenario would be to use a remote database. This is an interesting scenario that I’ve discussed in another blog about architectural patterns which describes the possibility of a ESME server that  is cloud-based but the database is in the DMZ behind the firewall.  This pattern is interesting, because corporations often have problems with cloud-based installations, because the data is usually not under their control. There may be performance problems associated with such a pattern but this is something that might be able to be lessened via hardware and/or network-related measures.

Once we get the ESME server up and running, I’ll post another blog with our experiences.

ESME (Enterprise Social Messaging Experiment) has been officially accepted into the prestigious Apache Incubator – http://incubator.apache.org/projects/esme.html.

ESME is a microblogging tool and community project which originated in conversations amongst members of the SAP Community, moved to Google Code, and was the first submission in the history of SAP TechEds which was present at all three Demo Jams (Las Vegas, Berlin and Bangalore) in one year.

The Apache Incubator is the entry path into The Apache Software Foundation (ASF) for projects and codebases wishing to become part of the Foundation’s efforts. All code donations from external organizations and existing external projects wishing to join Apache enter through the Incubator. The Apache Software Foundation (ASF) provides organizational, legal, and financial support for a broad range of open source software projects. The Foundation provides an established framework for intellectual property and financial contributions that simultaneously limits contributors potential legal exposure.

Apache projects are considered the top open-source projects and many of have become standards in software development (Tomcat, Struts, etc.)

Why the inclusion of ESME in the Apache Incubator is ImportantAs I described in my last blog about innovation and the SCN community, I’d like to suggest that the evolution of ESME proves that the potential of the SCN community lies in looking beyond borders of the community itself and must include the experience gained from other communities in order to evolve itself. Although I am very much an Apache novice, I’ve been fascinated by the processes and policies that are present in this organization. For example, there is much that we (SCN) can learn from how Apache deals with IP-related issues. This cross-pollination is especially relevant based on the various open-source-related discussions occurring in background in the SCN.

 As SAP’s participation (including Senior Vice President of the SAP Community Network Mark Yolton) in the Open Source Meets Business (OSMB) Congress demonstrates, there is an increasing awareness that the relationship of corporations to open-source software should be more than just as consumers but also as contributors. This changing role requires a great deal of re-thinking in terms of corporate policies (legal, organizational, etc.); thus, the experience of the ESME team (many of whom are SAP mentors) in Apache will flow back into our involvement with the SCN as well as assist our respective corporations to deal with this new opportunity.

Note: This blog was also posted on the SDN.

Lately, I’ve been thinking about the typical ROI-related arguments about the use of Web 2.0 technology within the enterprise.  One thing that you always hear is the importance of metrics to justify the use of such new technology. I’ve had some dealing with wiki-based metrics and I wanted to try and start an initial list of metrics that might be useful. The following list is very preliminary

• Messages / time period (hour, day, week, etc.)
• Number of links sent (internal vs. external)
• Which are clients are being used
• Number of users time period (hour, day, week, etc.)
• Number of users with 10, 50, 100, 200+ followers
• Number of users following 10, 50, 100, 200+ other users
• Percentage of following/followed ratio (For example, “Are there just a few opinion leaders who have many followers and the majority of users just have a few followers)
• Number of messages that have “retweeted”

Of course, these metrics just refer to information that is inherent in the micro-blogging platform itself.   Finding the ability to associate micro-blogging with more general corporate goals (for example, innovation) is obviously more difficult and requires more thought.

It is important to consider that such metrics are only available when enterprises have complete access to the data that lies at the heart of their micro-blogging platform. If their system is located in the cloud, then such access may be difficult. What is necessary is access to the database that acts as the archive for messages.   Once this access has been achieved, then the creation of tools should be relatively straight-forward. The fact that ESME runs behind your firewall in a database of your choice makes it easy to create such metrics.

I think that providing an application that provides such ESME-specific metrics is a perfect opportunity for a someone.

Last week, we received the amazing news that ESME will be participating in the Demo Jam at SAP’s TechEd in Bangalore. This will be the first time that any team has presented at all three Demo Jams.  Abesh Bhattacharjee and Mrinal Wadhwa will be on stage presenting ESME to the crowd. Athavan Raja Durairaj will backstage supporting the two.

Each time ESME has been presented at one of this year’s Demo Jam, “local” team members have presented / supported. This is definitely one of the advantages of having a global team.  

Good luck next week to the ESME team at the Demo Jam.

P.S. We will posting the link to the live Internet video stream once it is available.