ESME Blog

In a previous blog, I described our use of the Stax Cloud to perform load tests for ESME. Well, Daniel Koller has just finished his analysis of the results of this initial test series and has made a few interesting discoveries. For the ESME team, these initial tests were primarily focused on gaining experience with Stax as an environment to perform such tests (including the use of clusters) as well as creating a test bed for future ESME tests.  The tests used both the REST-API and the Web UI.

Now you may be thinking the main focus should be on measuring ESME performance. True. However, the ESME version that we used is based on a older Scala library with a known memory bug. Thus, the test results are not really representative of the current ESME code base.

If you look at other microblogging platforms (including Twitter) – irregardless of whether they are focused on the enterprise or not, you’ll be hard pressed to find any load test results that are published and available to potential users. However, such tests are usually mandatory for IT projects, especially for larger companies with many potential users.  Thus, we would like start publishing our results so that those interested in ESME can make better sizing decisions.

We will soon be starting a new round of load tests based on the Apache code base and a new Scala library, so expect a blog in the future with these results.

For those interested in performing the tests themselves, Daniel has also made the test scripts available

Now that ESME works in the cloud, I was thinking about ways to take this existing installation and going viral with it so that a variety of cloud-based ESME  installations could be easily created.   The following steps would be necessary:

1. Download existing ESME source code from Stax
2. Create new application in Stax via the application console
3. Download the new application into the local installation
4. Create new database in Stax via the application console for the new ESME application
5. Copy the existing ESME source into the deployment directories of the new application
6. Change the source code to meet the basic style configurations of the new ESME installation
7. Change data-source configuration of the new application to point to new database
8. Upload the new application

 One shortcoming with this description is that as owner of the original ESME application, only I have the ability to create copies of the current installation.  At the current time,  just individuals who I invite to join my Stax application team have access to the ESME source deployment. Ideal would be the ability to create public clones of Stax-based deployments.  Then, other Stax users could very easily create new cloud-based ESMEs.  Although ESME is open-source and the source is available to all via the Apache repository, it would be easier if this was possible via Stax itself.

As a ESME supplier (someone who creates, maintains and/or supports cloud-based ESME installations for others), these steps could be patched together to create a process to very rapidly create micro-blogging sites for customers. The only task that requires more effort would be the customer-specific adaptions regarding styles, etc.

In my last blog, I mentioned my tests with the Stax.net cloud as hosting opportunity for ESME.  Well, after fighting various issues dealing with proxies and request timeouts,  I finally got ESME installed at Stax, then logged in via OpenID and posted my first cloud-based ESME message.

I really think the Stax environment is great once you get the hang of it.

To get ESME working, I used the sample application from Stax and then copied the files from the ESME war file from the Google Code site. I copied all the esme files into the “webapp” directory and then did a “stax deploy” command to load it all up into the Stax system.  I had to make some adjustments to the ant build file for the Stax deployment, because I have a limitation on the size of files that I can send via HTTP ( Therefore, the source code currently available in Stax is missing all the Jar files.)  Besides that problem, everything worked out well.

I also was able to use the cloud-database in the Stax environment rather than the existing Derby implementation. If you wanted to use a remote DB (for example, behind the firewall), you just have to change the details.

I just added the following snippet to the web.xml file:

<resource-ref>
<res-ref-name>esme</res-ref-name>
<res-type>javax.sql.DataSource</res-type>
<res-auth>Container</res-auth>
</resource-ref>

and the following snippet to the stax-application.xml

<resource name=”esme” auth=”Container” type=”javax.sql.DataSource”>
<param name=”username” value=”esme_2″ />
<param name=”password” value=”xxx” /> [password is hidden for this blog]
<param name=”url” value=”jdbc:stax://esme” />
</resource>

Now, that ESME is installed, it is possible to perform operations-related tasks -all from my browser.

Still open

• There are still exceptions dealing with file-based activities.  For example. the search interface via Compass currently doesn’t work, because it uses file. These have to be changed as well to have a fully functional application.
• It would be cool to be able to clone this application(including database)  for others.
• It would be useful to have the ability to restrict access to particular IP addresses (maybe, this is a function for which Stax could charge?)

I’m been playing with Stax -  “the fastest way for developers to build, manage and scale Java applications on EC2″ as a means to get ESME into the cloud. I’ve already created an example application (not ESME) based on the standard functionality and am now exploring the environment to get ESME up and running. Currently, the main problem is associated with the fact that ESME uses a local Apache Derby database for test purposes. One requirement on Stax is that local files can’t be created, so I have to use a JDBC data-source to a DB hosted in the Amazon cloud or somewhere else. I’ve created a database in the Amazon cloud (via Stax – real easy by the way -) and am now looking at configuring ESME to use this database. Once this step is done, the next step/scenario would be to use a remote database. This is an interesting scenario that I’ve discussed in another blog about architectural patterns which describes the possibility of a ESME server that  is cloud-based but the database is in the DMZ behind the firewall.  This pattern is interesting, because corporations often have problems with cloud-based installations, because the data is usually not under their control. There may be performance problems associated with such a pattern but this is something that might be able to be lessened via hardware and/or network-related measures.

Once we get the ESME server up and running, I’ll post another blog with our experiences.