Tom DeMarco, the author of the popular book titled Controlling Software Projects: Management, Measurement and Estimation reflects on Software engineering nearly 3 decades later and now has a totally different view on the subject. The article “Software Engineering: An Idea whose time has come and gone?” is a interesting take on the subject based on practical experience. A must read for all those involved in Software development. You can find the article here.
More good news
The future of SaaS is getting brighter by the day. The following three news items are very encouraging and I hope will provide enough momemtum in diminishing the desktop and browser gap. The day of Linux, Windows, Solaris as the main platform is going to be history and the new platforms will be the browser and the desktop.
#TARGET=desktop
TARGET=browser
./configure –target=$TARGET ….
This is exactly what we need; a good competition to get the best out for the community. I hope more players join in the competition.
The role of Media types
I finally understand how media types should be used in REST<link>. I have not struggled as much to understand any of the REST constraints and principles as much as I had to understand the proper use of media types. Even HATEOAS<link> didn’t cause me such difficulty in understanding (though the ahh!! moment when I finally undetstood HATEOAS is still fresh in my mind). After the insight into HATEOAS, I felt I had mastered REST and went on to design a real application based on REST principles. When I started on the design of the application , I glided through the steps <link> of resource identification , methods supported at each URI’s but soon hit a snag when it came to the use of media types. The major problem or confusion I faced was the increasing number of media types that were getting defined in our application. I then posted a question here<link> and also exchanged few emails with experts<subbu> in the field hoping to understand where or what I was doing wrong. Even though I did get some really good responses to both my question and through emails, I still couldn’t get my head around the confusion. The main reason I never got around to understanding is due to the fact that I had a totally wrong assumption on the role of media types. After spending quite some time researching, trying to figure out where and what I was doing wrong, I think I might have finally found the missing link.
With this post, I hope to get across my understanding of media types and try to clarify the role of media types in REST for those who might have got into similar situation.
First some background, one of the goals of our team was to replace a legacy (proprietary) communication mechanism with standards based communication system. We choose HTTP and also made a conciouse decision to base our architecture on RESTful principles, and thats when we started looking at media types. Our initial understanding of the role of media types were that media types are used to describe the representation of the resources in lieu of a schema language i.e. we thought that each resource should have a media type which defines the processing model, the applicable methods on the resources and also the structure details of the resource representation. This led to us defining a media type for each resource in our application and finally to the realization that we were doing something wrong.
Recently, I came across the statement “media type for helping tune the hypermedia engine, schema for structure” <link> in a blog entry by Jim Webber. I then found this presentation<link> by Ian Robinson of Thoughtworks<link>. This presentation is one of the best that I have come across that provides a very clear understanding of the roles and responsibilities of media types and schema languages (the entire presentation is a treat and I highly recommend for all). Especially lookout for the slides titled “You’ve Chosen application/xml, you b*st*rd.” and “Custom media types”. Ian clearly explains the different roles of the schemas and the media types. In short, this is my take away from Ian’s presentation:
A media type description includes the processing model that identifies hypermedia controls and defines what methods are applicable for the resources of that type. Identifying hypermedia controls means “How do we identify links?” in XHTML, links are identified based on <a> tag and RDF has different semantics for the same. The next thing that media types help identify is what methods are applicable for resources of a given media type? A good example is ATOM (application/atom+xml) specification which gives a very rich description of hyper media controls; they tell us how the link element is defined? and what we can expect to be able to do when we dereference a URI so it actually tells something about the methods we can expect to be able to apply to the resource. The structural information of a resource represenation is NOT part of or NOT contained within the media type description but is provided as part of appropriate schema of the actual representation i.e the media type specification won’t necessarily dictate anything about the structure of the representation.
So what does this mean to us? simply that we dont need a separate media type for describing each resource as described here<link>. We just need one media type for the entire application. This could be a totally new custom media type or a custom media type which reuses existing standard media types or better still, simply a standard media type that can be reused without change in our application.
Now, armed with this knowledge, I do hope to progress smoothly in our design and as Jim says, “media type for helping tune the hypermedia engine, schema for structure” is going to be my mantra too.
I finally understand how media types should be used in REST. I have not struggled as much to understand any of the REST constraints and principles as much as I had to understand the proper use of media types. Even HATEOAS didn’t cause me such difficulty in understanding (though the ahh!! moment when I finally undetstood HATEOAS is still fresh in my mind). After the insight into HATEOAS, I assumed that I had mastered REST but my fallacy was exposed when I went on to design a real application based on REST principles. When I started on the design of the application , I glided through the steps of resource identification , supported methods, etc… but soon hit a snag when it came to the use of media types. The major problem or confusion I faced was the increasing number of media types that were getting defined in our application. I then posted a question here and also exchanged few emails with experts in the field hoping to understand where or what I was doing wrong. Even though I did get some really good responses to both my questions and through other channels, I still couldn’t get my head around the confusion. The main reason I never got around to understanding is due to the fact that I had a totally wrong assumption on the role of media types. After spending quite some time researching, trying to figure out where and what I was doing wrong, I think I might have finally found the missing link.
With this post, I hope to get across my understanding of media types and try to clarify the role of media types in REST for those who might have got into similar situation.
First some background, one of the goals of our team was to replace a legacy (proprietary) communication mechanism with standards based communication system. We choose HTTP and also made a conscious decision to base our architecture on RESTful principles, and thats when we started looking at media types. Our initial understanding of the role of media types were that media types are used to describe the representation of the resources in lieu of a schema language i.e. we thought that each resource should have a media type which defines the processing model, the applicable methods on the resources and also the structural details of the resource representation. This led us to defining a media type for each resource in our application and finally to the realization that we were doing something wrong.
Recently, I came across the statement “media type for helping tune the hypermedia engine, schema for structure” in a blog entry by Jim Webber. I then found this presentation by Ian Robinson of Thoughtworks. This presentation is one of the best that I have come across that provides a very clear understanding of the roles and responsibilities of media types and schema languages (the entire presentation is a treat and I highly recommend for all). Especially lookout for the slides titled “You’ve Chosen application/xml, you b*st*rd.” and “Custom media types”. Ian clearly explains the different roles of the schemas and the media types. In short, this is my take away from Ian’s presentation regarding media types:
A media type description includes the processing model that identifies hypermedia controls and defines what methods are applicable for the resources of that type. Identifying hypermedia controls means “How do we identify links?”, in XHTML links are identified based on <a> tag and RDF has different semantics for defining links. The next thing that media types help identify is what methods are applicable for resources of a given media type? A good example of these can be seen in the ATOM (application/atom+xml) specification which gives a very rich description of hyper media controls; they tell us how the link element is defined? and what we can expect to be able to do when we dereference a URI so it actually tells something about the methods we can expect to be able to apply to the resource. The structural information of a resource represenation is NOT part of or NOT contained within the media type description but is provided as part of appropriate schema of the actual representation i.e the media type specification won’t necessarily dictate anything about the structure of the representation.
So what does this mean to us? simply that we dont need a separate media type for describing each resource as described here. We just need one media type for the entire application. This could be a totally new custom media type or a custom media type which reuses existing standard media types or better still, simply a standard media type that can be reused without change in our application.
Now, armed with this knowledge, I do hope to progress smoothly in our design and as Jim says, “media type for helping tune the hypermedia engine, schema for structure” is going to be my mantra too.
The Chrome OS as the SaaS enabler
I am really excited about the announcement from google regarding their new OS. I was kind of expecting something similar to come out of Google after Chrome browser. I do see a lot of potential in a browser based OS. For starters, the distinction between desktop apps and browser based apps is going to blur in future. It is also encouraging for SaaS consumers since the Chrome OS addresses aspects core to SaaS like providing iinstant access to information from anywhere and everywhere, non intrusive and simple, enhanced security, removing the complexities involved in software upgrades, hardware configuration etc.. Imagine such an environment where the OS boots up and presents a browser and enables you to use the browser as your complete desktop (access to spreadsheets and other apps, the location bar acting as your command line etc..) , it enables the web to be your desktop and gives you immense freedom and possiblities.
The reason I say I was kind of expecting this from google is that after looking at Chrome and using it for almost from the day of its release, I naturally got a feeling that the Web is going to be next desktop platform. Chrome gives you an idea of the immense possibiliies and benefits of bringing the web and the desktop together. I have been a huge fan of the Chrome browser from the day I started using it, even though currently there are many short comings such as lack of extensions, themes etc.. (I really do miss the mouse gestures extension for Firefox though I haven’t touched Firefox or any other browser after my Chrome experience) However, I feel these things and many more things will get addressed soon and I am bearing this inadequacies for the simple reason that Chrome is one browser which hides itself and lets you concentrate on things that you really care. When using Chrome, I rarely feel that I am using a browser and I already get a feeling of working on the desktop with Chrome. It is blazingly fast and absolutely non intrusive and this is one of main reason I feel that Chrome OS is going to change the game, forever. I hope others will soon follow suit and we may ultimately be able to blur the distinction between Browser based apps and Desktop apps.
I am really excited about the announcement from google regarding their new OS. I was kind of expecting something similar to come out of Google after their Chrome browser. I do see a lot of potential in a browser based OS. For starters, the distinction between desktop apps and browser based apps is going to blur in future. It is also encouraging for SaaS consumers since the Chrome OS addresses aspects core to SaaS like providing instant access to information from anywhere and everywhere, non intrusive and simple, enhanced security, removing the complexities involved in software upgrades, hardware configuration etc. I do feel that this new OS is going to do a lot good for SaaS consumers and might in fact play a big role as a SaaS enabler. Imagine such an environment where the OS boots up and presents a browser and enables you to use the browser as your complete desktop (access to spreadsheets and other apps, the location bar acting as your command line etc..) , it enables the web to be your desktop and gives you immense freedom and possiblities.
The reason I say I was kind of expecting this from google is that after looking at Chrome and using it for almost from the day of its release, I naturally got a feeling that the Web is going to be next desktop platform. Chrome gives you an idea of the immense possibiliies and benefits of bringing the web and the desktop together. I have been a huge fan of the Chrome browser from the day I started using it, even though currently there are many short comings such as lack of extensions, themes etc.. (I really do miss the mouse gestures extension for Firefox though I haven’t touched Firefox or any other browser after my Chrome experience) However, I feel these things and many more things will get addressed soon and I am bearing this inadequacies for the simple reason that Chrome is one browser which hides itself and lets you concentrate on things that you really care. When using Chrome, I rarely feel that I am using a browser and I already get a feeling of working on the desktop with Chrome. It is blazingly fast and absolutely non intrusive and this is one of main reason I feel that Chrome OS is going to change the game, forever. I hope others will soon follow suit and we may ultimately be able to blur the distinction between Browser based apps and Desktop apps.
Multi tenancy vs Single tenancy
I often come across the debate whether SaaS should provide mult-tenancy or not. My personal take on this is that SaaS should mean multi-tenancy;or else it is not SaaS, may be ASP but not SaaS. The whole reason why ASP failed in the past and why now SaaS model is being proclaimed as the best bet in the future for software delivery model is the because of its ability to provide multi tenancy. A SaaS offering without multi-tenancy is no different from traditional ASP i.e multi tenancy is the differentiating factor for SaaS when compared to ASP. A multi tenant architecture allows all users and applications to share a single common infrastructure (Database, application etc..) that can be maintained centrally. For me the biggest advantage of multi-tenancy is the single codebase that eliminates the maintanance nightmare.
Of course the technical challenges of implementing Multi-tenant architecture is complex (partitions, security etc..) compared to Single tenant architecture but I feel the tradeoff here is beneficial in the long run.
I often come across the debate of Multi-tenancy vs Single-tenancy and I feel both have their place and have their own advantages and disadvantages but when it comes to SaaS, I feel that SaaS should mean multi-tenancy;or else it is not SaaS, may be ASP but not SaaS. The whole reason why ASP failed in the past and why now SaaS model is being proclaimed as the best bet in the future for software delivery model is the because of its ability to provide multi tenancy. A SaaS offering without multi-tenancy is no different from traditional ASP i.e multi tenancy is the differentiating factor for SaaS when compared to ASP. A multi tenant architecture allows all users and applications to share a single common infrastructure (Database, application etc..) that can be maintained centrally. For me the biggest advantage of multi-tenancy is the single codebase that eliminates the maintanance nightmare.
Of course the technical challenges of implementing Multi-tenant architecture is complex (partitions, security etc..) compared to Single tenant architecture but I feel the tradeoff here is beneficial in the long run.
UPDATE:
Just after posting this, I found this nice presentation which talks about the business case of multi tenancy in SaaS. Very informative.
There is no Hi-REST or Lo-REST
A question at stackoverflow regarding media type explosion resulted in someone commenting about Hi-REST and Lo-REST that got me completely confused(. i.e you are talking about hi-rest and i am talking about lo-rest blah blah). Well, NO. I am talking about REST and I know nothing about your Hi- and Lo- stuff. I am not sure why people invent new terminology and abuse the original ideas. As far as I am concerned, REST is what Roy’s dissertation says, everything else is ‘not’ REST. But that doesn’t mean it is bad or good, it just means it is not REST. I hope people stop abusing terms to create confusion.
A question at stackoverflow regarding media type explosion resulted in someone commenting about Hi-REST and Lo-REST that got me completely confused(. i.e you are talking about hi-rest and i am talking about lo-rest blah blah). Well, NO. I am talking about REST and I know nothing about your Hi- and Lo- stuff. I am not sure why people invent new terminology to abuse the original ideas. As far as I am concerned, REST is what Roy’s dissertation says, everything else is ‘not’ REST. But that doesn’t mean it is bad or good, it just means it is not REST. I hope people stop abusing terms to create confusion.
Scary SMS RTO service
Recently the RTO (Regional Transport Office) of India has unveiled a new and scary SMS service. With this service, you just need to send an SMS to 56006 along with the vehicle number (RTO XXXX-XX-XXXX ) and you get back an SMS with complete address of the vehicle owner. I fail to understand the value of this service. This is a great tool for stalkers though. Anybody can now just SMS my vehicle number and voila, they have my private details with them in seconds. I am not sure what the officials at RTO were thinking of as a use case for such a service? is it to enable somebody to pin down hit-and-run cases? isn’t it prudent to approach the police with the vehicle number instead of confronting the culprit at his residence. What else could be the RTO thinking with such a service? is this kind of service even legal? Hope somebody drives some sense into these guys, and fast.
Custom Realm in Tomcat
If you want to define your own mechanism to retrieve and check credentails instead of using the standard Realm available in Tomcat, you can extend org.apache.catalina.realm.RealmBase and overide the following methods
public class CustomRealm extends RealmBase {
@Override
protected String getName() {
return this.getClass().getSimpleName();
}
@Override
protected String getPassword(final String username) {
//myDao is your DataAccessObject
return myDao.getPasswordForUser(username);
}
@Override
protected Principal getPrincipal(final String username) {
final List<String> roles = myDao.getRolesForUser(username);
return new GenericPrincipal(this, username, getPassword(username), roles);
}
@Override
public Principal authenticate(String username, String credentials) {
String serverCredentials = getPassword(username);
//credential encrypt
boolean validated;
if (serverCredentials == null){
validated = false;
}
else if (hasMessageDigest()){
validated = serverCredentials.equalsIgnoreCase(digest(credentials));
}
else{
validated = serverCredentials.equals(credentials);
}
if (!validated) {
return null;
}
return getPrincipal(username);
}
}
In the above class, you can define your own mechanism to retrieve user credentials from a database or any other source as required. Once you have defined your own custom realm, you need to tell tomcat about it. You do that by specifying the following in server.xml (within the <Engine> element) of the tomcat server.
<Realm className=”com.test.tomcat.authentication.CustomRealm” debug=”0″/>
You can also one step further and provide your own Authenticator by extending BasicAuthenticator/DigestAuthenticator etc.. as shown below.
public class CustomAuthenticator extends DigestAuthenticator {
@Override
public boolean authenticate(HttpRequest req, HttpResponse res,
LoginConfig lcfg) throws IOException {
//perform pre processing here
boolean success = super.authenticate(req, res, lcfg);
if (success) {
//perform post processing here
}
return success;
}
}
You need to define a Valve under tomcat to ensure this class is looked up instead of the default. You do this by specifying the following in the server.xml file.
<Context docBase=”authentication” path=”/auth” reloadable=”true”>
<Valve className=”com.tomcat.authentication.CustomAuthenticator” debug=”0″/>
</Context>
I found that you need to define the Valve in an explicit Context (<Context/> for this to work.
Both custom realm and custom authenticator classes need to be in the Tomcat classpath (not in web app classpath). For adding classes to tomcat classpath, you can jar the class files and place it under the lib folder. i.e TOMCAT_HOME/server/lib for apache-tomcat-4.1.39 and TOMCAT_HOME/lib for apache-tomcat-6.0.18.
If you want to define your own mechanisms to retrieve and check credentails instead of using the standard Realms available in Tomcat, you can extend org.apache.catalina.realm.RealmBase and overide the following methods
public class CustomRealm extends RealmBase {
@Override
protected String getName() {
return this.getClass().getSimpleName();
}
@Override
protected String getPassword(final String username) {
//myDao is your DataAccessObject
return myDao.getPasswordForUser(username);
}
@Override
protected Principal getPrincipal(final String username) {
final List<String> roles = myDao.getRolesForUser(username);
return new GenericPrincipal(this, username, getPassword(username), roles);
}
@Override
public Principal authenticate(String username, String credentials) {
String serverCredentials = getPassword(username);
//credential encrypt
boolean validated;
if (serverCredentials == null){
validated = false;
}else if (hasMessageDigest()){
validated = serverCredentials.equalsIgnoreCase(digest(credentials));
}else{
validated = serverCredentials.equals(credentials);
}
if (!validated) {
return null;
}
return getPrincipal(username);
}
}
In the above class, you can define your own mechanism to retrieve user credentials from a database or any other data source as required. Once you have defined your own custom realm, you need to tell tomcat about it. You do that by specifying the following in server.xml (within the <Engine> element) of the tomcat server.
<Realm className=”com.test.tomcat.authentication.CustomRealm” debug=”0″/>
You can also one step further and provide your own Authenticator by extending BasicAuthenticator/DigestAuthenticator etc.. as shown below.
public class CustomAuthenticator extends DigestAuthenticator {
@Override
public boolean authenticate(HttpRequest req, HttpResponse res,
LoginConfig lcfg) throws IOException {
//perform pre processing here
boolean success = super.authenticate(req, res, lcfg);
if (success) {
//perform post processing here
}
return success;
}
}
You need to define a Valve under tomcat to ensure this class is looked up instead of the default. You do this by specifying the following in the server.xml file.
<Context docBase=”authentication” path=”/auth” reloadable=”true”>
<Valve className=”com.tomcat.authentication.CustomAuthenticator” debug=”0″/>
</Context>
I found that you need to define the Valve in an explicit Context (<Context/> for this to work.
Both custom realm and custom authenticator classes need to be in the Tomcat classpath (not in web app classpath). For adding classes to tomcat classpath, you can jar the class files and place it under the lib folder. i.e TOMCAT_HOME/server/lib for apache-tomcat-4.1.39 and TOMCAT_HOME/lib for apache-tomcat-6.0.18.
Setting up HTTP DIGEST authentication in Tomcat
In the previous post, I showed how to configure BASIC authentication in Tomcat. For DIGEST, follow the similar approach except replace BASIC as DIGEST in <auth-method> as shown below:
<login-config>
<!–DIGEST for HTTP Digest authentication–>
<auth-method>DIGEST</auth-method>
<!–Any name here, this is the string that will be displayed when the browser prompts the user for credentials–>
<realm-name>Admin</realm-name>
</login-config>
Note: The above doesn’t seem to work on apache-tomcat-6.0.18. I couldn’t find any bug reports or any kind of help to get it working on Tomcat 6.x. However, the above works on tomcat 4.x. I have tested it on apache-tomcat-4.1.39 version.
With this in place, when you try to access any protected area in your web application, the browser brings up a Digest authentication popup which is quite different from the one for Basic.
In the previous post, I showed how to configure BASIC authentication in Tomcat. For DIGEST, follow the similar approach except replace BASIC as DIGEST in <auth-method> as shown below:
<login-config>
<!–DIGEST for HTTP Digest authentication–>
<auth-method>DIGEST</auth-method>
<!–Any name here, this is the string that will be displayed when the browser prompts the user for credentials–>
<realm-name>Admin</realm-name>
</login-config>
Note: The above doesn’t seem to work on apache-tomcat-6.0.18. I couldn’t find any bug reports or any kind of help to get it working on Tomcat 6.x. However, the above works on tomcat 4.x. I have tested it on apache-tomcat-4.1.39 version.
With this in place, when you try to access any protected area in your web application, the browser brings up a Digest authentication popup which is quite different from the one for Basic.
Setting up HTTP Basic Authentication in Tomcat
Here are the basic steps required to setup HTTP Basic Authentication in Tomcat using the default realm (UserDatabaseRealm).
Note: I am using apache-tomcat-6.0.18
- Add the login configuration to your web application’s web.xml (anywhere under <web-app> element):
<login-config>
<!--BASIC for HTTP Basic authentication-->
<auth-method>BASIC</auth-method>
<!--Any name here, this is the string that will be displayed when the browser prompts the user for credentials-->
<realm-name>Admin</realm-name>
</login-config>
- Add the security constraint to your web application’s web.xml (anywhere under <web-app> element):
<security-constraint>
<web-resource-collection>
<web-resource-name>My Secure Area</web-resource-name>
<description>Security constraint /secure</description>
<!--Add the URI that you want to protect. The example below protects all resources starting from root -->
<url-pattern>/*</url-pattern>
<!--Add all the HTTP methods for which this security constraint should apply -->
<http-method>POST</http-method>
<http-method>GET</http-method>
</web-resource-collection>
<auth-constraint><description>only let the admin users login</description>
<!--The authentication occurs via the role name. This has to be mapped to users in tomcat-user.xml-->
<role-name>admin</role-name>
</auth-constraint>
</security-constraint> - Associate the role to a user in tomcat-users.xml (anywhere under <tomcat-users> element):
<!-- You can add multiple role and user elements -->
<role rolename="admin"/>
<user password="password" roles="admin" username="username"/> - Add the default Realm in server.xml (inside the <Engine> element)
<Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase"/>
