Fall 2010

Colors, Fall 2010, Vermont and Massachusetts.

Avoiding browser popup for 401

If you are writing a web application that consumes RESTful web services which enforce HTTP basic authentication, you may face a problem where the browser may pop up a dialog box on the authentication failure (HTTP status: 401) before even your error handler code is called. This happens especially when the web application does not have any controller on the server side. For example, an application written using a client-side JavaScript framework such as Ext-JS.

The browser's HTTP user-agent obviously follows the HTTP protocol which says the following for 401.

The request requires user authentication. The response MUST include a WWW-Authenticate header field (section 14.47) containing a challenge applicable to the requested resource.

14.47 WWW-Authenticate

The WWW-Authenticate response-header field MUST be included in 401 (Unauthorized) response messages. The field value consists of at least one challenge that indicates the authentication scheme(s) and parameters applicable to the Request-URI.

where the contents of a challenge may itself contain a comma-separated list of authentication parameters. The authentication parameter realm is defined for all authentication schemes:

With some trial and error, we found that the pop up is triggered not due to the presence of 401 but due to the presence of the challenge.



So, as a web service developer, if you want to help service consumers disable the pop up and still send 401, you could use a trick. Replace Basic with your own scheme, e.g. xBasic as shown below.



To do this with Spring Security, you would want to override the commence method of the default
BasicAuthenticationEntryPoint.



Write your own entry point, e.g. MyBasicAuthenticationEntryPoint as shown below.



Then plugin your entry point into the basic authentication filter as follows:



Thanks to Venkat Mantirraju in helping figure this out.

Bulk delete

Sometimes there is a need to send an entity body for HTTP DELETE. The use case is that of applying DELETE on one or more entity of the resource in one shot. For example, a Web GUI (consumer of a web service) might list one more entity of a resource and allow the user to select one or more of these for delete. If you are using Gmail, it allows you to select one or more email and offers a DELETE action on the selected emails.

Typically DELETE works as follows


where id is the id of the email entity.

If you are using Apache HTTP Client, it will not allow to send entity body with DELETE operation, correct me if I am wrong. In that case, you have to overload POST with query parameter such as following:



and then you could send entity body with the request.

On the server side, if you are using a JAX-RS container, based on which container you are using, the annotations you use would differ.

Apache CXF JAX-RS

Apache CXF automatically dispatches requests with query _method=delete to a method on the resource that is annotated with @DELETE. More on this could be found at Overriding HTTP methods.

RESTEasy
If you are using JBoss RESTEasy, you would need to write POST as follows



Note that EmailCollection could very well have just ids of emails to be deleted. It is received as an entity body.

Response

Typically, for POST, a location header is sent back with the URI that points to the id of the newly created resource. There would be deliberated departure from that style in this case. Because, it is a bulk delete operation tunneled over POST, we would not send any location header. Also, we would not respond back with 201 on success, instead we would send 200 assuming that a transaction was applied to delete all the entities. If the transaction fails, 500 should be sent and transaction should be rolled back. More on this later...

SEDA in Streametics

Recently, I came across Matt Welsh's "A Retrospective on SEDA". Matt writes

If I were to design SEDA today, I would decouple stages (i.e., code modules) from queues and thread pools (i.e., concurrency boundaries). Stages are still useful as a structuring primitive, but it is probably best to group multiple stages within a single "thread pool domain" where latency is critical.

SEDA was one of the research projects that had influenced Streametics' technology. Streametics was the company I had co-founded in 2006. To jump start on the SEDA based implementation, I had started with the code base of Commons Pipeline and made several modifications to it. One change relevant to the above discussion was that of an application handler. This handler was introduced to separate out the application logic from a queue and thread pool of the stage. The stage driver would call out to the application and pass on the event.

Second modification was about allowing the stage designer to receive events synchronously from the previous stage without the queue -> thread pool route. This was done to allow the application designer to group multiple stages such that the processing of the event passing through these stages happens in the same thread without any context switch.

We had also externalized the pipeline (or route) configuration in XML. One could design the whole Streametics application by designing pipelines of stages and connecting these with endpoints and adapters. This way, one could form a graph of stages in the application using a tool. We had a library of stages, endpoints and adapters. The application developers could also use simple APIs to write these components on their own. We used typed XML-Java binding (XMLBeans) at the time of deployment. In a hindsight, we could have used Spring beans framework to wire-in these components.

When we were developing Streametics' event engine, Apache Camel was in very early stage.Camel supports SEDA. However, it required writing routes in Java then. Since then it now allows to do the wiring using Spring Beans. Oh well, there goes a little retrospective on Streametics...

 

RESTful web services in CollectionSpace

I have left the CollectionSpace project. However, I occasionally check out what my former colleagues there are up to. We had started work on an early draft for RESTful API documentation a few months ago. Subsequent versions are getting better, you could find it over at here. Because we had to support schema extensibility, you may find multi part payloads in examples cited. However, the draft does a nice job in describing the request and response payloads as well as error codes.

Thanks to Aron Roberts who has been diligently maintaining and updating it regularly and to Patrick Schmitz in shepherding the work. I hope it becomes a good reference for other projects using the RESTful web services.

Considering Apache CXF ...

Recently, I completed my engagement with the CollectionSpace project and joined a small company in the south bay. Here I am again where we have to choose a web service framework to implement RESTful web services.

In CollectionSpace, we had selected JBoss's RESTEasy. One reason for that was that our deployment environment included JBoss due to the requirements driven by a major 3rd party software we had to use. I liked the simplicity of RESTEasy and also its quality. There were enough plugins to extend the framework. The proxy based client-side library was very useful in writing service consumers for testing purposes. Bill Burke and I had worked on a project more than 14 years ago. He then went JBoss route and I had joined BEA. Anyway, it was good to catch up with him. It was easy to rely on his work.

However, unlike CollectionSpace, here, there are legacy SOAP-based web services used for B2B transactions. These services are implemented using Metro - SUN's RI for JAX-WS. Metro does not support JAX-RS. For that, one would have to use Jersey. These are two different web service frameworks.

I have been investigating if Apache CXF could be used. Here is my impressions so far...will keep you posted. The following is not necessarily a comparison. These are just my notes with some perspective on using a web service framework.

1. CXF supports both JAX-WS and JAX-RS. Same stack.
2. It is also possible to use the same service implementation to support for JAX-WS and JAX-RS annotations. Don't know about complexities till I play with such an implementation
3. Spring integration. This is something I missed in RESTEasy. It is good to describe the meta data for a service outside of Java code.
4. CXF's advanced search capabilities looks really interesting. It has built-in support for FIQL. I look forward to using it. It'd be interesting to figuring out if SearchCondition could be represented as a resource to which access could be controlled.
5. There is a nice built-in support for sub-resource locators. Not very difficult to do without it but nice to have it in a framework.
6. Schema validation could be at service or at provider levels...again configurable with Spring beans.
7. CXF supports 3 types of clients : Proxy-based, HTTP-centric and XML-centric. Many times, it is good to have these choices as Proxy-based clients and JAXB could hide some real scenarios. e.g. using JAXB on the client does not require one to b64 encode content if proper XML type is used.
8. It is possible to configure implementation as singleton or prototype

Overall, these look promising to consider CXF. We will know more when we actually start using it.

Tunneling DELETE over POST for deleting specific relationships

One of the problems with the relationship management approach in  as described in Part II of my post earlier was that DELETE was very coarse grained. The following would delete all the relationships between the given permission (id:e62d2306-518b-414b-9591) and roles.



This might not be desirable. However, as I mentioned DELETE does not take any payload, so it is not possible to provide a list of relationships using DELETE. Here, the approach of tunneling DELETE over POST could be useful. The overloaded POST would look as follows:



Now we can send a payload for deleting specific relationships. Here, the ?_method=delete would indicate to the service that DELETE operation should be performed for the given payload which might look as follows :



This operation would delete a relationship between given permission (id:e62d2306-518b-414b-9591) and given role (id:814ed9d4-3315-44fc-993b) only.

User managed access

While attending the 10th Internet Identity Workshop last week, I came across an interesting initiative that I had absolutely no idea about. I am glad that I attended. It is called User Managed Access (UMA). It is about letting a user choose the access control policies for resources offering his/her personal data (personal information, address book, bookmarks, blog posts, comments, etc.). It is explained much better in UMA Explained.

"... a web user (authorizing user) can authorize a web app (requester) to gain one-time or ongoing access to a resource containing his home address stored at a "personal data store" service (host), by telling the host to act on access decisions made by his authorization decision-making service (authorization manager)."

If you are familiar with enterprise access control, the following might help.

"In enterprise settings, application access management often involves letting back-office applications serve only as policy enforcement points (PEPs), depending entirely on access decisions coming from a central policy decision point (PDP) to govern the access they give to requesters. This separation eases auditing and allows policy administration to scale in several dimensions. UMA makes use of this separation, letting the authorizing user serve as a policy administrator crafting authorization strategies on his or her own behalf."

The UMA work group has protocol spec, scenarios and reference implementation.

I plan to follow this work. Having just checked in the default set of permissions for resources for CollectionSpace, I wonder how and when a Host (such as CollectionSpace) would make permissions for a user's data available to an Authorization Manager (AM). UMA envisions that the AM could be a third party web-based service, but would that be practical from a performance perspective?

 

Relationship management in RESTful web services - Part II

In the Part I on this topic, I described an approach that makes relationship between two resources a first-class resource. In this post, I would describe the second approach I had mentioned in Part I, the relationship as a sub-resource.

Relationship as a sub-resource
Here, the relationship is managed as a sub resource of the resource that acts as object in the relationship. Let's take an example.

Roles ROLE_COLLECTIONMANAGER (id: 814ed9d4-3315-44fc-993b) has permission (id: e62d2306-518b-414b-9591) to access a resource named intakes

Permission as object in relationship

If the object of the relationship is a permission and subject being role, i.e. a permission to access resource named intakes is given to role ROLE_COLLECTIONMANAGER, it could be created using the following API. We assume here that both the permission and role are already created using respective POST operations and we are only trying to relate these over here.



Here, e62d2306-518b-414b-9591 is the id of the permission. The sub-resource is permroles that indicates the context of the relationship, i.e. the relationship is between permission and role and not permission and user for example.



As you might notice, we have added some additional data about the permission (resourceName) as well as the role (roleName). This is optional and for convenience purposes only. Also, it would be possible to associate one or more roles with the same permission in a single POST request. So, associating both ROLE_COLLECTIONMANAGER and ROLE_CURATOR (id: 3772624d-1ab3-4e47-a26d-191fc6437410) with the same permission would look like the following.



Role as object in the relationship

The same data structure could also be used with role as an object in the relationship. That is, ROLE_COLLECTIONMANAGER has permission(s) to access intakes (and collectionobjects) resource. This could be accomplished using the following API. Note that id 814ed9d4-3315-44fc-993b represents ROLE_COLLECTIONMANAGER.





Here, POST does return an id to comply with RESTful architecture, however, because relationship is not treated as a first-class resource, this id is meaningless for subsequent operations such as GET and DELETE. Let's assume that we received id 123 for now. Also, more subjects could be associated with the same object using the POST operation again as follows. That means, POST also acts as PUT.





The GET operation would use the id of the object (permission:e62d2306-518b-414b-9591 or role:814ed9d4-3315-44fc-993b) in the relationship to access the relationships with all the subject(s). The GET operation would return the same data as was posted but with a union of all subject(s).



Note the last element of the URI, 123, it is just a filler for an id as the relationship is not a first-class RESTful resource.



All 3 roles associated with a permission are returned.

According to the RESTful architecture, the DELETE should only take an id of the resource to be deleted. The following operation would delete relationships between the given object and all its subjects.



This is good and bad. It is good from efficiency purposes in the sense that all the relationships between subjects and given object could be deleted in one shot. However, it is bad that individual relationships between a subject and given object cannot be deleted as there is no way to uniquely identify such a relationship.

Overall, I like the sub-resource based approach as it is more convenient to navigate to relationship(s) from an object in the relationship. Also, it supports bulk operations, i.e. associating more than one subjects with an object with a single request is possible and GET returns relationship between an object and all the subjects that object is related to in the context of that relationship.

However, this is a bit un-RESTful approach as the identifier returned from POST is meaningless for the GET and DELETE operations. Also, DELETE is not fine-grained, i.e. deleting a relationship between an object and a subject is not possible. It would be better if there was a way to make that possible (see an alternative). The approach described in Part I does not have this problem. Perhaps a hybrid is possible? Would a hybrid approach more complex? I am eager to hear your viewpoints. Feel free to send me comments if you have taken an approach which does not suffer from the limitations I have described.

would you use webfinger?

I was at the Internet Identity Workshop #10 today. I have been to an unconference only once before. It looked like a chaos initially but it was organized chaos. I liked the format. Moreover, I liked how open the participation was both from the presenter and the listeners.

Anyway, while I was slightly familiar with OpenID and OAuth, I am just getting familiar with some of the problems of the initial versions of the Open ID and OAuth 1.0a. Came across several initiatives ... one of which is WebFinger.

"WebFinger is about making email addresses more
valuable, by letting people attach public metadata to them. That
metadata might include:

• public profile data
• pointer to identity provider (e.g. OpenID server)
• a public key
• other services used by that email address (e.g. Flickr, Picasa, Smugmug, Twitter, Facebook, and usernames for each)
• a URL to an avatar
• profile data (nickname, full name, etc)
• whether
  the email address is also a JID, or explicitly declare that it's NOT an
  email, and ONLY a JID, or any combination to disambiguate all the
  addresses that look like something@somewhere.com
• or even a public declaration that the email address doesn't have public metadata,
  but has a pointer to an endpoint that, provided authentication, will
  tell you some protected metadata, depending on who you authenticate as."
  

Eran Hammer-Lahav describes the rationale for the same over here. I like number of arguments he makes, however, I am stuck on the following ...

"The arguments against email as identifiers usually include concerns
over spam and privacy ..."

At least with the Http URI, I don't have to worry about spam. Indeed, there is a phishing problem, but as far as one knows how to protect against it, it might be manageable. How do I know that the email address I am giving to some site in order to enable it to fetch my public meta data won't be misused? Am I missing something here?

User profile management in a social application

In my opinion, user profile management is one of the most important aspects of any social application. Social graphs are made of people and relationships between people. Not only the user profile allows an application developer to capture the information about a person, it is also an important dimension in deducting various kinds of user and usage focused analytics. Such analytics is very important for a social application. If designed well, it could also help increase the application's user base. I will explain about this further later.

User profile management should include the following functionality.

1. Account management including lifecycle management
   
2. Support for multiple identity providers (IdP)
3. Login / Logout
4. Single sign-on (OpenID, Facebook Connect)
5. Profile management
6. Authorization (Native, OAuth)
7. ?
   

Account management
From an application developer's perspective, every user of the application must have an account regardless of how the user logs in (user/password, OpenId or Facebook auth). An account could have minimal information such as display name, email, status, timestamps for creation and modification times. Account may not necessarily have user's profile information.

The lifecycle events associated with an account could be registration, activation, deactivation and finally deletion.

Support multiple IdP
Any social application should support at least 3 types of identity providers in my opinion. A local IdP, OpenID IdP and Facebook.

The local IdP comes handy when potential users of the application do not have any OpenID or do want to create and maintain a profile with the application. The local IdP is usually implemented as identifying users with username and password. Database is generally used as a realm.

The application should also support one or more OpenID IdPs. Many users may not want to create one more online identity to login to the application. They may want to use one of their existing identities managed by a 3rd party identity provider (Yahoo!, Google, myOpenId, etc.) using the OpenID protocol. An application would act as a relying party (RP) that relies on identities asserted by the 3rd party IdPs.

Lastly, any social application developer may want to tap into Facebook's 400MM user base. Unfortunately, Facebook does not support open standard such as OpenID. So, the application has to support proprietary FB authentication protocol (FB Connect).  

See Stackoverflow or Plaxo login screens to check how these applications support multiple IdPs.

Login / Logout
This is a an obvious feature of any web application that would want to offer personalized services. No need to describe anything here except that a user should be able to login using an id managed by the local IdP or by a foreign IdP after due assertion of that id. Other functions would include ability to reset password (for local IdP only), "remember me" among other things. Session management (session expiration, persistent session, etc.) would be required as well as protection against session fixation attacks.

Logout for a user logged in using OpenID or FB Connect would require logging out locally from the application and destroying session context related to the application that is relevant to the user logging out. Note that for a social application, the short comings of a log out feature would not only expose the user but the user's social activities and social graph as well.  

Single sign-on
For a social application, user experience is very important. Offering single sign on could improve user experience right away. If the application user has logged into some other web application using OpenID or FB Connect, that user may not have to sign in again to your application (within the timeframe set by the OpenID IdP or FB) if you support OpenID or FB. Again, check out Stackoverflow or Plaxo.

Profile management

The profile data of a user is important for any social application as
it acts as a very important dimension in various analytical services
the application could provide to the users, insights to improve its own
services and to interested 3rd parties. Account may hold minimal information about the user. An application may have a user profile to hold other user-specific information. This depends on the application but it could include information such as first and last name, nick name, address(s) (land and web), land and mobile phone(s), email(s), instant messenger id(s) and other demographics information as required by your application.

If the user logs in using OpenID, it is possible to populate some of these using OpenID's attribute exchange protocol at the time of login. FB Connect also has APIs to retrieve FB user's profile data per user's privacy settings. Such data could also be retrieved securely with user's consent after the login using the OAuth protocol.

Authorization
Authorization deserves its own post. I will cover authorization in my subsequent post.

If you are using a Java based platform on the server side of your application, you may want to look at Apache Shiro based Nimble project. Nimble is a Grails plugin that uses Shiro underneath. It provides most of the features I have mentioned here except OAuth. It also provides customizable user interface and security tags to insert into the user interface.

Social identity theft - protecting your personal brand

I recently attended a talk given by an OpenId foundation member. I would not say here where and who gave the talk. What I came to know is that the big web-based identity providers (IdP) such as Google and Yahoo! have embraced OpenId for almost couple of years now and they would want you to use your id/account as many places where open id is supported. Facebook wants you to do the same but their protocol is very proprietary. Anyway, this is good news!

1. This helps a lot in increasing user registration at a relying party (a social media application) web site because users can sign in using their OpenId. 
2. It also helps those folks who want to use OpenId to sign in where ever they could because they don't want to keep track of and maintain various social identities and profiles at various social media web sites. 
3. Lastly it helps in further service authorization and sharing of content using OAuth.

However, this also makes these folks vulnerable to social identity theft. If your social identity is stolen, not only you are vulnerable but your social graph might be vulnerable too (without any reason). This is more dangerous.
I came across an article "How to Combat Social Identity Theft and Strengthen Your Online Personal Brand" where the author recommends creating a separate profile and identity at each social media site . This is like having a separate password for each web site...hard to remember and maintain but it restricts the vulnerability to a single profile/identity/website if stolen. Some folks even provide a service to manually go and create profiles in your name at some 150 web sites! Indeed, what guarantee they give that they won't misuse this information, disgrunted employees could be found everywhere right! And finally there are tools like KnowEm which automate the process by helping find availability of the username at>350 websites and also help create profiles (with subscription service). Many other such tools are listed here.

I would think that one should carefully use identities maintained by the big guys. I would also keep more than one OpenId handy to use at various social media web sites so if any one of these is stolen, at least the vulnerability is contained to only those sites where it was used. However, indeed there could be a better solution...looking forward to your comments, thoughts and suggestions.

Relationship management in RESTful web services - Part I

CollectionSpace offers various RESTful web services for managing meta data for physical/digital objects. Defining RESTful interfaces for various entities in the system is straight forward. Create, Read, Update, Delete, List and Search (CRUDLS) operations on these entities could be easily be mapped to the POST (create), GET (read, list and search), PUT (update) and DELETE (delete) methods of HTTP. For example, see the RESTful APIs for the Role service.

CollectionSpace also has a requirement to support relationships between several entities. For example, there could be one to many relationship between a collection object and a loan object. On a more domain-agnostic side of the services, a permission might be for one or more roles and a role might be related to one or more permissions. Implementing relationships between the RESTful resources is not so straight forward and might need some due diligence. Let's take each of the above two use cases to model relationships in two different ways and talk about pros and cons of each. The two approaches are :

1. Relationship as a first-class RESTful resource
2. Relationship as a sub-resource

I'll cover only the first part in this entry. Feel free to comment on the blog if you have implemented relationships in different and better way.

Relationship as a first-class RESTful resource With this approach, relationship becomes a RESTful resource that supports CRUDLS operations. It is a bit unintuitive to think about relationships like this but it is the most flexible approach. Here, each relationship meta data has the following components:

• Object of the relationship
• Subject of the relationship
• Type of the relationship

For example, a relationship between a collection object entity and a loan entity in English could be:

The object in the relationship is the collection object entity and the subject of the relationship is the loan entity. The types should be namespace qualified. This is derived from the RDF model but perhaps only in parts. The relationship web service would support the following methods :

The advantage of such a data structure is that one can relate anything to anything without worrying about semantics. It is also the biggest disadvantage. Relationship being a first-class RESTful resource, each relationship has a distinct id of its own using which it could be retrieved and updated. However, there are many problems.

1. Unlike the RDF model, the predicate is missing, so it is hard to machine learn how entities are related. The web service would have to do a lot of validation to make sure semantically the relationships are possible between the given entities and are correct.
2. It is not possible to determine cardinality in the relationship
3. If RESTful URIs are used, there is no need to provide separate type and id. For example, http://collectionspace.org/services/collectionobjects/814ed9d4-3315-44fc-993b would uniquely identify the object in this relationship.

I'll cover the 2nd approach in my subsequent blog entry.

Account/user management live in CollectionSpace

I am glad to see the Account service having a face on the CollectionSpace UI. Thanks to the TorontoU and the CambridgeU (UK) teams for pulling this off. It is available in the upcoming 0.6 release to play with at a test server. Eventually it will be moving to the demo. The project release documentation has a nice walk through with screenshots.

RESTful management interfaces for security services

In the past, I used WebLogic Security extensively while at BEA. We were building security in the "layered" products : WebLogic Integration and AquaLogic Service Bus. The runtime APIs at our disposal were the Java APIs weblogic.security.* and for management the JMX APIs in weblogic.management.security.*. The JMX APIs are remotable, indeed behind the firewall, but remotable.

Indeed, we had to build security console using JSP/JSF/Struts/etc. so the security administrators of these products could manage users, accounts, roles, user-role mappings, permissions/policies, keys and certificates, etc. The console implementation would use JMX APIs underneath. Alternately, application developers could build their own administration consoles by directly using the JMX APIs behind the firewall.

In the open source, there are good options available for enterprise security such as Spring Security and Apache Shiro. These have non-remotable management APIs in Java. However, I could not find any remotable management interfaces that could be easily accessed from a web-based console over HTTP. So, for CollectionSpace, we built management interfaces using REST. These include

3 entity resources

1. Account (also manages a simple IdP using DB realm)
2. Role 
3. Permission 

and 2 relationship resources

1. AccountRole a sub resource accessed from the account service
2. PermissionRole a sub resource accessed from the permission service

Your feedback

If you think these management interfaces would be useful in other projects or if you have suggestions, please send me an email at [sanjay dot dalal at gmail dot com]. We could perhaps extract these out from CollectionSpace and make them available through a separate open source project with Apache 2 license.

Authorization service RESTful interface

Yesterday, I checked in the last piece of functionality to complete the first pass at the RESTful interface of the authorization service in CollectionSpace. There is a short description of these APIs on the wiki. Yes, I am aware of the confusion behind the term authorization. I have described the terms used on the wiki.

1. Role
2. Permission
3. Role - Permission relationship (available from Permission service, /permissions/{id}/permroles)
4. Account - Role relationship (available from Account service, /accounts/{i}/accountroles)

Indeed, there is a separate web service for account management with its own RESTful interface. The CollectionSpace security runtime exposes APIs to enforce access control from the CollectionSpace services runtime and SPIs to plug in various providers. I will be using Spring Security ACLs as the underlying service provider.

More on the enforcement of the permissions in future entries. I will also write more about my experiences and approaches I took in implementing relationships between RESTful resources, e.g. relationship between Role and Permission.

Security Resource, Permission and Administration

First, let's clarify what is a resource in terms of security. According to XACML, a resource is data, service or system component for which access is requested. WebLogic Security defines resource as any software component, such as a server, a service, an application, or an application artifact that can be secured using security roles and policies. There are subtle differences between XACML's definition and WebLogic's definition, e.g. a method to secure on an EJB is considered a resource in WebLogic while for XACML that method is an action on an EJB resource.

While Spring Security supports several resources to protect such as a URL, a method invocation and an (domain) object, it lacks a uniform abstraction for representing the same in its architecture. It also lacks an abstraction to represent a permission on a resource. Let me give examples for what I mean.

Defining and enforcing permissions for URL resources requires configuring FilterSecurityInterceptor with permissions defined in the form of intercept-url elements including URL patterns and allowed roles.

1:    <bean id="filterInvocationInterceptor" class="org.springframework.security.web.access.intercept.FilterSecurityInterceptor">  
2:      ...  
3:      <!--property name="securityMetadataSource" ref="cspaceMetadataSource"/-->  
4:      <property name="securityMetadataSource">  
5:        <sec:filter-security-metadata-source>  
6:          <sec:intercept-url pattern="/**" access="ROLE_USERS"/>  
7:        </sec:filter-security-metadata-source>  
8:      </property>  
9:    </bean>  

Defining and enforcing permissions for method resources using MethodSecurityInterceptor requires listing permissions using a set of properties including qualified methods as names and allowed roles as value.

1:  <bean id="bankManagerSecurity"  
2:    class="org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor">  
3:  ...  
4:   <property name="securityMetadataSource">  
5:    <value>  
6:     com.mycompany.BankManager.delete*=ROLE_SUPERVISOR  
7:     com.mycompany.BankManager.getBalance=ROLE_TELLER,ROLE_SUPERVISOR  
8:    </value>  
9:   </property>  
10:  </bean>   

Spring ACL for Domain Object Security has an abstraction for Permission but actually the permission is required to be configured programatically using abstractions such as Permission, ObjectIdentity and Sid (security identity). It offers no management interfaces to configure or administer permissions declaratively.

1:  <security:global-method-security pre-post-annotations="enabled">  
2:    <security:expression-handler ref="expressionHandler"/>  
3:   </security:global-method-security>  
4:   <bean id="expressionHandler"  
5:     class="org.springframework.security.access.expression.method.DefaultMethodSecurityExpressionHandler">  
6:      <property name="permissionEvaluator" ref="myPermissionEvaluator"/>  
7:   </bean>  

The first two examples above show that Spring Security neatly allows to define permissions exactly where those permissions would be enforced. This makes it really easy to configure permissions. However, it also makes it really difficult to administer permissions independently as there are no distinct and unified administrative interfaces to do so. Now, if you had abstractions for Resource and Permission, you would be able to administer these separately and enforce them from FilterInvocation, MethodInvocation and PermissionEvaluator too.

A typical simple non-hierarchical resource would have the following information:

• Type of the resource (URL, method, object, class, db table, etc.)
• Id to uniquely identify the resource
• Pattern (e.g. com.mycompany.BankManager.delete* or /**)

A typical permission might have the following

• Resource on which access is defined
• A set of principals (roles and users) it is applicable to
• An effect (permit or deny)

How this can help in administering and enforcing permissions? I will write about that in my next post...

Spring ACL for RESTful web services

I had mentioned before that the easiest way to provision permissions for URIs using Spring Security is by adding intercept-urls and corresponding expressions in security metadata for a filter in a web application. It is also possible to define security metadata in file/database for these URIs so that permission could be changed dynamically after the webapp is deployed. Here is a sample using a property file named url.properties (credits: custom security metadata)for more details.

1:  <bean id="filterInvocationInterceptor" class="org.springframework.security.web.access.intercept.FilterSecurityInterceptor">  
2:      <property name="authenticationManager" ref="authenticationManager"/>  
3:      <property name="accessDecisionManager" ref="httpRequestAccessDecisionManager"/>  
4:      <property name="securityMetadataSource" ref="cspaceMetadataSource"/>  
5:    </bean>  
6:    <bean id="cspaceMetadataSource" class="org.collectionspace.services.authorization.spring.CSpaceSecurityMetadataSource">  
7:      <property name="urlProperties">  
8:        <util:properties location="classpath:urls.properties" />  
9:      </property>  
10:    </bean>  

The url.properties includes a list of permissions. It might look something like following.

1:  /accounts/**=ROLE_ADMINISTRATOR  

And the CSpaceSecurityMetadataResource looks as follows:

1:  import java.util.Collection;  
2:  import java.util.Properties;  
3:  import org.springframework.security.access.ConfigAttribute;  
4:  import org.springframework.security.access.SecurityConfig;  
5:  import org.springframework.security.web.FilterInvocation;  
6:  import org.springframework.security.web.access.intercept.FilterInvocationSecurityMetadataSource;  
7:  public class CSpaceSecurityMetadataSource implements FilterInvocationSecurityMetadataSource {  
8:    private Properties urlProperties;  
9:    public Collection<ConfigAttribute> getAllConfigAttributes() {  
10:      return null;  
11:    }  
12:    public Collection<ConfigAttribute> getAttributes(Object filter)  
13:        throws IllegalArgumentException {  
14:      FilterInvocation filterInvocation = (FilterInvocation) filter;  
15:      String url = filterInvocation.getRequestUrl();  
16:      String method = filterInvocation.getHttpRequest().getMethod();  
17:      //get the roles for requested page from the property file  
18:      String urlPropsValue = urlProperties.getProperty(url);  
19:      StringBuilder rolesStringBuilder = new StringBuilder();  
20:      //fetch roles from property file that could invoke the given http method on given url
21:      return SecurityConfig.createListFromCommaDelimitedString(rolesStringBuilder.toString());  
22:    }  
23:    public boolean supports(Class<?> arg0) {  
24:      return true;  
25:    }  
26:    public void setUrlProperties(Properties urlProperties) {  
27:      this.urlProperties = urlProperties;  
28:    }  
29:    public Properties getUrlProperties() {  
30:      return urlProperties;  
31:    }  
32:  }  

Limitations :


Http Method

Unlike a web application, for the RESTful web service, it is important to know the http method (e.g. POST is used for creating representation for a resource) in addition to the URI (e.g. /accounts/) from the invocation context. Using this two part information, you should retrieve a list of roles (comma separated) that are allowed to invoke the given HTTP method on the given URI. Note that for this reason the example url.properties file shown earlier is not enough as it does not include any HTTP methods in permission definition.

Sub-resources and permission inheritance

According to the HATEOS principle, it is very common to expect that a representation received from a resource could include additional URIs to retrieve representations of related resources. For example, to retrieve the user profile associated with an account, one could invoke a GET request on URI /accounts/{accountid}/users/{userid}. Security administrator may want to inherit permissions for the users resource from the accounts resource. Using a flat permission model such as shown above in the url.properties, it is not possible to define inheritance.

Therefore, I am exploring if a URI could be considered a domain object and Spring ACL could be used to define permissions. Will keep you posted ...

Shutting down log4j

When I integrated Spring Security into CollectionSpace, the dynamic deployment of cspace-services.war stopped working. We use Cargo's JBoss plugin for maven to dynamically deploy war on a running server instance. The redeploy (undeploy + deploy) would fail with an error "Shutting down log4j"! Only restart of the server will allow to deploy the war. We use slf4 with log4j binding.

While the process of coming up to the following list of steps was long, the list itself is short.

1. In our environment, JBoss uses log4j, cspace-services.war uses log4j and Spring used it as well. However, Spring was shutting down log4j logger when our war was undeploying due to the following listener. Remove Log4jConfigListener from web.xml.

1:  <listener>  
2:   <listener-class>org.springframework.web.util.Log4jConfigListener</listener-class>  
3:  </listener>  

2. Do not package log4j jar with your web app. JBoss ships with log4j.jar that is available in the system classpath.

3. Have your own log RepositorySelector to separate your log from other applications (here JBoss). This also helps in stopping JBoss's RepositorySelector from jamming application's log entry onto its own. Here is how repository selector looked.

1:  package org.collectionspace.services.common.log;  
2:  import java.util.Hashtable;  
3:  import javax.naming.Context;  
4:  import javax.naming.InitialContext;  
5:  import javax.naming.NamingException;  
6:  import org.apache.log4j.Hierarchy;  
7:  import org.apache.log4j.Level;  
8:  import org.apache.log4j.spi.LoggerRepository;  
9:  import org.apache.log4j.spi.RepositorySelector;  
10:  import org.apache.log4j.spi.RootLogger;  
11:  /**  
12:   * CollectionSpaceLog4jRepositorySelector is a CollectionSpace  
13:   * specific log4j repository selector. See Ceki's solution  
14:   * for more details  
15:   * Courtsey Ceki Gulcu http://articles.qos.ch/sc.html  
16:   */  
17:  /** JNDI based Repository selector */  
18:  public class CollectionSpaceLog4jRepositorySelector implements RepositorySelector {  
19:    // key: name of logging context,  
20:    // value: Hierarchy instance  
21:    private Hashtable ht;  
22:    private Hierarchy defaultHierarchy;  
23:    public CollectionSpaceLog4jRepositorySelector() {  
24:      ht = new Hashtable();  
25:      defaultHierarchy = new Hierarchy(new RootLogger(Level.DEBUG));  
26:    }  
27:    // the returned value is guaranteed to be non-null  
28:    public LoggerRepository getLoggerRepository() {  
29:      String loggingContextName = null;  
30:      try {  
31:        Context ctx = new InitialContext();  
32:        loggingContextName = (String) ctx.lookup("java:comp/env/cspace-logging-context");  
33:      } catch (NamingException ne) {  
34:        // we can't log here  
35:      }  
36:      if (loggingContextName == null) {  
37:        return defaultHierarchy;  
38:      } else {  
39:        Hierarchy h = (Hierarchy) ht.get(loggingContextName);  
40:        if (h == null) {  
41:          h = new Hierarchy(new RootLogger(Level.DEBUG));  
42:          ht.put(loggingContextName, h);  
43:        }  
44:        return h;  
45:      }  
46:    }  
47:    /**  
48:     * The Container should remove the entry when the web-application  
49:     * is removed or restarted.  
50:     * */  
51:    public void remove(ClassLoader cl) {  
52:      ht.remove(cl);  
53:    }  
54:  }  

The JNDI environment entry was registered as follows in web.xml:

1:    <env-entry>  
2:      <description>Sets the logging context for the Tiger web-app</description>  
3:      <env-entry-name>cspace-logging-context</env-entry-name>  
4:      <env-entry-value>CSpaceLoggingContext</env-entry-value>  
5:      <env-entry-type>java.lang.String</env-entry-type>  
6:    </env-entry>  

Here is how we initialized it in our ServletContextLIstener

1:  public class CollectionSpaceServiceContextListener implements ServletContextListener {  
2:    @Override  
3:    public void contextInitialized(ServletContextEvent event) {  
4:  ...  
5:        LogManager.setRepositorySelector(new CollectionSpaceLog4jRepositorySelector(),  
6:            null);  
7:  ...  
8:    }  

4. Remove log4j.properties or log4j.xml from your war. We could still share log4j configuration with JBoss by adding categories into the jboss-log4j.xml. This is nice because you could dynamically change the log level using JMX mbeans provided by JBoss. If you package your own configuration in the war, you will need to redeploy it if you want to change log level or will have to build your own JMX Mbeans to change the level dynamically.

As I said before, the debugging took longer than writing this post. Hopefully, it is useful...

What I would like to see in Spring Authorization...

I am trying to integrate Spring Security in CollectionSpace web services. I chose Spring Security because it is used quite widely in enterprise applications and has proven its utility and reliability. Also, it is a well thought out architecture that is easy to understand for the application developers. I like most of the features of Spring Security Authorization. This includes role-based access control, possibility for several types of voting in access decision manager and pre/post invocation hooks, etc.

In Spring Security, a resource to protect such as URI is treated differently from other resources (methods, domain objects, ...) both for permission administration as well as enforcement.  Easiest way to provision permissions for URI is by adding intercept-urls and corresponding expressions in security metadata for a filter in web application. This is metadata packaged with the war file.

1:  <bean id="filterSecurityInterceptor"  
2:      class="org.springframework.security.web.access.intercept.FilterSecurityInterceptor">  
3:   <property name="authenticationManager" ref="authenticationManager"/>  
4:   <property name="accessDecisionManager" ref="accessDecisionManager"/>  
5:   <property name="securityMetadataSource">  
6:    <security:filter-security-metadata-source>  
7:     <security:intercept-url pattern="/secure/super/**" access="ROLE_WE_DONT_HAVE"/>  
8:     <security:intercept-url pattern="/secure/**" access="ROLE_SUPERVISOR,ROLE_TELLER"/>  
9:    </security:filter-security-metadata-source>  
10:   </property>  
11:  </bean>  

It perhaps is possible to define security metadata in database for these URIs so that permission could be changed dynamically after the webapp is deployed. I am trying to do exactly the same for CollectionSpace. I will explain the why later. However, I am disappointed to see statements such as "The first thing you should ask yourself is if you really need to do this. If an application requires securing, then it also requires that the security be tested thoroughly based on a defined policy." to defend their approach (of embedding permission expressions in the application itself). This seems like a short-sighted argument to me. It is a bad practice to embed permissions in the application package as it requires redeploying the application if permissions change dynamically as they often do, especially if the web application is a multi-tenant software service or exposes web services.

Secondly, while it is good to see permission enforcement for resources such as URI, method and AspectJ JoinPoint "described" as metadata that is not "hard coded" into the application, it is disappointing to see permissions ("securityMetadataSource") associated with these at the same location. I would have loved to see permission provisioning aspect separated out from the permission enforcement points (interceptors). The permissions enforcement points should only know about which resources to protect and not what permissions are associated with those resources. There seems to be a nicer way to delegate the permission evaluation using the PermissionsEvaluator interface but that seems only applicable to method security.

Why I need such flexibility?

1. CollectionSpace is a multi-tenant application. For the same service URI, there could be different permissions per tenant. 
2. There are lots of URIs that need to be protected for a web service and for a collection of web services in CollectionSpace. CollectionSpace web services are RESTful web services.
3. Not all services might be packaged into a single war file. It would be good to centralize permission provisioning and administration out of the security interceptor meta data packaged with the war file.

Keep tuned for the next post describing how we do permission administration using Spring Security in CollectionSpace...

Spring security and JAAS provider

Recently, I checked in code (#1318, #1321) to integrate Spring Security into CollectionSpace. As you know, we already had implemented a working JAAS Login Module that acts as a default identity provider. The configuration of that login module is externalized into a JBoss application-policy in the login-config.xml. That is a good thing. Also, that login module inserts tenant information for the user into the security context. We plan to use this info. while making access control decisions in downstream request processing. So, my first task was to integrate that login module with Spring security, if it was possible.

I faced some problems (described at the end) but it was possible. The application-security.xml of our web application looks as follows.

1:  <beans xmlns="http://www.springframework.org/schema/beans"  
2:      xmlns:s="http://www.springframework.org/schema/security"  
3:      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
4:      xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd  
5:      http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-3.0.xsd">  
6:    <s:http use-expressions="true">  
7:      <s:intercept-url pattern="/**" access="permitAll" />  
8:      <s:http-basic />  
9:      <s:logout />  
10:    </s:http>  
11:    <s:authentication-manager>  
12:      <s:authentication-provider ref="jaasAuthenticationProvider" user-service-ref="userDetailsService"/>  
13:    </s:authentication-manager>  
14:    <bean id="jaasAuthenticationProvider"  
15:       class="org.springframework.security.authentication.jaas.JaasAuthenticationProvider">  
16:      <property name="loginContextName">  
17:        <value>cspace</value> <!-- value should be same as in application-policy in JBoss login-config.xml -->  
18:      </property>  
19:      <property name="loginConfig">  
20:        <value>/WEB-INF/login.conf</value> <!-- filler, not used at runtime -->  
21:      </property>  
22:      <property name="callbackHandlers">  
23:        <list>  
24:          <bean class="org.springframework.security.authentication.jaas.JaasNameCallbackHandler"/>  
25:          <bean class="org.springframework.security.authentication.jaas.JaasPasswordCallbackHandler"/>  
26:        </list>  
27:      </property>  
28:      <property name="authorityGranters">  
29:        <list>  
30:          <bean class="org.collectionspace.authentication.spring.CSpaceAuthorityGranter"/>  
31:        </list>  
32:      </property>  
33:    </bean>  
34:    <bean id="userDetailsService" class="org.collectionspace.authentication.spring.CSpaceUserDetailsService">  
35:    </bean>  
36:  </beans>  

Listing 1

As you see, authentication-manager refers to jaasAuthenticationProvider (#12). This JAAS authentication provider is using the same value cspace for loginContextName (#16-18) as the value of the attribute named name (#1 Listing 2) in application-policy of JBoss's login-config.xml as shown below. This is important. This is how Spring security framework finds out the required JAAS login module from the JBoss environment and ties it to its own JAAS authentication provider.

1:  <application-policy name="cspace">  
2:    <authentication>  
3:      <login-module code="org.collectionspace.authentication.jaas.CSpaceJBossDBLoginModule"  
4:             flag="required">  
5:        ...
6:      </login-module>  
7:    </authentication>  
8:  </application-policy>  

Listing 2

From application-security.xml in Listing 1, you may also notice that I have configured the following two entities for CollectionSpace:

1. CSpaceAuthorityGranter (#30). This class returns a set of strings. Each entry in the set represents a string for a role which is represented as Java Principal by the JAAS provider. Spring uses this string to create a corresponding AuthorityGranter.
2. CSpaceUserDetailsService (#34). Even if you do not have additional attributes for a user (e.g. user profile), this entity needs to be configured. It is useful to obtain application specific information attached to the user. For example, in CollectionSpace, this service could retrieve information such as first name, last name, contact details, etc. from services such as the Person sevice.

Problem

The CollectionSpace authentication service is packaged as a jar. This jar includes CS database realm classes, JAAS login module and the above mentioned Spring security specific classes. This jar is copied to JBoss's domain lib, e.g. (server/cspace/lib) and is available from system classpath at runtime.


The application-security.xml is packaged with the war file. This war file is built using maven. Maven could package all the Spring specific dependencies when the war is built.

At runtime, Spring framework would complain with the following:

"Cannot convert value of type org.collectionspace.authentication.spring.CSpaceAuthorityGranter to required type org.springframework.security.authentication.jaas.AuthorityGranter for property authorityGranters[0] : no matching editors or conversion strategy found"

To overcome this, I used the following dependency while building the authentication service jar.

1:      <spring.security.version>3.0.1.RELEASE</spring.security.version>  
2:      <dependency>  
3:        <groupId>org.springframework.security</groupId>  
4:        <artifactId>spring-security-core</artifactId>  
5:        <version>${spring.security.version}</version>  
6:        <scope>provided</scope>  
7:      </dependency>  

Listing 3


And the war was build using the following dependencies.

1:    <properties>  
2:      <spring.version>3.0.0.RELEASE</spring.version>  
3:      <spring.security.version>3.0.1.RELEASE</spring.security.version>  
4:    </properties>  
5:      <!-- dependencies on spring security & framework are runtime deps only -->  
6:      <!-- the following list is kept to make sure domain has these packages -->  
7:      <!-- in the cspace/lib directory -->  
8:      <dependency>  
9:        <groupId>org.springframework.security</groupId>  
10:        <artifactId>spring-security-core</artifactId>  
11:        <version>${spring.security.version}</version>  
12:        <scope>provided</scope>  
13:      </dependency>  
14:      <dependency>  
15:        <groupId>org.springframework.security</groupId>  
16:        <artifactId>spring-security-config</artifactId>  
17:        <version>${spring.security.version}</version>  
18:        <scope>provided</scope>  
19:      </dependency>  
20:      <dependency>  
21:        <groupId>org.springframework.security</groupId>  
22:        <artifactId>spring-security-web</artifactId>  
23:        <version>${spring.security.version}</version>  
24:        <scope>provided</scope>  
25:      </dependency>  
26:      <dependency>  
27:        <groupId>org.springframework.security</groupId>  
28:        <artifactId>spring-security-acl</artifactId>  
29:        <version>${spring.security.version}</version>  
30:        <scope>provided</scope>  
31:      </dependency>  
32:      <dependency>  
33:        <groupId>org.springframework</groupId>  
34:        <artifactId>spring-context</artifactId>  
35:        <version>${spring.version}</version>  
36:        <scope>provided</scope>  
37:      </dependency>  
38:      <dependency>  
39:        <groupId>org.springframework</groupId>  
40:        <artifactId>spring-web</artifactId>  
41:        <version>${spring.version}</version>  
42:        <scope>provided</scope>  
43:      </dependency>  
44:      <dependency>  
45:        <groupId>org.springframework</groupId>  
46:        <artifactId>spring-webmvc</artifactId>  
47:        <version>${spring.version}</version>  
48:        <scope>provided</scope>  
49:      </dependency>  
50:      <dependency>  
51:        <groupId>org.springframework</groupId>  
52:        <artifactId>spring-aop</artifactId>  
53:        <version>${spring.version}</version>  
54:        <scope>provided</scope>  
55:      </dependency>  

Listing 4

Note that I have used dependency scope for these dependencies. That means, I have to make these jars available at runtime, usually in the system classpath of JBoss. This is ugly, but if I did not do this, the Spring framework would complain with the error I mentioned earlier. In Spring security 3.0.1.RELEASE, it is coming from line #289 of org.springframework.beans.TypeConverterDelegate.java

My assumption here is that somehow the classes (esp. AuthorityGranter) pulled by maven while building the authentication provider jar and the war, both using default dependency scope, were not the same, even if I was using the same release versions for Spring security in both the poms. At one point I was using CI SNAPSHOTs for Spring security. Perhaps that might be the reason as the repository might be changing at the time. Dunno for sure.

If you know a better way to solve this problem, feel free to comment.

Authorization service in CollectionSpace

I have started design for the authorization service in CollectionSpace. While working on security services, I always provide some descriptions on terms used as well as core processes. Take a look at Authorization Service Description and Assumptions for more details about the service. There are some non-trivial requirements that I would like to highlight here.

1. Support multiple tenants from the same service. That is roles and policies/permissions should be tenant-qualified. One tenant's collectionmanager role may not have the same privileges as the other tenant's collectionmanager role.
2. Attribute-level access control. It appears that in the domain of collection management, attribute level access control might be more than uncommon requirement. How to enforce access control at attribute level in search functions so that performance is not degraded drastically is a big challenge.

Indeed, it is not in CollectionSpace's charter to build technology for authorization. So, we looked at a few options including XACML and Spring Security. I have taken rough notes on evaluation. Take a look and let me know if we missed something or have misinterpreted.