Provisioning Connectors and Grouper


Grouper provides virtual organizations with an easy and scalable framework for access control to collaborative web applications such as wikis, blogs, group calendars, and mailing lists. Provisioning users with correct information and privileges is vital in this framework. As a result, having a reliable set of connectors is an important requirement to ensure that data is provisioned in a reliable and correct manner from the Grouper registry to corresponding data stores. This report describes the work done in Newcastle University with respect to provisioning connectors, in particular web service based connectors and LDAP provisioning connectors.

Table of Contents

1........... Introduction

1.1 Controlling user access with Web Services

2........... Web Services Provisioning

2.1 Secure Web Services Provisioning with WS-Security

2.2 How users interact with the Grouper Web Service

2.2.1 How the system works

3........... LDAP Provisioning

4........... Overall Conclusions

4.1 Lessons learned

4.2 Future work


In September 2007, the gFivo reported on our research and findings on the Grouper Subject API. We followed this up with a report on the usage of the Grouper system to control access to admin controlled and self service web resources. Here, we report on the usage of reliable and secure connectors that provision user information to and from the Grouper Registry from a suitable data store, for example a LDAP data store, relation database (e.g. Mysql) or Active Directory. Provisioning in the Grouper context, enables group information in Grouper to be provisioned to a LDAP directory or Active Directory. It also enables user information such as attributes and permissions, which in Newcastle University is hosted in Active Directory, to be provisioned by Grouper. Grouper provisions this user information and sets up wikis, blogs and other collaborative tools with the relevant user permissions and requirements.

Controlling user access with Web Services

Provisioning is vitally important in Newcastle University with its large academic population and various student services. User details, permissions and requirements have to be provisioned to collaborative tools such as wikis and blogs in a secure and reliable manner. Grouper performs this task by retrieving group information from the Grouper registry and provisioning it user applications in a reliable and secure manner.

Thus far, we have researched two different provisioning connectors: Web Services provisioning and LDAP provisioning.

At the time of writing, Grouper did not provide a Web Service interface for accessing groups. However, after much activity and discussion within the Grouper community, a Web Service interface was deemed necessary because it allowed for easier user interaction. The Grouper UI can be intimidating to someone who is not familiar with Grouper concepts and design. A simpler approach based on a Web Service interface was developed to meet the Grouper community needs. Newcastle University contributed to the Web Service part of Grouper by implementing WS-Security functionality and enabling WS-Security authorisation and authentication on both the service and client end.

We also worked with Ldappc to provision users from our Grouper Registry to a LDAP server. However, this was more of a proof-of-concept installation than a complete and through installation. This is because Newcastle University uses Active Directory (AD) to provision user accounts. Ldappc is optimized towards openLDAP provisioning and any integration with Active Directory might have knock on effects to our applications. Newcastle University’s central Active Directory contains vital information related to users, devices, networks, in addition to system configuration information. Any misrepresentation of data can lead to network or system failure and any overloading of provisioning resources may also lead to system failure. The reasons mentioned above meant that it was too risky to provision to our Active Directory with Ldappc.

Nevertheless, we are well positioned to provision user accounts to openLDAP in the near future if need be. We are exploring the use of openLDAP as a separate data store because many applications like email clients and calendars for example, require the provisioning of user and group permissions from a central repository. This central repository can be in form of openLDAP, Active Directory, or any reliable data source for that matter. openLDAP would be an ideal compliment to our existing Active Directory installation because Active Directory is not totally compliant with LDAP specifications, hence breaking interoperability.

Web Services Provisioning

Web Services provisioning is important when you wish to present a Web Service in a customised manner to a specific set of users. Even in the case of Web Services developed for internal consumption, there will be varying users across the departments with different requirements and client interfaces.

Web Services can broadly be divided into two categories, anonymous and provisioned. A user can consume an anonymous Web Service without authentication prior to invoking the Web Service. For example, anyone can invoke a stock quote Web Service without prior authentication.

A provisioned service on the other hand, requires the management of user accounts and other related information with the service. The Grouper Web Service is an example of a provisioned service, whereby the user is associated to a particular role and assigned privileges accordingly. One of the main reasons we developed a Web Service interface to the Grouper API is because the current Grouper interface is fairly complex to someone who is not familiar to Grouper terminology. The usage of Stems, Subjects, data sources, and composite groups would probably intimidate most non-technical users (i.e. a research group leader). Although it's entirely possible to implement a simpler interface by configuring the Grouper UI code, we opted for a Web Services approach as we plan to implement more Web Services in the near future, and working with Grouper would provide us with valuable experience in Web Services development. This enabled us to create simple bespoke applications that manage group information in a given application by talking to Grouper via Web Services. It should also be noted that there is an ongoing project in the grouper community to simplify the grouper UI and make the terminology more accessible to the first time user.

We see this work as complementary to the existing Grouper UI, whereby the Grouper UI is deployed as a “power tool” for the use of system administrators, and as apotential tool for role out to the wider user population. The PHP, .NET and Java applications operated by Web Services is used for simple use cases like self registration, moderated registration, and updating of Grouper lists.

Secure Web Services Provisioning with WS-Security

Certain Grouper functions can be made available to everyone, i.e. listing all the members of a particular group. However, certain functions should only be performed by users who are authorised to do so, for example adding/deleting a user from a group. This requires an authentication and authorization framework for securing resources that include applications and Web Services. The preferred method of authentication in a Web Services environment, especially after the release of Axis2, is WS-Security. Although HTTP authentication is still supported, it is has been superseded by WS-Security. We hope to provide support for both HTTP authentication and WS-Security (in the form of separate servlets) because HTTP authentication is still a very viable method of securing web applications. Web application security via HTTP authentication is a mature, widely used technology and as such no new technologies or class libraries need to be mastered by developers.

Rampart is an Axis2 module that implements WS-Security functionality and can easily be added to a base Axis installation. WS-Security supports 4 different actions: Timestamping, Authentication, Encryption and Signature. At the moment, we are only concerned with Authentication. Authentication is based on a UsernameToken, and consists of a username and password.

There are two different configuration approaches to Rampart: the parameter based approach and policy based approach. The parameter based approach is extremely rigid in its implementation and incorporates more of a static configuration approach. The client's username/password has to be hardcoded on the client side, and a different configuration file is required for each possible client. This makes it unsuitable for many user cases. The policy based approach on the other hand, enables a more dynamic approach to configuring client based options. The Rampart enabled Grouper Web Service was deployed as a policy based service exactly for these reasons.

Figure 1 . SOAP request header with user credentials

Rampart can be engaged on either the service or client side or both. Rampart-1.3.mar and addressing-1.1.mar files have to be added to both the service and client installations if Rampart is enabled on both the service and client side. Rampart only has to be engaged on the client side if a Java Web Service client is used. If for example, a .NET client is used instead, Windows Communication Foundation (WCF) has to be used in place of Rampart to provide the Webservices-security functionality. The primary reason for using Rampart is that it promotes WS-Security standardisation within the Grouper community.

Rampart implements the java.auth.CallbackHandler interface to set/retrieve the username/password of a particular user. This class might access an LDAP directory or use other methods for associating a username with a password, but thus far we have a simple callback class that returns arbitrary values as a proof-of-concept. The callback class keeps the actual service and authentication method separate. The service doesn't need to know how the user was authenticated. The callback class implements the authentication method and it is upto the developer to implement this class. This is a lightweight integration and as such, very little code needs to be integrated into the Grouper Web Service.

Figure 1 show how user credentials are placed in a SOAP request message when Rampart is enabled. The Grouper Web Service than checks the user credentials against the Callback class to ensure the user is authorised to access the service.

Rampart can also extract the results of security processing at any state of the execution flow. The UsernameToken information can be extracted at the service side in a similar manner to REMOTE-USER (with HTTP authentication). This is especially useful when deciding if a Grouper user is authorised to perform a certain task (for example, if the user can add/delete members to/from a group).

So in brief, WS-Security standards were successfully used to secure Grouper Web Services without too much difficulty using Rampart. The different configuration approaches to setting up Rampart enabled services and clients were initially confusing because there is not that much material on the topic. However, it was fairly straightforward to implement dynamic client/service configuration with the policy based approach.

How users interact with the Grouper Web Service

As part of the development process, Java, PHP, and .NET clients were deployed to consume the Grouper Web Service. From our perspective, Java, PHP, and .NET clients are integral to the uptake of Grouper Web Services by Newcastle University. We support numerous platforms and clients should exist for these platforms. Much of our testing was done with Java clients before porting most of the functionality to PHP and .NET.

How the system works

The Newcastle University wiki service was used to trial the integration of Web Services with Grouper. Our wiki service was used as a trial the Grouper system because it’s a relatively new service and also proved to be the service with the most group fluctuation (members joining/leaving groups at a user's request).

Currently, Newcastle University has in the region of 30 wikis deployed in a pilot wiki service to support the research community. Historically, access control was achieved by the use of hand crafted .htaccess files for each wiki. These .htaccess files contain Apache httpd specific commands that restrict user access to the wikis. The group memberships for each wiki range from 3 users to 50+ users and a great deal of user churn is exhibited by the research groups. The high levels of change exhibited by research group membership make it an ideal candidate for testing the use of Grouper Web Services. Both the user community and the administrators realise that hand edited .htaccess file are not scalable in terms of staff time and present a barrier to further take up of the wiki service.

The general URL format for Wikis at Newcastle University is (where group name is a name provided by the Wiki administrator prior to setup). An admin script that interfaces with the Grouper Web Service was then installed in the group’s directory with the URL format of as shown in the screen shot in Figure 2.

Figure 2 . Screen shot of admin script.

As shown in the screen shot, there is an “Action” field in the form which defines whether a user should be added or deleted from a group. The “Username” field in turn defines the user who shall be added or deleted from a group. The greyed out area at the bottom lists the users who are currently members of the group (if the access control list has been updated, this is read from the .htaccess file after the update from the web service).

Figure 3 . Interaction flow between admin script and Grouper Web Service

When the user submits the form, the Grouper Web Service is called. The Grouper Web Service in turn interacts with the Grouper system in the form of a SOAP request. The Grouper Web Service then returns the results of the call in the form of a SOAP response. This SOAP response is then forwarded to the admin page.

The admin page then reloads itself and shows the result (success/failure) of the command issued and the new list of users that belong to the group. The flow of interaction is shown in Figure 3.

Fault tolerance was also introduced whereby the Grouper Web Service is called for a maximum of 10 times before the PHP client gives up. A Web Service call only returns an error after the Grouper Web Service has been invoked for more than 10 times unsuccessfully. This fault tolerance was introduced into the system to make it more robust to persistent Web Service calls.

LDAP Provisioning

LDAP Provisioning Connector (Ldappc) automates the provisioning of groups, memberships, and permissions from the Grouper Registry into an LDAP directory. Ldappc supports a whole range of target LDAP schemas for representing information about relationships and roles about users in an organization. It makes use of the Grouper API to perform LDAP provisioning operations.

Members of Grouper groups can only be provisioned to a LDAP server using Ldappc. Ldappc will not create or delete person entries in LDAP. That's presumed to be the province of the existing IdM operation. As such, a LDAP source containing users that are members of the Grouper groups should already be in place. It *will* add/delete/modify group entries that correspond to grouper groups, and it will add/delete/modify a membership attribute in existing person entries (and any other entry types that are associated with group memberships).Ldappc looks for users in LDAP to determine the DN of entries corresponding to grouper group memberships to add that value to each group's member attribute.

It is assumed that the LDAP server has been installed and is in working order before proceeding with the rest of the Ldappc deployment. Configuring OpenLDAP as described above is more to do with tweaking with the ACL part of slapd.conf. As default, Ldappc binds to the LDAP server as rootDN so it should be able to read/write anything.

Newcastle University uses Active Directory (AD) to provision user accounts. Active Directory is an implementation of LDAP directory services and as such, and it would be fairly straightforward to provision user accounts into our existing Active Directory implementation. However, this is not an option we wish to pursue at the moment because it because it is a mission critical service and we require it to be online at all times. At the moment, Ldappc is optimized towards openLDAP provisioning and we cannot be certain of the end result. A test LDAP server was installed and configured before proceeding with the rest of the Ldappc deployment. Configuring OpenLDAP is more to do with tweaking the Access Control List (ACL) part of slapd.conf. As default, Ldappc binds to the LDAP server as rootDN so it is configured to read/write anything

Most of the installation and configuration of Ldappc was the result of tips from the Grouper mailing list and from personal experience, and so figuring out the steps that worked were bit of a challenge.

One of the main hindrance to a complete testing of Ldappc was the fact almost all of our user accounts are stored in a JDBC database. Much of the data provisioning from the Grouper Registry was to a test LDAP server. This test LDAP server did not contain enough user accounts to perform a complete and through load testing in terms of indices, query caches, and connection handling. However, as a proof-of-concept, the installation and configuration of Ldappc was successful.

Overall Conclusions

The gFivo project has successfully demonstrated the use of Web Services to provision user details from Grouper to collaborative tools, especially wikis. The use of Web Services quickened the administration of Wikis considerably and meant that .htaccess files for wikis are updated almost instantaneously.

We have also contributed to the Web Services codebase for Grouper by implementing WS-Security standards at both service and client level. It is envisaged that the code developed for Grouper will be used by other departments in Newcastle University to secure their own services.

We also worked with Ldappc to provision group information from Grouper to openLDAP. This was more of a proof of concept installation since our central repository is Active Directory. However, openLDAP is an ideal compliment to our Active Directory repository, and it is hoped any experience gained during the provisioning of data to openLDAP servers will be beneficial in the future.

Lessons learned

The Web Service interface developed for Grouper and to which we contributed to, is much simpler than the Grouper UI. It integrates well with our admin scripts and presents the user with a more accessible interface. The Web Service coupled with our admin scripts considerably reduces the administration time for a group update.

Currently, much of the Web Services being developed at Newcastle University are secured by HTTP authentication. This is primarily because HTTP authentication is a mature and widely used technology and our developers are comfortable working with it. However, WS-Security standards are actively being promoted by the Web Services community and supersedes HTTP authentication. The work done in securing Grouper Web Services will go a great way in implementing WS-Security standards for other web applications in Newcastle University. Some of our services are .NET based and .NET supports WS-Security standards in the form of Windows Communication Foundation (WCF). SAP Web Services is also in consideration by Newcastle University. It istherefore strategically imperative that our services, in the form of Java, PHP, .NET and SAP, are capable of employing a security mechanism based on WS-Security standards, as this is likely to enhance interoperability and increase the relevance of grouper as a group management product in the university context.

It is also hoped that the experience gained while working with Ldappc will help us if and when we use openLDAP as an additional data repository. Unlike Active Directory, openLDAP conforms to LDAP standards and protocol and promotes interoperability.

Future work

Currently, the LDAP provisioning connector, Ldappc, assumes LDAP functionality is built into applications at every level. However, this is not the case in Newcastle University since our primary repositories are Active Directory and MySQL relational database. We would like the option of provisioning data from Grouper to a relational database as we are more comfortable working with relational databases. Relational Database Provisioning Connector (RDBPC) which is currently being developed by University of Kansas, and will be released soon, is seen as an alternative to Ldappc. It is able to provision both LDAP and relational databases using flexible, extensible downstream provisioning connectors. We plan to work with RDBPC far more extensively than Ldappc because it will be able to perform a very important functionality from our perspective, i.e. provision Grouper data into a relational database.

We also plan to research the provisioning of group information into Sympa. Sympa is a mailing list software and will provide us with a different backend to LDAP or relational databases. Sympa has it’s own web service based interface so integration may well be a relatively simple matter.