DataSnap in Action!

By Bob Swart, Bob Swart Training & Consultancy (Bob@eBob42.com) DataSnap is the multi-tier application development framework of RAD Studio (which includes Delphi, C++Builder, Delphi Prism and RadPHP). In this article, the power and functionality of DataSnap is demonstrated using Delphi XE by describing a small but real-world DataSnap application. The sample application is an issue monitoring and tracking system, called D.I.R.T. (Development Issue Report Tool). The system is used by remote developers in different locations to enter issue reports for projects. The new DataSnap features that we use to produce high quality multi-tier applications include: DataSnap Server Wizards producing application skeletons (server and client) HTTPS support for secure ISAPI DLLs deployment RSA and PC1 encryption filters (for example when HTTPS is not an option) Enhanced TAuthenticationManager for login support Role-based Authentication capabilities for server methods and datasets

Data Model
The data model describing the Users, the Reports and the Comments on these reported issues is as follows:

-1-

The User table contains the list of developers, testers and others that access the application. Required information are a username, (hashed) password, and e-mail address. A user also has a single role, like developer, tester or manager. We dont store the real password of the user, but only the HASH value of the password. During logon, only the hashed version of the password is sent over the (secure) connection from the client to the server. The Report table stores issue report and brief summary, plus the Project, Version (optional), Module (optional), Type of issue and Priority. A special field is used to contain the Status of the issue (reported, assigned, opened, solved, tested, deployed, closed). Also important are the date of the report, the user who reported it, and (optionally) the user who is assigned to the issue. The Comment table contains the workflow of whats being reported and done about the issue. These three tables, linked together by the different ReportID, ReporterID, AssignedTo and UserID fields, are used as basis for the DataSnap Server application.

DataSnap Server
Delphi XE Enterprise (and Architect) supports three Wizards to create DataSnap Servers: a DataSnap REST Application, a DataSnap Server, and a DataSnap WebBroker Application. For the D.I.R.T. application, we need a DataSnap Server type accessed in a secure and safe way. To allow access from mobile devices and wireless connections, the HTTPS protocol was chosen, using an SSL/TLS certificate for the encrypted communication and secure identification of the server. HTTPS means an ISAPI DLL produced by either the DataSnap WebBroker or the DataSnap REST Application wizard. The REST wizard generates additional files for the REST application, which are not needed at this time, so the choice was made to use the DataSnap WebBroker Application wizard. The ISAPI DLL is deployed on Microsoft Internet Information Services (IIS), using an SSL certificate for HTTPS support. The New DataSnap WebBroker Application wizard includes options to use Authentication and Authorization, and the choice of a Server Methods Class with Sample Methods. Authentication means checking if a user is who he or she claims he is, while authorization controls which functionality is explicitly allowed or forbidden based on a user role. Our example uses both Authentication and Authorization, as well as a Server Methods Class, but no Sample Methods:

-2-

TDSServerModule is used as ancestor for the Server Methods class, allowing the export of server methods and datasets (using TDataSetProviders).

Login and Authentication

The TDSAuthenticationManager component on the Web Module controls the authentication and authorization.

-3-

In the OnUserAuthenticate event handler we verify the User, Password as well as the used protocol (to make sure its HTTS and not HTTP). To get access to the DIRT database with usernames and passwords, an TSQLConnection component and SQL query is used, performing a SELECT of the UserID and Role from the User table. The Role retrieved by the SELECT command is added to the UserRoles collection.
procedure TWebModule1.DSAuthenticationManager1UserAuthenticate( Sender: TObject; const Protocol, Context, User, Password: string; var valid: Boolean; UserRoles: TStrings); <snipped> SQLQuery.CommandText := 'SELECT UserID, Role FROM [User] WHERE ' + ' (Name = :UserName) AND (PasswordHASH = :Password)'; <snipped>

Authorization
DataSnap connects user roles with server classes and server methods. In the DSAuthenticationManager OnUserAuthorize event the authorization can be implemented in an optimistic or pessimistic way, depending on your situation. Optimistic means that any operation is allowed, unless explicitly forbidden, while pessimistic means no operation is allowed, unless explicitly allowed. For our example, security is important, so we used the pessimistic approach. Assignment of authorizations by role can be done at the server methods level. Admin has the right add new users, a manager can view all issues but only in a read-only way, a tester can report new issues, and both the tester and developer can add comments to issues.

Server Methods for the Client

Extend the TServerMethods1 class with custom server methods, like GetCurrentUserRoles to return the role that the current user belongs to.

Server Method to Get All Issues

A more useful server method, only available for the Manager role, returns the list of all reported issues, within a range of MinStatus and MaxStatus.
[TRoleAuth('Manager')] // Return all issues (read-only) between MinStatus..MaxStatus function GetIssues(MinStatus,MaxStatus: Integer): TDataSet;

This server method needs a TSQLConnection and TSQLDataSet component on the server module. The TSQLDataSet selects all fields from the Report, using the Status field in the WHERE clause. The query returns a read-only list, and is joined with the User table replacing ReporterID and AssignedTo with actual names.

-4-

SELECT "ReportID", "Project", "Version", "Module", "IssueType", "Priority", "Status", "ReportDate", UReporterID.Name AS "Reporter", UAssignedTO.Name AS Assigned, "Summary", "Report" FROM "Report" LEFT OUTER JOIN [User] UReporterID ON UReporterID.UserID = ReporterID LEFT OUTER JOIN [User] UAssignedTO ON UAssignedTO.UserID = AssignedTO WHERE Status >= :MinStatus AND Status <= :MaxStatus ORDER BY Status

Implement GetIssues using the SQL statement above to return a DataSet.

Exporting Data: Open Issues

We can also export a TDataSetProvider to the client, where changes can be made to the data (insert, update and/or delete), and applied back to the server. A developer working on an issue report should also see all comments, so the TDataSetProvider exports records from the Reports and Comments tables in a master-detail relationship. This is implemented using two TSQLDataSet components sqlReportUser and sqlComments, and a TDataSource component dsReportUser pointing to sqlReportUser and used as DataSource for sqlComments. The TDataSetProvider called dspReportUserWithComments exposes the master sqlReportUser.

The sqlReportUser retrieves Report records with either the ReporterID or the AssignedTo fields equal to the current user:
SELECT "ReportID", "Project", "Version", "Module", "IssueType", "Priority", "Status", "ReportDate", "ReporterID", "AssignedTo", "Summary", "Report" FROM "Report" WHERE (Status >= :MinStatus) AND (Status <= :MaxStatus) AND ((AssignedTo = :AssignedTo) OR (ReporterID = :ReporterID))

This SQL command returns 12 fields, configurable using the Fields Editor.

-5-

ReportID is an autoincrement key field, and needs pfInUpdate of the ProviderFlags set to False, but the pfInKey set to True. The SQL command has four parameters.

The MinStatus and MaxStatus input parameters are provided by the client and should be kept. AssignedTo and ReportedID must be removed from the Params collection, so they will not show up at the client side, and will be filled in by the server itself, based on the UserID value in the session of the current user. The detail sqlComments uses a parameter ReportID that connects the comments to the master report record:
SELECT CommentID, ReportID, UserID, CommentDate, Comment FROM Comment WHERE (ReportID = :ReportID)

Modify the ProviderFlags for the primary key CommentID to set pfInUpdate to False, and pfInKey to True, ensuring the primary key CommentID is never sent to the server as assignment, but used in the WHERE clause to locate records. Stepping through the result of the master, each time we move one record in the master record set, we must reset the detail by closing and reopening sqlComments.

-6-

This enforces the detail query to be executed for every master record, filling the nested dataset at the server side before exporting the TDataSetProvider to the client. The event is named AS_sqlReportUserAfterScroll and not the default sqlReportUserAfterScroll to ensure its not visible as server method. We need to assign the AssignedTo and ReporterID parameters in the BeforeGetRecords event handler of the TDataSetProviderm, using the UserID of the current user. This ensures that all parameters of sqlReportUser are filled before the query is opened, so any user who requests the Report and Comments records from dspReportUserWithComments will only get the reports where the user was either the reporter (ReporterID) or the one assigned to it (AssignedTo).

TDataSetProvider Role Based Authorization

The Roles property of the TDSAuthenticationManager contains a collection of Role items with ApplyTo, AuthorizedRoles and DeniedRoles properties. Use ApplyTo to specify a method name, class name, or class name followed by a dot and a method name. Only the role Developer should be allowed to call the 7 pre-defined IAppServerFastCall methods, so add these 7 methods to the Apply property as follows:

Server Deployment
The DataSnap D.I.R.T. Server is deployed as ISAPI DLL in Microsofts Internet Information Services (IIS) using the HTTPS protocol. Alternately, it can be deployed as stand-alone DataSnap server with RSA and PC1 filters to encrypt the transport channels. In the latter case, use TCP/IP as transport protocol and not HTTP, since TCP/IP is a lot faster.

-7-

In IIS we can configure application pools with automatic load balancing, recycling, and limitation of the CPU and memory usage of the server. An SSL certificate is required for the HTTPS option, like done for domain www.bobswart.nl. A virtual directory DataSnapXE and application pool are configured, and the DirtServer.dll is available as https://www.bobswart.nl/DataSnapXE/DirtServer.dll Apart from the DirtServer.dll, the required database driver: dbxmss.dll for SQL Server 2008 must also be deployed (but with the MidasLib unit in the uses clause, the MIDAS.dll does not have to be deployed). Deploying the DataSnap standalone server, using TCP/IP and the RSA and PC1 filters, also requires two Indy SSL DLLs: libeay32.dll and ssleay32.dll., needed for the RSA filter (which encrypts the password used by the PC1 filter). Theese DLLs are also required by the DataSnap client, whether connected to the TCP/IP server using the RSA and PC1 filters, or to the ISAPI filter using HTTPS.

DataSnap Client
The DataSnap client uses a TSQLConnection with Driver property set to Datasnap, https as CommunicationProtocol, Port 443 and HostName set to www.bobswart.nl (feel free to connect to your own server). URLPath is /DataSnapXE/DirtServer.dll, and LoginPrompt set to False.

Login
The username and password can be assigned dynamically to the TSQLConnection Params, hashing the password:
procedure TFormClient.Login1Click(Sender: TObject); <snipped> SQLConnection1.Params.Values['DSAuthenticationUser'] := Username; MD5 := TIdHashMessageDigest5.Create; try SQLConnection1.Params.Values['DSAuthenticationPassword'] := LowerCase(MD5.HashStringAsHex(Password, TEncoding.UTF8)); SQLConnection1.Connected := True; // try to login... Server := TServerMethods1Client.Create(SQLConnection1.DBXConnection); try UserRoles := Server.GetCurrentUserRoles; <snipped>

Data Module and server methods

A TsqlServerMethod, TDataSetProvider, TClientDataSet and TDataSource component are needed next.

-8-

The SqlServerMethodGetIssues calls TServerMethods1.GetIssues server only allowed by Manager role, returning a read-only dataset with all issues. Assign a value to MinStatus and MaxStatus parameters as follows:
procedure TFormClient.ViewAllIssues1Click(Sender: TObject); // Manager - all reports (read-only) begin <snipped> DataModule1.SqlServerMethodGetIssues.Params. ParamByName('MinStatus').AsInteger := MinStatus; DataModule1.SqlServerMethodGetIssues.Params. ParamByName('MaxStatus').AsInteger := MaxStatus; <snipped>

The resulting dataset contains IssueType, Priority and Status as integer values. Map these integer values into more human-friendly string values using OnGetText events.

DSProviderConnection
Reported issues for users with the Developer or Tester role are retrieved from the dspReportUserWithComments TDataSetProvider, using a TDSProviderConnection, two TClientDataSets and one TDataSource:

-9-

DSProviderConnection connects to TServerMethods1 (in the ServerClassName property). The cdsReports retrieves dspReportUserWithComments with two parameters: MinStatus and MaxStatus. Using the Fields Editor on cdsReports, Add all Fields to show 13 fields including sqlComments.

The sqlComments contains the nested detail records, used as DataSetField of cdsComments, containing detail records for the current master record in cdsReports. Both TClientDataSets need configuration for their autoincrement primary keys (ReportID and CommentID), setting ProviderFlags pfInUpdate to False and pfInKey to True.

- 10 -

Adding Lookup Fields

The cdsReports includes integer fields that can be translated, like IssueType, Priority and Status, using OnGetText event handlers shown before. Two other fields, containing a UserID value, should also be replaced using a cdsUserNames lookup table, to produce ReporterName and AssignedName lookup fields:

Autoincrement Fields
We cannot send a value for ReportID and CommentID primary keys to the server, but we still need to assign a temporary value at the client to allow us to create new issues and new comments. For new records, use negative key values, implemented using the AfterInsert event of the two TClientDataSets:
cdsReportsReportID.AsInteger := -1;

The client application should send updates back to the server right after a post or delete. Implement the AfterDelete and AfterPost event handlers of both TClientDataSets, which can be shared in one event handler. This ensures that ApplyUpdates is called to send changes to the server, followed by a Refresh to het fresh data from the server (including the actual autoincrement primary key values). When implementing view issues for a user with the role Developer or Tester, you may want to hide the nested dataset field
// hide nested dataset field DBGridReports.Columns[DBGridReports.Columns.Count-1].Visible := False;

The code to display the reported issues is similar to the code for the Managers call to the GetIssues server method, exposed by the SqlServerMethodGetIssues. However, where the Managers result is read-only, the Developer and Tester get a dataset where modifications can be made.

- 11 -

Reported Issue Form

The Reported Issue Form can be used to edit a new or existing report, and consists of two TDataSources (dsComments and dsReports) and a number of data-aware controls.

Comments are displayed in a TDBGrid, with the current comment details shown in the TDBMemo. Adding a comment is not be done in the grid but in a new form (not shown here), which also offers the option to change the status of the reported issue, and hence get actual progress.

Client Deployment
With the MidasLib unit to the uses clause of the DataSnap Client, we end up with an almost stand-alone executable. However, since the ISAPI DLL uses HTTPS, and the TCP/IP stand-alone server uses the RSA and PC1 filters, the libeay32.dll and ssleay32.dll files must also be deployed with the DataSnap Client.

Summary
This article described a small but real-world secure DataSnap Server application, deployed on Microsofts Internet Information Services as an ISAPI DLL. Security was implemented using authentication and authorization as well as HTTPS (or RSA/PC1 filters for a stand-alone server) for a secure transport channel. You can find actual code for the application at http://www.embarcadero.com/rad-in- 12 -

action/datasnap-xe. Techniques were shown to connect the DataSnap Client to the DataSnap Server in order to call server methods and work with exported DataSetProviders. Details included how to work with autoincrement fields, especially in combination with master-detail and nested datasets. The application would not have been possible without the many new features and enhancements in found in DataSnap XE. With the release of Delphi XE, DataSnap was again substantially expanded and enhanced compared to DataSnap 2010 or 2009, and a lot improved since the COM-based original versions of DataSnap and MIDAS.

References
For code sample of the application describe in this article, see http://www.embarcadero.com/rad-in-action/datasnap-xe RAD Studio in Action DataSnap 2010 white paper http://www.embarcadero-info.com/in_action/radstudio/db.html Delphi XE DataSnap Development courseware manual http://www.ebob42.com/courseware/

- 13 -