Java dumps

I recently had to debug a problem with the MDM Workbench where exporting a tailoring project for Information Server didn’t do anything. In fact it didn’t even report any problems!

Unfortunately the code in question likes to put a brave face on things and just reports that everything was OK, even when something goes wrong. This was the perfect opportunity to try out some of the diagnostic tools available for the IBM Java runtime, which I’ve been meaning to try for ages. I had an idea where the problem was likely to be but to find out for sure I started the workbench using the following command line:

eclipsec -vmargs -Xdump:system:events=catch,filter=java/lang/AbstractMethodError#com/ibm/mdm/tools/export/infoserver/job/MDMDatabaseDAO.queryDatabaseWithoutFilter

Sure enough the failing export produced a dump which I could check using the Memory Analyzer tool. You can get the IBM version via IBM Support Assistant but it’s probably easier to get the standard Eclipse Memory Analyzer and add the required IBM plugins from the DTFJ update site.

I’m fortunate enough to work in Hursley so I could pester someone who works on IBM Java runtime diagnostics, but there’s also a helpful article on developerWorks with details of how to trigger dumps, and how to run queries using OQL:

Debugging from dumps: Diagnose more than memory leaks with Memory Analyzer

So mystery solved- if you have an Oracle database and want to exporting tailoring projects for Information Server, make sure you set up the database connection with a more recent JDBC driver than the defaults.

Advertisement

Getting Technical: Resources for MDM Developers

One reason for the lack of new posts here is that I’ve been attempting to write my first article for the Mastering Data Management blog. After several false starts, and falling back to old school pencil and paper to finally get going, it’s finally done! So, for anyone interested in master data management, here is the very latest Mastering Data Management front page hot off the press:

New home for MDM Workbench on developerWorks

If you’ve visited any developerWorks spaces recently, you’ll have noticed there’s been a bit of a change; the old spaces application has been sunset. All is not lost though, and redirects are in place for the various new homes for old spaces, including the MDM Workbench space.

Instead of just migrating from spaces to a group on My developerWorks, I took the chance to open it up to a wider community of MDM Developers. While I do miss some of the features of the old spaces, one big advantage of the new group is that it’s much easier for anyone to participate.

Apart migrating information from the old MDM Workbench space, most of the activity in the group so far has been new people joining, which is a great start. I’m also looking forward seeing who the first person to add a new bookmark or feed will be. (I’m almost tempted to have another little competition to encourage people with a much prized Hursley postcard, just for a bit of fun!)

Anyway, if you’re a developer working with any of IBM’s MDM products, have a look at the new MDM Developers group, and let me know if you have any comments or suggestions.

MDM Workbench developerWorks blog

I’m very excited to announce that the MDM Workbench now has its own dedicated blog on developerWorks! If you’ve found any of my MDM Workbench posts useful, I would definitely recommend taking a look. Iain might have been the last person in my team to start blogging but he’s already posted two great articles on the new blog:

• Installing MDM Workbench
• Using JUnit to test Web Services Generated by the MDM Workbench

I’ll also be using the new blog to post MDM Workbench news which I was previously featuring on the MDM Workbench space.

That isn’t the end of MDM posts here either; I’ll still be writing about my own views of MDM and the workbench, and hopefully continuing my Social Bookmarking example.

Master Information Hub: Getting Started

Like InfoSphere Master Data Management (MDM) Server, Master Information Hub (MIH) comes with Rational based model-driven tools to develop custom data domains. Without the rich data model which comes with MDM, the MIH Workbench will be essential to make use of MIH to master your own information. The blank canvas also makes MIH an ideal base for simple examples; I’ve been experimenting with a social bookmarking data domain which I hope will be a good way to demo some of the new workbench features, including the ability to create a complete domain with data stewardship user interface without writing a single line of code.

Although I plan to use the MIH Workbench for this example, the same features are available in the MDM Workbench. There are a few things to do before getting to the interesting bit though, and that’s what this post is about.

Installing the workbench

Unfortunately there are some pitfalls which can catch the unwary when installing the MIH Workbench. For example, there are some very specific prereqs which do not get checked at install time, so it’s worth spending a bit of extra time checking the documentation before you install. The readme, which does include workbench information, is available on the external information center, however the workbench installation documentation is only available in the PDFs and stand-alone information center provided with the product. (I’ll post a link if I do find it available on the internet in the future.)

If you do encounter any strange problems later (missing workbench functionality or a hang when generating web service clients being the most likely) the software requirements should be the first thing you check. In particular, because there can be problems getting the web service client generation ifix applied properly, I’ve used the latest fix pack level for RSA instead.

Prepare a new workspace

Once you have the workbench installed successfully, I recommend creating a clean new workspace. It’s important to use a short path length, for example C:\ibm\mih, to avoid Windows length limits when importing EAR files in to RSA. If you are not prompted to choose a workspace location when starting RSA, use the File > Switch Workspace… menu option instead.

Next open the Java EE perspective (click on the Show all option to find it in the list); this should prompt you to enable the required capability.

And finally, make sure the default JRE is WebSphere v7.0 JRE and the compiler compliance level is set to 1.6 in the workspace preferences.

Create a WebSphere profile

As well as a fresh workspace, I also create a new WebSphere profile for testing MIH applications. Launch the profile management tool from the Server > WebSphere Application Server workspace preferences page.

Note: the tool which will configure the WebSphere profile for MIH has very specific ideas about where this profile will be. Make sure you are using the WebSphere Application Server Test Environment installed using Installation Manager, and create your profile in the default location.

Use the advanced profile creation option to specify the following configuration settings:

• create the server using the development template
• disable administrative security
• disable the option to run application server processes as a Windows service

After creating a new profile, add the server to the RSA Servers view to make it easier to publish the MIH application later. Open the server in the Servers view to change a couple of settings:

• run with resources on server
• never publish automatically

You can set the first of these while adding the server but when I do, I always forget to turn off automatic publishing- it just takes too long on my development machine!

Set up the development environment

After double checking the readme (there’s a specific section with Additional notes on the Development Environment Setup Tool) everything should be ready for a smooth run through the MIH Development and Test Environment (DEST) Wizard. Rather than running all the operations provided by the wizard in one go, I find it more reliable to complete the setup in five separate steps, with some checking along the way to confirm everything is going well.

This is the sequence I follow:

When using the DEST wizard, double check all the values it asks for. The defaults may not be correct for your system and not all the values are validated. These are the settings I used, where they differ from the defaults:

WebSphere version : WebSphere Application Server v7.0
Profile name : [This should be the profile you created earlier]
Backup profile at : C:/MIHTEMP
WebSphere home : C:/Program Files/IBM/SDP75/runtimes/base_v7
Application distribution file : C:/Temp/MIH900_WAS_AIX.tar.gz
Unpack distribution file to : C:/MIHUNPACK
Deployment name : MIH
Temporary output folder : C:/MIHTEMP/MIH.ear
Database name : MIHDB

To find the DEST wizard for the steps below, open the File menu and choose New > Other…, then look for InfoSphere Master Information Hub > Development and Test Environment.

Step 1:

Run the Restore the Development Environment Setup Tool operation.

Step 2:

Run the first five operations:

• Import Application Projects into Workspace
• Customize Configuration Files
• Backup WebSphere Profile
• Configure WebSphere Profile
• Create an Application Database

Check that the database was created correctly, for example using the Data perspective in RSA, and make sure that the data sources are working on the server:

• Right click the server in the Servers view and choose Administration > Run administrative console
• Open Resources > JDBC > Data sources in the navigation pane
• Use the Test Connection option to verify the DWLConfig, DWLCustomer and EventManager data sources

Logs from the create database operation can be found in the following folders of the MIHDatabase project:

• configRepository\DB2\Standard\ddl\CM\localrun
  
• CoreData\DB2\Standard\ddl\CUST\localrun
• WCCEM\DB2\Standard\ddl\EM\localrun

Step 3:

Run the next three operations :

• Clean Projects
• Build Projects
• Export Application EAR

Hopefully you won’t hit any problems with these ones.

Step 4:

To make the development process a bit simpler, skip the Deploy Application to WebSphere Profile operation and use the standard tools provided with RSA instead. Right click on your server in the Servers view, choose Add and Remove Projects… and add the MIH project (you can use Add All >> if it’s the only project available).

Step 5:

Back to the DEST wizard for the final three operations :

• Deploy Configuration
• Import Installation Verification Project
• Validate Installation

The first thing to check is the result of the install verification. Refresh the InstallVerification project in your workspace to pick up changes on the file system. You should find some response files in the testCases\xml_mih\response folder and, if all went well, they should contain ‘SUCCESS’ result codes!

If the response files contain an error, the Deploy Configuration operation is the most likely one to have gone wrong at this stage. Check whether the CONFIGELEMENT table contains any configuration data. If this table is empty, modify the Management Agent settings to connect to the database directly and rerun the Deploy Configuration operation; see this developerWorks forum post for details.

If rerunning the Deploy Configuration operation doesn’t fix the CONFIGELEMENT table or there were other errors, check the server and MIH logs. The MIH logs can be found in a logs folder in the CustomerResources project.

Next steps

Turn on automatic builds. (The DEST wizard turns this option off and it drives me crazy every time I forget to turn it back on!)

Please leave a comment and let me know if I’ve missed anything from these instructions so that I can improve them, however if you need help with a specific problem, the MDM Workbench forum on developerWorks is definitely the best place to ask.

Updated: added link to developerWorks forum post for tips on fixing configuration deployment problems. (4 February 2010)

Updated: added link to the next post in the series for creating your first simple MIH application. (19 March 2010)

Updated: added link to Installing MDM Workbench post on developerWorks. (17 April 2010)

Updated: added note about restrictions on WebSphere profile location. (25 June 2010)

Master Data Management links: August

Another random selection of MDM related links, this time all turning up in my inbox at about the same time. The first one is from a five part series on SearchSOA.com which reminded me of earlier musings on the relationship between MDM and SOA:

• SOA with MDM prevents messaging confusion

The definition of MDM still seems to me to be quite subjective, with subtle differences depending on who you talk to. (Someone I know rather unkindly suggests it’s just a glorified database!) The next two links both have something to say about what problems MDM is trying to address:

• Is smart DM an alternative to MDM?
• Can you “do” MDM without data quality?

What’s the best definition/example of MDM you’ve seen?

And finally, now seems like a good time to mention the Information on Demand conference. As well as meeting people with real world experience of this kind of thing, there are technical sessions for the MDM Workbench which is what I actually work on.

Pic and Mix

Unfortunately decorating the bathroom is higher up the to do list than blogging at the moment — I’d rather be blogging as I’ve yet to gas myself typing on a computer! — so I haven’t had a chance to mention some cool and interesting things that have been cluttering up my list of open browser tabs. While I wait for the paint fumes to subside before going to bed, here are a few of them, in no particular order…

First from the Mix and Mash Blog, and giving this post its title, Pic and Mix project from Kent County Council: I wonder if Eastleigh do anything similar.

From John’s Random Musings, Exposing your WebSphere logs as ATOM feeds: definitely want to give this a try with MDM Server.

From knolleary.net, Twitterlogue: wish twitter had been around when I was in New Zealand. Brilliant.

From developerWorks, Leverage DataPower SOA Appliances to extend InfoSphere Master Data Management Server security capabilities: looks interesting but I haven’t had a chance to read it in detail yet.

And finally, also from developerWorks, two new articles for the user interface generator:

• Build a user model with Rational Software Architect and the User Interface Generator
• Connect your user interface to existing back-end services

MDM Workbench white paper

If you’ve read my Setting up an MDM Server development environment post in the past, there is now a new white paper on the MDM Workbench developerWorks forum which I would definitely recommend taking a look at.

The White paper on using the MDM Workbench has updated information on setting up a development environment, with screen shots of the wizard used. It then takes you through subsequent tasks to create a working extension to the MDM Server, with essential information about working on the code in a team environment.

The information should provide a quick start for anyone new to the MDM Workbench. Please provide any feedback on the white paper in the forum thread.

The IET gets sociable

I recently read Yes, we can twitter while catching up on some E&T reading. Probably the most interesting bit for me was seeing @TheIET is also on twitter, so I tore off the bottom of the page with the link on to check out. Web 0.1 bookmarking then; I still like reading on paper.

The IET twitter account doesn’t look like it’s progressed beyond getting their brand on there; they don’t follow anyone, have a surprisingly small 168 followers and don’t seem to be talking to anyone. Still, hopefully it’s just a small beginning and, amongst the links to their web site, I did spot a press release about the launch of the new IET social networking site! That news somehow passed me by until now, so I’ve been investigating to see what it offers. To start with, I have yet another profile, which is not a big surprise. Earlier today I was scratching my head over a spiced up developerWorks profile. It has a bookmarking service which, for anyone in the IET new to such things is great. While I already use delicious for my own bookmarks, IET Discover combines bookmarks with groups, in what looks quite a similar way to Lotus Connections. There’s already a good selection of groups, although I’ve not found any that appear that active yet. Groups have always been a bit of a mystery to me in things like Facebook, never quite fulfilling their apparent potential, mostly ending up little more than a way to tag yourself as being interested in something.

Talking of tagging, from what I can tell on first look, I can tag my own profile, but other people can’t tag me, which seems like a missed opportunity. I think there’s much more value in tagging other people. In networks where you can tag yourself, I tend to have a poor attempt to start with, and then never return to keep the tags up-to-date.

And finally, I can watch people… except so far I’ve not found anyone to watch. I’m guessing it’s much like adding people to your delicious network.

Overall, it’s an interesting foray into the world of social networking. Like LinkedIn, it has a more professional focus, but it feels more limited by association with a single professional body. With recent homecamp, arduino and related projects in mind, I joined the electronic circuits group, but there are already more established social networks around those topics, whether IET members or not. Having said that, I think there is a place for more focused social networks. For example, I’m a big fan of developerWorks, where I’ve been trying to get some momentum for a community around the MDM Workbench, which is after all a pretty niche topic. So IET Discover looks interesting, and it has the potential to get me more involved in the IET. Time will tell how it turns out… maybe @TheIET will share their view…

UML cats

I’ve been using UML a fair bit since I started working on the user interface generator but I still sometimes need reminding what the different associations are getting at. I’ve just been checking out the next user modeling demystified that will feature on the MDM Workbench developerWorks space and it has a nice example to help:

Mind you, that’s not the best description of the cats I’ve lived with: none of them have had collars, the word ‘owner’ is a bit suspect, and they all tended to play with mice before letting them escape under the cooker!

Update: I recently found an article describing UML class relationships in more detail, which looks quite useful. (8 February 2011)